summaryrefslogtreecommitdiffstats
path: root/docs/ands_ansible.txt
blob: 70800e12c37262d379c75221da98bfb85f8f1ca8 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
Ands (Advanced aNalysis and Data Services)

Ansible groups
==============
 - OpenShift
    nodes [enforced]                            All OpenShift Nodes
    storage_nodes                               All OpenShift Nodes with attached storage
    masters [enforced]                          All OpenShift Nodes with attached storage and acting as masters
    
    Additional building blocks:
        simple_nodes                            OpenShift Node (without Storage)
        simple_storage_nodes                    OpenShift Node with Storage atached, but not acting as Master

 - oVirt
    ? ovirt_servers                             All oVirt servers
    ? ovirt_masters                             All oVirt masters

 - General
    ands_servers                                All server hardware running in the server rooms
    ands_storage_servers                        All servers with storage attached (OpenShift storage, oVirt storage, PDV storage)
    ands_cuda_servers                           All servers with NVIDIA hardware
    ands_opencl_servers                         All servers expected to run OpenCL SDK
    ands_desktops                               Destkop computers outside of the server room
    ands                                        Everything

Ansible roles
=============
 - Preparation of Vagrant nodes
    ands_vagrant_vm     vagrant                 Configures Vagrantfile and starts virtual machines if necessary
 
 - Ands-specific
    ands_facts          ands                    Detect current Ands configuration
    ands_storage        ands_storage_servers    Detects large storage device and configures LVM, sets aside space for Heketi if ands_heketi_* is set

 - General (abstract roles)
    common              ands                    Standard repositories & software
    keepalived          masters                 Keepalived Service
    openvpn             nodes                   Configures required tunnels (contains templates for all planned tunnels)
    docker              nodes                   Installs docker and configures LVM-based docker storage backend
    glusterfs           servers                 Installs glusterfs, provisions requested volumes, and mounts them

 - OpenShift
    openshift_resource  include                 Populates OpenShift resources (should be executed on master nodes)
    ands_openshift      nodes                   Configures /etc/hosts (to be independent of DNS), user accounts, GlusterFS endpoints in all projects, Heketi service if needed
    ands_kaas           masters                 For all projects configure Gluster volumes, populates OpenShift resources and configuration user configuration

 - Katrin-specific
    kaas

Ansible parameters (global)
==================
    We are listing here only variables which have the global scope, i.e. affect multiple roles. The role parameters are documented 
    within the role itself. General (non ands_*) roles should not rely on ands_* variables directly (except in 'defaults').
        => Instead the playbooks are used to set role-specific parameters based on ands_* variables. 

 - Domains
    ands_openshift_subdomain    group_vars      The default subdomain of the OpenShift cluster (kaas.kit.edu)
    ands_cluster_domain         group_vars      The default domain name of all cluster servers (.ipe.kit.edu)

 - High Availability
    ands_openshift_lb           group_vars      The FQDN of load-balancer of the OpenShift cluster (katrin.kit.edu)
    ands_ipfailover_interface   group_vars      Interface to run IPFailover service
    ands_ipfailover_vips        group_vars      Virtual IPs to maintain by IPFailover service
 
 - Networks 
    ands_storage_network           group_vars   CIDR of storage network
    ands_openshift_network         group_vars   CIDR of OpenShift network (normally, we use storage network)
    ands_openshift_public_network  group_vars   CIDR of OpenShift public network (normally, we use infrastructure network)
    ands_openshift_ingress_network group_vars   CIDR of OpenShift ingress network

 - Host names / IPs
    ands_host_id                inventory       ID is equivalent to the IP in the infrastructure network
    ands_storage_hostname       computed        IP or hostname of the host in the Infiniband storage network
    ands_hpc_hostname           computed        IP or hostname of the host in the Infiniband HPC network

 - Storage
    ands_data_device            ands_storage    The raid device where Ands LVM will be populated. Normally will be autodetected based on size.
    ands_data_vg                ands_storage    The LVM VG on the main storage device
    ands_data_path              ands_storage    The path where the large storage is mounted
    ands_data_volume_size       group_vars      If not specified, all disks space will be imideately used. Oherwise, it is possible to keep it for other uses and grow as needed.
    ands_heketi_volume_size     group_vars      If specified, some space will set aside on LVM some space for Heketi volume
    ands_storage_domains        ands_storage    List of GlusterFS volumens and mount points (configured in ands_facts and may add additional heketi domain in ands_storage)

 - Docker specific
    docker_storage_device       ands_storage    Storage device to create Docker LVM on (if not existing yet)
    docker_storage_vg           ands_storage    LVM VG for Docker volumes (autodetected if not specified)

 - GlusterFS specific 
    glusterfs_version           group_vars
    glusterfs_transport         group_vars

 - OpenShift specific
    ands_openshift_labels       setup/configs   Labels to assign to the nodes
    ands_openshift_projects     setup/configs   List of projects to configure (with GlusterFS endpoints, etc.)
    ands_openshift_users        setup/configs   Optional list of user names with contacts
    ands_openshift_roles        setup/configs   User-permissions. Configures list of admins, project users, etc.
    ands_secrets                setup/configs   Passwords, etc.

    ands_paths                  setup/configs   Default mount point paths
    ands_storage_domains        setup/configs   List of GlusterFS volumes along with hosting servers     
    ands_openshift_volumes      setup/configs   List of OpenShift volumes linked to GlusterFS volumes and paths. Used to generate PV/PVC. Can be restricted per project.
    ands_openshift_files        setup/configs   List of directories/files on the GlusterFS we want to create, ensure permissions/owners.

    kaas_glusterfs_endpoints    <hard-corded>   Specifies the name of GlusterFS endpoint, should be configured by the ands_openshift service (but currently hardcorded)

 - Project specific 
    volumes                     project         Overrides list of PV/PVC to populate in the project
    files                       project         Additional files to ensure are existing and have correct permissions
    file_owner                  project         Default user owning files
    file_group                  project         Default group owning files
    pods                        project         Configuration of the expected pods
    oc                          project         If necessary may give detailed instructions in which prioritu resources should be populated

 - KATRIN specific
    katrin_node                  group_vars     FQDN of KaaS router (katrin.kit.edu, but now katrin.ipe.kit.edu or something)
    katrin_openvpn_network       group_vars     OpenVPN network encompassing all subnets to Katrin control network
    katrin_openvpn_subnet_bits   group_vars     Subnet size
    katrin_openvpn_subnet_offset group_vars   Offset of a first subnet (according to the specified size)