ansible-playbook -i inventory/dr playbooks/openstack-provision.yml PLAY [prepend logical cluster name for all cluster resources] ****************** TASK [pre-checks : check for bad hostname] ************************************* changed: [loadbal] changed: [gfs2] changed: [gfs1] changed: [gfs3] changed: [gfs4] changed: [node1] changed: [lma] changed: [master1] TASK [pre-checks : fail] ******************************************************* skipping: [gfs1] skipping: [lma] skipping: [master1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [loadbal] skipping: [node1] TASK [pre-checks : checking openstack credentials] ***************************** changed: [gfs1] TASK [set_fact] **************************************************************** ok: [loadbal] ok: [gfs1] ok: [lma] ok: [master1] ok: [gfs3] ok: [gfs4] ok: [gfs2] ok: [node1] PLAY [OpenStack Key] *********************************************************** TASK [openstack-key : OpenStack Key] ******************************************* ok: [gfs1] TASK [openstack-key : Save new key] ******************************************** skipping: [gfs1] TASK [openstack-key : Set Hosts Keyfile] *************************************** skipping: [gfs1] PLAY [OpenStack LoadBalancer Security Group] *********************************** TASK [openstack-securitygroup-open : Security Group] *************************** ok: [loadbal] TASK [openstack-securitygroup-open : Security Group Wide Open V6] ************** ok: [loadbal] TASK [openstack-securitygroup-open : Security Group Wide Open V4] ************** ok: [loadbal] PLAY [NDSLabs OpenStack Cluster] *********************************************** TASK [openstack-system : OpenStack System] ************************************* ok: [lma] ok: [gfs4] ok: [gfs3] ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [master1] ok: [node1] TASK [openstack-system : Grab Private IP] ************************************** ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [node1] ok: [lma] ok: [gfs3] ok: [loadbal] ok: [master1] TASK [openstack-system : Wait for host up] ************************************* changed: [gfs1 -> localhost] changed: [lma -> localhost] changed: [node1 -> localhost] changed: [loadbal -> localhost] changed: [master1 -> localhost] changed: [gfs3 -> localhost] changed: [gfs2 -> localhost] changed: [gfs4 -> localhost] TASK [openstack-system : register key for host] ******************************** ok: [gfs1] ok: [lma] ok: [master1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] PLAY [CoreOS Setup] ************************************************************ TASK [pre-ansible : Get os_version from /etc/os-release] *********************** ok: [gfs1] ok: [lma] ok: [gfs2] ok: [loadbal] ok: [master1] ok: [gfs3] ok: [node1] ok: [gfs4] TASK [pre-ansible : Get distro name from /etc/os-release] ********************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [lma] ok: [loadbal] ok: [gfs4] ok: [node1] ok: [master1] TASK [pre-ansible : Init the is_coreos fact] *********************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [master1] ok: [gfs4] ok: [node1] ok: [lma] ok: [loadbal] TASK [pre-ansible : Set the is_coreos fact] ************************************ ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [loadbal] ok: [node1] ok: [lma] ok: [master1] TASK [pre-ansible : Set the bin directory path for CoreOS] ********************* ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [gfs2] ok: [node1] ok: [lma] ok: [master1] ok: [loadbal] TASK [pre-ansible : include] *************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/pre-ansible/tasks/coreos.yml for gfs1, gfs2, gfs3, gfs4, node1, master1, loadbal, lma TASK [pre-ansible : CoreOS | Create bootstrap directory] *********************** ok: [lma] ok: [loadbal] ok: [gfs1] ok: [master1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [node1] TASK [pre-ansible : CoreOS | Download pypy source] ***************************** ok: [gfs3] ok: [master1] ok: [lma] ok: [node1] ok: [gfs2] ok: [gfs1] ok: [gfs4] ok: [loadbal] TASK [pre-ansible : CoreOS | Extract pypy tar to tmp] ************************** ok: [gfs4] ok: [loadbal] ok: [gfs1] ok: [lma] ok: [gfs2] ok: [master1] ok: [node1] ok: [gfs3] TASK [pre-ansible : CoreOS | Move pypy source to python bootstrap directory] *** ok: [loadbal] ok: [gfs1] ok: [master1] ok: [lma] ok: [gfs3] ok: [gfs2] ok: [node1] ok: [gfs4] TASK [pre-ansible : CoreOS | Make pypy lib directory and link ncurses .so] ***** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [lma] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [master1] TASK [pre-ansible : CoreOS | Add python exec script to bootstrap directory] **** ok: [loadbal] ok: [lma] ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [gfs2] ok: [master1] TASK [pre-ansible : CoreOS | Add exec permission to python exec script] ******** ok: [loadbal] ok: [gfs1] ok: [lma] ok: [gfs3] ok: [gfs4] ok: [gfs2] ok: [node1] ok: [master1] TASK [pre-ansible : CoreOS | Check python executable linkage and mark .bootstrapped] *** ok: [gfs2] ok: [lma] ok: [master1] ok: [gfs3] ok: [loadbal] ok: [gfs1] ok: [node1] ok: [gfs4] TASK [pre-ansible : CoreOS | Add opt-path.sh to profile.d for adding python exec PATH] *** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [master1] ok: [gfs4] ok: [gfs3] ok: [node1] ok: [lma] TASK [pre-ansible : CoreOS | Change permissions and ownership for opt-path.sh to run as root] *** ok: [gfs1] ok: [master1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [loadbal] ok: [lma] ok: [node1] TASK [pre-ansible : Determine if Atomic] *************************************** ok: [gfs1] ok: [gfs2] ok: [lma] ok: [gfs4] ok: [master1] ok: [node1] ok: [loadbal] ok: [gfs3] TASK [pre-ansible : Set the is_atomic fact] ************************************ skipping: [loadbal] skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [lma] skipping: [node1] TASK [pre-ansible : include] *************************************************** skipping: [lma] skipping: [master1] skipping: [gfs4] skipping: [loadbal] skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [node1] PLAY [Recache facts] *********************************************************** TASK [setup] ******************************************************************* ok: [gfs2] ok: [gfs1] ok: [loadbal] ok: [lma] ok: [gfs3] ok: [node1] ok: [master1] ok: [gfs4] PLAY [NDSLabs environment setup] *********************************************** TASK [cluster-addrs-etc-hosts : write a hosts file for the cluster] ************ ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [gfs2] ok: [master1] ok: [loadbal] ok: [lma] ok: [node1] PLAY [CoreOS NDSLabs Preferences] ********************************************** TASK [coreos-ndslabs-devsystem : Set Update Strategy] ************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [node1] ok: [gfs4] ok: [loadbal] ok: [master1] ok: [lma] TASK [coreos-ndslabs-devsystem : CoreOS no locksmithd] ************************* ok: [gfs1] ok: [master1] ok: [gfs2] ok: [lma] ok: [loadbal] ok: [gfs3] ok: [node1] ok: [gfs4] TASK [coreos-ndslabs-devsystem : fix coreos docker BIP and MTU - make service dir] *** ok: [gfs1] ok: [gfs2] ok: [node1] ok: [master1] ok: [gfs4] ok: [loadbal] ok: [gfs3] ok: [lma] TASK [coreos-ndslabs-devsystem : fix coreos BIP and MTU - copy the service file] *** ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [node1] ok: [master1] ok: [loadbal] ok: [gfs3] ok: [lma] TASK [coreos-ndslabs-devsystem : Update CoreOS - may take a few minutes] ******* fatal: [gfs1]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.826127", "end": "2016-09-07 22:30:35.502974", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.676847", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.694023 3112 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.697065 3112 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.501948 3112 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [gfs2]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.864668", "end": "2016-09-07 22:30:35.597057", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.732389", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.748118 3228 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.749961 3228 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.596634 3228 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [lma]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.837703", "end": "2016-09-07 22:30:35.656112", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.818409", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.838140 2891 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.840601 2891 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.655560 2891 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [node1]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.905454", "end": "2016-09-07 22:30:35.703602", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.798148", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.819335 3072 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.823341 3072 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.703127 3072 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [gfs4]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.937222", "end": "2016-09-07 22:30:35.705202", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.767980", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.821951 2988 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.824023 2988 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.704726 2988 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [master1]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.909211", "end": "2016-09-07 22:30:35.715101", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.805890", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.828017 3011 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.829643 3011 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.714710 3011 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [gfs3]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:04.974158", "end": "2016-09-07 22:30:35.733699", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.759541", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.778662 3028 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.780266 3028 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.733211 3028 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring fatal: [loadbal]: FAILED! => {"changed": true, "cmd": ["update_engine_client", "-update"], "delta": "0:00:05.126864", "end": "2016-09-07 22:30:35.916808", "failed": true, "rc": 1, "start": "2016-09-07 22:30:30.789944", "stderr": "WARNING: Logging before InitGoogleLogging() is written to STDERR\nI0907 22:30:30.809908 3029 update_engine_client.cc:243] Initiating update check and install.\nI0907 22:30:30.811954 3029 update_engine_client.cc:248] Waiting for update to complete.\nE0907 22:30:35.916488 3029 update_engine_client.cc:189] Update failed.", "stdout": "LAST_CHECKED_TIME=0\nPROGRESS=0.000000\nCURRENT_OP=UPDATE_STATUS_IDLE\nNEW_VERSION=0.0.0.0\nNEW_SIZE=0", "stdout_lines": ["LAST_CHECKED_TIME=0", "PROGRESS=0.000000", "CURRENT_OP=UPDATE_STATUS_IDLE", "NEW_VERSION=0.0.0.0", "NEW_SIZE=0"], "warnings": []} ...ignoring TASK [coreos-ndslabs-devsystem : Reboot the system] **************************** skipping: [master1] skipping: [gfs1] skipping: [gfs4] skipping: [gfs2] skipping: [node1] skipping: [lma] skipping: [gfs3] skipping: [loadbal] TASK [coreos-ndslabs-devsystem : Wait for host up after reboot] **************** skipping: [gfs1] skipping: [loadbal] skipping: [gfs2] skipping: [gfs3] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [master1] TASK [coreos-ndslabs-devsystem : Fix hosts file] ******************************* changed: [gfs1] changed: [gfs2] changed: [node1] changed: [gfs3] changed: [loadbal] changed: [gfs4] changed: [master1] changed: [lma] TASK [coreos-ndslabs-devsystem : CoreOS set our timezone] ********************** changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [master1] changed: [lma] changed: [gfs3] changed: [gfs4] changed: [node1] PLAY [OpenStack GLFS provision volume and attach] ****************************** TASK [openstack-volume-attached : OpenStack Volume] **************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [gfs3] TASK [openstack-volume-attached : Attach OS volume] **************************** changed: [gfs4] changed: [gfs3] changed: [gfs1] changed: [gfs2] TASK [openstack-volume-attached : Register block_dev for mount] **************** ok: [gfs1] ok: [gfs3] ok: [gfs2] ok: [gfs4] PLAY [GLFS brick dir] ********************************************************** TASK [file] ******************************************************************** changed: [gfs1] changed: [gfs2] changed: [gfs3] changed: [gfs4] PLAY [CoreOS Mount GLFS] ******************************************************* TASK [coreos-mount : create /media/brick0] ************************************* ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [gfs3] TASK [coreos-mount : Format Brick dev] ***************************************** changed: [gfs4] changed: [gfs1] changed: [gfs2] changed: [gfs3] TASK [coreos-mount : create mount service media-brick0.mount job] ************** changed: [gfs2] changed: [gfs3] changed: [gfs1] changed: [gfs4] TASK [coreos-mount : notify systemd of config changes] ************************* changed: [gfs1] changed: [gfs3] changed: [gfs4] changed: [gfs2] TASK [coreos-mount : start media-brick0.mount and set to start on startup] ***** changed: [gfs1] changed: [gfs2] changed: [gfs3] changed: [gfs4] PLAY [keep original hosts file] ************************************************ TASK [copy] ******************************************************************** ok: [localhost] PLAY [start with original /etc/hosts] ****************************************** TASK [command] ***************************************************************** changed: [localhost] PLAY [Add cluster host entries] ************************************************ TASK [lineinfile] ************************************************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [master1] changed: [loadbal] changed: [node1] changed: [lma] changed: [gfs3] PLAY [replace /etc/hosts] ****************************************************** TASK [command] ***************************************************************** changed: [localhost] PLAY [ssh identity for cluster] ************************************************ TASK [lineinfile] ************************************************************** changed: [gfs1] PLAY RECAP ********************************************************************* gfs1 : ok=45 changed=14 unreachable=0 failed=0 gfs2 : ok=42 changed=12 unreachable=0 failed=0 gfs3 : ok=42 changed=12 unreachable=0 failed=0 gfs4 : ok=42 changed=12 unreachable=0 failed=0 lma : ok=33 changed=5 unreachable=0 failed=0 loadbal : ok=36 changed=5 unreachable=0 failed=0 localhost : ok=3 changed=2 unreachable=0 failed=0 master1 : ok=33 changed=5 unreachable=0 failed=0 node1 : ok=33 changed=5 unreachable=0 failed=0 root@fde4df001c3d:~# ansible-playbook -i inventory/dr playbooks/k8s-install.ymlsion.yml [DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and make sure become_method is 'sudo' (default). This feature will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. PLAY [setup ansible host vars] ************************************************* TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [gfs3] ok: [lma] ok: [gfs2] ok: [node1] ok: [loadbal] ok: [gfs4] ok: [master1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [loadbal] ok: [gfs3] ok: [lma] ok: [gfs2] ok: [node1] ok: [gfs4] ok: [master1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [loadbal] ok: [gfs4] ok: [node1] ok: [lma] ok: [gfs2] ok: [gfs3] ok: [master1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [gfs2] ok: [master1] ok: [lma] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] PLAY [all] ********************************************************************* TASK [pre-ansible : Get os_version from /etc/os-release] *********************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [loadbal] ok: [node1] ok: [lma] ok: [master1] TASK [pre-ansible : Get distro name from /etc/os-release] ********************** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [node1] ok: [gfs3] ok: [master1] ok: [lma] ok: [gfs4] TASK [pre-ansible : Init the is_coreos fact] *********************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [lma] ok: [gfs4] ok: [node1] ok: [master1] ok: [loadbal] TASK [pre-ansible : Set the is_coreos fact] ************************************ ok: [gfs1] ok: [gfs2] ok: [node1] ok: [gfs4] ok: [gfs3] ok: [lma] ok: [loadbal] ok: [master1] TASK [pre-ansible : Set the bin directory path for CoreOS] ********************* ok: [gfs1] ok: [gfs4] ok: [loadbal] ok: [gfs2] ok: [gfs3] ok: [node1] ok: [lma] ok: [master1] TASK [pre-ansible : include] *************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/pre-ansible/tasks/coreos.yml for gfs1, gfs3, gfs2, gfs4, master1, node1, loadbal, lma TASK [pre-ansible : CoreOS | Create bootstrap directory] *********************** ok: [master1] ok: [lma] ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [loadbal] ok: [node1] TASK [pre-ansible : CoreOS | Download pypy source] ***************************** ok: [gfs3] ok: [gfs1] ok: [gfs4] ok: [master1] ok: [gfs2] ok: [lma] ok: [node1] ok: [loadbal] TASK [pre-ansible : CoreOS | Extract pypy tar to tmp] ************************** ok: [gfs2] ok: [master1] ok: [gfs1] ok: [gfs4] ok: [lma] ok: [gfs3] ok: [node1] ok: [loadbal] TASK [pre-ansible : CoreOS | Move pypy source to python bootstrap directory] *** ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] ok: [gfs3] TASK [pre-ansible : CoreOS | Make pypy lib directory and link ncurses .so] ***** ok: [gfs1] ok: [node1] ok: [loadbal] ok: [gfs2] ok: [lma] ok: [master1] ok: [gfs4] ok: [gfs3] TASK [pre-ansible : CoreOS | Add python exec script to bootstrap directory] **** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] TASK [pre-ansible : CoreOS | Add exec permission to python exec script] ******** ok: [gfs1] ok: [node1] ok: [gfs3] ok: [gfs2] ok: [gfs4] ok: [loadbal] ok: [lma] ok: [master1] TASK [pre-ansible : CoreOS | Check python executable linkage and mark .bootstrapped] *** ok: [gfs1] ok: [node1] ok: [loadbal] ok: [gfs3] ok: [gfs2] ok: [gfs4] ok: [lma] ok: [master1] TASK [pre-ansible : CoreOS | Add opt-path.sh to profile.d for adding python exec PATH] *** ok: [master1] ok: [lma] ok: [gfs1] ok: [node1] ok: [loadbal] ok: [gfs2] ok: [gfs3] ok: [gfs4] TASK [pre-ansible : CoreOS | Change permissions and ownership for opt-path.sh to run as root] *** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [loadbal] ok: [gfs4] ok: [lma] ok: [master1] ok: [node1] TASK [pre-ansible : Determine if Atomic] *************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [node1] ok: [loadbal] ok: [master1] ok: [gfs4] ok: [lma] TASK [pre-ansible : Set the is_atomic fact] ************************************ skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [master1] TASK [pre-ansible : include] *************************************************** skipping: [master1] skipping: [gfs1] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] PLAY [etcd] ******************************************************************** TASK [common : Determine if Atomic] ******************************************** ok: [master1] TASK [common : Init the is_atomic fact] **************************************** ok: [master1] TASK [common : Set the is_atomic fact] ***************************************** skipping: [master1] TASK [common : Determine if CoreOS] ******************************************** ok: [master1] TASK [common : Init the is_coreos fact] **************************************** ok: [master1] TASK [common : Set the is_coreos fact] ***************************************** ok: [master1] TASK [common : Set the bin directory path for CoreOS] ************************** ok: [master1] TASK [common : Create the directory used to store binaries] ******************** ok: [master1] TASK [common : Create Ansible temp directory] ********************************** changed: [master1] TASK [common : Determine if has rpm] ******************************************* ok: [master1] TASK [common : Init the has_rpm fact] ****************************************** ok: [master1] TASK [common : Set the has_rpm fact] ******************************************* skipping: [master1] TASK [common : Init the has_firewalld fact] ************************************ ok: [master1] TASK [common : Init the has_iptables fact] ************************************* ok: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : Create systemd dropin directory] ******************************** skipping: [master1] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [master1] TASK [etcd : Install etcd] ***************************************************** skipping: [master1] TASK [etcd : Force etcd_source_type to github when packages are not available] * skipping: [master1] TASK [etcd : Install etcd from github] ***************************************** skipping: [master1] TASK [etcd : Create etcd config directory] ************************************* changed: [master1] TASK [etcd : Write etcd config file] ******************************************* skipping: [master1] TASK [etcd : Write etcd systemd unit file for Debian] ************************** skipping: [master1] TASK [etcd : Enable etcd] ****************************************************** skipping: [master1] TASK [etcd : Start etcd] ******************************************************* skipping: [master1] TASK [etcd : include] ********************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/etcd/tasks/coreos.yml for master1 TASK [etcd : CoreOS | Create etcd dropin directory] **************************** changed: [master1] TASK [etcd : CoreOS | Write etcd dropin file] ********************************** changed: [master1] TASK [etcd : CoreOS | Write etcd config file] ********************************** changed: [master1] TASK [etcd : CoreOS | Enable etcd2] ******************************************** changed: [master1] TASK [etcd : CoreOS | Start etcd2] ********************************************* changed: [master1] TASK [etcd : include] ********************************************************** skipping: [master1] TASK [etcd : include] ********************************************************** skipping: [master1] RUNNING HANDLER [common : reload systemd] ************************************** changed: [master1] RUNNING HANDLER [etcd : restart etcd2] ***************************************** skipping: [master1] PLAY [all] ********************************************************************* TASK [common : Determine if Atomic] ******************************************** ok: [gfs1] ok: [gfs2] ok: [node1] ok: [gfs3] ok: [master1] ok: [lma] ok: [loadbal] ok: [gfs4] TASK [common : Init the is_atomic fact] **************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [master1] ok: [gfs4] ok: [node1] ok: [lma] ok: [loadbal] TASK [common : Set the is_atomic fact] ***************************************** skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [gfs1] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Determine if CoreOS] ******************************************** ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [node1] ok: [lma] ok: [loadbal] ok: [master1] TASK [common : Init the is_coreos fact] **************************************** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [lma] ok: [master1] ok: [gfs3] ok: [gfs4] ok: [node1] TASK [common : Set the is_coreos fact] ***************************************** ok: [gfs1] ok: [lma] ok: [master1] ok: [gfs2] ok: [gfs3] ok: [loadbal] ok: [gfs4] ok: [node1] TASK [common : Set the bin directory path for CoreOS] ************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] TASK [common : Create the directory used to store binaries] ******************** ok: [master1] ok: [lma] ok: [loadbal] ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [node1] ok: [gfs3] TASK [common : Create Ansible temp directory] ********************************** changed: [gfs1] changed: [loadbal] changed: [gfs2] changed: [gfs3] ok: [master1] changed: [gfs4] changed: [node1] changed: [lma] TASK [common : Determine if has rpm] ******************************************* ok: [gfs1] ok: [loadbal] ok: [gfs2] ok: [master1] ok: [gfs3] ok: [lma] ok: [node1] ok: [gfs4] TASK [common : Init the has_rpm fact] ****************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [loadbal] ok: [gfs4] ok: [node1] ok: [master1] ok: [lma] TASK [common : Set the has_rpm fact] ******************************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Init the has_firewalld fact] ************************************ ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] TASK [common : Init the has_iptables fact] ************************************* ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [gfs2] ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] TASK [common : Create systemd dropin directory] ******************************** skipping: [gfs1] skipping: [gfs4] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [master1] skipping: [loadbal] skipping: [lma] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [node1] skipping: [master1] skipping: [lma] TASK [docker : Override docker config file directory for Debian] *************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [lma] skipping: [node1] skipping: [loadbal] skipping: [master1] TASK [docker : Create config file directory] *********************************** changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [gfs4] changed: [gfs3] changed: [node1] changed: [master1] changed: [lma] TASK [docker : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [master1] TASK [docker : include] ******************************************************** skipping: [master1] skipping: [lma] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [docker : include] ******************************************************** skipping: [master1] skipping: [lma] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [node1] TASK [docker : include] ******************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/docker/tasks/coreos.yml for gfs1, gfs3, gfs2, node1, gfs4, loadbal, lma, master1 TASK [docker : CoreOS | Create docker dropin directory] ************************ ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [gfs3] ok: [master1] ok: [lma] TASK [docker : CoreOS | Write docker dropin file] ****************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [loadbal] changed: [gfs3] changed: [lma] changed: [master1] changed: [node1] TASK [docker : set_fact] ******************************************************* ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [lma] ok: [loadbal] ok: [node1] ok: [master1] TASK [docker : Local vars for systemd installs] ******************************** ok: [gfs1] ok: [loadbal] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [lma] ok: [master1] TASK [docker : Local vars for upstart installs] ******************************** skipping: [gfs1] skipping: [loadbal] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [lma] skipping: [node1] skipping: [master1] TASK [docker : Verify docker config files exists] ****************************** ok: [master1] => (item=docker) ok: [lma] => (item=docker) ok: [gfs1] => (item=docker) ok: [loadbal] => (item=docker) ok: [gfs2] => (item=docker) ok: [gfs4] => (item=docker) ok: [gfs3] => (item=docker) ok: [node1] => (item=docker) ok: [master1] => (item=docker-network) ok: [lma] => (item=docker-network) ok: [loadbal] => (item=docker-network) ok: [gfs4] => (item=docker-network) ok: [gfs2] => (item=docker-network) ok: [gfs1] => (item=docker-network) ok: [gfs3] => (item=docker-network) ok: [node1] => (item=docker-network) TASK [docker : Turn down docker logging] *************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [master1] TASK [docker : Install http_proxy into docker(-network)] *********************** skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [master1] TASK [docker : Install https_proxy into docker(-network)] ********************** skipping: [master1] skipping: [gfs1] skipping: [node1] skipping: [gfs3] skipping: [gfs2] skipping: [loadbal] skipping: [gfs4] skipping: [lma] TASK [docker : Install no-proxy into docker(-network)] ************************* changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [master1] changed: [lma] changed: [node1] changed: [gfs4] changed: [gfs3] TASK [docker : Add any insecure registrys to docker config] ******************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [docker : Enable Docker] ************************************************** changed: [loadbal] changed: [gfs1] changed: [master1] changed: [gfs2] changed: [gfs3] changed: [lma] changed: [gfs4] changed: [node1] RUNNING HANDLER [common : reload systemd] ************************************** changed: [gfs2] changed: [master1] changed: [loadbal] changed: [gfs1] changed: [lma] changed: [gfs3] changed: [gfs4] changed: [node1] RUNNING HANDLER [docker : restart docker] ************************************** changed: [gfs1] changed: [gfs2] changed: [master1] changed: [loadbal] changed: [lma] changed: [node1] changed: [gfs3] changed: [gfs4] RUNNING HANDLER [docker : start docker] **************************************** changed: [loadbal] changed: [master1] changed: [gfs1] changed: [gfs2] changed: [node1] changed: [gfs3] changed: [lma] changed: [gfs4] RUNNING HANDLER [docker : stop docker] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [master1] skipping: [lma] skipping: [node1] skipping: [loadbal] skipping: [gfs3] skipping: [gfs4] RUNNING HANDLER [docker : delete docker0] ************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [master1] skipping: [gfs3] skipping: [lma] skipping: [loadbal] skipping: [gfs4] RUNNING HANDLER [docker : docker graph driver] ********************************* ok: [loadbal] ok: [lma] ok: [gfs3] ok: [gfs2] ok: [master1] ok: [gfs4] ok: [node1] ok: [gfs1] RUNNING HANDLER [docker : docker fix start on btrfs] *************************** skipping: [loadbal] skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [node1] skipping: [gfs3] skipping: [gfs4] RUNNING HANDLER [docker : docker start service] ******************************** changed: [master1] changed: [loadbal] changed: [gfs3] changed: [node1] changed: [gfs2] changed: [lma] changed: [gfs4] changed: [gfs1] PLAY [masters,nodes] *********************************************************** TASK [common : Determine if Atomic] ******************************************** ok: [master1] ok: [loadbal] ok: [gfs2] ok: [node1] ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [lma] TASK [common : Init the is_atomic fact] **************************************** ok: [gfs1] ok: [master1] ok: [gfs3] ok: [gfs2] ok: [node1] ok: [lma] ok: [gfs4] ok: [loadbal] TASK [common : Set the is_atomic fact] ***************************************** skipping: [master1] skipping: [node1] skipping: [gfs1] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [gfs2] skipping: [lma] TASK [common : Determine if CoreOS] ******************************************** ok: [loadbal] ok: [node1] ok: [lma] ok: [master1] ok: [gfs2] ok: [gfs1] ok: [gfs4] ok: [gfs3] TASK [common : Init the is_coreos fact] **************************************** ok: [master1] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [lma] ok: [gfs4] ok: [node1] ok: [loadbal] TASK [common : Set the is_coreos fact] ***************************************** ok: [lma] ok: [loadbal] ok: [master1] ok: [node1] ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [gfs2] TASK [common : Set the bin directory path for CoreOS] ************************** ok: [node1] ok: [loadbal] ok: [lma] ok: [master1] ok: [gfs2] ok: [gfs1] ok: [gfs4] ok: [gfs3] TASK [common : Create the directory used to store binaries] ******************** ok: [master1] ok: [gfs2] ok: [gfs1] ok: [gfs3] ok: [loadbal] ok: [node1] ok: [gfs4] ok: [lma] TASK [common : Create Ansible temp directory] ********************************** ok: [master1] ok: [node1] ok: [loadbal] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [gfs1] ok: [lma] TASK [common : Determine if has rpm] ******************************************* ok: [master1] ok: [loadbal] ok: [gfs1] ok: [gfs2] ok: [lma] ok: [gfs3] ok: [gfs4] ok: [node1] TASK [common : Init the has_rpm fact] ****************************************** ok: [master1] ok: [gfs2] ok: [gfs1] ok: [loadbal] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [lma] TASK [common : Set the has_rpm fact] ******************************************* skipping: [master1] skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Init the has_firewalld fact] ************************************ ok: [master1] ok: [gfs2] ok: [node1] ok: [gfs4] ok: [gfs1] ok: [gfs3] ok: [loadbal] ok: [lma] TASK [common : Init the has_iptables fact] ************************************* ok: [node1] ok: [master1] ok: [loadbal] ok: [lma] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] TASK [common : include] ******************************************************** skipping: [loadbal] skipping: [lma] skipping: [master1] skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] TASK [common : include] ******************************************************** skipping: [master1] skipping: [loadbal] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [gfs2] skipping: [gfs1] skipping: [gfs3] TASK [common : Create systemd dropin directory] ******************************** skipping: [master1] skipping: [gfs1] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [lma] skipping: [gfs4] skipping: [loadbal] skipping: [node1] TASK [flannel : Prepare and write Flannel configuration to etcd] *************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/flannel/tasks/config.yml for master1, loadbal, lma, gfs1, gfs2, gfs3, gfs4, node1 TASK [flannel : Set facts about etcdctl command] ******************************* ok: [master1 -> dr-master1] TASK [flannel : Create flannel config file to go in etcd] ********************** changed: [master1 -> dr-master1] TASK [flannel : Load the flannel config file into etcd] ************************ changed: [master1 -> dr-master1] TASK [flannel : Clean up the flannel config file] ****************************** ok: [master1 -> dr-master1] TASK [flannel : Install Flannel] *********************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/flannel/tasks/client.yml for master1, gfs1, gfs2, node1, gfs3, gfs4, loadbal, lma TASK [flannel : Force to use github-release when packages are not available] *** ok: [master1] ok: [gfs1] ok: [gfs2] ok: [node1] ok: [gfs3] ok: [gfs4] ok: [loadbal] ok: [lma] TASK [flannel : Install Flannel with package manager] ************************** skipping: [lma] skipping: [master1] skipping: [gfs1] skipping: [node1] skipping: [loadbal] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] TASK [flannel : set_fact] ****************************************************** ok: [master1] ok: [node1] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [loadbal] ok: [lma] TASK [flannel : set_fact] ****************************************************** skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [gfs4] skipping: [node1] skipping: [lma] TASK [flannel : Install Flannel with github release] *************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/flannel/tasks/github-release.yml for master1, loadbal, lma, gfs1, gfs4, node1, gfs2, gfs3 TASK [flannel : stat] ********************************************************** ok: [master1] ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [lma] ok: [gfs3] ok: [gfs4] ok: [node1] TASK [flannel : Download tar file] ********************************************* changed: [gfs1] changed: [gfs2] changed: [master1] changed: [loadbal] changed: [gfs3] changed: [gfs4] changed: [lma] changed: [node1] TASK [flannel : Extract tar file] ********************************************** changed: [gfs2] changed: [master1] changed: [node1] changed: [lma] changed: [gfs3] changed: [gfs4] changed: [gfs1] changed: [loadbal] TASK [flannel : Create symlinks] *********************************************** changed: [loadbal] => (item=flanneld) changed: [lma] => (item=flanneld) changed: [master1] => (item=flanneld) changed: [node1] => (item=flanneld) changed: [gfs4] => (item=flanneld) changed: [gfs1] => (item=flanneld) changed: [gfs3] => (item=flanneld) changed: [loadbal] => (item=mk-docker-opts.sh) changed: [lma] => (item=mk-docker-opts.sh) changed: [master1] => (item=mk-docker-opts.sh) changed: [gfs1] => (item=mk-docker-opts.sh) changed: [gfs2] => (item=flanneld) changed: [gfs3] => (item=mk-docker-opts.sh) changed: [gfs4] => (item=mk-docker-opts.sh) changed: [node1] => (item=mk-docker-opts.sh) changed: [gfs2] => (item=mk-docker-opts.sh) TASK [flannel : Setup systemd service] ***************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/flannel/tasks/systemd-service.yml for master1, gfs1, gfs2, gfs3, gfs4, node1, loadbal, lma TASK [flannel : assert] ******************************************************** ok: [master1] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] TASK [flannel : Add Flanneld Systemd Unit File] ******************************** changed: [gfs2] changed: [master1] changed: [gfs1] changed: [lma] changed: [gfs3] changed: [gfs4] changed: [node1] changed: [loadbal] TASK [flannel : Create Docker systemd dropin directory] ************************ ok: [node1] ok: [loadbal] ok: [master1] ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [lma] ok: [gfs3] TASK [flannel : Add Docker drop-in with dependency on Flannel] ***************** changed: [master1] changed: [gfs2] changed: [gfs1] changed: [gfs4] changed: [node1] changed: [loadbal] changed: [lma] changed: [gfs3] TASK [flannel : Setup upstart service] ***************************************** skipping: [master1] skipping: [gfs2] skipping: [loadbal] skipping: [gfs1] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [lma] TASK [flannel : Set Flannel config file directory] ***************************** ok: [loadbal] ok: [lma] ok: [master1] ok: [gfs1] ok: [gfs4] ok: [node1] ok: [gfs2] ok: [gfs3] TASK [flannel : Override Flannel config file directory for Ubuntu] ************* skipping: [lma] skipping: [master1] skipping: [node1] skipping: [loadbal] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] TASK [flannel : Create config file directory] ********************************** ok: [master1] ok: [node1] ok: [loadbal] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [lma] ok: [gfs4] TASK [flannel : Install Flannel config file] *********************************** changed: [master1] changed: [gfs2] changed: [loadbal] changed: [gfs3] changed: [gfs1] changed: [lma] changed: [gfs4] changed: [node1] TASK [flannel : include] ******************************************************* skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [flannel : include] ******************************************************* skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [node1] TASK [flannel : include] ******************************************************* skipping: [master1] skipping: [loadbal] skipping: [lma] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [flannel : Enable flannel] ************************************************ changed: [master1] changed: [gfs2] changed: [lma] changed: [loadbal] changed: [gfs1] changed: [node1] changed: [gfs3] changed: [gfs4] TASK [flannel : Start flannel] ************************************************* changed: [loadbal] changed: [gfs3] changed: [gfs1] changed: [gfs2] changed: [master1] changed: [lma] changed: [node1] changed: [gfs4] RUNNING HANDLER [common : reload systemd] ************************************** changed: [gfs2] changed: [master1] changed: [gfs1] changed: [node1] changed: [loadbal] changed: [gfs3] changed: [gfs4] changed: [lma] RUNNING HANDLER [flannel : restart flannel] ************************************ changed: [master1] changed: [gfs2] changed: [gfs1] changed: [node1] changed: [loadbal] changed: [lma] changed: [gfs4] changed: [gfs3] RUNNING HANDLER [flannel : restart docker] ************************************* changed: [loadbal] changed: [master1] changed: [gfs2] changed: [gfs3] changed: [lma] changed: [gfs1] changed: [node1] changed: [gfs4] RUNNING HANDLER [flannel : start docker] *************************************** changed: [loadbal] changed: [gfs2] changed: [gfs3] changed: [master1] changed: [lma] changed: [gfs1] changed: [node1] changed: [gfs4] RUNNING HANDLER [flannel : check docker existence] ***************************** ok: [gfs2] ok: [loadbal] ok: [master1] ok: [lma] ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [node1] RUNNING HANDLER [flannel : stop docker] **************************************** skipping: [loadbal] skipping: [master1] skipping: [gfs2] skipping: [gfs1] skipping: [gfs3] skipping: [gfs4] skipping: [lma] skipping: [node1] RUNNING HANDLER [flannel : delete docker0] ************************************* skipping: [loadbal] skipping: [master1] skipping: [gfs2] skipping: [gfs3] skipping: [lma] skipping: [gfs1] skipping: [node1] skipping: [gfs4] RUNNING HANDLER [flannel : docker graph driver] ******************************** ok: [loadbal] ok: [gfs2] ok: [node1] ok: [master1] ok: [gfs1] ok: [gfs3] ok: [gfs4] ok: [lma] RUNNING HANDLER [flannel : docker fix start on btrfs] ************************** skipping: [loadbal] skipping: [gfs2] skipping: [gfs3] skipping: [master1] skipping: [gfs1] skipping: [node1] skipping: [lma] skipping: [gfs4] RUNNING HANDLER [flannel : docker start service] ******************************* changed: [master1] changed: [gfs4] changed: [gfs3] changed: [loadbal] changed: [gfs2] changed: [lma] changed: [node1] changed: [gfs1] PLAY [all] ********************************************************************* TASK [common : Determine if Atomic] ******************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [master1] skipping: [loadbal] skipping: [lma] TASK [common : Init the is_atomic fact] **************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Set the is_atomic fact] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [master1] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Determine if CoreOS] ******************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Init the is_coreos fact] **************************************** skipping: [gfs1] skipping: [master1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [loadbal] skipping: [node1] skipping: [lma] TASK [common : Set the is_coreos fact] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [node1] TASK [common : Set the bin directory path for CoreOS] ************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [master1] skipping: [lma] TASK [common : Create the directory used to store binaries] ******************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [lma] skipping: [node1] skipping: [loadbal] skipping: [master1] TASK [common : Create Ansible temp directory] ********************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs4] skipping: [gfs2] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Determine if has rpm] ******************************************* skipping: [gfs1] skipping: [master1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [gfs4] skipping: [node1] TASK [common : Init the has_rpm fact] ****************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Set the has_rpm fact] ******************************************* skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [common : Init the has_firewalld fact] ************************************ skipping: [gfs1] skipping: [gfs4] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Init the has_iptables fact] ************************************* skipping: [gfs1] skipping: [lma] skipping: [master1] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [gfs4] skipping: [loadbal] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [master1] skipping: [gfs2] skipping: [lma] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [common : Create systemd dropin directory] ******************************** skipping: [gfs1] skipping: [gfs4] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [node1] skipping: [lma] skipping: [gfs4] skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [lma] skipping: [master1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [kubernetes : Update /usr/libexec/kubernetes if this is atomic] *********** skipping: [gfs1] skipping: [node1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [kubernetes : Initialize the python_bin fact] ***************************** skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [kubernetes : Update facts if this is CoreOS or Ubuntu 14.04] ************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [kubernetes : Create kubernetes config directory] ************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [gfs4] skipping: [node1] skipping: [master1] TASK [kubernetes : Create kubernetes script directory] ************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] skipping: [node1] TASK [kubernetes : Create bin dir] ********************************************* skipping: [gfs1] skipping: [lma] skipping: [master1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [kubernetes : write the global config file] ******************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [master1] skipping: [lma] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [master1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [lma] skipping: [loadbal] skipping: [node1] TASK [opencontrail : Build kernel module] ************************************** skipping: [gfs1] skipping: [lma] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [gfs2] skipping: [node1] skipping: [loadbal] TASK [opencontrail : Install vrouter] ****************************************** skipping: [gfs1] skipping: [master1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [node1] TASK [opencontrail : Install compute nodes] ************************************ skipping: [gfs1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [master1] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [opencontrail : Install gateways] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [lma] skipping: [loadbal] skipping: [master1] TASK [opencontrail : Install master] ******************************************* skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] skipping: [master1] PLAY [masters] ***************************************************************** TASK [common : Determine if Atomic] ******************************************** skipping: [master1] TASK [common : Init the is_atomic fact] **************************************** skipping: [master1] TASK [common : Set the is_atomic fact] ***************************************** skipping: [master1] TASK [common : Determine if CoreOS] ******************************************** skipping: [master1] TASK [common : Init the is_coreos fact] **************************************** skipping: [master1] TASK [common : Set the is_coreos fact] ***************************************** skipping: [master1] TASK [common : Set the bin directory path for CoreOS] ************************** skipping: [master1] TASK [common : Create the directory used to store binaries] ******************** skipping: [master1] TASK [common : Create Ansible temp directory] ********************************** skipping: [master1] TASK [common : Determine if has rpm] ******************************************* skipping: [master1] TASK [common : Init the has_rpm fact] ****************************************** skipping: [master1] TASK [common : Set the has_rpm fact] ******************************************* skipping: [master1] TASK [common : Init the has_firewalld fact] ************************************ skipping: [master1] TASK [common : Init the has_iptables fact] ************************************* skipping: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : Create systemd dropin directory] ******************************** skipping: [master1] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [master1] TASK [etcd : Install etcd] ***************************************************** skipping: [master1] TASK [etcd : Force etcd_source_type to github when packages are not available] * skipping: [master1] TASK [etcd : Install etcd from github] ***************************************** skipping: [master1] TASK [etcd : Create etcd config directory] ************************************* skipping: [master1] TASK [etcd : Write etcd config file] ******************************************* skipping: [master1] TASK [etcd : Write etcd systemd unit file for Debian] ************************** skipping: [master1] TASK [etcd : Enable etcd] ****************************************************** skipping: [master1] TASK [etcd : Start etcd] ******************************************************* skipping: [master1] TASK [etcd : include] ********************************************************** skipping: [master1] TASK [etcd : include] ********************************************************** skipping: [master1] TASK [etcd : include] ********************************************************** skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [master1] TASK [kubernetes : Update /usr/libexec/kubernetes if this is atomic] *********** skipping: [master1] TASK [kubernetes : Initialize the python_bin fact] ***************************** skipping: [master1] TASK [kubernetes : Update facts if this is CoreOS or Ubuntu 14.04] ************* skipping: [master1] TASK [kubernetes : Create kubernetes config directory] ************************* skipping: [master1] TASK [kubernetes : Create kubernetes script directory] ************************* skipping: [master1] TASK [kubernetes : Create bin dir] ********************************************* skipping: [master1] TASK [kubernetes : write the global config file] ******************************* skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [master1] TASK [contiv : Ensure bin_dir exists] ****************************************** skipping: [master1] TASK [contiv : include] ******************************************************** skipping: [master1] TASK [contiv : include] ******************************************************** skipping: [master1] TASK [contiv : include] ******************************************************** skipping: [master1] PLAY [masters] ***************************************************************** TASK [common : Determine if Atomic] ******************************************** ok: [master1] TASK [common : Init the is_atomic fact] **************************************** ok: [master1] TASK [common : Set the is_atomic fact] ***************************************** skipping: [master1] TASK [common : Determine if CoreOS] ******************************************** ok: [master1] TASK [common : Init the is_coreos fact] **************************************** ok: [master1] TASK [common : Set the is_coreos fact] ***************************************** ok: [master1] TASK [common : Set the bin directory path for CoreOS] ************************** ok: [master1] TASK [common : Create the directory used to store binaries] ******************** ok: [master1] TASK [common : Create Ansible temp directory] ********************************** ok: [master1] TASK [common : Determine if has rpm] ******************************************* ok: [master1] TASK [common : Init the has_rpm fact] ****************************************** ok: [master1] TASK [common : Set the has_rpm fact] ******************************************* skipping: [master1] TASK [common : Init the has_firewalld fact] ************************************ ok: [master1] TASK [common : Init the has_iptables fact] ************************************* ok: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : include] ******************************************************** skipping: [master1] TASK [common : Create systemd dropin directory] ******************************** skipping: [master1] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [master1] TASK [kubernetes : include] **************************************************** skipping: [master1] TASK [kubernetes : Update /usr/libexec/kubernetes if this is atomic] *********** skipping: [master1] TASK [kubernetes : Initialize the python_bin fact] ***************************** ok: [master1] TASK [kubernetes : Update facts if this is CoreOS or Ubuntu 14.04] ************* ok: [master1] TASK [kubernetes : Create kubernetes config directory] ************************* changed: [master1] TASK [kubernetes : Create kubernetes script directory] ************************* changed: [master1] TASK [kubernetes : Create bin dir] ********************************************* ok: [master1] TASK [kubernetes : write the global config file] ******************************* changed: [master1] TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/download_bins.yml for master1 TASK [kubernetes : Create directory for current Kubernetes release] ************ changed: [master1] TASK [kubernetes : Download Kubernetes binaries] ******************************* changed: [master1] => (item=kube-apiserver) changed: [master1] => (item=kube-controller-manager) changed: [master1] => (item=kube-proxy) changed: [master1] => (item=kube-scheduler) changed: [master1] => (item=kubectl) changed: [master1] => (item=kubelet) TASK [kubernetes : Create symlinks] ******************************************** changed: [master1] => (item=kube-apiserver) changed: [master1] => (item=kube-controller-manager) changed: [master1] => (item=kube-proxy) changed: [master1] => (item=kube-scheduler) changed: [master1] => (item=kubectl) changed: [master1] => (item=kubelet) TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/secrets.yml for master1 TASK [kubernetes : Create system kube-cert groups] ***************************** changed: [master1] TASK [kubernetes : Create system kube user] ************************************ changed: [master1] TASK [kubernetes : make sure the certificate directory exits] ****************** changed: [master1] TASK [kubernetes : make sure the tokens directory exits] *********************** changed: [master1] TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/gen_certs.yml for master1 TASK [kubernetes : Install openssl for easy-rsa stuff] ************************* skipping: [master1] => (item=openssl) skipping: [master1] => (item=curl) TASK [kubernetes : HACK | overwrite make-ca-cert.sh from local copy] *********** ok: [master1] TASK [kubernetes : Run create cert script on master] *************************** changed: [master1] TASK [kubernetes : Verify certificate permissions] ***************************** changed: [master1] => (item=/etc/kubernetes/certs/ca.crt) changed: [master1] => (item=/etc/kubernetes/certs/server.crt) changed: [master1] => (item=/etc/kubernetes/certs/server.key) changed: [master1] => (item=/etc/kubernetes/certs/kubecfg.crt) changed: [master1] => (item=/etc/kubernetes/certs/kubecfg.key) TASK [kubernetes : Read back the CA certificate] ******************************* ok: [master1 -> dr-master1] TASK [kubernetes : Register the CA certificate as a fact so it can be used later] *** ok: [master1] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** ok: [master1] TASK [kubernetes : Read back the kubecfg key] ********************************** ok: [master1 -> dr-master1] TASK [kubernetes : Register the cfg key as a fact so it can be used later] ***** ok: [master1] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** ok: [master1] TASK [kubernetes : Read back the kubecfg cert] ********************************* ok: [master1 -> dr-master1] TASK [kubernetes : Register the cfg cert as a fact so it can be used later] **** ok: [master1] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** ok: [master1] TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/gen_tokens.yml for master1 TASK [kubernetes : Copy the token gen script] ********************************** changed: [master1] TASK [kubernetes : Generate tokens for master components] ********************** changed: [master1] => (item=[u'system:controller_manager', u'master1']) changed: [master1] => (item=[u'system:scheduler', u'master1']) changed: [master1] => (item=[u'system:kubectl', u'master1']) TASK [kubernetes : Generate tokens for node components] ************************ changed: [master1] => (item=[u'system:kubelet', u'gfs1']) changed: [master1] => (item=[u'system:kubelet', u'gfs2']) changed: [master1] => (item=[u'system:kubelet', u'gfs3']) changed: [master1] => (item=[u'system:kubelet', u'gfs4']) changed: [master1] => (item=[u'system:kubelet', u'node1']) changed: [master1] => (item=[u'system:kubelet', u'loadbal']) changed: [master1] => (item=[u'system:kubelet', u'lma']) changed: [master1] => (item=[u'system:proxy', u'gfs1']) changed: [master1] => (item=[u'system:proxy', u'gfs2']) changed: [master1] => (item=[u'system:proxy', u'gfs3']) changed: [master1] => (item=[u'system:proxy', u'gfs4']) changed: [master1] => (item=[u'system:proxy', u'node1']) changed: [master1] => (item=[u'system:proxy', u'loadbal']) changed: [master1] => (item=[u'system:proxy', u'lma']) TASK [master : include] ******************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/master/tasks/coreos.yml for master1 TASK [master : CoreOS | Force source_type to github] *************************** ok: [master1] TASK [master : CoreOS | Get Systemd Unit Files from Kubernetes repository] ***** changed: [master1] => (item=kube-apiserver) changed: [master1] => (item=kube-controller-manager) changed: [master1] => (item=kube-scheduler) TASK [master : CoreOS | Create dropin directories for Kubernetes Master services] *** changed: [master1] => (item=kube-apiserver) changed: [master1] => (item=kube-controller-manager) changed: [master1] => (item=kube-scheduler) TASK [master : CoreOS | Write Kubernetes Master dropin files] ****************** changed: [master1] => (item=kube-apiserver) changed: [master1] => (item=kube-controller-manager) changed: [master1] => (item=kube-scheduler) TASK [master : include] ******************************************************** skipping: [master1] TASK [master : include] ******************************************************** skipping: [master1] TASK [master : include] ******************************************************** skipping: [master1] TASK [master : write the config file for the api server] *********************** changed: [master1] TASK [master : Ensure that a token auth file exists (addons may populate it)] ** ok: [master1] TASK [master : add cap_net_bind_service to kube-apiserver] ********************* skipping: [master1] TASK [master : Enable apiserver] *********************************************** changed: [master1] TASK [master : Get the master token values] ************************************ ok: [master1 -> dr-master1] => (item=system:controller_manager) ok: [master1 -> dr-master1] => (item=system:scheduler) ok: [master1 -> dr-master1] => (item=system:kubectl) TASK [master : Set token facts] ************************************************ ok: [master1] TASK [master : write the config file for the controller-manager] *************** changed: [master1] TASK [master : write the kubecfg (auth) file for controller-manager] *********** changed: [master1] TASK [master : Enable controller-manager] ************************************** changed: [master1] TASK [master : write the config file for the scheduler] ************************ changed: [master1] TASK [master : write the kubecfg (auth) file for scheduler] ******************** changed: [master1] TASK [master : Enable scheduler] *********************************************** changed: [master1] TASK [master : write the kubecfg (auth) file for kubectl] ********************** changed: [master1] TASK [master : write the config files for kubelet] ***************************** skipping: [master1] TASK [master : Enable kubelet] ************************************************* skipping: [master1] TASK [master : write the delay-master-services target] ************************* skipping: [master1] TASK [master : Enable delay-master-services] *********************************** skipping: [master1] TASK [master : include] ******************************************************** skipping: [master1] TASK [master : include] ******************************************************** skipping: [master1] RUNNING HANDLER [common : reload systemd] ************************************** changed: [master1] RUNNING HANDLER [master : restart daemons] ************************************* changed: [master1] RUNNING HANDLER [master : restart apiserver] *********************************** changed: [master1] RUNNING HANDLER [master : restart controller-manager] ************************** changed: [master1] RUNNING HANDLER [master : restart scheduler] *********************************** changed: [master1] PLAY [nodes] ******************************************************************* TASK [common : Determine if Atomic] ******************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [lma] ok: [loadbal] ok: [gfs4] ok: [node1] TASK [common : Init the is_atomic fact] **************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] TASK [common : Set the is_atomic fact] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Determine if CoreOS] ******************************************** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [gfs3] ok: [node1] ok: [lma] ok: [gfs4] TASK [common : Init the is_coreos fact] **************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [loadbal] ok: [node1] ok: [gfs4] ok: [lma] TASK [common : Set the is_coreos fact] ***************************************** ok: [gfs1] ok: [lma] ok: [loadbal] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] TASK [common : Set the bin directory path for CoreOS] ************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [node1] ok: [loadbal] ok: [lma] ok: [gfs4] TASK [common : Create the directory used to store binaries] ******************** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [node1] ok: [lma] ok: [gfs3] ok: [gfs4] TASK [common : Create Ansible temp directory] ********************************** ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [gfs4] ok: [node1] ok: [gfs3] ok: [lma] TASK [common : Determine if has rpm] ******************************************* ok: [gfs1] ok: [gfs2] ok: [loadbal] ok: [node1] ok: [gfs4] ok: [gfs3] ok: [lma] TASK [common : Init the has_rpm fact] ****************************************** ok: [gfs1] ok: [loadbal] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [node1] ok: [lma] TASK [common : Set the has_rpm fact] ******************************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Init the has_firewalld fact] ************************************ ok: [gfs1] ok: [gfs2] ok: [node1] ok: [loadbal] ok: [lma] ok: [gfs3] ok: [gfs4] TASK [common : Init the has_iptables fact] ************************************* ok: [gfs1] ok: [loadbal] ok: [gfs2] ok: [node1] ok: [gfs4] ok: [gfs3] ok: [lma] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [node1] skipping: [lma] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Create systemd dropin directory] ******************************** skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [gfs4] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [lma] skipping: [loadbal] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [kubernetes : Update /usr/libexec/kubernetes if this is atomic] *********** skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [gfs3] skipping: [node1] skipping: [gfs4] skipping: [loadbal] TASK [kubernetes : Initialize the python_bin fact] ***************************** ok: [gfs1] ok: [node1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [lma] ok: [loadbal] TASK [kubernetes : Update facts if this is CoreOS or Ubuntu 14.04] ************* ok: [lma] ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] TASK [kubernetes : Create kubernetes config directory] ************************* changed: [gfs2] changed: [gfs3] changed: [gfs1] changed: [loadbal] changed: [gfs4] changed: [node1] changed: [lma] TASK [kubernetes : Create kubernetes script directory] ************************* changed: [gfs2] changed: [gfs3] changed: [gfs1] changed: [gfs4] changed: [loadbal] changed: [node1] changed: [lma] TASK [kubernetes : Create bin dir] ********************************************* ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [loadbal] ok: [lma] ok: [node1] TASK [kubernetes : write the global config file] ******************************* changed: [gfs2] changed: [gfs1] changed: [lma] changed: [node1] changed: [gfs4] changed: [loadbal] changed: [gfs3] TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/download_bins.yml for gfs1, gfs2, gfs3, lma, gfs4, node1, loadbal TASK [kubernetes : Create directory for current Kubernetes release] ************ changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [gfs3] changed: [loadbal] changed: [lma] changed: [node1] TASK [kubernetes : Download Kubernetes binaries] ******************************* changed: [gfs2] => (item=kube-apiserver) changed: [gfs4] => (item=kube-apiserver) changed: [loadbal] => (item=kube-apiserver) changed: [gfs1] => (item=kube-apiserver) changed: [lma] => (item=kube-apiserver) changed: [gfs4] => (item=kube-controller-manager) changed: [loadbal] => (item=kube-controller-manager) changed: [gfs2] => (item=kube-controller-manager) changed: [gfs1] => (item=kube-controller-manager) changed: [loadbal] => (item=kube-proxy) changed: [gfs2] => (item=kube-proxy) changed: [gfs4] => (item=kube-proxy) changed: [lma] => (item=kube-controller-manager) changed: [gfs1] => (item=kube-proxy) changed: [gfs2] => (item=kube-scheduler) changed: [loadbal] => (item=kube-scheduler) changed: [gfs4] => (item=kube-scheduler) changed: [gfs3] => (item=kube-apiserver) changed: [gfs1] => (item=kube-scheduler) changed: [loadbal] => (item=kubectl) changed: [gfs2] => (item=kubectl) changed: [gfs4] => (item=kubectl) changed: [gfs1] => (item=kubectl) changed: [gfs3] => (item=kube-controller-manager) changed: [gfs2] => (item=kubelet) changed: [loadbal] => (item=kubelet) changed: [node1] => (item=kube-apiserver) changed: [gfs4] => (item=kubelet) changed: [gfs3] => (item=kube-proxy) changed: [gfs1] => (item=kubelet) changed: [node1] => (item=kube-controller-manager) changed: [gfs3] => (item=kube-scheduler) changed: [node1] => (item=kube-proxy) changed: [gfs3] => (item=kubectl) changed: [node1] => (item=kube-scheduler) changed: [lma] => (item=kube-proxy) changed: [node1] => (item=kubectl) changed: [gfs3] => (item=kubelet) changed: [node1] => (item=kubelet) changed: [lma] => (item=kube-scheduler) changed: [lma] => (item=kubectl) changed: [lma] => (item=kubelet) TASK [kubernetes : Create symlinks] ******************************************** changed: [gfs2] => (item=kube-apiserver) changed: [gfs3] => (item=kube-apiserver) changed: [gfs1] => (item=kube-apiserver) changed: [loadbal] => (item=kube-apiserver) changed: [lma] => (item=kube-apiserver) changed: [gfs4] => (item=kube-apiserver) changed: [node1] => (item=kube-apiserver) changed: [gfs2] => (item=kube-controller-manager) changed: [gfs3] => (item=kube-controller-manager) changed: [loadbal] => (item=kube-controller-manager) changed: [lma] => (item=kube-controller-manager) changed: [gfs1] => (item=kube-controller-manager) changed: [gfs4] => (item=kube-controller-manager) changed: [node1] => (item=kube-controller-manager) changed: [gfs2] => (item=kube-proxy) changed: [loadbal] => (item=kube-proxy) changed: [lma] => (item=kube-proxy) changed: [gfs1] => (item=kube-proxy) changed: [gfs3] => (item=kube-proxy) changed: [node1] => (item=kube-proxy) changed: [gfs4] => (item=kube-proxy) changed: [gfs2] => (item=kube-scheduler) changed: [loadbal] => (item=kube-scheduler) changed: [gfs1] => (item=kube-scheduler) changed: [gfs3] => (item=kube-scheduler) changed: [gfs4] => (item=kube-scheduler) changed: [node1] => (item=kube-scheduler) changed: [lma] => (item=kube-scheduler) changed: [gfs2] => (item=kubectl) changed: [loadbal] => (item=kubectl) changed: [gfs1] => (item=kubectl) changed: [gfs3] => (item=kubectl) changed: [gfs4] => (item=kubectl) changed: [node1] => (item=kubectl) changed: [gfs2] => (item=kubelet) changed: [loadbal] => (item=kubelet) changed: [lma] => (item=kubectl) changed: [gfs1] => (item=kubelet) changed: [gfs4] => (item=kubelet) changed: [node1] => (item=kubelet) changed: [gfs3] => (item=kubelet) changed: [lma] => (item=kubelet) TASK [kubernetes : include] **************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/kubernetes/tasks/secrets.yml for gfs1, gfs2, gfs3, gfs4, node1, loadbal, lma TASK [kubernetes : Create system kube-cert groups] ***************************** changed: [loadbal] changed: [gfs3] changed: [node1] changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [lma] TASK [kubernetes : Create system kube user] ************************************ changed: [gfs1] changed: [gfs2] changed: [gfs3] changed: [node1] changed: [loadbal] changed: [gfs4] changed: [lma] TASK [kubernetes : make sure the certificate directory exits] ****************** changed: [lma] changed: [loadbal] changed: [gfs3] changed: [gfs4] changed: [gfs2] changed: [node1] changed: [gfs1] TASK [kubernetes : make sure the tokens directory exits] *********************** changed: [lma] changed: [gfs1] changed: [gfs4] changed: [gfs2] changed: [gfs3] changed: [node1] changed: [loadbal] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [kubernetes : Read back the CA certificate] ******************************* ok: [gfs1 -> dr-master1] TASK [kubernetes : Register the CA certificate as a fact so it can be used later] *** ok: [gfs1] ok: [gfs2] ok: [gfs4] ok: [gfs3] ok: [node1] ok: [loadbal] ok: [lma] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [node1] changed: [gfs3] changed: [gfs4] changed: [lma] TASK [kubernetes : Read back the kubecfg key] ********************************** ok: [gfs1 -> dr-master1] TASK [kubernetes : Register the cfg key as a fact so it can be used later] ***** ok: [gfs1] ok: [gfs3] ok: [gfs2] ok: [loadbal] ok: [lma] ok: [gfs4] ok: [node1] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** changed: [gfs1] changed: [gfs2] changed: [node1] changed: [loadbal] changed: [gfs3] changed: [lma] changed: [gfs4] TASK [kubernetes : Read back the kubecfg cert] ********************************* ok: [gfs1 -> dr-master1] TASK [kubernetes : Register the cfg cert as a fact so it can be used later] **** ok: [gfs1] ok: [gfs2] ok: [node1] ok: [loadbal] ok: [lma] ok: [gfs3] ok: [gfs4] TASK [kubernetes : Place CA certificate and kube_cfg credentials everywhere] *** changed: [gfs2] changed: [node1] changed: [gfs3] changed: [gfs4] changed: [loadbal] changed: [lma] changed: [gfs1] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] TASK [node : include] ********************************************************** included: /usr/local/lib/kubernetes/contrib/ansible/roles/node/tasks/coreos.yml for gfs1, gfs2, loadbal, gfs3, lma, gfs4, node1 TASK [node : CoreOS | Force source_type to github] ***************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] ok: [node1] ok: [loadbal] ok: [lma] TASK [node : CoreOS | Get Systemd Unit Files from Kubernetes repository] ******* changed: [gfs1] => (item=kubelet) changed: [gfs2] => (item=kubelet) changed: [gfs3] => (item=kubelet) changed: [gfs4] => (item=kubelet) changed: [node1] => (item=kubelet) changed: [loadbal] => (item=kubelet) changed: [lma] => (item=kubelet) changed: [gfs1] => (item=kube-proxy) changed: [gfs2] => (item=kube-proxy) changed: [gfs3] => (item=kube-proxy) changed: [gfs4] => (item=kube-proxy) changed: [node1] => (item=kube-proxy) changed: [loadbal] => (item=kube-proxy) changed: [lma] => (item=kube-proxy) TASK [node : CoreOS | Create dropin directories for Kubernetes Node services] ** changed: [loadbal] => (item=kubelet) changed: [gfs1] => (item=kubelet) changed: [gfs2] => (item=kubelet) changed: [gfs4] => (item=kubelet) changed: [node1] => (item=kubelet) changed: [lma] => (item=kubelet) changed: [gfs3] => (item=kubelet) changed: [loadbal] => (item=kube-proxy) changed: [gfs4] => (item=kube-proxy) changed: [lma] => (item=kube-proxy) changed: [gfs2] => (item=kube-proxy) changed: [gfs3] => (item=kube-proxy) changed: [node1] => (item=kube-proxy) changed: [gfs1] => (item=kube-proxy) TASK [node : CoreOS | Write Kubernetes Node dropin files] ********************** changed: [gfs2] => (item=kubelet) changed: [gfs1] => (item=kubelet) changed: [node1] => (item=kubelet) changed: [loadbal] => (item=kubelet) changed: [lma] => (item=kubelet) changed: [gfs3] => (item=kubelet) changed: [gfs4] => (item=kubelet) changed: [gfs2] => (item=kube-proxy) changed: [loadbal] => (item=kube-proxy) changed: [gfs1] => (item=kube-proxy) changed: [node1] => (item=kube-proxy) changed: [gfs4] => (item=kube-proxy) changed: [lma] => (item=kube-proxy) changed: [gfs3] => (item=kube-proxy) TASK [node : include] ********************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [gfs4] skipping: [node1] skipping: [lma] TASK [node : Set selinux permissive because tokens and selinux don't work together] *** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [node : include] ********************************************************** skipping: [gfs1] skipping: [lma] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [loadbal] TASK [node : include] ********************************************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [lma] skipping: [loadbal] TASK [node : Make sure manifest directory exists] ****************************** changed: [gfs1] changed: [gfs2] changed: [lma] changed: [node1] changed: [loadbal] changed: [gfs3] changed: [gfs4] TASK [node : Install fluentd pod into each node] ******************************* changed: [gfs1] changed: [gfs4] changed: [gfs2] changed: [loadbal] changed: [gfs3] changed: [node1] changed: [lma] TASK [node : Get the node token values] **************************************** ok: [node1 -> dr-master1] => (item=system:kubelet) ok: [gfs3 -> dr-master1] => (item=system:kubelet) ok: [loadbal -> dr-master1] => (item=system:kubelet) ok: [gfs1 -> dr-master1] => (item=system:kubelet) ok: [gfs2 -> dr-master1] => (item=system:kubelet) ok: [gfs4 -> dr-master1] => (item=system:kubelet) ok: [lma -> dr-master1] => (item=system:kubelet) ok: [node1 -> dr-master1] => (item=system:proxy) ok: [gfs3 -> dr-master1] => (item=system:proxy) ok: [loadbal -> dr-master1] => (item=system:proxy) ok: [gfs1 -> dr-master1] => (item=system:proxy) ok: [gfs4 -> dr-master1] => (item=system:proxy) ok: [gfs2 -> dr-master1] => (item=system:proxy) ok: [lma -> dr-master1] => (item=system:proxy) TASK [node : Set token facts] ************************************************** ok: [gfs1] ok: [gfs4] ok: [gfs3] ok: [gfs2] ok: [lma] ok: [loadbal] ok: [node1] TASK [node : Create the kubelet working directory] ***************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [gfs3] changed: [lma] changed: [loadbal] changed: [node1] TASK [node : write the config files for kubelet] ******************************* changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [lma] changed: [gfs3] changed: [node1] changed: [gfs4] TASK [node : write the kubecfg (auth) file for kubelet] ************************ changed: [gfs2] changed: [gfs1] changed: [loadbal] changed: [lma] changed: [gfs4] changed: [gfs3] changed: [node1] TASK [node : Enable kubelet] *************************************************** changed: [gfs1] changed: [node1] changed: [gfs3] changed: [lma] changed: [gfs2] changed: [loadbal] changed: [gfs4] TASK [node : write the config files for proxy] ********************************* changed: [gfs1] changed: [gfs2] changed: [lma] changed: [gfs3] changed: [gfs4] changed: [node1] changed: [loadbal] TASK [node : write the kubecfg (auth) file for kube-proxy] ********************* changed: [gfs2] changed: [gfs1] changed: [lma] changed: [loadbal] changed: [node1] changed: [gfs3] changed: [gfs4] TASK [node : Enable proxy] ***************************************************** changed: [gfs3] changed: [gfs4] changed: [gfs2] changed: [node1] changed: [loadbal] changed: [lma] changed: [gfs1] TASK [node : write the delay-node-services target] ***************************** skipping: [gfs1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [node1] skipping: [gfs4] TASK [node : Enable delay-node-services] *************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [gfs4] TASK [node : include] ********************************************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [loadbal] skipping: [node1] skipping: [lma] TASK [node : include] ********************************************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] RUNNING HANDLER [common : reload systemd] ************************************** changed: [gfs1] changed: [loadbal] changed: [gfs2] changed: [lma] changed: [gfs4] changed: [node1] changed: [gfs3] RUNNING HANDLER [node : restart daemons] *************************************** changed: [gfs2] changed: [gfs1] changed: [lma] changed: [node1] changed: [loadbal] changed: [gfs4] changed: [gfs3] RUNNING HANDLER [node : restart kubelet] *************************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [gfs3] changed: [node1] changed: [lma] changed: [loadbal] RUNNING HANDLER [node : restart proxy] ***************************************** changed: [gfs1] changed: [gfs2] changed: [loadbal] changed: [lma] changed: [gfs3] changed: [node1] changed: [gfs4] PLAY [masters[0],nodes] ******************************************************** TASK [opencontrail-provision : Pull config image] ****************************** skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [loadbal] skipping: [lma] skipping: [gfs4] skipping: [node1] TASK [opencontrail-provision : controller provisioning] ************************ skipping: [master1] skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [lma] skipping: [loadbal] TASK [opencontrail-provision : node provisioning] ****************************** skipping: [master1] skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] PLAY [nodes] ******************************************************************* TASK [common : Determine if Atomic] ******************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Init the is_atomic fact] **************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Set the is_atomic fact] ***************************************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] TASK [common : Determine if CoreOS] ******************************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Init the is_coreos fact] **************************************** skipping: [gfs1] skipping: [node1] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] TASK [common : Set the is_coreos fact] ***************************************** skipping: [gfs1] skipping: [gfs4] skipping: [gfs3] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [node1] TASK [common : Set the bin directory path for CoreOS] ************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [common : Create the directory used to store binaries] ******************** skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] TASK [common : Create Ansible temp directory] ********************************** skipping: [gfs1] skipping: [node1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [common : Determine if has rpm] ******************************************* skipping: [gfs1] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] TASK [common : Init the has_rpm fact] ****************************************** skipping: [gfs1] skipping: [gfs4] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Set the has_rpm fact] ******************************************* skipping: [gfs1] skipping: [gfs4] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : Init the has_firewalld fact] ************************************ skipping: [gfs1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [lma] skipping: [loadbal] TASK [common : Init the has_iptables fact] ************************************* skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [common : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [lma] skipping: [gfs4] skipping: [loadbal] TASK [common : Create systemd dropin directory] ******************************** skipping: [gfs1] skipping: [lma] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [gfs3] TASK [common : Enable cpu and memory accounting for systemd services] ********** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Install etcd] ***************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Force etcd_source_type to github when packages are not available] * skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Install etcd from github] ***************************************** skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [etcd : Create etcd config directory] ************************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Write etcd config file] ******************************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Write etcd systemd unit file for Debian] ************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Enable etcd] ****************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : Start etcd] ******************************************************* skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [etcd : include] ********************************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [etcd : include] ********************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [etcd : include] ********************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [kubernetes : Update /opt/bin/kubernetes if this is atomic] *************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [kubernetes : Initialize the python_bin fact] ***************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] TASK [kubernetes : Update facts if this is CoreOS or Ubuntu 14.04] ************* skipping: [gfs1] skipping: [gfs2] skipping: [node1] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] TASK [kubernetes : Create kubernetes config directory] ************************* skipping: [gfs1] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] TASK [kubernetes : Create kubernetes script directory] ************************* skipping: [gfs1] skipping: [gfs2] skipping: [lma] skipping: [gfs3] skipping: [node1] skipping: [gfs4] skipping: [loadbal] TASK [kubernetes : Create bin dir] ********************************************* skipping: [gfs1] skipping: [gfs2] skipping: [loadbal] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [lma] TASK [kubernetes : write the global config file] ******************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs3] skipping: [gfs2] skipping: [gfs4] skipping: [node1] skipping: [lma] skipping: [loadbal] TASK [kubernetes : include] **************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [contiv : Ensure bin_dir exists] ****************************************** skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] skipping: [loadbal] skipping: [lma] TASK [contiv : include] ******************************************************** skipping: [gfs1] skipping: [gfs2] skipping: [loadbal] skipping: [lma] skipping: [gfs3] skipping: [gfs4] skipping: [node1] TASK [contiv : include] ******************************************************** skipping: [gfs1] skipping: [node1] skipping: [gfs2] skipping: [gfs4] skipping: [gfs3] skipping: [lma] skipping: [loadbal] TASK [contiv : include] ******************************************************** skipping: [gfs1] skipping: [loadbal] skipping: [lma] skipping: [gfs2] skipping: [gfs3] skipping: [gfs4] skipping: [node1] PLAY [nodes] ******************************************************************* TASK [epilogue : Install kubernetes runtime dependencies] ********************** skipping: [gfs1] => (item=socat) skipping: [gfs1] => (item=tar) skipping: [gfs1] => (item=git) skipping: [gfs2] => (item=socat) skipping: [gfs2] => (item=tar) skipping: [gfs3] => (item=socat) skipping: [gfs3] => (item=tar) skipping: [node1] => (item=socat) skipping: [gfs4] => (item=socat) skipping: [loadbal] => (item=socat) skipping: [gfs2] => (item=git) skipping: [gfs3] => (item=git) skipping: [node1] => (item=tar) skipping: [gfs4] => (item=tar) skipping: [gfs4] => (item=git) skipping: [lma] => (item=socat) skipping: [loadbal] => (item=tar) skipping: [loadbal] => (item=git) skipping: [lma] => (item=tar) skipping: [node1] => (item=git) skipping: [lma] => (item=git) PLAY [kube-system namespace] *************************************************** TASK [command] ***************************************************************** changed: [master1] PLAY [start DNS and implicitly kube-system namespace] ************************** TASK [cluster-dns : DNS copy yaml] ********************************************* changed: [master1] => (item=kube-dns.yaml) TASK [cluster-dns : DNS status] ************************************************ changed: [master1] TASK [cluster-dns : Cluster Addon - SkyDNS| create kube-dns] ******************* changed: [master1] PLAY RECAP ********************************************************************* gfs1 : ok=141 changed=51 unreachable=0 failed=0 gfs2 : ok=138 changed=51 unreachable=0 failed=0 gfs3 : ok=138 changed=51 unreachable=0 failed=0 gfs4 : ok=138 changed=51 unreachable=0 failed=0 lma : ok=138 changed=51 unreachable=0 failed=0 loadbal : ok=138 changed=51 unreachable=0 failed=0 master1 : ok=179 changed=67 unreachable=0 failed=0 node1 : ok=138 changed=51 unreachable=0 failed=0 root@fde4df001c3d:~# ansible-playbook -i inventory/dr playbooks/ndslabs-k8s-install.yml PLAY [setup ansible host vars] ************************************************* TASK [pre-checks : check for bad hostname] ************************************* changed: [gfs1] changed: [lma] changed: [master1] changed: [gfs2] changed: [gfs3] changed: [node1] changed: [gfs4] changed: [loadbal] TASK [pre-checks : fail] ******************************************************* skipping: [gfs1] skipping: [gfs2] skipping: [gfs3] skipping: [node1] skipping: [master1] skipping: [gfs4] skipping: [loadbal] skipping: [lma] TASK [pre-checks : checking openstack credentials] ***************************** changed: [gfs1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [node1] ok: [loadbal] ok: [gfs4] ok: [lma] ok: [master1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [node1] ok: [gfs2] ok: [loadbal] ok: [gfs3] ok: [gfs4] ok: [lma] ok: [master1] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [lma] ok: [master1] ok: [gfs1] ok: [node1] ok: [gfs3] ok: [gfs2] ok: [gfs4] ok: [loadbal] TASK [ansible_host_vars_setup : set_fact] ************************************** ok: [gfs1] ok: [lma] ok: [master1] ok: [gfs4] ok: [gfs2] ok: [node1] ok: [gfs3] ok: [loadbal] PLAY [label K8s nodes] ********************************************************* TASK [ndslabs-k8-init-labels : NDSLabs initial system labels] ****************** changed: [gfs1 -> dr-master1] => (item=ndslabs-node-role=glfs) changed: [lma -> dr-master1] => (item=ndslabs-node-role=lma) changed: [loadbal -> dr-master1] => (item=ndslabs-node-role=loadbalancer) changed: [gfs3 -> dr-master1] => (item=ndslabs-node-role=glfs) changed: [gfs2 -> dr-master1] => (item=ndslabs-node-role=glfs) changed: [gfs4 -> dr-master1] => (item=ndslabs-node-role=glfs) changed: [node1 -> dr-master1] => (item=ndslabs-node-role=compute) PLAY [glfs state directories] ************************************************** TASK [file] ******************************************************************** changed: [gfs1] changed: [gfs2] changed: [gfs4] changed: [loadbal] changed: [gfs3] changed: [master1] changed: [node1] changed: [lma] TASK [file] ******************************************************************** changed: [gfs1] changed: [gfs2] changed: [gfs3] changed: [master1] changed: [gfs4] changed: [loadbal] changed: [node1] changed: [lma] TASK [file] ******************************************************************** changed: [gfs1] changed: [master1] changed: [gfs2] changed: [loadbal] changed: [gfs4] changed: [node1] changed: [gfs3] changed: [lma] TASK [file] ******************************************************************** changed: [gfs1] changed: [master1] changed: [lma] changed: [gfs2] changed: [node1] changed: [loadbal] changed: [gfs4] changed: [gfs3] PLAY [Global FS k8s objects] *************************************************** TASK [k8-glfs-server-pods : Global FS ConfigFile] ****************************** changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Global FS create configmap] ************************ changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Server Spec Deploy File] ******************* changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Server Pods Deploy] ************************ changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Server Wait DS ready] ********************** FAILED - RETRYING: TASK: k8-glfs-server-pods : Gluster Server Wait DS ready (49 retries left). FAILED - RETRYING: TASK: k8-glfs-server-pods : Gluster Server Wait DS ready (48 retries left). FAILED - RETRYING: TASK: k8-glfs-server-pods : Gluster Server Wait DS ready (47 retries left). changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Pod Name for peer probe] ******************* changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Set glpod] ***************************************** ok: [gfs1] ok: [gfs2] ok: [gfs3] ok: [gfs4] TASK [k8-glfs-server-pods : Gluster Init Peer Probe] *************************** changed: [gfs4 -> dr-master1] changed: [gfs1 -> dr-master1] changed: [gfs3 -> dr-master1] changed: [gfs2 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Full Cluster FS Volume 2x2 strip/mirror] *** changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Volume NFS off] **************************** changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Start volume] ****************************** changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster Global volume check] *********************** changed: [gfs1 -> dr-master1] TASK [k8-glfs-server-pods : Gluster enable quota] ****************************** changed: [gfs1 -> dr-master1] PLAY [Global FS k8s objects] *************************************************** TASK [k8-glfs-server-pods : Global FS ConfigFile] ****************************** ok: [master1 -> dr-master1] TASK [k8-glfs-server-pods : Global FS create configmap] ************************ skipping: [master1] TASK [k8-glfs-server-pods : Gluster Server Spec Deploy File] ******************* skipping: [master1] TASK [k8-glfs-server-pods : Gluster Server Pods Deploy] ************************ skipping: [master1] TASK [k8-glfs-server-pods : Gluster Server Wait DS ready] ********************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Pod Name for peer probe] ******************* skipping: [master1] TASK [k8-glfs-server-pods : Set glpod] ***************************************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Init Peer Probe] *************************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Full Cluster FS Volume 2x2 strip/mirror] *** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Volume NFS off] **************************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Start volume] ****************************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster Global volume check] *********************** skipping: [master1] TASK [k8-glfs-server-pods : Gluster enable quota] ****************************** skipping: [master1] TASK [k8-glfs-client-set : Gluster Clients DaemonSet Spec] ********************* changed: [master1] TASK [k8-glfs-client-set : Gluster Clients Deploy DaemonSet] ******************* changed: [master1] TASK [k8-glfs-client-set : Wait for global FS ready] *************************** FAILED - RETRYING: TASK: k8-glfs-client-set : Wait for global FS ready (49 retries left). FAILED - RETRYING: TASK: k8-glfs-client-set : Wait for global FS ready (48 retries left). changed: [master1] PLAY [LoadBalancer public interface] ******************************************* TASK [os_floating_ip] ********************************************************** changed: [loadbal] TASK [command] ***************************************************************** changed: [loadbal] PLAY [Generate TLS certificates] *********************************************** TASK [generate-tls-certs : Store certificate directory] ************************ ok: [gfs1] TASK [generate-tls-certs : file] *********************************************** ok: [gfs1] TASK [generate-tls-certs : stat] *********************************************** ok: [gfs1] TASK [generate-tls-certs : Generate TLS certificates| Generate .key] *********** skipping: [gfs1] TASK [generate-tls-certs : copy] *********************************************** skipping: [gfs1] TASK [generate-tls-certs : stat] *********************************************** ok: [gfs1] TASK [generate-tls-certs : Generate TLS certificates| Generate .cert] ********** skipping: [gfs1] TASK [generate-tls-certs : stat] *********************************************** ok: [gfs1] TASK [generate-tls-certs : Generate TLS certificates| Generate .info] ********** skipping: [gfs1] TASK [generate-tls-certs : stat] *********************************************** ok: [gfs1] TASK [generate-tls-certs : Generate TLS certificates| Generate .pem] *********** skipping: [gfs1] TASK [generate-tls-certs : copy] *********************************************** skipping: [gfs1] PLAY [Create TLS secret] ******************************************************* TASK [k8-tls-secret : NDSLabs TLS Secret| copy certs] ************************** changed: [master1] => (item=ndslabs.key) changed: [master1] => (item=ndslabs.cert) changed: [master1] => (item=ndslabs.info) changed: [master1] => (item=ndslabs.pem) TASK [k8-tls-secret : Secret status - default] ********************************* changed: [master1] TASK [k8-tls-secret : NDSLabs TLS Secret| create ndslabs-tls-secret - default] * changed: [master1] PLAY [Kubernetes Cluster Addons] *********************************************** TASK [cluster-addons : Cluster Addons| copy yaml] ****************************** changed: [master1] => (item=kube-ui.yaml) changed: [master1] => (item=kube-dash.yaml) changed: [master1] => (item=kibana.yaml) changed: [master1] => (item=grafana.yaml) changed: [master1] => (item=prometheus-all.json) changed: [master1] => (item=prometheus-service.json) changed: [master1] => (item=pushgateway-service.json) TASK [cluster-addons : Cluster Addons| populate template] ********************** changed: [master1] => (item=kube-system-ingress.yaml) TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/init.yml for master1 TASK [cluster-addons : Secret status - kube-system] **************************** changed: [master1] TASK [cluster-addons : NDSLabs TLS Secret| create ndslabs-tls-secret - kube-system] *** changed: [master1] TASK [cluster-addons : Kube-System ingress status] ***************************** changed: [master1] TASK [cluster-addons : Cluster Addon init| create kube-lma-ingress rules] ****** changed: [master1] TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/kubeui.yml for master1 TASK [cluster-addons : Kube UI status] ***************************************** changed: [master1] TASK [cluster-addons : Cluster Addon - KubeUI| create kube-ui] ***************** skipping: [master1] TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/kubedash.yml for master1 TASK [cluster-addons : Kube Dashboard status] ********************************** changed: [master1] TASK [cluster-addons : Cluster Addon - Kubernetes Dashboard| create kube-dash] * changed: [master1] TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/grafana.yml for master1 TASK [cluster-addons : Grafana status] ***************************************** changed: [master1] TASK [cluster-addons : InfluxDB status] **************************************** changed: [master1] TASK [cluster-addons : Cluster Profiling - Grafana| create grafana stack] ****** changed: [master1] TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/kibana.yml for master1 TASK [cluster-addons : Elasticsearch status] *********************************** changed: [master1] TASK [cluster-addons : Kibana status] ****************************************** changed: [master1] TASK [cluster-addons : Cluster Logging - Kibana| create kibana stack] ********** skipping: [master1] TASK [cluster-addons : include] ************************************************ included: /usr/local/lib/ndslabs/ansible/roles/cluster-addons/tasks/promdash.yml for master1 TASK [cluster-addons : Prometheus Dashboard status] **************************** changed: [master1] TASK [cluster-addons : PushGateway status] ************************************* changed: [master1] TASK [cluster-addons : Cluster Metrics - Prometheus Dashboard| create prometheus stack] *** skipping: [master1] PLAY [Start NDSLabs ApiServer and WebUI] *************************************** TASK [ndslabs-api-gui : NDSLabs| copy yaml] ************************************ changed: [master1] => (item=ndslabs-ingress.yaml) changed: [master1] => (item=ndslabs-apiserver.yaml) changed: [master1] => (item=ndslabs-webui.yaml) TASK [ndslabs-api-gui : Ingress status] **************************************** changed: [master1] TASK [ndslabs-api-gui : NDSLabs| create ndslabs-ingress] *********************** changed: [master1] TASK [ndslabs-api-gui : ApiServer status] ************************************** changed: [master1] TASK [ndslabs-api-gui : NDSLabs| create ndslabs-apiserver] ********************* changed: [master1] TASK [ndslabs-api-gui : WebUI status] ****************************************** changed: [master1] TASK [ndslabs-api-gui : NDSLabs| create ndslabs-webui] ************************* changed: [master1] TASK [ndslabs-loadbalancer : Loadbalancer| copy yaml] ************************** changed: [master1] => (item=default-backend.yaml) changed: [master1] => (item=loadbalancer.yaml) TASK [ndslabs-loadbalancer : Loadbalancer status] ****************************** changed: [master1] TASK [ndslabs-loadbalancer : Default-http-backend status] ********************** changed: [master1] TASK [ndslabs-loadbalancer : Loadbalancer| create default-http-backend] ******** changed: [master1] TASK [ndslabs-loadbalancer : Loadbalancer| create loadbalancer] **************** changed: [master1] PLAY [registry storage] ******************************************************** TASK [file] ******************************************************************** changed: [node1] PLAY [cluster-local-docker-registry] ******************************************* TASK [k8s-local-docker-registry : deploy local registry yaml] ****************** changed: [master1] => (item=docker-registry-rc.yaml) changed: [master1] => (item=docker-registry-svc.yaml) changed: [master1] => (item=docker-registry-proxy-ds.yaml) TASK [k8s-local-docker-registry : run local registry yaml] ********************* changed: [master1] => (item=docker-registry-rc.yaml) changed: [master1] => (item=docker-registry-svc.yaml) changed: [master1] => (item=docker-registry-proxy-ds.yaml) PLAY RECAP ********************************************************************* gfs1 : ok=30 changed=19 unreachable=0 failed=0 gfs2 : ok=12 changed=7 unreachable=0 failed=0 gfs3 : ok=12 changed=7 unreachable=0 failed=0 gfs4 : ok=12 changed=7 unreachable=0 failed=0 lma : ok=10 changed=6 unreachable=0 failed=0 loadbal : ok=12 changed=8 unreachable=0 failed=0 master1 : ok=52 changed=41 unreachable=0 failed=0 node1 : ok=11 changed=7 unreachable=0 failed=0 root@fde4df001c3d:~#