Uploaded image for project: 'National Data Service'
  1. National Data Service
  2. NDS-715

Kubernetes addons terminate themselves automatically in Kubernetes 1.5

    XMLWordPrintableJSON

Details

    • Bug
    • Resolution: Fixed
    • Major
    • None
    • None
    • Infrastructure
    • None
    • NDS Sprint 18, NDS Sprint 19

    Description

      After deploying a Kubernetes 1.5 cluster at SDSC, we noticed that the addons were not all running as they should be. Performing a kubectl create manually on the .yml file showed the services would run properly for several minutes before terminating themselves for seemingly no reason.

      Step to recreate:

      1. Deploy a Kubernetes 1.5 cluster via deploy-tools
      2. Create grafana via kubectl

        kubectl create -f /tmp/grafana.yml

      3. Watch kube get pods for changes (~5 min)

        watch kubectl get pods

        • You should see the grafana / heapster / influx pods all go into a "Terminating" state.

      This may be related to the addons and their associated update script, but more investigation will certainly be needed.

      This ticket is complete when the addons (grafana and kibana, at minimum) continue to run for longer than 10 minutes on a multi-node Kubernetes 1.5 cluster without terminating themselves.

      Gliffy Diagrams

        Attachments

          Activity

            People

              raila David Raila
              lambert8 Sara Lambert
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 6 hours
                  6h
                  Remaining:
                  Time Spent - 3 hours Remaining Estimate - 3 hours
                  3h
                  Logged:
                  Time Spent - 3 hours Remaining Estimate - 3 hours
                  3h

                  Tasks