PR #7189 - 11-07 15:25

Job: hypershift
FAILURE
← Back to Test Grid

Test Summary

99
Total Tests
63
Passed
21
Failed
15
Skipped

Failed Tests

TestAutoscaling
48m17s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-w7jnk/autoscaling-d4f4x in 2m17s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster autoscaling-d4f4x util.go:2835: Successfully waited for HostedCluster e2e-clusters-w7jnk/autoscaling-d4f4x to have valid conditions in 50ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestAutoscaling/ValidateHostedCluster
35m30.16s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-w7jnk/autoscaling-d4f4x in 6m15.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-d4f4x.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-d4f4x.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF util.go:359: Successfully waited for a successful connection to the guest API server in 23.425s eventually.go:258: Failed to get **v1.Node: the server was unable to return a response in the time allotted, but may still be processing the request (get nodes) util.go:541: Successfully waited for 1 nodes to become ready in 20m12.025s util.go:574: Successfully waited for HostedCluster e2e-clusters-w7jnk/autoscaling-d4f4x to rollout in 8m33.05s util.go:2835: Successfully waited for HostedCluster e2e-clusters-w7jnk/autoscaling-d4f4x to have valid conditions in 50ms
TestAutoscaling/ValidateHostedCluster/EnsureNoCrashingPods
260ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-w7jnk/autoscaling-d4f4x in 50ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1) util.go:758: Container openshift-controller-manager in pod openshift-controller-manager-556768f5dc-wj5k2 has a restartCount > 0 (1) util.go:758: Container openshift-oauth-apiserver in pod openshift-oauth-apiserver-75759d6cd7-dnb8c has a restartCount > 0 (1)
TestAzureScheduler
49m9.5s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-84wrn/azure-scheduler-5t2kb in 2m5s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster azure-scheduler-5t2kb util.go:2835: Successfully waited for HostedCluster e2e-clusters-84wrn/azure-scheduler-5t2kb to have valid conditions in 25ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestAzureScheduler/ValidateHostedCluster
36m18.24s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-84wrn/azure-scheduler-5t2kb in 6m6.075s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-azure-scheduler-5t2kb.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout util.go:359: Successfully waited for a successful connection to the guest API server in 23.275s eventually.go:258: Failed to get **v1.Node: Get "https://api-azure-scheduler-5t2kb.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes": stream error: stream ID 395; INTERNAL_ERROR; received from peer util.go:541: Successfully waited for 2 nodes to become ready in 20m54.2s util.go:574: Successfully waited for HostedCluster e2e-clusters-84wrn/azure-scheduler-5t2kb to rollout in 8m48.05s util.go:2835: Successfully waited for HostedCluster e2e-clusters-84wrn/azure-scheduler-5t2kb to have valid conditions in 50ms
TestAzureScheduler/ValidateHostedCluster/EnsureNoCrashingPods
250ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-84wrn/azure-scheduler-5t2kb in 75ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1) util.go:758: Container openshift-oauth-apiserver in pod openshift-oauth-apiserver-7df8cdccdb-vjj64 has a restartCount > 0 (1)
TestCreateCluster
46m7.07s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-gqbhq/create-cluster-sgcj5 in 2m25s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster create-cluster-sgcj5 util.go:2835: Successfully waited for HostedCluster e2e-clusters-gqbhq/create-cluster-sgcj5 to have valid conditions in 75ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestCreateCluster/ValidateHostedCluster
32m33.63s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-gqbhq/create-cluster-sgcj5 in 5m12.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout util.go:359: Successfully waited for a successful connection to the guest API server in 59.7s eventually.go:258: Failed to get **v1.Node: Get "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes": stream error: stream ID 397; INTERNAL_ERROR; received from peer eventually.go:258: Failed to get **v1.Node: Get "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes": stream error: stream ID 399; INTERNAL_ERROR; received from peer eventually.go:258: Failed to get **v1.Node: Get "https://api-create-cluster-sgcj5.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes": stream error: stream ID 401; INTERNAL_ERROR; received from peer util.go:541: Successfully waited for 2 nodes to become ready in 21m9.15s util.go:574: Successfully waited for HostedCluster e2e-clusters-gqbhq/create-cluster-sgcj5 to rollout in 5m6.075s util.go:2835: Successfully waited for HostedCluster e2e-clusters-gqbhq/create-cluster-sgcj5 to have valid conditions in 50ms
TestCreateCluster/ValidateHostedCluster/EnsureNoCrashingPods
390ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-gqbhq/create-cluster-sgcj5 in 50ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container cloud-controller-manager in pod azure-cloud-controller-manager-55cb9b65cd-pzfnm has a restartCount > 0 (1) util.go:758: Container cluster-node-tuning-operator in pod cluster-node-tuning-operator-7dc84445fc-xjlnr has a restartCount > 0 (1) util.go:758: Container cluster-version-operator in pod cluster-version-operator-79484ff6bb-jztzc has a restartCount > 0 (1) util.go:758: Container snapshot-controller in pod csi-snapshot-controller-8986b7755-2z82h has a restartCount > 0 (1) util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1) util.go:758: Container kube-scheduler in pod kube-scheduler-77bf998f48-j4j59 has a restartCount > 0 (1) util.go:758: Container machine-approver in pod machine-approver-678f45f6bd-6gnvj has a restartCount > 0 (1) util.go:758: Container openshift-route-controller-manager in pod openshift-route-controller-manager-7d49bdb89d-9nfdd has a restartCount > 0 (1)
TestCreateClusterCustomConfig
46m59.53s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-xbjs9/custom-config-knbjc in 2m26s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster custom-config-knbjc util.go:2835: Successfully waited for HostedCluster e2e-clusters-xbjs9/custom-config-knbjc to have valid conditions in 25ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestCreateClusterCustomConfig/ValidateHostedCluster
34m43.04s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-xbjs9/custom-config-knbjc in 6m6.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-knbjc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-knbjc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-knbjc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-knbjc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout util.go:359: Successfully waited for a successful connection to the guest API server in 1m12.35s eventually.go:258: Failed to get **v1.Node: Get "https://api-custom-config-knbjc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes": stream error: stream ID 317; INTERNAL_ERROR; received from peer util.go:541: Successfully waited for 2 nodes to become ready in 19m27.075s util.go:574: Successfully waited for HostedCluster e2e-clusters-xbjs9/custom-config-knbjc to rollout in 7m51.075s util.go:2835: Successfully waited for HostedCluster e2e-clusters-xbjs9/custom-config-knbjc to have valid conditions in 25ms
TestCreateClusterCustomConfig/ValidateHostedCluster/EnsureNoCrashingPods
180ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-xbjs9/custom-config-knbjc in 50ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1)
TestNodePool
0s
TestNodePool/HostedCluster0
38m42.55s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-kvcpp/node-pool-jgxsw in 2m25s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster node-pool-jgxsw util.go:2835: Successfully waited for HostedCluster e2e-clusters-kvcpp/node-pool-jgxsw to have valid conditions in 25ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestNodePool/HostedCluster0/ValidateHostedCluster
24m38.58s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-kvcpp/node-pool-jgxsw in 5m18.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-jgxsw.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-jgxsw.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-jgxsw.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout util.go:359: Successfully waited for a successful connection to the guest API server in 51.625s util.go:541: Successfully waited for 0 nodes to become ready in 75ms util.go:2835: Successfully waited for HostedCluster e2e-clusters-kvcpp/node-pool-jgxsw to have valid conditions in 18m21.05s
TestNodePool/HostedCluster0/ValidateHostedCluster/EnsureNoCrashingPods
310ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-kvcpp/node-pool-jgxsw in 50ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1) util.go:758: Container openshift-oauth-apiserver in pod openshift-oauth-apiserver-674b7f88d7-d5j5c has a restartCount > 0 (1) util.go:758: Container packageserver in pod packageserver-6959648cb5-cz2jp has a restartCount > 0 (1)
TestNodePool/HostedCluster2
39m8.98s
hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-5pxsb/node-pool-zzr5j in 2m30s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster node-pool-zzr5j util.go:2835: Failed to wait for HostedCluster e2e-clusters-5pxsb/node-pool-zzr5j to have valid conditions in 2s: context deadline exceeded eventually.go:224: observed *v1beta1.HostedCluster e2e-clusters-5pxsb/node-pool-zzr5j invalid at RV 44909 after 2s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(hosted-cluster-config-operator deployment has 1 unavailable replicas) hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestNodePool/HostedCluster2/ValidateHostedCluster
25m54.62s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-5pxsb/node-pool-zzr5j in 5m15.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-zzr5j.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-zzr5j.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-zzr5j.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": net/http: TLS handshake timeout util.go:359: Successfully waited for a successful connection to the guest API server in 39.425s util.go:541: Successfully waited for 0 nodes to become ready in 75ms eventually.go:104: Failed to get *v1beta1.HostedCluster: client rate limiter Wait returned an error: context deadline exceeded util.go:2835: Failed to wait for HostedCluster e2e-clusters-5pxsb/node-pool-zzr5j to have valid conditions in 20m0s: context deadline exceeded eventually.go:224: observed *v1beta1.HostedCluster e2e-clusters-5pxsb/node-pool-zzr5j invalid at RV 44909 after 20m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(hosted-cluster-config-operator deployment has 1 unavailable replicas)
TestUpgradeControlPlane
52m4.97s
control_plane_upgrade_test.go:26: Starting control plane upgrade test. FromImage: registry.build01.ci.openshift.org/ci-op-0dc9fs32/release@sha256:041dd273257b740f13889eee749cbfc771010e6051a5bab782f93f4422390af6, toImage: registry.build01.ci.openshift.org/ci-op-0dc9fs32/release@sha256:1981ca0c3376b70b2c0a2b9f3740dc069ed79e2eb45bf2c01d1ed3c49e6764ea hypershift_framework.go:423: Successfully created hostedcluster e2e-clusters-4cld4/control-plane-upgrade-svc9b in 2m22s hypershift_framework.go:119: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-svc9b util.go:2835: Successfully waited for HostedCluster e2e-clusters-4cld4/control-plane-upgrade-svc9b to have valid conditions in 50ms hypershift_framework.go:242: skipping postTeardown() hypershift_framework.go:223: skipping teardown, already called
TestUpgradeControlPlane/ValidateHostedCluster
38m0.65s
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-4cld4/control-plane-upgrade-svc9b in 7m0.05s util.go:297: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:104: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-svc9b.aks-e2e.hypershift.azure.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF util.go:359: Successfully waited for a successful connection to the guest API server in 33.025s util.go:541: Successfully waited for 2 nodes to become ready in 21m33.075s util.go:574: Successfully waited for HostedCluster e2e-clusters-4cld4/control-plane-upgrade-svc9b to rollout in 8m48.05s util.go:2835: Successfully waited for HostedCluster e2e-clusters-4cld4/control-plane-upgrade-svc9b to have valid conditions in 50ms
TestUpgradeControlPlane/ValidateHostedCluster/EnsureNoCrashingPods
190ms
util.go:280: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-4cld4/control-plane-upgrade-svc9b in 25ms util.go:297: Successfully waited for kubeconfig secret to have data in 25ms util.go:758: Container etcd in pod etcd-0 has a restartCount > 0 (1)