PR #8853 - 07-14 18:02

Job: hypershift
FAILURE
← Back to Test Grid

Test Summary

160
Total Tests
100
Passed
40
Failed
20
Skipped

Failed Tests

TestAutoscaling
45m26.43s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-7cs29/autoscaling-gxnrg in 2m32s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster autoscaling-gxnrg util.go:3158: Successfully waited for HostedCluster e2e-clusters-7cs29/autoscaling-gxnrg to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestAutoscaling/ValidateHostedCluster
34m50.43s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-7cs29/autoscaling-gxnrg in 48s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-gxnrg.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-autoscaling-gxnrg.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-gxnrg.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 18.207.3.29:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-gxnrg.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.213.15.217:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m15.025s util.go:583: Successfully waited for 1 nodes to become ready in 21m36s util.go:620: Successfully waited for HostedCluster e2e-clusters-7cs29/autoscaling-gxnrg to rollout in 10m6s util.go:3158: Successfully waited for HostedCluster e2e-clusters-7cs29/autoscaling-gxnrg to have valid conditions in 0s
TestAutoscaling/ValidateHostedCluster/EnsureNoCrashingPods
150ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-ccc697648-wjtxh has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-6994bbcdbb-vbhnk util.go:829: Container manager in pod cluster-api-5dc6c55cc7-g6xgq has a restartCount > 0 (2) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-7f45d496b9-r9p94 has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-54fbdf45c8-r2vkt has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-cc6d5f677-ss5zq has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-69d95fb449-cg2jn has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-d446b5fc7-npvrg has a restartCount > 0 (7)
TestCreateCluster
31m43.96s
create_cluster_test.go:47: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-thmxt/create-cluster-2rrgr in 52s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster create-cluster-2rrgr util.go:3158: Successfully waited for HostedCluster e2e-clusters-thmxt/create-cluster-2rrgr to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateCluster/ValidateHostedCluster
22m2.45s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-thmxt/create-cluster-2rrgr in 51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-2rrgr.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-create-cluster-2rrgr.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-2rrgr.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 13.222.78.167:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-2rrgr.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.209.89.240:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m42.175s util.go:583: Successfully waited for 3 nodes to become ready in 12m3s util.go:620: Successfully waited for HostedCluster e2e-clusters-thmxt/create-cluster-2rrgr to rollout in 6m21s util.go:3158: Successfully waited for HostedCluster e2e-clusters-thmxt/create-cluster-2rrgr to have valid conditions in 0s
TestCreateCluster/ValidateHostedCluster/EnsureNoCrashingPods
50ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-86db7dc4c8-cqrr9 has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-bd66677b5-j444s util.go:829: Container manager in pod cluster-api-b6c598d85-r4prm has a restartCount > 0 (6) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-676897db8d-86f85 has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-bb6877d6c-48jqb has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-f947fc848-5ksfp has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-64c94d9c8b-2fb6j has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-7cb88986c5-6fwsx has a restartCount > 0 (7)
TestCreateClusterCustomConfig
23m45.99s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-p8vwj/custom-config-xm7mb in 1m31s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster custom-config-xm7mb eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-p8vwj/hostedclusters/custom-config-xm7mb?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-p8vwj/custom-config-xm7mb to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterCustomConfig/Teardown
10m6.83s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io custom-config-xm7mb) journals.go:77: No machines associated with infra id custom-config-xm7mb were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-p8vwj, name: custom-config-xm7mb hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterCustomConfig/ValidateHostedCluster
12m6.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-p8vwj/custom-config-xm7mb in 2m6s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.226.10.157:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 23.20.51.75:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-xm7mb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded
TestCreateClusterHABreakGlassCredentials
26m45.68s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-9btwj/ha-break-glass-creds-t26mv in 1m5s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster ha-break-glass-creds-t26mv util.go:3158: Successfully waited for HostedCluster e2e-clusters-9btwj/ha-break-glass-creds-t26mv to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster
20m39.59s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-9btwj/ha-break-glass-creds-t26mv in 4m6s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-ha-break-glass-creds-t26mv.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF util.go:384: Successfully waited for a successful connection to the guest API server in 12.225s util.go:583: Successfully waited for 0 nodes to become ready in 10.05s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-9btwj/hostedclusters/ha-break-glass-creds-t26mv?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io ha-break-glass-creds-t26mv) util.go:3158: Successfully waited for HostedCluster e2e-clusters-9btwj/ha-break-glass-creds-t26mv to have valid conditions in 16m6s
TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster/EnsureNoCrashingPods
160ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-8df976f99-89shx has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-b4b9cbb76-nhp94 util.go:829: Container manager in pod cluster-api-6bbdb46944-jnzvz has a restartCount > 0 (6) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-76dd6864f7-srxfz has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-68477d6b94-whlfz has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-7d56666846-nb22h has a restartCount > 0 (5) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-7dbf69598c-bj9vd has a restartCount > 0 (5) util.go:826: Leader election failure detected in container etcd-defrag in pod etcd-2 util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-84c44675bb-4khbw has a restartCount > 0 (7)
TestCreateClusterPrivate
41m49.64s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-swg9b/private-rbknb in 44s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-rbknb util.go:3158: Failed to wait for HostedCluster e2e-clusters-swg9b/private-rbknb to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-swg9b/private-rbknb invalid at RV 102265 after 2s: incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorDegraded(Cluster operator ingress is degraded) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterPrivate/ValidateHostedCluster
30m45.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-swg9b/private-rbknb in 45s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: the server was unable to return a response in the time allotted, but may still be processing the request (get nodepools.hypershift.openshift.io) eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-swg9b/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded util.go:744: Failed to wait for NodePools for HostedCluster e2e-clusters-swg9b/private-rbknb to have all of their desired nodes in 30m0s: context deadline exceeded eventually.go:384: observed invalid **v1beta1.NodePool state after 30m0s eventually.go:401: - observed **v1beta1.NodePool e2e-clusters-swg9b/private-rbknb-us-east-1a invalid: expected 2 replicas, got 1 util.go:744: *v1beta1.NodePool e2e-clusters-swg9b/private-rbknb-us-east-1a conditions: util.go:744: AutoscalingEnabled=False: AsExpected util.go:744: UpdateManagementEnabled=True: AsExpected util.go:744: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-iy3n0ff2/release@sha256:8c7d29fca314b9a4ff7deef5d78f5b5d1b66666ae1b0dee20981e38a9b75997d) util.go:744: ValidArchPlatform=True: AsExpected util.go:744: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:744: SupportedVersionSkew=True: AsExpected(Release image version is valid) util.go:744: ValidMachineConfig=True: AsExpected util.go:744: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 0d04d416) util.go:744: UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 5.0.0-0.ci-2026-07-14-200835-test-ci-op-iy3n0ff2-latest) util.go:744: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:744: ReachedIgnitionEndpoint=True: AsExpected util.go:744: AllMachinesReady=True: AsExpected(All is well) util.go:744: AllNodesHealthy=False: NodeProvisioning(1 of 2 machines are not healthy Machine private-rbknb-us-east-1a-fxpv8-rn4xn: NodeProvisioning: Waiting for a node with matching ProviderID to exist ) util.go:744: ValidPlatformConfig=True: AsExpected(All is well) util.go:744: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0fbc8be8796dc1df5") util.go:744: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) util.go:744: ValidTuningConfig=True: AsExpected util.go:744: UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: private-rbknb-us-east-1a-b9461ceb) util.go:744: AutorepairEnabled=False: AsExpected util.go:744: Ready=False: WaitingForAvailableMachines(Minimum availability requires 2 replicas, current 1 available)
TestCreateClusterPrivateWithRouteKAS
42m40.28s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-khl4f/private-dngcv in 1m22s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-dngcv util.go:3158: Failed to wait for HostedCluster e2e-clusters-khl4f/private-dngcv to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-khl4f/private-dngcv invalid at RV 102201 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, node-tuning, openshift-samples, service-ca, storage are not available) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-200835-test-ci-op-iy3n0ff2-latest: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterPrivateWithRouteKAS/ValidateHostedCluster
31m51.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-khl4f/private-dngcv in 1m51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-khl4f/nodepools?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: the server was unable to return a response in the time allotted, but may still be processing the request (get nodepools.hypershift.openshift.io) eventually.go:259: Failed to get **v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded util.go:744: Failed to wait for NodePools for HostedCluster e2e-clusters-khl4f/private-dngcv to have all of their desired nodes in 30m0s: context deadline exceeded eventually.go:384: observed invalid **v1beta1.NodePool state after 30m0s eventually.go:401: - observed **v1beta1.NodePool e2e-clusters-khl4f/private-dngcv-us-east-1b invalid: expected 2 replicas, got 0 util.go:744: *v1beta1.NodePool e2e-clusters-khl4f/private-dngcv-us-east-1b conditions: util.go:744: AutoscalingEnabled=False: AsExpected util.go:744: UpdateManagementEnabled=True: AsExpected util.go:744: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-iy3n0ff2/release@sha256:8c7d29fca314b9a4ff7deef5d78f5b5d1b66666ae1b0dee20981e38a9b75997d) util.go:744: ValidArchPlatform=True: AsExpected util.go:744: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:744: SupportedVersionSkew=True: AsExpected(Release image version is valid) util.go:744: ValidMachineConfig=True: AsExpected util.go:744: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 0df29d52) util.go:744: UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 5.0.0-0.ci-2026-07-14-200835-test-ci-op-iy3n0ff2-latest) util.go:744: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:744: ReachedIgnitionEndpoint=True: AsExpected util.go:744: AllMachinesReady=True: AsExpected(All is well) util.go:744: AllNodesHealthy=False: NodeProvisioning(2 of 2 machines are not healthy Machine private-dngcv-us-east-1b-vj7qf-72h4c: NodeProvisioning: Waiting for a node with matching ProviderID to exist Machine private-dngcv-us-east-1b-vj7qf-zpx98: NodeProvisioning: Waiting for a node with matching ProviderID to exist ) util.go:744: ValidPlatformConfig=True: AsExpected(All is well) util.go:744: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0fbc8be8796dc1df5") util.go:744: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) util.go:744: ValidTuningConfig=True: AsExpected util.go:744: UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: private-dngcv-us-east-1b-f390ddc6) util.go:744: AutorepairEnabled=False: AsExpected util.go:744: Ready=False: WaitingForAvailableMachines(Minimum availability requires 2 replicas, current 0 available)
TestCreateClusterProxy
41m4.69s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-r5bx6/proxy-vt29j in 1m24s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster proxy-vt29j util.go:3158: Successfully waited for HostedCluster e2e-clusters-r5bx6/proxy-vt29j to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterProxy/ValidateHostedCluster
30m12.63s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-r5bx6/proxy-vt29j in 2m0s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-vt29j.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-proxy-vt29j.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-vt29j.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.166.1.49:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-vt29j.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.26.168.235:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-vt29j.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": EOF util.go:384: Successfully waited for a successful connection to the guest API server in 1m40.325s util.go:583: Successfully waited for 2 nodes to become ready in 23m21s util.go:620: Successfully waited for HostedCluster e2e-clusters-r5bx6/proxy-vt29j to rollout in 3m6s util.go:3158: Successfully waited for HostedCluster e2e-clusters-r5bx6/proxy-vt29j to have valid conditions in 0s
TestCreateClusterProxy/ValidateHostedCluster/EnsureNoCrashingPods
90ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-7d56f47c78-kpdqx has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-6d4dd5bd68-wztvz util.go:829: Container manager in pod cluster-api-66965948fb-tvcgn has a restartCount > 0 (6) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-84799df967-28twm has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-6f6cc456f9-plw6b has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-5f7dd446-rlll9 has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-6974964875-cxr95 has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-5d96f97dc8-5s6sd has a restartCount > 0 (7)
TestCreateClusterRequestServingIsolation
46m54.92s
requestserving.go:105: Created request serving nodepool clusters/e6133134a9-mgmt-reqserving-2j9zq requestserving.go:105: Created request serving nodepool clusters/e6133134a9-mgmt-reqserving-gjxvj requestserving.go:113: Created non request serving nodepool clusters/e6133134a9-mgmt-non-reqserving-xpcls requestserving.go:113: Created non request serving nodepool clusters/e6133134a9-mgmt-non-reqserving-htltt requestserving.go:113: Created non request serving nodepool clusters/e6133134a9-mgmt-non-reqserving-l6g2x util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/e6133134a9-mgmt-reqserving-2j9zq in 4m36s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/e6133134a9-mgmt-reqserving-gjxvj in 0s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/e6133134a9-mgmt-non-reqserving-xpcls in 100ms util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/e6133134a9-mgmt-non-reqserving-htltt in 18s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/e6133134a9-mgmt-non-reqserving-l6g2x in 100ms create_cluster_test.go:299: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-kwj5k/request-serving-isolation-m965b in 20s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster request-serving-isolation-m965b util.go:3158: Successfully waited for HostedCluster e2e-clusters-kwj5k/request-serving-isolation-m965b to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() requestserving.go:132: Tearing down custom nodepool clusters/e6133134a9-mgmt-reqserving-2j9zq requestserving.go:132: Tearing down custom nodepool clusters/e6133134a9-mgmt-reqserving-gjxvj requestserving.go:132: Tearing down custom nodepool clusters/e6133134a9-mgmt-non-reqserving-xpcls requestserving.go:132: Tearing down custom nodepool clusters/e6133134a9-mgmt-non-reqserving-htltt requestserving.go:132: Tearing down custom nodepool clusters/e6133134a9-mgmt-non-reqserving-l6g2x hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterRequestServingIsolation/ValidateHostedCluster
33m14.32s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-kwj5k/request-serving-isolation-m965b in 1m9s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-m965b.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-request-serving-isolation-m965b.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-m965b.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 13.217.251.190:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m12.025s util.go:583: Successfully waited for 3 nodes to become ready in 22m3s util.go:620: Successfully waited for HostedCluster e2e-clusters-kwj5k/request-serving-isolation-m965b to rollout in 6m54s util.go:3158: Successfully waited for HostedCluster e2e-clusters-kwj5k/request-serving-isolation-m965b to have valid conditions in 51s
TestCreateClusterRequestServingIsolation/ValidateHostedCluster/EnsureNoCrashingPods
60ms
util.go:826: Leader election failure detected in container manager in pod capi-provider-6844d79588-bxk2c util.go:829: Container manager in pod cluster-api-58cc74dcc8-5dgp5 has a restartCount > 0 (6) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-598b649479-x8d7d has a restartCount > 0 (6) util.go:829: Container control-plane-operator in pod control-plane-operator-b7bc4bd9f-fls7k has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-9fb777c9f-cxl7k has a restartCount > 0 (5) util.go:826: Leader election failure detected in container etcd-defrag in pod etcd-0
TestKarpenter
46m18.39s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-v9hp7/karpenter-h87f5 in 1m57s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-h87f5 util.go:3158: Successfully waited for HostedCluster e2e-clusters-v9hp7/karpenter-h87f5 to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenter/ValidateHostedCluster
35m40.33s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-v9hp7/karpenter-h87f5 in 1m3s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-h87f5.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-h87f5.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-h87f5.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 32.195.235.154:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m20.025s util.go:583: Successfully waited for 2 nodes to become ready in 23m6s util.go:620: Successfully waited for HostedCluster e2e-clusters-v9hp7/karpenter-h87f5 to rollout in 9m6s util.go:3158: Successfully waited for HostedCluster e2e-clusters-v9hp7/karpenter-h87f5 to have valid conditions in 0s
TestKarpenter/ValidateHostedCluster/EnsureNoCrashingPods
70ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-74c8fb79fb-7kkdk has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-55f4859fc6-jxvbz util.go:829: Container manager in pod cluster-api-6db845575b-4gpbv has a restartCount > 0 (2) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-7bc74dbcc4-hqj8j has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-65958997b6-t2nsq has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-7b9758b89-wfrjs has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-6fd7b56bd8-kwc4c has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-8469dc7f54-4xn4w has a restartCount > 0 (7)
TestKarpenterUpgradeControlPlane
45m59.7s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-vlzdh/karpenter-upgrade-control-plane-7xd6z in 57s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-upgrade-control-plane-7xd6z util.go:3158: Successfully waited for HostedCluster e2e-clusters-vlzdh/karpenter-upgrade-control-plane-7xd6z to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenterUpgradeControlPlane/ValidateHostedCluster
35m12.41s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-vlzdh/karpenter-upgrade-control-plane-7xd6z in 1m24s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-7xd6z.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-upgrade-control-plane-7xd6z.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-7xd6z.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.211.18.31:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 1m49.05s util.go:583: Successfully waited for 2 nodes to become ready in 24m27s util.go:620: Successfully waited for HostedCluster e2e-clusters-vlzdh/karpenter-upgrade-control-plane-7xd6z to rollout in 2m39s util.go:3158: Successfully waited for HostedCluster e2e-clusters-vlzdh/karpenter-upgrade-control-plane-7xd6z to have valid conditions in 4m48s
TestKarpenterUpgradeControlPlane/ValidateHostedCluster/EnsureNoCrashingPods
100ms
util.go:826: Leader election failure detected in container manager in pod capi-provider-f5bbc697f-rw7vg util.go:829: Container manager in pod cluster-api-849d57f4cb-crl7c has a restartCount > 0 (2) util.go:826: Leader election failure detected in container cluster-node-tuning-operator in pod cluster-node-tuning-operator-54ddb785d5-qbdxs util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-7556fd4449-l6gz9 has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-5c8f5c9cd5-hcqqg has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-8675df94b5-lqjm7 has a restartCount > 0 (5) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-6f987bcd8d-q8xht has a restartCount > 0 (5) util.go:826: Leader election failure detected in container etcd-defrag in pod etcd-0
TestNodePool
0s
TestNodePool/HostedCluster0
26m45.6s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-wm75l/node-pool-z66bc in 1m2s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-z66bc util.go:3158: Successfully waited for HostedCluster e2e-clusters-wm75l/node-pool-z66bc to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePool/HostedCluster0/ValidateHostedCluster
20m56.29s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-wm75l/node-pool-z66bc in 1m33s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-z66bc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-z66bc.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-z66bc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.204.17.202:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-z66bc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.165.75.253:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-z66bc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.204.17.202:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m27.025s util.go:583: Successfully waited for 0 nodes to become ready in 25ms eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-wm75l/hostedclusters/node-pool-z66bc?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io node-pool-z66bc) util.go:3158: Successfully waited for HostedCluster e2e-clusters-wm75l/node-pool-z66bc to have valid conditions in 16m51s
TestNodePool/HostedCluster0/ValidateHostedCluster/EnsureNoCrashingPods
60ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-5b6d56dd54-xb9j6 has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-c4cb5774c-pks97 util.go:829: Container manager in pod cluster-api-cdd9df97-pvg7k has a restartCount > 0 (6) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-68fbf69ff7-p22wm has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-7c5d9995ff-lthg4 has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-84c9d76c57-dspz5 has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-795ff4d7c-vhhdl has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-7cc67bf6d6-ms527 has a restartCount > 0 (7)
TestNodePool/HostedCluster2
26m20.91s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-6jk68/node-pool-wjdsp in 27s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-wjdsp util.go:3158: Successfully waited for HostedCluster e2e-clusters-6jk68/node-pool-wjdsp to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePool/HostedCluster2/ValidateHostedCluster
21m2.3s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-6jk68/node-pool-wjdsp in 51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-wjdsp.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-wjdsp.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-wjdsp.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.208.226.230:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-wjdsp.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 98.90.56.205:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m21.025s util.go:583: Successfully waited for 0 nodes to become ready in 25ms eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a7639eaa1cd3041fcbe0bce124838aeb-aada29e6fa092241.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-6jk68/hostedclusters/node-pool-wjdsp?timeout=5m0s": dial tcp 35.175.105.113:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io node-pool-wjdsp) util.go:3158: Successfully waited for HostedCluster e2e-clusters-6jk68/node-pool-wjdsp to have valid conditions in 17m45s
TestNodePool/HostedCluster2/ValidateHostedCluster/EnsureNoCrashingPods
70ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-b775d8744-b5j6p has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod capi-provider-d46dffcb6-sjxph util.go:829: Container manager in pod cluster-api-7b84c8744c-47zl4 has a restartCount > 0 (2) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-556c5c8955-bzxd8 has a restartCount > 0 (5) util.go:829: Container control-plane-operator in pod control-plane-operator-86996b8bd6-v9d6r has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-5f787d5d7b-767fz has a restartCount > 0 (4) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-69b5f889bc-g5rh4 has a restartCount > 0 (5) util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-5d56698cd7-j7nbp has a restartCount > 0 (7)
TestNodePoolAutoscalingScaleFromZero
1h0m44.5s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-lt8x5/scale-from-zero-wtn2q in 34s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster scale-from-zero-wtn2q util.go:3158: Failed to wait for HostedCluster e2e-clusters-lt8x5/scale-from-zero-wtn2q to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-lt8x5/scale-from-zero-wtn2q invalid at RV 137494 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, node-tuning, openshift-samples, service-ca, storage are not available) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-200835-test-ci-op-iy3n0ff2-latest: some cluster operators are not available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePoolAutoscalingScaleFromZero/ValidateHostedCluster
49m2.04s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-lt8x5/scale-from-zero-wtn2q in 2m30s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-wtn2q.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-scale-from-zero-wtn2q.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-wtn2q.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 34.206.82.97:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 1m32.025s util.go:583: Failed to wait for 1 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 1 nodes, got 0
TestUpgradeControlPlane
42m35.47s
control_plane_upgrade_test.go:25: Starting control plane upgrade test. FromImage: registry.build01.ci.openshift.org/ci-op-iy3n0ff2/release@sha256:622e3aad9f1316badf4e7c2069a15bb6e480d3a5783727abee78031623fec157, toImage: registry.build01.ci.openshift.org/ci-op-iy3n0ff2/release@sha256:8c7d29fca314b9a4ff7deef5d78f5b5d1b66666ae1b0dee20981e38a9b75997d hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-69qsx/control-plane-upgrade-25h5x in 53s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-25h5x util.go:3158: Successfully waited for HostedCluster e2e-clusters-69qsx/control-plane-upgrade-25h5x to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestUpgradeControlPlane/ValidateHostedCluster
33m36.45s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-69qsx/control-plane-upgrade-25h5x in 1m12s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-25h5x.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-control-plane-upgrade-25h5x.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-25h5x.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 32.198.181.124:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m16.15s util.go:583: Successfully waited for 2 nodes to become ready in 23m27s util.go:620: Successfully waited for HostedCluster e2e-clusters-69qsx/control-plane-upgrade-25h5x to rollout in 3m54s util.go:3158: Successfully waited for HostedCluster e2e-clusters-69qsx/control-plane-upgrade-25h5x to have valid conditions in 2m42s
TestUpgradeControlPlane/ValidateHostedCluster/EnsureNoCrashingPods
80ms
util.go:826: Leader election failure detected in container manager in pod capi-provider-59c7975bd9-wwvg6 util.go:829: Container manager in pod cluster-api-5cb4cb6785-hg4br has a restartCount > 0 (6) util.go:829: Container cluster-node-tuning-operator in pod cluster-node-tuning-operator-584855cbd-hnnh9 has a restartCount > 0 (1) util.go:829: Container cluster-storage-operator in pod cluster-storage-operator-557658c9f5-lvfh7 has a restartCount > 0 (6) util.go:829: Container control-plane-operator in pod control-plane-operator-77fb86d989-8nhmm has a restartCount > 0 (7) util.go:829: Container control-plane-pki-operator in pod control-plane-pki-operator-6bcd7c87d5-8dlzd has a restartCount > 0 (5) util.go:829: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-757cc4f45d-94dtn has a restartCount > 0 (5) util.go:826: Leader election failure detected in container etcd-defrag in pod etcd-1