PR #8338 - 09-02 17:37

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-xq9ww/autoscaling-lxp9d in 22s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster autoscaling-lxp9d util.go:3215: Successfully waited for HostedCluster e2e-clusters-xq9ww/autoscaling-lxp9d to have valid conditions in 25ms hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-xq9ww/autoscaling-lxp9d in 1m3.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-lxp9d.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-autoscaling-lxp9d.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-lxp9d.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.6.22.138:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-lxp9d.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 34.238.71.235:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m25.075s util.go:590: Successfully waited for 1 nodes to become ready in 7m48.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xq9ww/hostedclusters/autoscaling-lxp9d?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xq9ww/hostedclusters/autoscaling-lxp9d?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xq9ww/hostedclusters/autoscaling-lxp9d?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xq9ww/hostedclusters/autoscaling-lxp9d?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xq9ww/hostedclusters/autoscaling-lxp9d?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-xq9ww/autoscaling-lxp9d to rollout in 4m51.025s util.go:3215: Successfully waited for HostedCluster e2e-clusters-xq9ww/autoscaling-lxp9d to have valid conditions in 3.025s

util.go:836: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-749656bbb7-hdprz has a restartCount > 0 (3) util.go:836: Container manager in pod capi-provider-5478cd4fc4-k89t7 has a restartCount > 0 (5) util.go:836: Container manager in pod cluster-api-64b8ccb98f-qdbxt has a restartCount > 0 (5) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-85b886f658-cnws4 has a restartCount > 0 (3) util.go:836: Container control-plane-operator in pod control-plane-operator-7cd6fcb4cc-nvsh9 has a restartCount > 0 (5) util.go:833: Leader election failure detected in container control-plane-pki-operator in pod control-plane-pki-operator-7f9bcb99db-77p45 util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-7677c5bf87-zjq6x has a restartCount > 0 (3) util.go:836: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-7774d54b56-g4vjn has a restartCount > 0 (5)

create_cluster_test.go:47: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-xmv9f/create-cluster-rq67t in 21s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster create-cluster-rq67t util.go:3215: Failed to wait for HostedCluster e2e-clusters-xmv9f/create-cluster-rq67t to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-xmv9f/create-cluster-rq67t invalid at RV 91754 after 2s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(capi-provider deployment has 1 unavailable replicas) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-xmv9f/create-cluster-rq67t in 54.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-rq67t.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-create-cluster-rq67t.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-rq67t.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.173.7.64:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m21.125s util.go:590: Successfully waited for 3 nodes to become ready in 8m21.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xmv9f/hostedclusters/create-cluster-rq67t?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xmv9f/hostedclusters/create-cluster-rq67t?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-xmv9f/hostedclusters/create-cluster-rq67t?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-xmv9f/create-cluster-rq67t to rollout in 4m48.025s util.go:3215: Successfully waited for HostedCluster e2e-clusters-xmv9f/create-cluster-rq67t to have valid conditions in 1m18.825s

util.go:1951: failed to ensure guest webhooks validated, violating webhook test-webhook was not deleted: context deadline exceeded

util.go:836: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-76f5b69c84-k6pkf has a restartCount > 0 (3) util.go:836: Container manager in pod capi-provider-dfc96d494-6kwrl has a restartCount > 0 (6) util.go:836: Container manager in pod cluster-api-b867b4c7c-nxxbv has a restartCount > 0 (5) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-5984bd8dbd-4d2vc has a restartCount > 0 (3) util.go:836: Container control-plane-operator in pod control-plane-operator-5987bfd579-pbbbs has a restartCount > 0 (5) util.go:833: Leader election failure detected in container control-plane-pki-operator in pod control-plane-pki-operator-f44c5f4c5-w6582 util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-84446f95db-w2m75 has a restartCount > 0 (3) util.go:836: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-f698bfcdd-fvdxs has a restartCount > 0 (5)

util.go:994: Timed out after 300.000s. ca-bundle.crt file should be available in openshift-apiserver pod Expected success, but got an error: <*errors.errorString | 0x2aff03ba99c0>: unable to upgrade connection: pod does not exist { s: "unable to upgrade connection: pod does not exist", }

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-q8f5t/custom-config-f7758 in 10s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster custom-config-f7758 eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-q8f5t/custom-config-f7758 to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: i/o timeout fixture.go:427: Failed to dump machine journals; this is nonfatal: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:427: Failed to dump machine journals; this is nonfatal: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused journals.go:319: Successfully copied machine journals to /logs/artifacts/TestCreateClusterCustomConfig/machine-journals hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-q8f5t, name: custom-config-f7758 hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1": dial tcp 54.243.96.163:6443: connect: connection refused

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-q8f5t/custom-config-f7758 in 57.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-f7758.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-custom-config-f7758.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-f7758.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.210.200.220:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m11.2s util.go:590: Successfully waited for 2 nodes to become ready in 7m45.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 479; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-q8f5t/custom-config-f7758 to rollout in 21m33.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-q8f5t/hostedclusters/custom-config-f7758?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-q8f5t/custom-config-f7758 to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-q8f5t/custom-config-f7758 invalid at RV 98820 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(capi-provider deployment has 1 unavailable replicas)

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-99p52/ha-break-glass-creds-89trq in 50s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster ha-break-glass-creds-89trq util.go:3215: Failed to wait for HostedCluster e2e-clusters-99p52/ha-break-glass-creds-89trq to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-99p52/ha-break-glass-creds-89trq invalid at RV 145753 after 2s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(ignition-server-proxy deployment has 1 unavailable replicas) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-99p52/ha-break-glass-creds-89trq in 25ms util.go:329: Successfully waited for kubeconfig secret to have data in 25ms util.go:391: Successfully waited for a successful connection to the guest API server in 25ms create_cluster_test.go:170: Successfully waited for KAS deployment to have 3 ready replicas in 25ms create_cluster_test.go:187: fetching mgmt kubeconfig util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-99p52/ha-break-glass-creds-89trq in 25ms util.go:329: Successfully waited for kubeconfig secret to have data in 25ms

control_plane_pki_operator.go:101: generating new break-glass credentials for more than one signer pki.go:76: loading certificate/key pair from disk for signer customer-break-glass, use $REGENERATE_PKI to generate new ones control_plane_pki_operator.go:210: creating CSR "2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k" for signer "customer-break-glass", requesting client auth usages control_plane_pki_operator.go:220: creating CSRA e2e-clusters-99p52-ha-break-glass-creds-89trq/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k to trigger automatic approval of the CSR control_plane_pki_operator.go:227: Successfully waited for CSR "2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k" to be approved and signed in 3.025s control_plane_pki_operator.go:139: validating that the client certificate provides the appropriate access control_plane_pki_operator.go:125: amending the existing kubeconfig to use break-glass client certificate credentials control_plane_pki_operator.go:142: issuing SSR to identify the subject we are given using the client certificate control_plane_pki_operator.go:162: ensuring that the SSR identifies the client certificate as having system:masters power and correct username pki.go:76: loading certificate/key pair from disk for signer sre-break-glass, use $REGENERATE_PKI to generate new ones control_plane_pki_operator.go:210: creating CSR "1dfxm9u7ey7eao247glk1ipa2v4qrmcjz8bu97l2swmb" for signer "sre-break-glass", requesting client auth usages control_plane_pki_operator.go:220: creating CSRA e2e-clusters-99p52-ha-break-glass-creds-89trq/1dfxm9u7ey7eao247glk1ipa2v4qrmcjz8bu97l2swmb to trigger automatic approval of the CSR control_plane_pki_operator.go:227: Successfully waited for CSR "1dfxm9u7ey7eao247glk1ipa2v4qrmcjz8bu97l2swmb" to be approved and signed in 3.025s control_plane_pki_operator.go:139: validating that the client certificate provides the appropriate access control_plane_pki_operator.go:125: amending the existing kubeconfig to use break-glass client certificate credentials control_plane_pki_operator.go:142: issuing SSR to identify the subject we are given using the client certificate control_plane_pki_operator.go:162: ensuring that the SSR identifies the client certificate as having system:masters power and correct username control_plane_pki_operator.go:105: revoking the "customer-break-glass" signer pki.go:76: loading certificate/key pair from disk for signer customer-break-glass, use $REGENERATE_PKI to generate new ones control_plane_pki_operator.go:262: creating CRR e2e-clusters-99p52-ha-break-glass-creds-89trq/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k to trigger signer certificate revocation eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1alpha1.CertificateRevocationRequest: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/certificates.hypershift.openshift.io/v1alpha1/namespaces/e2e-clusters-99p52-ha-break-glass-creds-89trq/certificaterevocationrequests/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k?timeout=5m0s": context deadline exceeded - error from a previous attempt: http2: client connection lost control_plane_pki_operator.go:270: Failed to wait for CRR e2e-clusters-99p52-ha-break-glass-creds-89trq/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k to complete in 10m0s: context deadline exceeded eventually.go:225: observed *v1alpha1.CertificateRevocationRequest e2e-clusters-99p52-ha-break-glass-creds-89trq/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k invalid at RV 90953 after 10m0s: incorrect condition: wanted PreviousCertificatesRevoked=True, got PreviousCertificatesRevoked=False: WaitingForAvailable(Previous signer certificate not yet revoked.) control_plane_pki_operator.go:270: *v1alpha1.CertificateRevocationRequest e2e-clusters-99p52-ha-break-glass-creds-89trq/2ospauvka3odiohyi0t684n8eztmiesonfjtdruz6o0k conditions: control_plane_pki_operator.go:270: LeafCertificatesRegenerated=True: AsExpected(All leaf certificates are re-generated.) control_plane_pki_operator.go:270: PreviousCertificatesRevoked=False: WaitingForAvailable(Previous signer certificate not yet revoked.) control_plane_pki_operator.go:270: NewCertificatesTrusted=True: AsExpected(New signer certificate e2e-clusters-99p52-ha-break-glass-creds-89trq/customer-system-admin-signer trusted.) control_plane_pki_operator.go:270: RootCertificatesRegenerated=True: AsExpected(Signer certificate e2e-clusters-99p52-ha-break-glass-creds-89trq/customer-system-admin-signer regenerated.) control_plane_pki_operator.go:270: SignerClassValid=True: AsExpected(Signer class "customer-break-glass" known.)

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-tv7b4/private-9wmn5 in 20s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-9wmn5 eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-tv7b4/private-9wmn5 to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: i/o timeout fixture.go:427: Failed to dump machine journals; this is nonfatal: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused journals.go:319: Successfully copied machine journals to /logs/artifacts/TestCreateClusterPrivate/machine-journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-tv7b4, name: private-9wmn5 hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1": dial tcp 54.243.96.163:6443: connect: connection refused

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-tv7b4/private-9wmn5 in 45.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 467; INTERNAL_ERROR; received from peer util.go:751: Successfully waited for NodePools for HostedCluster e2e-clusters-tv7b4/private-9wmn5 to have all of their desired nodes in 21m57.175s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-tv7b4/private-9wmn5 to rollout in 9m36.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-tv7b4/hostedclusters/private-9wmn5?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-tv7b4/private-9wmn5 to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-tv7b4/private-9wmn5 invalid at RV 98823 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(capi-provider deployment has 1 unavailable replicas)

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-bpgrk/private-9h79m in 41s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-9h79m util.go:3215: Successfully waited for HostedCluster e2e-clusters-bpgrk/private-9h79m to have valid conditions in 25ms hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-bpgrk/private-9h79m in 1m18.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/nodepools?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 459; INTERNAL_ERROR; received from peer util.go:751: Successfully waited for NodePools for HostedCluster e2e-clusters-bpgrk/private-9h79m to have all of their desired nodes in 21m3.95s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/hostedclusters/private-9h79m?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-bpgrk/hostedclusters/private-9h79m?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-bpgrk/private-9h79m to rollout in 13m17.9s util.go:3215: Successfully waited for HostedCluster e2e-clusters-bpgrk/private-9h79m to have valid conditions in 4.125s

util.go:836: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-75c76b55c7-hncl2 has a restartCount > 0 (6) util.go:833: Leader election failure detected in container manager in pod capi-provider-7c65fb5db5-rxhkw util.go:836: Container manager in pod cluster-api-54747f55b9-x4v2x has a restartCount > 0 (10) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-787c4479b6-5756v has a restartCount > 0 (6) util.go:836: Container control-plane-operator in pod control-plane-operator-74f9b67b78-wh67b has a restartCount > 0 (7) util.go:833: Leader election failure detected in container control-plane-pki-operator in pod control-plane-pki-operator-84cf6d85dd-c2jdl util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-cbf6f8f5b-wqbjb has a restartCount > 0 (6) util.go:836: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-b59c6b768-rt4vf has a restartCount > 0 (8)

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-t64lq/proxy-k95pb in 23s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster proxy-k95pb eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-t64lq/proxy-k95pb to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-t64lq/proxy-k95pb in 1m3.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-k95pb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-proxy-k95pb.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-k95pb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.52.55.101:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m1.25s util.go:590: Successfully waited for 2 nodes to become ready in 8m45.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-t64lq/proxy-k95pb to rollout in 4m33.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 471; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-t64lq/hostedclusters/proxy-k95pb?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-t64lq/proxy-k95pb to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-t64lq/proxy-k95pb invalid at RV 91389 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas(capi-provider deployment has 1 unavailable replicas)

requestserving.go:105: Created request serving nodepool clusters/8afb22bfd4-mgmt-reqserving-9cvjs requestserving.go:105: Created request serving nodepool clusters/8afb22bfd4-mgmt-reqserving-p8dql requestserving.go:113: Created non request serving nodepool clusters/8afb22bfd4-mgmt-non-reqserving-v57zn requestserving.go:113: Created non request serving nodepool clusters/8afb22bfd4-mgmt-non-reqserving-hvrvn requestserving.go:113: Created non request serving nodepool clusters/8afb22bfd4-mgmt-non-reqserving-stq5j util.go:590: Successfully waited for 1 nodes to become ready for NodePool clusters/8afb22bfd4-mgmt-reqserving-9cvjs in 3m42.025s util.go:590: Successfully waited for 1 nodes to become ready for NodePool clusters/8afb22bfd4-mgmt-reqserving-p8dql in 9.025s util.go:590: Successfully waited for 1 nodes to become ready for NodePool clusters/8afb22bfd4-mgmt-non-reqserving-v57zn in 48.025s util.go:590: Successfully waited for 1 nodes to become ready for NodePool clusters/8afb22bfd4-mgmt-non-reqserving-hvrvn in 3.025s util.go:590: Successfully waited for 1 nodes to become ready for NodePool clusters/8afb22bfd4-mgmt-non-reqserving-stq5j in 100ms create_cluster_test.go:298: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd in 12s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster request-serving-isolation-sqzsd eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() requestserving.go:132: Tearing down custom nodepool clusters/8afb22bfd4-mgmt-reqserving-9cvjs requestserving.go:132: Tearing down custom nodepool clusters/8afb22bfd4-mgmt-reqserving-p8dql requestserving.go:132: Tearing down custom nodepool clusters/8afb22bfd4-mgmt-non-reqserving-v57zn requestserving.go:132: Tearing down custom nodepool clusters/8afb22bfd4-mgmt-non-reqserving-hvrvn requestserving.go:132: Tearing down custom nodepool clusters/8afb22bfd4-mgmt-non-reqserving-stq5j hypershift_framework.go:327: skipping teardown, already called

fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: i/o timeout fixture.go:427: Failed to dump machine journals; this is nonfatal: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: i/o timeout hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused journals.go:319: Successfully copied machine journals to /logs/artifacts/TestCreateClusterRequestServingIsolation/machine-journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-2fwd4, name: request-serving-isolation-sqzsd hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1": dial tcp 54.243.96.163:6443: connect: connection refused

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd in 1m15.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-sqzsd.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-request-serving-isolation-sqzsd.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-sqzsd.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.4.21.115:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 1m44.05s util.go:590: Successfully waited for 3 nodes to become ready in 15m33.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd to rollout in 9m12.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-2fwd4/hostedclusters/request-serving-isolation-sqzsd?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-2fwd4/request-serving-isolation-sqzsd invalid at RV 99697 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, kube-controller-manager deployment has 1 unavailable replicas, openshift-apiserver deployment has 1 unavailable replicas])

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-8l6rw/karpenter-sk59g in 26s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-sk59g util.go:3215: Successfully waited for HostedCluster e2e-clusters-8l6rw/karpenter-sk59g to have valid conditions in 25ms hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-8l6rw/karpenter-sk59g in 1m15.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-sk59g.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-sk59g.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-sk59g.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.54.140.180:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m23.225s util.go:590: Successfully waited for 2 nodes to become ready in 8m42.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8l6rw/hostedclusters/karpenter-sk59g?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8l6rw/hostedclusters/karpenter-sk59g?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8l6rw/hostedclusters/karpenter-sk59g?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused util.go:627: Successfully waited for HostedCluster e2e-clusters-8l6rw/karpenter-sk59g to rollout in 3m36.025s util.go:3215: Successfully waited for HostedCluster e2e-clusters-8l6rw/karpenter-sk59g to have valid conditions in 3.025s

util.go:836: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-686dbf744-77pwn has a restartCount > 0 (3) util.go:836: Container manager in pod capi-provider-76b58f6b6-jvl46 has a restartCount > 0 (5) util.go:836: Container manager in pod cluster-api-8558cfc684-4zz7x has a restartCount > 0 (5) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-76f55b5b49-qjq7r has a restartCount > 0 (3) util.go:836: Container control-plane-operator in pod control-plane-operator-6df4669754-ltcgl has a restartCount > 0 (5) util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-748c4fc989-p5fvr has a restartCount > 0 (3) util.go:836: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-64c6d574b6-dbm8t has a restartCount > 0 (5)

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk in 34s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-upgrade-control-plane-kg8vk util.go:3215: Failed to wait for HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk invalid at RV 99703 after 2s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, openshift-apiserver deployment has 1 unavailable replicas]) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk in 1m24.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-kg8vk.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-upgrade-control-plane-kg8vk.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-kg8vk.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.55.171.33:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-kg8vk.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.157.135.129:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m12.075s util.go:590: Successfully waited for 2 nodes to become ready in 7m54.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 481; INTERNAL_ERROR; received from peer util.go:627: Successfully waited for HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk to rollout in 12m21.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io karpenter-upgrade-control-plane-kg8vk) eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8r8z9/hostedclusters/karpenter-upgrade-control-plane-kg8vk?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-8r8z9/karpenter-upgrade-control-plane-kg8vk invalid at RV 99703 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, openshift-apiserver deployment has 1 unavailable replicas])

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-npphz/node-pool-z6dnq in 31s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-z6dnq util.go:3215: Failed to wait for HostedCluster e2e-clusters-npphz/node-pool-z6dnq to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-npphz/node-pool-z6dnq invalid at RV 149327 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=False, got ClusterVersionAvailable=True: FromClusterVersion(Done applying 5.1.0-0.ci-2026-09-02-180235-test-ci-op-gcbxwih8-latest) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=False, got ClusterVersionSucceeding=True: FromClusterVersion eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=Unknown, got ControlPlaneConnectionAvailable=True: AsExpected(All is well) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=Unknown, got DataPlaneConnectionAvailable=True: AsExpected(All is well) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=True, got ClusterVersionProgressing=False: FromClusterVersion(Cluster version is 5.1.0-0.ci-2026-09-02-180235-test-ci-op-gcbxwih8-latest) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-npphz/node-pool-z6dnq in 25ms util.go:329: Successfully waited for kubeconfig secret to have data in 25ms util.go:391: Successfully waited for a successful connection to the guest API server in 0s

nodepool_kms_root_volume_test.go:44: Starting test KMSRootVolumeTest nodepool_kms_root_volume_test.go:56: retrieved KMS ARN: arn:aws:kms:us-east-1:820196288204:key/d3cdd9e0-3fd1-47a4-a559-72ae3672c5a6 nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:590: Failed to wait for 1 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-kms-root-volume in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 1 nodes, got 0

nodepool_nto_machineconfig_test.go:67: Starting test NTOMachineConfigRolloutTest nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:590: Failed to wait for 2 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-inplace in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 2 nodes, got 0

nodepool_nto_machineconfig_test.go:67: Starting test NTOMachineConfigRolloutTest nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=InstanceProvisionStarted,WaitingForInfrastructure util.go:590: Successfully waited for 2 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace in 8m0.025s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace to have correct status in 15.025s util.go:506: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace to start config update in 15.025s util.go:522: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace to finish config update in 7m0.025s nodepool_machineconfig_test.go:165: Successfully waited for all pods in the DaemonSet kube-system/node-pool-z6dnq-test-ntomachineconfig-replace to be ready in 10.025s util.go:590: Successfully waited for 2 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace in 0s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-ntomachineconfig-replace to have correct status in 25ms

util.go:836: Container manager in pod capi-provider-798d46b796-v9cq8 has a restartCount > 0 (11) util.go:836: Container manager in pod cluster-api-54bf544769-5l6qd has a restartCount > 0 (14) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-db694859f-nbhgk has a restartCount > 0 (11) util.go:836: Container control-plane-operator in pod control-plane-operator-8c75cdb87-2w5nd has a restartCount > 0 (14) util.go:836: Container control-plane-pki-operator in pod control-plane-pki-operator-774d69c75c-5wngd has a restartCount > 0 (5) util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-75c64b4c6-ztjcf has a restartCount > 0 (11)

nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure util.go:590: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-day2-tags in 8m57.025s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-day2-tags to have correct status in 36.25s nodepool_day2_tags_test.go:91: Updating NodePool with day-2 labels and taints nodepool_day2_tags_test.go:147: Timed out after 145.564s. The function passed to Eventually failed at /hypershift/test/e2e/nodepool_day2_tags_test.go:142 with: Expected <[]types.Tag | len:9, cap:16>: [ { Key: "MachineName", Value: "e2e-clusters-npphz-node-pool-z6dnq/node-pool-z6dnq-test-day2-tags-vbxsz-kvc2w", noSmithyDocumentSerde: {}, }, { Key: "kubernetes.io/cluster/node-pool-z6dnq", Value: "owned", noSmithyDocumentSerde: {}, }, { Key: "red-hat-clustertype", Value: "rosa", noSmithyDocumentSerde: {}, }, { Key: "expirationDate", Value: "2026-09-02T22:48+00:00", noSmithyDocumentSerde: {}, }, { Key: "sigs.k8s.io/cluster-api-provider-aws/role", Value: "node", noSmithyDocumentSerde: {}, }, { Key: "Name", Value: "node-pool-z6dnq-test-day2-tags-vbxsz-kvc2w", noSmithyDocumentSerde: {}, }, { Key: "red-hat-managed", Value: "true", noSmithyDocumentSerde: {}, }, { Key: "architecture", Value: "x86_64", noSmithyDocumentSerde: {}, }, { Key: "sigs.k8s.io/cluster-api-provider-aws/cluster/node-pool-z6dnq", Value: "owned", noSmithyDocumentSerde: {}, }, ] to contain element matching <types.Tag>: { Key: "test-day2-tag", Value: "test-day2-value", noSmithyDocumentSerde: {}, }

nodepool_machineconfig_test.go:54: Starting test NodePoolMachineconfigRolloutTest nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure util.go:590: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig in 8m33.025s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig to have correct status in 15.025s util.go:506: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig to start config update in 15.025s util.go:522: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig to finish config update in 6m20.025s nodepool_machineconfig_test.go:165: Successfully waited for all pods in the DaemonSet kube-system/machineconfig-update-checker-replace to be ready in 40.025s util.go:590: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig in 0s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-machineconfig to have correct status in 25ms

util.go:836: Container manager in pod capi-provider-798d46b796-v9cq8 has a restartCount > 0 (11) util.go:836: Container manager in pod cluster-api-54bf544769-5l6qd has a restartCount > 0 (14) util.go:836: Container cluster-storage-operator in pod cluster-storage-operator-db694859f-nbhgk has a restartCount > 0 (11) util.go:836: Container control-plane-operator in pod control-plane-operator-8c75cdb87-2w5nd has a restartCount > 0 (14) util.go:836: Container control-plane-pki-operator in pod control-plane-pki-operator-774d69c75c-5wngd has a restartCount > 0 (5) util.go:836: Container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-75c64b4c6-ztjcf has a restartCount > 0 (11)

nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure util.go:590: Successfully waited for 2 nodes to become ready for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade in 8m57.025s nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade to have correct status in 36.25s nodepool_rolling_upgrade_test.go:111: Successfully waited for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade to start the rolling upgrade in 6.1s eventually.go:105: Failed to get *v1beta1.NodePool: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 457; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-npphz/nodepools/node-pool-z6dnq-test-rolling-upgrade?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.NodePool: context deadline exceeded nodepool_rolling_upgrade_test.go:125: Failed to wait for NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade to finish the rolling upgrade in 30m0s: context deadline exceeded eventually.go:225: observed *v1beta1.NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade invalid at RV 102576 after 30m0s: incorrect condition: wanted UpdatingPlatformMachineTemplate=False, got UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: node-pool-z6dnq-test-rolling-upgrade-6681849f) nodepool_rolling_upgrade_test.go:125: *v1beta1.NodePool e2e-clusters-npphz/node-pool-z6dnq-test-rolling-upgrade conditions: nodepool_rolling_upgrade_test.go:125: AutoscalingEnabled=False: AsExpected nodepool_rolling_upgrade_test.go:125: UpdateManagementEnabled=True: AsExpected nodepool_rolling_upgrade_test.go:125: ValidReleaseImage=True: AsExpected(Using release image: registry.build09.ci.openshift.org/ci-op-gcbxwih8/release@sha256:cfeb6fdb26ab90cf5be3ac115968e475961d48e36910990f06788cbecf12f04c) nodepool_rolling_upgrade_test.go:125: ValidArchPlatform=True: AsExpected nodepool_rolling_upgrade_test.go:125: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) nodepool_rolling_upgrade_test.go:125: SupportedVersionSkew=True: AsExpected(Release image version is valid) nodepool_rolling_upgrade_test.go:125: ValidMachineConfig=True: AsExpected nodepool_rolling_upgrade_test.go:125: UpdatingConfig=False: AsExpected nodepool_rolling_upgrade_test.go:125: UpdatingVersion=False: AsExpected nodepool_rolling_upgrade_test.go:125: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) nodepool_rolling_upgrade_test.go:125: ReachedIgnitionEndpoint=True: AsExpected nodepool_rolling_upgrade_test.go:125: AllMachinesReady=True: AsExpected(All is well) nodepool_rolling_upgrade_test.go:125: AllNodesHealthy=False: NodeProvisioning(1 of 3 machines are not healthy Machine node-pool-z6dnq-test-rolling-upgrade-pwhmc-6nhgq: NodeProvisioning: Waiting for a node with matching ProviderID to exist ) nodepool_rolling_upgrade_test.go:125: ValidPlatformConfig=True: AsExpected(All is well) nodepool_rolling_upgrade_test.go:125: AWSResourceTagConflict=False: NoTagConflict(No AWS resource tag conflicts detected) nodepool_rolling_upgrade_test.go:125: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0c6ace7ae6dea6e9d") nodepool_rolling_upgrade_test.go:125: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) nodepool_rolling_upgrade_test.go:125: ValidTuningConfig=True: AsExpected nodepool_rolling_upgrade_test.go:125: UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: node-pool-z6dnq-test-rolling-upgrade-6681849f) nodepool_rolling_upgrade_test.go:125: AutorepairEnabled=False: AsExpected nodepool_rolling_upgrade_test.go:125: Ready=True: AsExpected

journals.go:308: Error copying machine journals to artifacts directory: signal: killed hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: client rate limiter Wait returned an error: context canceled fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: client rate limiter Wait returned an error: context canceled fixture.go:427: Failed to dump machine journals; this is nonfatal: client rate limiter Wait returned an error: context canceled hypershift_framework.go:589: Failed to destroy cluster: context canceled util.go:220: Failed to delete namespace: e2e-clusters-npphz, will retry: context canceled hypershift_framework.go:605: Failed to delete test namespace: failed to delete namespace: context canceled fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: client rate limiter Wait returned an error: context canceled fixture.go:427: Failed to dump machine journals; this is nonfatal: client rate limiter Wait returned an error: context canceled hypershift_framework.go:546: archiving /logs/artifacts/TestNodePool_HostedCluster0/hostedcluster-node-pool-z6dnq to /logs/artifacts/TestNodePool_HostedCluster0/hostedcluster.tar.gz

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-8mz5p/node-pool-szc84 in 40s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-szc84 util.go:3215: Failed to wait for HostedCluster e2e-clusters-8mz5p/node-pool-szc84 to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-8mz5p/node-pool-szc84 invalid at RV 126145 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=True, got ClusterVersionProgressing=False: FromClusterVersion(Cluster version is 5.1.0-0.ci-2026-09-02-180235-test-ci-op-gcbxwih8-latest) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=Unknown, got DataPlaneConnectionAvailable=True: AsExpected(All is well) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=Unknown, got ControlPlaneConnectionAvailable=True: AsExpected(All is well) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=False, got ClusterVersionSucceeding=True: FromClusterVersion eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=False, got ClusterVersionAvailable=True: FromClusterVersion(Done applying 5.1.0-0.ci-2026-09-02-180235-test-ci-op-gcbxwih8-latest) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-8mz5p/node-pool-szc84 in 25ms util.go:329: Successfully waited for kubeconfig secret to have data in 25ms util.go:391: Successfully waited for a successful connection to the guest API server in 25ms

nodepool_additionalTrustBundlePropagation_test.go:40: Starting AdditionalTrustBundlePropagationTest. nodepool_test.go:376: Validating CAPI v1 condition message bubbling during machine provisioning nodepool_test.go:512: CAPI bubbling confirmed for AllNodesHealthy: Status=False, Reason=WaitingForNodeRef nodepool_test.go:512: CAPI bubbling confirmed for AllMachinesReady: Status=False, Reason=WaitingForInfrastructure util.go:590: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation in 9m24.025s eventually.go:105: Failed to get *v1beta1.NodePool: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 483; INTERNAL_ERROR; received from peer nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation to have correct status in 6m15.025s eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused nodepool_test.go:436: Successfully waited for NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation to have correct status in 11m45.025s

nodepool_additionalTrustBundlePropagation_test.go:82: Updating hosted cluster with additional trust bundle. Bundle: additional-trust-bundle nodepool_additionalTrustBundlePropagation_test.go:90: Successfully waited for Waiting for NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation to begin updating in 15.025s eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.NodePool: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-8mz5p/nodepools/node-pool-szc84-test-additional-trust-bundle-propagation?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused nodepool_additionalTrustBundlePropagation_test.go:104: Failed to wait for Waiting for NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation to stop updating in 20m0s: context deadline exceeded eventually.go:225: observed *v1beta1.NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation invalid at RV 104732 after 20m0s: eventually.go:228: - incorrect condition: wanted UpdatingConfig=False, got UpdatingConfig=True: AsExpected(Updating config in progress. Target config: de5fe49f) eventually.go:228: - incorrect condition: wanted AllNodesHealthy=True, got AllNodesHealthy=False: Deleting(1 of 2 machines are not healthy Machine node-pool-szc84-test-additional-trust-bundle-propagation-kk8ff4: Deleting ) nodepool_additionalTrustBundlePropagation_test.go:104: *v1beta1.NodePool e2e-clusters-8mz5p/node-pool-szc84-test-additional-trust-bundle-propagation conditions: nodepool_additionalTrustBundlePropagation_test.go:104: AutoscalingEnabled=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: UpdateManagementEnabled=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: ValidReleaseImage=True: AsExpected(Using release image: registry.build09.ci.openshift.org/ci-op-gcbxwih8/release@sha256:cfeb6fdb26ab90cf5be3ac115968e475961d48e36910990f06788cbecf12f04c) nodepool_additionalTrustBundlePropagation_test.go:104: ValidArchPlatform=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) nodepool_additionalTrustBundlePropagation_test.go:104: SupportedVersionSkew=True: AsExpected(Release image version is valid) nodepool_additionalTrustBundlePropagation_test.go:104: ValidMachineConfig=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: de5fe49f) nodepool_additionalTrustBundlePropagation_test.go:104: UpdatingVersion=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) nodepool_additionalTrustBundlePropagation_test.go:104: ReachedIgnitionEndpoint=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: AllMachinesReady=True: AsExpected(All is well) nodepool_additionalTrustBundlePropagation_test.go:104: AllNodesHealthy=False: Deleting(1 of 2 machines are not healthy Machine node-pool-szc84-test-additional-trust-bundle-propagation-kk8ff4: Deleting ) nodepool_additionalTrustBundlePropagation_test.go:104: ValidPlatformConfig=True: AsExpected(All is well) nodepool_additionalTrustBundlePropagation_test.go:104: AWSResourceTagConflict=False: NoTagConflict(No AWS resource tag conflicts detected) nodepool_additionalTrustBundlePropagation_test.go:104: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0c6ace7ae6dea6e9d") nodepool_additionalTrustBundlePropagation_test.go:104: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) nodepool_additionalTrustBundlePropagation_test.go:104: ValidTuningConfig=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: UpdatingPlatformMachineTemplate=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: AutorepairEnabled=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:104: Ready=True: AsExpected

fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: i/o timeout fixture.go:427: Failed to dump machine journals; this is nonfatal: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused fixture.go:419: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/api": dial tcp 54.243.96.163:6443: connect: connection refused journals.go:319: Successfully copied machine journals to /logs/artifacts/TestNodePoolAutoscalingScaleFromZero/machine-journals hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-clqwp, name: scale-from-zero-mvxbd hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1": dial tcp 54.243.96.163:6443: i/o timeout

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-clqwp/scale-from-zero-mvxbd in 1m21.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-mvxbd.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-scale-from-zero-mvxbd.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-mvxbd.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.198.73.28:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-mvxbd.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.57.241.126:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 2m21.075s util.go:590: Successfully waited for 1 nodes to become ready in 8m30.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 463; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-clqwp/scale-from-zero-mvxbd to rollout in 19m36.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-clqwp/hostedclusters/scale-from-zero-mvxbd?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-clqwp/scale-from-zero-mvxbd to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-clqwp/scale-from-zero-mvxbd invalid at RV 99556 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, cluster-api deployment has 1 unavailable replicas])

control_plane_upgrade_test.go:25: Starting control plane upgrade test. FromImage: registry.build09.ci.openshift.org/ci-op-gcbxwih8/release@sha256:ccf472c18d1103af06a56d0a5a06f27835305d660ea0700e9e009bc4689743ec, toImage: registry.build09.ci.openshift.org/ci-op-gcbxwih8/release@sha256:cfeb6fdb26ab90cf5be3ac115968e475961d48e36910990f06788cbecf12f04c hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs in 17s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-k6sgs eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-g2jdx/hostedclusters/control-plane-upgrade-k6sgs?timeout=5m0s": context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:312: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs in 1m36.025s util.go:329: Successfully waited for kubeconfig secret to have data in 25ms eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-k6sgs.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-control-plane-upgrade-k6sgs.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-k6sgs.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.200.75.105:443: i/o timeout util.go:391: Successfully waited for a successful connection to the guest API server in 1m37.125s util.go:590: Successfully waited for 2 nodes to become ready in 8m45.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-g2jdx/hostedclusters/control-plane-upgrade-k6sgs?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-g2jdx/hostedclusters/control-plane-upgrade-k6sgs?timeout=5m0s": dial tcp 54.243.96.163:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://a3275916aaa7442f4868e67823ae5b0b-0685ab7a1972736b.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-g2jdx/hostedclusters/control-plane-upgrade-k6sgs?timeout=5m0s": dial tcp 54.243.96.163:6443: i/o timeout util.go:627: Successfully waited for HostedCluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs to rollout in 4m27.025s eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 475; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: context deadline exceeded util.go:3215: Failed to wait for HostedCluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-g2jdx/control-plane-upgrade-k6sgs invalid at RV 91246 after 10m0s: incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, openshift-apiserver deployment has 1 unavailable replicas])