PR #8804 - 07-14 19:12

Job: hypershift
FAILURE
← Back to Test Grid

Test Summary

108
Total Tests
44
Passed
44
Failed
20
Skipped

Failed Tests

TestAutoscaling
1h30m26.78s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-fwdll/autoscaling-jrtgb in 2m18s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster autoscaling-jrtgb eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-fwdll/hostedclusters/autoscaling-jrtgb?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-fwdll/autoscaling-jrtgb to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestAutoscaling/Teardown
40m0.07s
journals.go:77: No machines associated with infra id autoscaling-jrtgb were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:77: No machines associated with infra id autoscaling-jrtgb were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-fwdll, name: autoscaling-jrtgb hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestAutoscaling/ValidateHostedCluster
48m7.03s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-fwdll/autoscaling-jrtgb in 45s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-jrtgb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-autoscaling-jrtgb.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-jrtgb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.31.222.109:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-autoscaling-jrtgb.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.200.173.111:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m22.025s eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:583: Failed to wait for 1 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 1 nodes, got 0
TestCreateCluster
1h59m58.94s
create_cluster_test.go:47: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-nkmvp/create-cluster-4lrs2 in 1m2s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster create-cluster-4lrs2 eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-nkmvp/hostedclusters/create-cluster-4lrs2?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-nkmvp/create-cluster-4lrs2 to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateCluster/Teardown
1h10m38.07s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io create-cluster-4lrs2) journals.go:294: Error copying machine journals to artifacts directory: exit status 1 hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:294: Error copying machine journals to artifacts directory: signal: killed hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:589: Failed to destroy cluster: context canceled util.go:213: Failed to delete namespace: e2e-clusters-nkmvp, will retry: client rate limiter Wait returned an error: context canceled hypershift_framework.go:605: Failed to delete test namespace: failed to delete namespace: context canceled fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: client rate limiter Wait returned an error: context canceled fixture.go:417: Failed to dump machine journals; this is nonfatal: client rate limiter Wait returned an error: context canceled hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateCluster/ValidateHostedCluster
48m17.12s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-nkmvp/create-cluster-4lrs2 in 1m39s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-4lrs2.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-create-cluster-4lrs2.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-4lrs2.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 98.91.83.218:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 1m38.1s util.go:583: Failed to wait for 3 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 3 nodes, got 1
TestCreateClusterCustomConfig
1h35m49.68s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-7jt82/custom-config-l5xkt in 21s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster custom-config-l5xkt util.go:3158: Failed to wait for HostedCluster e2e-clusters-7jt82/custom-config-l5xkt to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-7jt82/custom-config-l5xkt invalid at RV 81300 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators dns, kube-storage-version-migrator, monitoring, service-ca are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-201636-test-ci-op-0wfvijy8-latest: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: redhat-marketplace-catalog, certified-operators-catalog, community-operators-catalog, redhat-operators-catalog) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterCustomConfig/Teardown
37m14.36s
journals.go:305: Successfully copied machine journals to /logs/artifacts/TestCreateClusterCustomConfig/machine-journals hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api": dial tcp 34.237.174.37:6443: i/o timeout - error from a previous attempt: http2: client connection lost journals.go:305: Successfully copied machine journals to /logs/artifacts/TestCreateClusterCustomConfig/machine-journals hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to add client finalizer to hosted cluster: Patch "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt": http2: client connection lost fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api": net/http: TLS handshake timeout fixture.go:417: Failed to dump machine journals; this is nonfatal: the server was unable to return a response in the time allotted, but may still be processing the request (get secrets custom-config-l5xkt-ssh-key) hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-7jt82, name: custom-config-l5xkt hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterCustomConfig/ValidateHostedCluster
58m12.05s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-7jt82/custom-config-l5xkt in 51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-l5xkt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-custom-config-l5xkt.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-l5xkt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.30.154.8:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-l5xkt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 98.95.2.17:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-l5xkt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.20.180.43:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m42.025s util.go:583: Successfully waited for 2 nodes to become ready in 24m39s eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 2623; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io custom-config-l5xkt) eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: connect: connection refused - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": dial tcp 34.237.174.37:6443: connect: connection refused eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io custom-config-l5xkt) eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-7jt82/hostedclusters/custom-config-l5xkt?timeout=5m0s": stream error: stream ID 3235; INTERNAL_ERROR; received from peer util.go:620: Failed to wait for HostedCluster e2e-clusters-7jt82/custom-config-l5xkt to rollout in 30m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-7jt82/custom-config-l5xkt invalid at RV 81300 after 30m0s: eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: redhat-marketplace-catalog, certified-operators-catalog, community-operators-catalog, redhat-operators-catalog) eventually.go:228: - wanted most recent version history to have state Completed, has state Partial util.go:620: *v1beta1.HostedCluster e2e-clusters-7jt82/custom-config-l5xkt conditions: util.go:620: ValidAWSIdentityProvider=True: AsExpected(All is well) util.go:620: ClusterVersionAvailable=False: FromClusterVersion util.go:620: ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators dns, kube-storage-version-migrator, monitoring, service-ca are not available) util.go:620: ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-201636-test-ci-op-0wfvijy8-latest: some cluster operators are not available) util.go:620: ClusterVersionReleaseAccepted=True: PayloadLoaded(Payload loaded version="5.0.0-0.ci-2026-07-14-201636-test-ci-op-0wfvijy8-latest" image="registry.build01.ci.openshift.org/ci-op-0wfvijy8/release@sha256:3d1bc26e2d6e79c35de018299e4e8a4dce11b5d4b7d22bafc4c25ad230bf4c1f" architecture="amd64") util.go:620: ClusterVersionUpgradeable=False: ClusterVersionUpdating(An update is already in progress and the details are in the Progressing condition.) util.go:620: Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) util.go:620: EtcdAvailable=True: QuorumAvailable util.go:620: KubeAPIServerAvailable=True: AsExpected(Kube APIServer deployment is available) util.go:620: InfrastructureReady=True: AsExpected(All is well) util.go:620: ExternalDNSReachable=True: AsExpected(All is well) util.go:620: ValidHostedControlPlaneConfiguration=True: AsExpected(Configuration passes validation) util.go:620: ValidReleaseInfo=True: AsExpected(All is well) util.go:620: ValidIDPConfiguration=True: IDPConfigurationValid(Identity provider configuration is valid) util.go:620: HostedClusterRestoredFromBackup=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) util.go:620: ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) util.go:620: EtcdBackupSucceeded=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: ConfigOperatorReconciliationSucceeded=True: AsExpected(All is well) util.go:620: AutoNodeEnabled=False: AutoNodeNotConfigured(AutoNode provisioner is not configured) util.go:620: Available=False: ComponentsNotAvailable(Waiting for components to be available: redhat-marketplace-catalog, certified-operators-catalog, community-operators-catalog, redhat-operators-catalog) util.go:620: AWSEndpointAvailable=True: AWSSuccess(All is well) util.go:620: AWSEndpointServiceAvailable=True: AWSSuccess(All is well) util.go:620: ValidConfiguration=True: AsExpected(Configuration passes validation) util.go:620: SupportedHostedCluster=True: AsExpected(HostedCluster is supported by operator configuration) util.go:620: ValidProxyConfiguration=True: AsExpected(No proxy CA bundle configured) util.go:620: IgnitionEndpointAvailable=True: AsExpected(Ignition server deployment is available) util.go:620: ReconciliationActive=True: AsExpected(Reconciliation active on resource) util.go:620: ValidReleaseImage=True: AsExpected(Release image is valid) util.go:620: Progressing=False: AsExpected(HostedCluster is at expected version) util.go:620: PlatformCredentialsFound=True: AsExpected(Required platform credentials are found) util.go:620: ValidOIDCConfiguration=True: AsExpected(OIDC configuration is valid) util.go:620: ReconciliationSucceeded=True: ReconciliatonSucceeded(Reconciliation completed successfully) util.go:620: ValidAWSKMSConfig=True: AsExpected(All is well) util.go:620: EtcdDataEncryptionUpToDate=True: ReEncryptionCompleted(Encryption key initialized) util.go:620: ClusterVersionRetrievedUpdates=False: NoChannel(The update channel has not been configured.) util.go:620: AWSDefaultSecurityGroupCreated=True: AsExpected(All is well)
TestCreateClusterHABreakGlassCredentials
1h30m25.25s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq in 21s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster ha-break-glass-creds-jh5gq util.go:3158: Failed to wait for HostedCluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq invalid at RV 80940 after 2s: eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: community-operators-catalog, redhat-operators-catalog, certified-operators-catalog, hosted-cluster-config-operator, redhat-marketplace-catalog) eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterHABreakGlassCredentials/Teardown
1h6m59.78s
journals.go:77: No machines associated with infra id ha-break-glass-creds-jh5gq were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:77: No machines associated with infra id ha-break-glass-creds-jh5gq were found. Skipping journal dump. journals.go:77: No machines associated with infra id ha-break-glass-creds-jh5gq were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-n6q2k, name: ha-break-glass-creds-jh5gq hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster
23m2.11s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq in 57s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-ha-break-glass-creds-jh5gq.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-ha-break-glass-creds-jh5gq.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-ha-break-glass-creds-jh5gq.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.214.239.240:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m5.075s util.go:583: Successfully waited for 0 nodes to become ready in 25ms eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-n6q2k/hostedclusters/ha-break-glass-creds-jh5gq?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-n6q2k/hostedclusters/ha-break-glass-creds-jh5gq?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-n6q2k/hostedclusters/ha-break-glass-creds-jh5gq?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io ha-break-glass-creds-jh5gq) util.go:3158: Failed to wait for HostedCluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq to have valid conditions in 20m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-n6q2k/ha-break-glass-creds-jh5gq invalid at RV 80940 after 20m0s: eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: community-operators-catalog, redhat-operators-catalog, certified-operators-catalog, hosted-cluster-config-operator, redhat-marketplace-catalog) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed
TestCreateClusterPrivate
1h30m22.06s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-rwckl/private-qpsng in 22s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-qpsng eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/hostedclusters/private-qpsng?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-rwckl/private-qpsng to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterPrivate/Teardown
58m58.29s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost journals.go:77: No machines associated with infra id private-qpsng were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io private-qpsng) journals.go:77: No machines associated with infra id private-qpsng were found. Skipping journal dump. hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request journals.go:77: No machines associated with infra id private-qpsng were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-rwckl, name: private-qpsng hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterPrivate/ValidateHostedCluster
31m0.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-rwckl/private-qpsng in 1m0s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/nodepools?timeout=5m0s": net/http: TLS handshake timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/nodepools?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: the server was unable to return a response in the time allotted, but may still be processing the request (get nodepools.hypershift.openshift.io) eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-rwckl/nodepools?timeout=5m0s": context deadline exceeded util.go:744: Failed to wait for NodePools for HostedCluster e2e-clusters-rwckl/private-qpsng to have all of their desired nodes in 30m0s: context deadline exceeded eventually.go:384: observed invalid **v1beta1.NodePool state after 30m0s eventually.go:401: - observed **v1beta1.NodePool e2e-clusters-rwckl/private-qpsng-us-east-1c invalid: expected 2 replicas, got 0 util.go:744: *v1beta1.NodePool e2e-clusters-rwckl/private-qpsng-us-east-1c conditions: util.go:744: AutoscalingEnabled=False: AsExpected util.go:744: UpdateManagementEnabled=True: AsExpected util.go:744: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-0wfvijy8/release@sha256:3d1bc26e2d6e79c35de018299e4e8a4dce11b5d4b7d22bafc4c25ad230bf4c1f) util.go:744: ValidArchPlatform=True: AsExpected util.go:744: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:744: SupportedVersionSkew=True: AsExpected(Release image version is valid) util.go:744: ValidMachineConfig=True: AsExpected util.go:744: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: ec07cadc) util.go:744: UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 5.0.0-0.ci-2026-07-14-201636-test-ci-op-0wfvijy8-latest) util.go:744: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:744: ReachedIgnitionEndpoint=False: ignitionNotReached util.go:744: AllMachinesReady=False: NotFound(No Machines are created) util.go:744: AllNodesHealthy=False: NotFound(No Machines are created) util.go:744: ValidPlatformConfig=True: AsExpected(All is well) util.go:744: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0fbc8be8796dc1df5") util.go:744: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) util.go:744: ValidTuningConfig=True: AsExpected util.go:744: UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: private-qpsng-us-east-1c-3bbe0143) util.go:744: AutorepairEnabled=False: AsExpected
TestCreateClusterPrivateWithRouteKAS
1h36m30.06s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-l6mwc/private-ljghp in 26s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-ljghp eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/hostedclusters/private-ljghp?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-l6mwc/private-ljghp to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterPrivateWithRouteKAS/Teardown
1h5m5.16s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost journals.go:305: Successfully copied machine journals to /logs/artifacts/TestCreateClusterPrivateWithRouteKAS/machine-journals hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io private-ljghp) journals.go:305: Successfully copied machine journals to /logs/artifacts/TestCreateClusterPrivateWithRouteKAS/machine-journals hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request journals.go:305: Successfully copied machine journals to /logs/artifacts/TestCreateClusterPrivateWithRouteKAS/machine-journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-l6mwc, name: private-ljghp hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterPrivateWithRouteKAS/ValidateHostedCluster
30m57.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-l6mwc/private-ljghp in 57s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": net/http: TLS handshake timeout eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:259: Failed to get **v1beta1.NodePool: the server was unable to return a response in the time allotted, but may still be processing the request (get nodepools.hypershift.openshift.io) eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": stream error: stream ID 2161; INTERNAL_ERROR; received from peer eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:259: Failed to get **v1beta1.NodePool: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-l6mwc/nodepools?timeout=5m0s": context deadline exceeded util.go:744: Failed to wait for NodePools for HostedCluster e2e-clusters-l6mwc/private-ljghp to have all of their desired nodes in 30m0s: context deadline exceeded eventually.go:384: observed invalid **v1beta1.NodePool state after 30m0s eventually.go:401: - observed **v1beta1.NodePool e2e-clusters-l6mwc/private-ljghp-us-east-1b invalid: expected 2 replicas, got 0 util.go:744: *v1beta1.NodePool e2e-clusters-l6mwc/private-ljghp-us-east-1b conditions: util.go:744: AutoscalingEnabled=False: AsExpected util.go:744: UpdateManagementEnabled=True: AsExpected util.go:744: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-0wfvijy8/release@sha256:3d1bc26e2d6e79c35de018299e4e8a4dce11b5d4b7d22bafc4c25ad230bf4c1f) util.go:744: ValidArchPlatform=True: AsExpected util.go:744: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:744: SupportedVersionSkew=True: AsExpected(Release image version is valid) util.go:744: ValidMachineConfig=True: AsExpected util.go:744: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 76b82f83) util.go:744: UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 5.0.0-0.ci-2026-07-14-201636-test-ci-op-0wfvijy8-latest) util.go:744: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:744: ReachedIgnitionEndpoint=True: AsExpected util.go:744: AllMachinesReady=True: AsExpected(All is well) util.go:744: AllNodesHealthy=False: NodeConditionsFailed(2 of 2 machines are not healthy Machine private-ljghp-us-east-1b-9q648-fwxsb: NodeConditionsFailed: Node condition Ready is False Machine private-ljghp-us-east-1b-9q648-s9pmq: NodeConditionsFailed: Node condition Ready is False ) util.go:744: ValidPlatformConfig=True: AsExpected(All is well) util.go:744: ValidPlatformImage=True: AsExpected(Bootstrap AMI is "ami-0fbc8be8796dc1df5") util.go:744: AWSSecurityGroupAvailable=True: AsExpected(NodePool has a default security group) util.go:744: ValidTuningConfig=True: AsExpected util.go:744: UpdatingPlatformMachineTemplate=True: AsExpected(platform machine template update in progress. Target template: private-ljghp-us-east-1b-a396a886) util.go:744: AutorepairEnabled=False: AsExpected util.go:744: Ready=False: WaitingForAvailableMachines(Minimum availability requires 2 replicas, current 0 available)
TestCreateClusterProxy
1h32m5.41s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-fg2gt/proxy-w5rwn in 32s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster proxy-w5rwn eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-fg2gt/hostedclusters/proxy-w5rwn?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-fg2gt/proxy-w5rwn to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterProxy/Teardown
43m23.53s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io proxy-w5rwn) journals.go:294: Error copying machine journals to artifacts directory: exit status 1 hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:294: Error copying machine journals to artifacts directory: exit status 1 hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-fg2gt, name: proxy-w5rwn hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterProxy/ValidateHostedCluster
48m8.06s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-fg2gt/proxy-w5rwn in 1m3.025s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-w5rwn.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-proxy-w5rwn.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-proxy-w5rwn.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 54.85.176.72:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m5.025s util.go:583: Failed to wait for 2 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 2 nodes, got 0
TestCreateClusterRequestServingIsolation
1h33m25.26s
requestserving.go:105: Created request serving nodepool clusters/5028e29cdd-mgmt-reqserving-2gwwd requestserving.go:105: Created request serving nodepool clusters/5028e29cdd-mgmt-reqserving-6v4r4 requestserving.go:113: Created non request serving nodepool clusters/5028e29cdd-mgmt-non-reqserving-4qjvj requestserving.go:113: Created non request serving nodepool clusters/5028e29cdd-mgmt-non-reqserving-whfb8 requestserving.go:113: Created non request serving nodepool clusters/5028e29cdd-mgmt-non-reqserving-k6kpp util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5028e29cdd-mgmt-reqserving-2gwwd in 3m48s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5028e29cdd-mgmt-reqserving-6v4r4 in 18s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5028e29cdd-mgmt-non-reqserving-4qjvj in 9s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5028e29cdd-mgmt-non-reqserving-whfb8 in 33s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5028e29cdd-mgmt-non-reqserving-k6kpp in 21s create_cluster_test.go:299: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-6hzkz/request-serving-isolation-2bl4d in 12s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster request-serving-isolation-2bl4d util.go:3158: Failed to wait for HostedCluster e2e-clusters-6hzkz/request-serving-isolation-2bl4d to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-6hzkz/request-serving-isolation-2bl4d invalid at RV 80967 after 2s: eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: cluster-network-operator, csi-snapshot-controller-operator, dns-operator, cloud-credential-operator, hosted-cluster-config-operator, kube-controller-manager, kube-storage-version-migrator, redhat-operators-catalog, cluster-policy-controller, community-operators-catalog, cluster-image-registry-operator, cluster-node-tuning-operator, cluster-version-operator, control-plane-pki-operator, cluster-storage-operator, oauth-openshift, aws-cloud-controller-manager, catalog-operator, konnectivity-agent, olm-operator, ingress-operator, machine-approver, openshift-oauth-apiserver, openshift-route-controller-manager, packageserver, redhat-marketplace-catalog, certified-operators-catalog, ignition-server, ignition-server-proxy, openshift-controller-manager) eventually.go:228: - incorrect condition: wanted IgnitionEndpointAvailable=True, got IgnitionEndpointAvailable=False: WaitingForAvailable(Ignition server deployment is not yet available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=Unknown: StatusUnknown(Condition not found in the CVO.) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: StatusUnknown(Condition not found in the HCP) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: StatusUnknown(Condition not found in the HCP) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=Unknown: StatusUnknown(Condition not found in the CVO.) eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([aws-cloud-controller-manager deployment has 1 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, konnectivity-agent deployment has 2 unavailable replicas, machine-approver deployment has 1 unavailable replicas]) eventually.go:228: - incorrect condition: wanted ClusterVersionReleaseAccepted=True, got ClusterVersionReleaseAccepted=Unknown: StatusUnknown(Condition not found in the CVO.) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=Unknown: StatusUnknown(Condition not found in the CVO.) eventually.go:228: - incorrect condition: wanted ConfigOperatorReconciliationSucceeded=True, got ConfigOperatorReconciliationSucceeded=Unknown: StatusUnknown(Condition not found in the HCP) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed hypershift_framework.go:349: skipping postTeardown() requestserving.go:132: Tearing down custom nodepool clusters/5028e29cdd-mgmt-reqserving-2gwwd requestserving.go:132: Tearing down custom nodepool clusters/5028e29cdd-mgmt-reqserving-6v4r4 requestserving.go:132: Tearing down custom nodepool clusters/5028e29cdd-mgmt-non-reqserving-4qjvj requestserving.go:132: Tearing down custom nodepool clusters/5028e29cdd-mgmt-non-reqserving-whfb8 requestserving.go:132: Tearing down custom nodepool clusters/5028e29cdd-mgmt-non-reqserving-k6kpp hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterRequestServingIsolation/Teardown
39m59.71s
journals.go:77: No machines associated with infra id request-serving-isolation-2bl4d were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: the server was unable to return a response in the time allotted, but may still be processing the request journals.go:77: No machines associated with infra id request-serving-isolation-2bl4d were found. Skipping journal dump. hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request journals.go:77: No machines associated with infra id request-serving-isolation-2bl4d were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to clean up secrets in e2e-clusters-6hzkz namespace: Delete "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/e2e-clusters-6hzkz/secrets?labelSelector=hypershift.openshift.io%2Fauto-created-for-infra%3Drequest-serving-isolation-2bl4d": http2: client connection lost journals.go:77: No machines associated with infra id request-serving-isolation-2bl4d were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-6hzkz, name: request-serving-isolation-2bl4d hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestCreateClusterRequestServingIsolation/ValidateHostedCluster
48m2.04s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-6hzkz/request-serving-isolation-2bl4d in 1m3s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-2bl4d.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-request-serving-isolation-2bl4d.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-request-serving-isolation-2bl4d.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.227.145.166:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 1m59.025s eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:583: Failed to wait for 3 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 3 nodes, got 0
TestKarpenter
1h33m15.08s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-dk4lh/karpenter-n4qtz in 1m10s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-n4qtz eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-dk4lh/hostedclusters/karpenter-n4qtz?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-dk4lh/karpenter-n4qtz to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenter/Teardown
43m32s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io karpenter-n4qtz) fixture.go:417: Failed to dump machine journals; this is nonfatal: failed to ensure bastion keypair: failed to import keypair: operation error EC2: ImportKeyPair, https response error StatusCode: 400, RequestID: 124bdf38-0a5f-4e63-91fe-3b13f770dc20, api error KeyPairLimitExceeded: Maximum of 5000 keypairs reached. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:417: Failed to dump machine journals; this is nonfatal: failed to ensure bastion keypair: failed to import keypair: operation error EC2: ImportKeyPair, https response error StatusCode: 400, RequestID: 94776ba3-b885-4fa0-8007-f86dfa43d4fe, api error KeyPairLimitExceeded: Maximum of 5000 keypairs reached. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-dk4lh, name: karpenter-n4qtz hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestKarpenter/ValidateHostedCluster
48m31.04s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-dk4lh/karpenter-n4qtz in 1m24s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-n4qtz.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-n4qtz.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-n4qtz.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 34.195.128.63:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-n4qtz.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 52.7.38.227:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m7.025s eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:583: Failed to wait for 2 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 2 nodes, got 0
TestKarpenterUpgradeControlPlane
1h33m7.07s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-kfp2m/karpenter-upgrade-control-plane-zplh6 in 43s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-upgrade-control-plane-zplh6 eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-kfp2m/hostedclusters/karpenter-upgrade-control-plane-zplh6?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-kfp2m/karpenter-upgrade-control-plane-zplh6 to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenterUpgradeControlPlane/Teardown
43m22.08s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io karpenter-upgrade-control-plane-zplh6) fixture.go:417: Failed to dump machine journals; this is nonfatal: failed to ensure bastion keypair: failed to import keypair: operation error EC2: ImportKeyPair, https response error StatusCode: 400, RequestID: 6e99e8a7-95e6-4a3d-b3ce-079fb91262d6, api error KeyPairLimitExceeded: Maximum of 5000 keypairs reached. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:417: Failed to dump machine journals; this is nonfatal: failed to ensure bastion keypair: failed to import keypair: operation error EC2: ImportKeyPair, https response error StatusCode: 400, RequestID: 10f1ffe7-981e-4e0d-85fb-e355ebe1a29e, api error KeyPairLimitExceeded: Maximum of 5000 keypairs reached. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-kfp2m, name: karpenter-upgrade-control-plane-zplh6 hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestKarpenterUpgradeControlPlane/ValidateHostedCluster
49m0.04s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-kfp2m/karpenter-upgrade-control-plane-zplh6 in 2m0s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-zplh6.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-upgrade-control-plane-zplh6.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-zplh6.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.82.185.209:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-zplh6.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.218.106.162:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-zplh6.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.82.185.209:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m0.025s eventually.go:259: Failed to get **v1.Node: context deadline exceeded util.go:583: Failed to wait for 2 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 2 nodes, got 0
TestNodePool
0s
TestNodePool/HostedCluster0
1h30m27.54s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-qcc89/node-pool-cv66h in 2m31s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-cv66h util.go:3158: Successfully waited for HostedCluster e2e-clusters-qcc89/node-pool-cv66h to have valid conditions in 150ms hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePool/HostedCluster0/Teardown
1h7m2.85s
journals.go:77: No machines associated with infra id node-pool-cv66h were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:77: No machines associated with infra id node-pool-cv66h were found. Skipping journal dump. journals.go:77: No machines associated with infra id node-pool-cv66h were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-qcc89, name: node-pool-cv66h hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestNodePool/HostedCluster0/ValidateHostedCluster
20m53.25s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-qcc89/node-pool-cv66h in 42s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-cv66h.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-cv66h.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-cv66h.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 34.230.100.13:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-cv66h.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 100.56.163.58:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m24.075s util.go:583: Successfully waited for 0 nodes to become ready in 25ms eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qcc89/hostedclusters/node-pool-cv66h?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qcc89/hostedclusters/node-pool-cv66h?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qcc89/hostedclusters/node-pool-cv66h?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io node-pool-cv66h) util.go:3158: Successfully waited for HostedCluster e2e-clusters-qcc89/node-pool-cv66h to have valid conditions in 12m57.075s
TestNodePool/HostedCluster0/ValidateHostedCluster/EnsureGuestWebhooksValidated
1m0.02s
util.go:1920: failed to ensure guest webhooks validated, violating webhook test-webhook was not deleted: context deadline exceeded
TestNodePool/HostedCluster0/ValidateHostedCluster/EnsureNoCrashingPods
150ms
util.go:829: Container aws-ebs-csi-driver-operator in pod aws-ebs-csi-driver-operator-7789fc5bcd-shhpj has a restartCount > 0 (3) util.go:829: Container manager in pod capi-provider-75649675f7-4f9dn has a restartCount > 0 (5) util.go:826: Leader election failure detected in container manager in pod cluster-api-666b8584b4-w7bvk util.go:826: Leader election failure detected in container cluster-storage-operator in pod cluster-storage-operator-74d676bdf8-nt9h8 util.go:829: Container control-plane-operator in pod control-plane-operator-77cbb994d9-5nxzw has a restartCount > 0 (6) util.go:826: Leader election failure detected in container csi-snapshot-controller-operator in pod csi-snapshot-controller-operator-7c5c787d68-hrwl2 util.go:829: Container hosted-cluster-config-operator in pod hosted-cluster-config-operator-74b4f466f5-zp7kl has a restartCount > 0 (6)
TestNodePool/HostedCluster2
1h31m37.73s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-qpwtx/node-pool-l8282 in 43s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-l8282 util.go:3158: Failed to wait for HostedCluster e2e-clusters-qpwtx/node-pool-l8282 to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-qpwtx/node-pool-l8282 invalid at RV 81319 after 2s: eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: redhat-operators-catalog, community-operators-catalog, redhat-marketplace-catalog, certified-operators-catalog) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePool/HostedCluster2/ValidateHostedCluster
23m48.06s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-qpwtx/node-pool-l8282 in 54s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-l8282.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-l8282.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-l8282.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.220.85.14:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m54.025s util.go:583: Successfully waited for 0 nodes to become ready in 25ms eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qpwtx/hostedclusters/node-pool-l8282?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qpwtx/hostedclusters/node-pool-l8282?timeout=5m0s": net/http: TLS handshake timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qpwtx/hostedclusters/node-pool-l8282?timeout=5m0s": dial tcp 34.237.174.37:6443: i/o timeout eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io node-pool-l8282) eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-qpwtx/hostedclusters/node-pool-l8282?timeout=5m0s": stream error: stream ID 2159; INTERNAL_ERROR; received from peer eventually.go:105: Failed to get *v1beta1.HostedCluster: client rate limiter Wait returned an error: context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-qpwtx/node-pool-l8282 to have valid conditions in 20m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-qpwtx/node-pool-l8282 invalid at RV 81319 after 20m0s: eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([certified-operators-catalog deployment has 2 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas]) eventually.go:228: - incorrect condition: wanted Available=True, got Available=False: ComponentsNotAvailable(Waiting for components to be available: redhat-operators-catalog, community-operators-catalog, redhat-marketplace-catalog, certified-operators-catalog) eventually.go:228: - controlPlaneVersion state is Partial, expected Completed
TestNodePoolAutoscalingScaleFromZero
1h38m17.87s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-5xd8h/scale-from-zero-jtpzf in 59s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster scale-from-zero-jtpzf util.go:3158: Failed to wait for HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf invalid at RV 81247 after 2s: controlPlaneVersion state is Partial, expected Completed hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePoolAutoscalingScaleFromZero/Teardown
1h8m33.59s
journals.go:305: Successfully copied machine journals to /logs/artifacts/TestNodePoolAutoscalingScaleFromZero/machine-journals hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: failed to get server groups: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/api": dial tcp 34.237.174.37:6443: i/o timeout - error from a previous attempt: http2: client connection lost fixture.go:417: Failed to dump machine journals; this is nonfatal: the server was unable to return a response in the time allotted, but may still be processing the request (get secrets scale-from-zero-jtpzf-ssh-key) hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:576: Failed to destroy cluster, will retry: failed to get hostedcluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-5xd8h/hostedclusters/scale-from-zero-jtpzf": stream error: stream ID 99; INTERNAL_ERROR; received from peer journals.go:305: Successfully copied machine journals to /logs/artifacts/TestNodePoolAutoscalingScaleFromZero/machine-journals hypershift_framework.go:581: Failed to dump cluster during destroy; this is nonfatal: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:305: Successfully copied machine journals to /logs/artifacts/TestNodePoolAutoscalingScaleFromZero/machine-journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-5xd8h, name: scale-from-zero-jtpzf hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestNodePoolAutoscalingScaleFromZero/ValidateHostedCluster
28m43.03s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf in 1m36s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-jtpzf.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-scale-from-zero-jtpzf.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-jtpzf.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 44.213.184.213:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 1m32.15s util.go:583: Successfully waited for 1 nodes to become ready in 11m48s eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io scale-from-zero-jtpzf) util.go:620: Successfully waited for HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf to rollout in 3m46.85s eventually.go:105: Failed to get *v1beta1.HostedCluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io scale-from-zero-jtpzf) eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-5xd8h/hostedclusters/scale-from-zero-jtpzf?timeout=5m0s": net/http: TLS handshake timeout - error from a previous attempt: http2: client connection lost eventually.go:105: Failed to get *v1beta1.HostedCluster: client rate limiter Wait returned an error: context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf to have valid conditions in 10m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-5xd8h/scale-from-zero-jtpzf invalid at RV 81247 after 10m0s: controlPlaneVersion state is Partial, expected Completed
TestUpgradeControlPlane
1h34m24.74s
control_plane_upgrade_test.go:25: Starting control plane upgrade test. FromImage: registry.build01.ci.openshift.org/ci-op-0wfvijy8/release@sha256:622e3aad9f1316badf4e7c2069a15bb6e480d3a5783727abee78031623fec157, toImage: registry.build01.ci.openshift.org/ci-op-0wfvijy8/release@sha256:3d1bc26e2d6e79c35de018299e4e8a4dce11b5d4b7d22bafc4c25ad230bf4c1f hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-dztjq/control-plane-upgrade-hjklt in 10s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-hjklt eventually.go:105: Failed to get *v1beta1.HostedCluster: Get "https://ac52d90767a5a432db90b1467b6de825-c55ed9af18ced436.elb.us-east-1.amazonaws.com:6443/apis/hypershift.openshift.io/v1beta1/namespaces/e2e-clusters-dztjq/hostedclusters/control-plane-upgrade-hjklt?timeout=5m0s": context deadline exceeded util.go:3158: Failed to wait for HostedCluster e2e-clusters-dztjq/control-plane-upgrade-hjklt to have valid conditions in 2s: context deadline exceeded hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestUpgradeControlPlane/Teardown
45m14.71s
fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: the server was unable to return a response in the time allotted, but may still be processing the request (get hostedclusters.hypershift.openshift.io control-plane-upgrade-hjklt) journals.go:77: No machines associated with infra id control-plane-upgrade-hjklt were found. Skipping journal dump. hypershift_framework.go:576: Failed to destroy cluster, will retry: hostedcluster wasn't finalized, aborting delete: context deadline exceeded journals.go:77: No machines associated with infra id control-plane-upgrade-hjklt were found. Skipping journal dump. hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-dztjq, name: control-plane-upgrade-hjklt hypershift_framework.go:605: Failed to delete test namespace: namespace still exists after deletion timeout: context deadline exceeded fixture.go:409: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: hostedclusters.hypershift.openshift.io "control-plane-upgrade-hjklt" not found fixture.go:417: Failed to dump machine journals; this is nonfatal: secrets "control-plane-upgrade-hjklt-ssh-key" not found hypershift_framework.go:613: Failed to dump cluster: failed to dump hosted cluster: failed to dump cluster: the server is currently unable to handle the request
TestUpgradeControlPlane/ValidateHostedCluster
48m58.13s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-dztjq/control-plane-upgrade-hjklt in 1m0s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-hjklt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-control-plane-upgrade-hjklt.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-hjklt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 50.17.181.2:443: i/o timeout eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-hjklt.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp 3.227.44.171:443: i/o timeout util.go:384: Successfully waited for a successful connection to the guest API server in 2m58.125s util.go:583: Failed to wait for 2 nodes to become ready in 45m0s: context deadline exceeded eventually.go:384: observed invalid **v1.Node state after 45m0s eventually.go:401: - observed **v1.Node collection invalid: expected 2 nodes, got 0