PR #8305 - 07-17 12:11

Job: hypershift
FAILURE
← Back to Test Grid

Test Summary

337
Total Tests
300
Passed
14
Failed
23
Skipped

Failed Tests

TestCreateCluster
29m39.38s
create_cluster_test.go:47: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-qxnhh/create-cluster-jp6hc in 5m48s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster create-cluster-jp6hc util.go:3158: Failed to wait for HostedCluster e2e-clusters-qxnhh/create-cluster-jp6hc to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-qxnhh/create-cluster-jp6hc invalid at RV 81129 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-17-122535-test-ci-op-bdrm5qhq-latest: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-create-cluster-jp6hc.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, openshift-samples, service-ca are not available) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=False: KASAccessFailed(Data plane to control plane connection is not available: no successful connectivity check recorded) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateCluster/ValidateHostedCluster
10m54.02s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-qxnhh/create-cluster-jp6hc in 54s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-create-cluster-jp6hc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-create-cluster-jp6hc.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded
TestCreateClusterPrivateWithRouteKAS
1h23m25.17s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-tcv5j/private-fhdjn in 7m57s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-fhdjn util.go:3158: Failed to wait for HostedCluster e2e-clusters-tcv5j/private-fhdjn to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-tcv5j/private-fhdjn invalid at RV 136099 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator console is not available) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-17-122535-test-ci-op-bdrm5qhq-latest: the cluster operator console is not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestCreateClusterPrivateWithRouteKAS/ValidateHostedCluster
41m57.02s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-tcv5j/private-fhdjn in 42s util.go:322: Successfully waited for kubeconfig secret to have data in 0s util.go:744: Successfully waited for NodePools for HostedCluster e2e-clusters-tcv5j/private-fhdjn to have all of their desired nodes in 11m15s eventually.go:105: Failed to get *v1beta1.HostedCluster: client rate limiter Wait returned an error: context deadline exceeded util.go:620: Failed to wait for HostedCluster e2e-clusters-tcv5j/private-fhdjn to rollout in 30m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-tcv5j/private-fhdjn invalid at RV 136099 after 30m0s: wanted most recent version history to have state Completed, has state Partial util.go:620: *v1beta1.HostedCluster e2e-clusters-tcv5j/private-fhdjn conditions: util.go:620: ValidAWSIdentityProvider=True: AsExpected(All is well) util.go:620: ClusterVersionUpgradeable=False: ClusterVersionUpdating(An update is already in progress and the details are in the Progressing condition.) util.go:620: ClusterVersionAvailable=False: FromClusterVersion util.go:620: ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator console is not available) util.go:620: ClusterVersionProgressing=True: ClusterOperatorNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-17-122535-test-ci-op-bdrm5qhq-latest: the cluster operator console is not available) util.go:620: ClusterVersionReleaseAccepted=True: PayloadLoaded(Payload loaded version="5.0.0-0.ci-2026-07-17-122535-test-ci-op-bdrm5qhq-latest" image="registry.build01.ci.openshift.org/ci-op-bdrm5qhq/release@sha256:c351b6e3188ba4c1ebbc8acdd153c2b943d31f18b4624db1e2d527306d1801f5" architecture="amd64") util.go:620: Degraded=False: AsExpected(The hosted cluster is not degraded) util.go:620: EtcdAvailable=True: QuorumAvailable util.go:620: KubeAPIServerAvailable=True: AsExpected(Kube APIServer deployment is available) util.go:620: InfrastructureReady=True: AsExpected(All is well) util.go:620: ExternalDNSReachable=Unknown: StatusUnknown(External DNS is not configured) util.go:620: ValidHostedControlPlaneConfiguration=True: AsExpected(Configuration passes validation) util.go:620: ValidReleaseInfo=True: AsExpected(All is well) util.go:620: ValidIDPConfiguration=True: IDPConfigurationValid(Identity provider configuration is valid) util.go:620: HostedClusterRestoredFromBackup=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: DataPlaneConnectionAvailable=True: AsExpected(All is well) util.go:620: ControlPlaneConnectionAvailable=True: AsExpected(All is well) util.go:620: EtcdBackupSucceeded=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: ConfigOperatorReconciliationSucceeded=True: AsExpected(All is well) util.go:620: AutoNodeEnabled=False: AutoNodeNotConfigured(AutoNode provisioner is not configured) util.go:620: Available=True: AsExpected(The hosted control plane is available) util.go:620: AWSEndpointAvailable=True: AWSSuccess(All is well) util.go:620: AWSEndpointServiceAvailable=True: AWSSuccess(All is well) util.go:620: ValidConfiguration=True: AsExpected(Configuration passes validation) util.go:620: SupportedHostedCluster=True: AsExpected(HostedCluster is supported by operator configuration) util.go:620: ValidProxyConfiguration=True: AsExpected(No proxy CA bundle configured) util.go:620: IgnitionEndpointAvailable=True: AsExpected(Ignition server deployment is available) util.go:620: ReconciliationActive=True: AsExpected(Reconciliation active on resource) util.go:620: ValidReleaseImage=True: AsExpected(Release image is valid) util.go:620: Progressing=False: AsExpected(HostedCluster is at expected version) util.go:620: PlatformCredentialsFound=True: AsExpected(Required platform credentials are found) util.go:620: ValidOIDCConfiguration=True: AsExpected(OIDC configuration is valid) util.go:620: ReconciliationSucceeded=True: ReconciliatonSucceeded(Reconciliation completed successfully) util.go:620: ValidAWSKMSConfig=Unknown: StatusUnknown(AWS KMS is not configured) util.go:620: EtcdDataEncryptionUpToDate=True: ReEncryptionCompleted(Encryption key initialized) util.go:620: ClusterVersionRetrievedUpdates=False: NoChannel(The update channel has not been configured.) util.go:620: AWSDefaultSecurityGroupCreated=True: AsExpected(All is well)
TestCreateClusterRequestServingIsolation
26m21.91s
requestserving.go:105: Created request serving nodepool clusters/9149a201ac-mgmt-reqserving-xdg4r requestserving.go:105: Created request serving nodepool clusters/9149a201ac-mgmt-reqserving-jrmc9 requestserving.go:113: Created non request serving nodepool clusters/9149a201ac-mgmt-non-reqserving-r9jmn requestserving.go:113: Created non request serving nodepool clusters/9149a201ac-mgmt-non-reqserving-lk42m requestserving.go:113: Created non request serving nodepool clusters/9149a201ac-mgmt-non-reqserving-p8k4n util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/9149a201ac-mgmt-reqserving-xdg4r in 3m30s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/9149a201ac-mgmt-reqserving-jrmc9 in 39s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/9149a201ac-mgmt-non-reqserving-r9jmn in 24s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/9149a201ac-mgmt-non-reqserving-lk42m in 12.025s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/9149a201ac-mgmt-non-reqserving-p8k4n in 100ms create_cluster_test.go:299: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:518: failed to create cluster, tearing down: failed to create infra: operation error Route 53: ChangeResourceRecordSets, exceeded maximum number of attempts, 21, https response error StatusCode: 400, RequestID: 42073d12-3e9e-4641-892c-9f8bd390f355, api error Throttling: Rate exceeded requestserving.go:132: Tearing down custom nodepool clusters/9149a201ac-mgmt-reqserving-xdg4r requestserving.go:132: Tearing down custom nodepool clusters/9149a201ac-mgmt-reqserving-jrmc9 requestserving.go:132: Tearing down custom nodepool clusters/9149a201ac-mgmt-non-reqserving-r9jmn requestserving.go:132: Tearing down custom nodepool clusters/9149a201ac-mgmt-non-reqserving-lk42m requestserving.go:132: Tearing down custom nodepool clusters/9149a201ac-mgmt-non-reqserving-p8k4n fixture.go:417: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: hostedclusters.hypershift.openshift.io "request-serving-isolation-4kdzc" not found fixture.go:425: Failed to dump machine journals; this is nonfatal: no SSH secret specified for cluster, cannot dump journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-xq8kl, name: request-serving-isolation-4kdzc
TestKarpenter
36m46.67s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-dbbqz/karpenter-wtq5w in 18m59s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-wtq5w util.go:3158: Failed to wait for HostedCluster e2e-clusters-dbbqz/karpenter-wtq5w to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-dbbqz/karpenter-wtq5w invalid at RV 130281 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-17-122535-test-ci-op-bdrm5qhq-latest: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-karpenter-wtq5w.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, openshift-samples, service-ca are not available) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenter/ValidateHostedCluster
10m54.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-dbbqz/karpenter-wtq5w in 54s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-wtq5w.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-wtq5w.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded
TestKarpenterUpgradeControlPlane
33m58.31s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-t9dhd/karpenter-upgrade-control-plane-t9lp5 in 3m12s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster karpenter-upgrade-control-plane-t9lp5 util.go:3158: Failed to wait for HostedCluster e2e-clusters-t9dhd/karpenter-upgrade-control-plane-t9lp5 to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-t9dhd/karpenter-upgrade-control-plane-t9lp5 invalid at RV 88975 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, openshift-samples, service-ca are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-070042: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-karpenter-upgrade-control-plane-t9lp5.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestKarpenterUpgradeControlPlane/ValidateHostedCluster
16m57.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-t9dhd/karpenter-upgrade-control-plane-t9lp5 in 6m57s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-karpenter-upgrade-control-plane-t9lp5.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-karpenter-upgrade-control-plane-t9lp5.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded
TestNodePool
0s
TestNodePool/HostedCluster0
32m39.23s
hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-q727c/node-pool-p4n2x in 11m43s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-p4n2x util.go:3158: Failed to wait for HostedCluster e2e-clusters-q727c/node-pool-p4n2x to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-q727c/node-pool-p4n2x invalid at RV 99169 after 2s: eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-node-pool-p4n2x.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted AWSEndpointAvailable=True, got AWSEndpointAvailable=False: AWSError(operation error Route 53: ChangeResourceRecordSets, exceeded maximum number of attempts, 3, https response error StatusCode: 400, RequestID: 4e5de84c-8098-4b99-82a2-88a783180704, api error Throttling: Rate exceeded) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestNodePool/HostedCluster0/ValidateHostedCluster
10m42.02s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-q727c/node-pool-p4n2x in 42s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-p4n2x.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-p4n2x.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded
TestUpgradeControlPlane
32m50.11s
control_plane_upgrade_test.go:25: Starting control plane upgrade test. FromImage: registry.build01.ci.openshift.org/ci-op-bdrm5qhq/release@sha256:a9d15ae23ed23b9f5b242b35049eff2e830d9819d532643a99fa7d3f899b08f1, toImage: registry.build01.ci.openshift.org/ci-op-bdrm5qhq/release@sha256:c351b6e3188ba4c1ebbc8acdd153c2b943d31f18b4624db1e2d527306d1801f5 hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-c6cwn/control-plane-upgrade-kqvpf in 8m34s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-kqvpf util.go:3158: Failed to wait for HostedCluster e2e-clusters-c6cwn/control-plane-upgrade-kqvpf to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-c6cwn/control-plane-upgrade-kqvpf invalid at RV 88881 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-14-070042: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, openshift-samples, service-ca are not available) eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-control-plane-upgrade-kqvpf.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called
TestUpgradeControlPlane/ValidateHostedCluster
11m18.01s
util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-c6cwn/control-plane-upgrade-kqvpf in 1m18s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-control-plane-upgrade-kqvpf.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-control-plane-upgrade-kqvpf.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded