PR #8792 - 07-15 11:08

create_cluster_test.go:47: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:518: failed to create cluster, tearing down: failed to create infra: failed to tag hosted zone: operation error Route 53: ChangeTagsForResource, https response error StatusCode: 400, RequestID: b9fae936-cef5-47f8-8179-0125b682f051, api error Throttling: Rate exceeded failed to get rate limit token, retry quota exceeded, 2 available, 5 requested fixture.go:417: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: hostedclusters.hypershift.openshift.io "create-cluster-299v9" not found fixture.go:425: Failed to dump machine journals; this is nonfatal: no SSH secret specified for cluster, cannot dump journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-5cz2k, name: create-cluster-299v9

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-226b5/custom-config-29cwk in 28m8s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster custom-config-29cwk util.go:3158: Failed to wait for HostedCluster e2e-clusters-226b5/custom-config-29cwk to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-226b5/custom-config-29cwk invalid at RV 105025 after 2s: eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators dns, kube-storage-version-migrator, monitoring, service-ca are not available) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-custom-config-29cwk.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted AWSEndpointAvailable=True, got AWSEndpointAvailable=False: AWSError(operation error Route 53: ChangeResourceRecordSets, exceeded maximum number of attempts, 3, https response error StatusCode: 400, RequestID: f540a508-e986-4239-800c-4c22540d34e2, api error Throttling: Rate exceeded) eventually.go:228: - incorrect condition: wanted ValidAWSKMSConfig=True, got ValidAWSKMSConfig=Unknown: StatusUnknown(failed to create token for KMS provider service account: failed to get serviceaccount: Get "https://api-custom-config-29cwk.service.ci.hypershift.devcluster.openshift.com:443/api/v1/namespaces/kube-system/serviceaccounts/kms-provider": dial tcp: lookup api-custom-config-29cwk.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-15-111914-test-ci-op-lyn9s9ns-latest: some cluster operators are not available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-226b5/custom-config-29cwk in 42s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-custom-config-29cwk.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-custom-config-29cwk.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-9wgwz/ha-break-glass-creds-kx4fs in 33m20s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster ha-break-glass-creds-kx4fs util.go:3158: Failed to wait for HostedCluster e2e-clusters-9wgwz/ha-break-glass-creds-kx4fs to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-9wgwz/ha-break-glass-creds-kx4fs invalid at RV 107156 after 2s: incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-ha-break-glass-creds-kx4fs.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-9wgwz/ha-break-glass-creds-kx4fs in 48s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-ha-break-glass-creds-kx4fs.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-ha-break-glass-creds-kx4fs.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-djk76/private-f85dc in 52s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-nhkcp/private-xbb98 in 1m14s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster private-xbb98 util.go:3158: Failed to wait for HostedCluster e2e-clusters-nhkcp/private-xbb98 to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-nhkcp/private-xbb98 invalid at RV 118055 after 2s: eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted Degraded=False, got Degraded=True: UnavailableReplicas([hosted-cluster-config-operator deployment has 1 unavailable replicas, kube-controller-manager deployment has 1 unavailable replicas]) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-15-111914-test-ci-op-lyn9s9ns-latest: the cluster operator console is not available) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator console is not available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-nhkcp/private-xbb98 in 45s util.go:322: Successfully waited for kubeconfig secret to have data in 0s util.go:744: Successfully waited for NodePools for HostedCluster e2e-clusters-nhkcp/private-xbb98 to have all of their desired nodes in 12m27s util.go:620: Failed to wait for HostedCluster e2e-clusters-nhkcp/private-xbb98 to rollout in 30m0s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-nhkcp/private-xbb98 invalid at RV 117769 after 30m0s: wanted most recent version history to have state Completed, has state Partial util.go:620: *v1beta1.HostedCluster e2e-clusters-nhkcp/private-xbb98 conditions: util.go:620: ValidAWSIdentityProvider=True: AsExpected(All is well) util.go:620: ClusterVersionProgressing=True: ClusterOperatorNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-15-111914-test-ci-op-lyn9s9ns-latest: the cluster operator console is not available) util.go:620: ClusterVersionReleaseAccepted=True: PayloadLoaded(Payload loaded version="5.0.0-0.ci-2026-07-15-111914-test-ci-op-lyn9s9ns-latest" image="registry.build01.ci.openshift.org/ci-op-lyn9s9ns/release@sha256:cbb7f9f1b359ab46f23b33759f9a5cea5b3b43f3f4882dd3440f6c2271ff1b38" architecture="amd64") util.go:620: ClusterVersionUpgradeable=False: ClusterVersionUpdating(An update is already in progress and the details are in the Progressing condition.) util.go:620: ClusterVersionAvailable=False: FromClusterVersion util.go:620: ClusterVersionSucceeding=False: ClusterOperatorNotAvailable(Cluster operator console is not available) util.go:620: Degraded=True: UnavailableReplicas([hosted-cluster-config-operator deployment has 1 unavailable replicas, ignition-server-proxy deployment has 1 unavailable replicas, kube-controller-manager deployment has 1 unavailable replicas]) util.go:620: EtcdAvailable=True: QuorumAvailable util.go:620: KubeAPIServerAvailable=True: AsExpected(Kube APIServer deployment is available) util.go:620: InfrastructureReady=True: AsExpected(All is well) util.go:620: ExternalDNSReachable=Unknown: StatusUnknown(External DNS is not configured) util.go:620: ValidHostedControlPlaneConfiguration=True: AsExpected(Configuration passes validation) util.go:620: ValidReleaseInfo=True: AsExpected(All is well) util.go:620: ValidIDPConfiguration=True: IDPConfigurationValid(Identity provider configuration is valid) util.go:620: HostedClusterRestoredFromBackup=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: DataPlaneConnectionAvailable=True: AsExpected(All is well) util.go:620: ControlPlaneConnectionAvailable=True: AsExpected(All is well) util.go:620: EtcdBackupSucceeded=Unknown: StatusUnknown(Condition not found in the HCP) util.go:620: ConfigOperatorReconciliationSucceeded=True: AsExpected(All is well) util.go:620: AutoNodeEnabled=False: AutoNodeNotConfigured(AutoNode provisioner is not configured) util.go:620: Available=True: AsExpected(The hosted control plane is available) util.go:620: AWSEndpointAvailable=False: AWSError(operation error Route 53: ChangeResourceRecordSets, exceeded maximum number of attempts, 3, https response error StatusCode: 400, RequestID: 8613ccc6-488f-4668-bea8-2746cb7550eb, api error Throttling: Rate exceeded) util.go:620: AWSEndpointServiceAvailable=True: AWSSuccess(All is well) util.go:620: ValidConfiguration=True: AsExpected(Configuration passes validation) util.go:620: SupportedHostedCluster=True: AsExpected(HostedCluster is supported by operator configuration) util.go:620: ValidProxyConfiguration=True: AsExpected(No proxy CA bundle configured) util.go:620: IgnitionEndpointAvailable=True: AsExpected(Ignition server deployment is available) util.go:620: ReconciliationActive=True: AsExpected(Reconciliation active on resource) util.go:620: ValidReleaseImage=True: AsExpected(Release image is valid) util.go:620: Progressing=False: AsExpected(HostedCluster is at expected version) util.go:620: PlatformCredentialsFound=True: AsExpected(Required platform credentials are found) util.go:620: ValidOIDCConfiguration=True: AsExpected(OIDC configuration is valid) util.go:620: ReconciliationSucceeded=True: ReconciliatonSucceeded(Reconciliation completed successfully) util.go:620: ValidAWSKMSConfig=Unknown: StatusUnknown(AWS KMS is not configured) util.go:620: EtcdDataEncryptionUpToDate=True: ReEncryptionCompleted(Encryption key initialized) util.go:620: ClusterVersionRetrievedUpdates=False: NoChannel(The update channel has not been configured.) util.go:620: AWSDefaultSecurityGroupCreated=True: AsExpected(All is well)

requestserving.go:105: Created request serving nodepool clusters/5b00e935c2-mgmt-reqserving-fd6j8 requestserving.go:105: Created request serving nodepool clusters/5b00e935c2-mgmt-reqserving-x6qcn requestserving.go:113: Created non request serving nodepool clusters/5b00e935c2-mgmt-non-reqserving-bnwxr requestserving.go:113: Created non request serving nodepool clusters/5b00e935c2-mgmt-non-reqserving-mmjfw requestserving.go:113: Created non request serving nodepool clusters/5b00e935c2-mgmt-non-reqserving-s5d8n util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5b00e935c2-mgmt-reqserving-fd6j8 in 4m3s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5b00e935c2-mgmt-reqserving-x6qcn in 0s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5b00e935c2-mgmt-non-reqserving-bnwxr in 9s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5b00e935c2-mgmt-non-reqserving-mmjfw in 18s util.go:583: Successfully waited for 1 nodes to become ready for NodePool clusters/5b00e935c2-mgmt-non-reqserving-s5d8n in 0s create_cluster_test.go:299: Sufficient zones available for InfrastructureAvailabilityPolicy HighlyAvailable hypershift_framework.go:518: failed to create cluster, tearing down: failed to create infra: operation error Route 53: ListResourceRecordSets, exceeded maximum number of attempts, 21, https response error StatusCode: 400, RequestID: cce86cf0-3988-4674-a1f7-4e062a3db808, api error Throttling: Rate exceeded requestserving.go:132: Tearing down custom nodepool clusters/5b00e935c2-mgmt-reqserving-fd6j8 requestserving.go:132: Tearing down custom nodepool clusters/5b00e935c2-mgmt-reqserving-x6qcn requestserving.go:132: Tearing down custom nodepool clusters/5b00e935c2-mgmt-non-reqserving-bnwxr requestserving.go:132: Tearing down custom nodepool clusters/5b00e935c2-mgmt-non-reqserving-mmjfw requestserving.go:132: Tearing down custom nodepool clusters/5b00e935c2-mgmt-non-reqserving-s5d8n fixture.go:417: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: hostedclusters.hypershift.openshift.io "request-serving-isolation-8lfrw" not found fixture.go:425: Failed to dump machine journals; this is nonfatal: no SSH secret specified for cluster, cannot dump journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-bzgzc, name: request-serving-isolation-8lfrw

hypershift_framework.go:518: failed to create cluster, tearing down: failed to create infra: failed to tag hosted zone: operation error Route 53: ChangeTagsForResource, exceeded maximum number of attempts, 21, https response error StatusCode: 400, RequestID: a2cd9f07-1e20-4336-84cd-05b5f572c266, api error Throttling: Rate exceeded fixture.go:417: Failed saving machine console logs; this is nonfatal: failed to get machine console logs: failed to get hostedcluster: hostedclusters.hypershift.openshift.io "karpenter-6jbzw" not found fixture.go:425: Failed to dump machine journals; this is nonfatal: no SSH secret specified for cluster, cannot dump journals hypershift_framework.go:591: Destroyed cluster. Namespace: e2e-clusters-cf2jv, name: karpenter-6jbzw

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-cfn8c/node-pool-8rvlv in 11m40s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-8rvlv util.go:3158: Failed to wait for HostedCluster e2e-clusters-cfn8c/node-pool-8rvlv to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-cfn8c/node-pool-8rvlv invalid at RV 68893 after 2s: incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-node-pool-8rvlv.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-cfn8c/node-pool-8rvlv in 51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-8rvlv.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-8rvlv.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-qgtx5/node-pool-6mkrv in 17m57s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster node-pool-6mkrv util.go:3158: Failed to wait for HostedCluster e2e-clusters-qgtx5/node-pool-6mkrv to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-qgtx5/node-pool-6mkrv invalid at RV 77071 after 2s: incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-node-pool-6mkrv.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-qgtx5/node-pool-6mkrv in 42s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-node-pool-6mkrv.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-node-pool-6mkrv.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded

hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-c97h7/scale-from-zero-bdvlc in 4m9s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster scale-from-zero-bdvlc util.go:3158: Failed to wait for HostedCluster e2e-clusters-c97h7/scale-from-zero-bdvlc to have valid conditions in 2s: context deadline exceeded eventually.go:225: observed *v1beta1.HostedCluster e2e-clusters-c97h7/scale-from-zero-bdvlc invalid at RV 55064 after 2s: eventually.go:228: - incorrect condition: wanted AWSEndpointAvailable=True, got AWSEndpointAvailable=False: AWSError(operation error Route 53: ChangeResourceRecordSets, exceeded maximum number of attempts, 3, https response error StatusCode: 400, RequestID: 3ef153b4-c49e-4670-86c6-288c38ca3ead, api error Throttling: Rate exceeded) eventually.go:228: - incorrect condition: wanted ControlPlaneConnectionAvailable=True, got ControlPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available to verify control plane connectivity) eventually.go:228: - incorrect condition: wanted ClusterVersionProgressing=False, got ClusterVersionProgressing=True: ClusterOperatorsNotAvailable(Unable to apply 5.0.0-0.ci-2026-07-15-111914-test-ci-op-lyn9s9ns-latest: some cluster operators are not available) eventually.go:228: - incorrect condition: wanted ClusterVersionAvailable=True, got ClusterVersionAvailable=False: FromClusterVersion eventually.go:228: - incorrect condition: wanted DataPlaneConnectionAvailable=True, got DataPlaneConnectionAvailable=Unknown: NoWorkerNodesAvailable(No worker nodes available) eventually.go:228: - incorrect condition: wanted ExternalDNSReachable=True, got ExternalDNSReachable=False: ExternalDNSHostNotReachable(lookup api-scale-from-zero-bdvlc.service.ci.hypershift.devcluster.openshift.com on 172.29.0.10:53: no such host) eventually.go:228: - incorrect condition: wanted ClusterVersionSucceeding=True, got ClusterVersionSucceeding=False: ClusterOperatorsNotAvailable(Cluster operators console, dns, image-registry, ingress, insights, kube-storage-version-migrator, monitoring, node-tuning, openshift-samples, service-ca, storage are not available) hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-c97h7/scale-from-zero-bdvlc in 51s util.go:322: Successfully waited for kubeconfig secret to have data in 0s eventually.go:105: Failed to get *v1.SelfSubjectReview: Post "https://api-scale-from-zero-bdvlc.service.ci.hypershift.devcluster.openshift.com:443/apis/authentication.k8s.io/v1/selfsubjectreviews": dial tcp: lookup api-scale-from-zero-bdvlc.service.ci.hypershift.devcluster.openshift.com on 172.30.0.10:53: no such host eventually.go:105: Failed to get *v1.SelfSubjectReview: context deadline exceeded util.go:384: Failed to wait for a successful connection to the guest API server in 10m0s: context deadline exceeded

control_plane_upgrade_test.go:25: Starting control plane upgrade test. FromImage: registry.build01.ci.openshift.org/ci-op-lyn9s9ns/release@sha256:622e3aad9f1316badf4e7c2069a15bb6e480d3a5783727abee78031623fec157, toImage: registry.build01.ci.openshift.org/ci-op-lyn9s9ns/release@sha256:cbb7f9f1b359ab46f23b33759f9a5cea5b3b43f3f4882dd3440f6c2271ff1b38 hypershift_framework.go:530: Successfully created hostedcluster e2e-clusters-mbbrg/control-plane-upgrade-55z8r in 27s hypershift_framework.go:179: Summarizing unexpected conditions for HostedCluster control-plane-upgrade-55z8r upgrade_safe_check.go:87: UpgradeSafeFieldCheck: latch engaged for status.controlPlaneVersion util.go:3158: Successfully waited for HostedCluster e2e-clusters-mbbrg/control-plane-upgrade-55z8r to have valid conditions in 0s hypershift_framework.go:349: skipping postTeardown() hypershift_framework.go:327: skipping teardown, already called

util.go:305: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-mbbrg/control-plane-upgrade-55z8r in 0s util.go:322: Successfully waited for kubeconfig secret to have data in 0s util.go:384: Successfully waited for a successful connection to the guest API server in 0s control_plane_upgrade_test.go:43: Starting version: 5.0.0-0.ci-2026-07-10-071205 control_plane_upgrade_test.go:53: Updating cluster image. Image: registry.build01.ci.openshift.org/ci-op-lyn9s9ns/release@sha256:cbb7f9f1b359ab46f23b33759f9a5cea5b3b43f3f4882dd3440f6c2271ff1b38 util.go:620: Successfully waited for HostedCluster e2e-clusters-mbbrg/control-plane-upgrade-55z8r to rollout in 0s

util.go:829: Container etcd-defrag in pod etcd-1 has a restartCount > 0 (1)