PR #5792 - 04-14 17:58

Job: hypershift
FAILURE
← Back to Test Grid

Test Summary

94
Total Tests
63
Passed
13
Failed
18
Skipped

Failed Tests

TestAzureScheduler
1h3m3.21s
hypershift_framework.go:316: Successfully created hostedcluster e2e-clusters-n429l/azure-scheduler-8lczm in 1m34s hypershift_framework.go:115: Summarizing unexpected conditions for HostedCluster azure-scheduler-8lczm util.go:2123: Successfully waited for HostedCluster e2e-clusters-n429l/azure-scheduler-8lczm to have valid conditions in 25ms hypershift_framework.go:194: skipping postTeardown() hypershift_framework.go:175: skipping teardown, already called
TestAzureScheduler/Main
30m0.18s
util.go:218: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-n429l/azure-scheduler-8lczm in 25ms util.go:235: Successfully waited for kubeconfig secret to have data in 0s util.go:281: Successfully waited for a successful connection to the guest API server in 0s util.go:462: Successfully waited for 2 nodes to become ready in 25ms azure_scheduler_test.go:111: Updated clusterSizingConfig. azure_scheduler_test.go:158: Successfully waited for HostedCluster size label and annotations updated in 25ms azure_scheduler_test.go:150: Scaled Nodepool. Namespace: e2e-clusters-n429l, name: azure-scheduler-8lczm, replicas: 0xc001de34ac util.go:462: Failed to wait for 3 nodes to become ready in 30m0s: context deadline exceeded eventually.go:383: observed invalid **v1.Node state after 30m0s eventually.go:400: - observed **v1.Node collection invalid: expected 3 nodes, got 2
TestNodePool
0s
TestNodePool/HostedCluster0
1h9m36.35s
hypershift_framework.go:316: Successfully created hostedcluster e2e-clusters-msxnr/node-pool-95spg in 1m44s hypershift_framework.go:115: Summarizing unexpected conditions for HostedCluster node-pool-95spg util.go:2123: Failed to wait for HostedCluster e2e-clusters-msxnr/node-pool-95spg to have valid conditions in 2s: context deadline exceeded eventually.go:224: observed *v1beta1.HostedCluster e2e-clusters-msxnr/node-pool-95spg invalid at RV 57514 after 2s: eventually.go:227: - incorrect condition: wanted ClusterVersionProgressing=True, got ClusterVersionProgressing=False: FromClusterVersion(Cluster version is 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) eventually.go:227: - incorrect condition: wanted ClusterVersionSucceeding=False, got ClusterVersionSucceeding=True: FromClusterVersion eventually.go:227: - incorrect condition: wanted ClusterVersionAvailable=False, got ClusterVersionAvailable=True: FromClusterVersion(Done applying 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) hypershift_framework.go:194: skipping postTeardown() hypershift_framework.go:175: skipping teardown, already called
TestNodePool/HostedCluster0/Main
180ms
util.go:218: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-msxnr/node-pool-95spg in 75ms util.go:235: Successfully waited for kubeconfig secret to have data in 25ms util.go:281: Successfully waited for a successful connection to the guest API server in 25ms
TestNodePool/HostedCluster0/Main/TestNTOMachineConfigAppliedInPlace
25m21.55s
nodepool_nto_machineconfig_test.go:68: Starting test NTOMachineConfigRolloutTest util.go:462: Successfully waited for 2 nodes to become ready for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-inplace in 24m16.05s nodepool_test.go:350: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-inplace to have correct status in 5.025s util.go:378: Failed to wait for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-inplace to start config update in 1m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-inplace invalid at RV 42878 after 1m0s: incorrect condition: wanted UpdatingConfig=True, got UpdatingConfig=False: AsExpected util.go:378: *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-inplace conditions: util.go:378: UpdatingVersion=False: AsExpected util.go:378: ReachedIgnitionEndpoint=True: AsExpected util.go:378: AllMachinesReady=True: AsExpected(All is well) util.go:378: AllNodesHealthy=True: AsExpected(All is well) util.go:378: AutoscalingEnabled=False: AsExpected util.go:378: ValidArchPlatform=True: AsExpected util.go:378: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:378: ValidMachineConfig=True: AsExpected util.go:378: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:378: UpdateManagementEnabled=True: AsExpected util.go:378: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722) util.go:378: UpdatingConfig=False: AsExpected util.go:378: ValidTuningConfig=True: AsExpected util.go:378: UpdatingPlatformMachineTemplate=False: AsExpected util.go:378: Ready=True: AsExpected util.go:378: AutorepairEnabled=False: AsExpected
TestNodePool/HostedCluster0/Main/TestNTOMachineConfigGetsRolledOut
26m41.58s
nodepool_nto_machineconfig_test.go:68: Starting test NTOMachineConfigRolloutTest util.go:462: Successfully waited for 2 nodes to become ready for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-replace in 24m26.15s nodepool_test.go:350: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-replace to have correct status in 1m15.025s eventually.go:104: Failed to get *v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded util.go:378: Failed to wait for NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-replace to start config update in 1m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-replace invalid at RV 44117 after 1m0s: incorrect condition: wanted UpdatingConfig=True, got UpdatingConfig=False: AsExpected util.go:378: *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-ntomachineconfig-replace conditions: util.go:378: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:378: ValidMachineConfig=True: AsExpected util.go:378: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:378: ReachedIgnitionEndpoint=True: AsExpected util.go:378: AllMachinesReady=True: AsExpected(All is well) util.go:378: AllNodesHealthy=True: AsExpected(All is well) util.go:378: AutoscalingEnabled=False: AsExpected util.go:378: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722) util.go:378: ValidArchPlatform=True: AsExpected util.go:378: UpdatingConfig=False: AsExpected util.go:378: UpdatingVersion=False: AsExpected util.go:378: UpdateManagementEnabled=True: AsExpected util.go:378: ValidTuningConfig=True: AsExpected util.go:378: UpdatingPlatformMachineTemplate=False: AsExpected util.go:378: AutorepairEnabled=False: AsExpected util.go:378: Ready=True: AsExpected
TestNodePool/HostedCluster0/Main/TestNodePoolReplaceUpgrade
46m46.28s
nodepool_upgrade_test.go:100: starting test NodePoolUpgradeTest util.go:462: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade in 24m27.15s nodepool_test.go:350: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade to have correct status in 59.025s nodepool_upgrade_test.go:160: Validating all Nodes have the synced labels and taints nodepool_upgrade_test.go:163: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade to have version 4.19.0-0.ci-2025-04-11-105126 in 25ms nodepool_upgrade_test.go:180: Updating NodePool image. Image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722 nodepool_upgrade_test.go:187: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade to start the upgrade in 1m19.025s eventually.go:104: Failed to get *v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded nodepool_upgrade_test.go:200: Failed to wait for NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade to have version 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest in 20m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade invalid at RV 48142 after 20m0s: eventually.go:227: - wanted version 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest, got 4.19.0-0.ci-2025-04-11-105126 eventually.go:227: - incorrect condition: wanted UpdatingVersion=False, got UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) nodepool_upgrade_test.go:200: *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-replaceupgrade conditions: nodepool_upgrade_test.go:200: ValidArchPlatform=True: AsExpected nodepool_upgrade_test.go:200: UpdatingConfig=False: AsExpected nodepool_upgrade_test.go:200: UpdatingVersion=True: AsExpected(Updating version in progress. Target version: 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) nodepool_upgrade_test.go:200: ReachedIgnitionEndpoint=False: ignitionNotReached nodepool_upgrade_test.go:200: AllMachinesReady=False: Creating(1 of 2 machines are not ready Machine node-pool-95spg-test-replaceupgrade-hbbgf-hrbhn: Creating: virtualmachine creating or updating ) nodepool_upgrade_test.go:200: AllNodesHealthy=False: WaitingForNodeRef(Machine node-pool-95spg-test-replaceupgrade-hbbgf-hrbhn: WaitingForNodeRef ) nodepool_upgrade_test.go:200: AutoscalingEnabled=False: AsExpected nodepool_upgrade_test.go:200: UpdateManagementEnabled=True: AsExpected nodepool_upgrade_test.go:200: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722) nodepool_upgrade_test.go:200: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) nodepool_upgrade_test.go:200: ValidMachineConfig=True: AsExpected nodepool_upgrade_test.go:200: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) nodepool_upgrade_test.go:200: ValidTuningConfig=True: AsExpected nodepool_upgrade_test.go:200: UpdatingPlatformMachineTemplate=False: AsExpected nodepool_upgrade_test.go:200: AutorepairEnabled=False: AsExpected nodepool_upgrade_test.go:200: Ready=True: AsExpected
TestNodePool/HostedCluster0/Main/TestNodepoolMachineconfigGetsRolledout
26m33.58s
nodepool_machineconfig_test.go:55: Starting test NodePoolMachineconfigRolloutTest util.go:462: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-msxnr/node-pool-95spg-test-machineconfig in 24m20.4s nodepool_test.go:350: Successfully waited for NodePool e2e-clusters-msxnr/node-pool-95spg-test-machineconfig to have correct status in 1m13.025s eventually.go:104: Failed to get *v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded util.go:378: Failed to wait for NodePool e2e-clusters-msxnr/node-pool-95spg-test-machineconfig to start config update in 1m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-machineconfig invalid at RV 44088 after 1m0s: incorrect condition: wanted UpdatingConfig=True, got UpdatingConfig=False: AsExpected util.go:378: *v1beta1.NodePool e2e-clusters-msxnr/node-pool-95spg-test-machineconfig conditions: util.go:378: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722) util.go:378: ValidArchPlatform=True: AsExpected util.go:378: ValidMachineConfig=True: AsExpected util.go:378: UpdatingConfig=False: AsExpected util.go:378: UpdatingVersion=False: AsExpected util.go:378: ReachedIgnitionEndpoint=True: AsExpected util.go:378: AllMachinesReady=True: AsExpected(All is well) util.go:378: AllNodesHealthy=True: AsExpected(All is well) util.go:378: UpdateManagementEnabled=True: AsExpected util.go:378: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) util.go:378: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) util.go:378: AutoscalingEnabled=False: AsExpected util.go:378: ValidTuningConfig=True: AsExpected util.go:378: UpdatingPlatformMachineTemplate=False: AsExpected util.go:378: Ready=True: AsExpected util.go:378: AutorepairEnabled=False: AsExpected
TestNodePool/HostedCluster2
1h0m45.22s
hypershift_framework.go:316: Successfully created hostedcluster e2e-clusters-q4p65/node-pool-n7tkc in 2m8s hypershift_framework.go:115: Summarizing unexpected conditions for HostedCluster node-pool-n7tkc util.go:2123: Failed to wait for HostedCluster e2e-clusters-q4p65/node-pool-n7tkc to have valid conditions in 2s: context deadline exceeded eventually.go:224: observed *v1beta1.HostedCluster e2e-clusters-q4p65/node-pool-n7tkc invalid at RV 49335 after 2s: eventually.go:227: - incorrect condition: wanted ClusterVersionSucceeding=False, got ClusterVersionSucceeding=True: FromClusterVersion eventually.go:227: - incorrect condition: wanted ClusterVersionAvailable=False, got ClusterVersionAvailable=True: FromClusterVersion(Done applying 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) eventually.go:227: - incorrect condition: wanted ClusterVersionProgressing=True, got ClusterVersionProgressing=False: FromClusterVersion(Cluster version is 4.19.0-0.ci-2025-04-14-183759-test-ci-op-z7rvrscy-latest) hypershift_framework.go:194: skipping postTeardown() hypershift_framework.go:175: skipping teardown, already called
TestNodePool/HostedCluster2/Main
110ms
util.go:218: Successfully waited for kubeconfig to be published for HostedCluster e2e-clusters-q4p65/node-pool-n7tkc in 25ms util.go:235: Successfully waited for kubeconfig secret to have data in 0s util.go:281: Successfully waited for a successful connection to the guest API server in 50ms
TestNodePool/HostedCluster2/Main/TestAdditionalTrustBundlePropagation
42m58.23s
nodepool_additionalTrustBundlePropagation_test.go:36: Starting AdditionalTrustBundlePropagationTest. eventually.go:258: Failed to get **v1.Node: Get "https://api-node-pool-n7tkc.aks-e2e.hypershift.azure.devcluster.openshift.com:443/api/v1/nodes?labelSelector=hypershift.openshift.io%2FnodePool%3Dnode-pool-n7tkc-test-additional-trust-bundle-propagation": net/http: TLS handshake timeout util.go:462: Successfully waited for 1 nodes to become ready for NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation in 12m39.025s nodepool_test.go:350: Successfully waited for NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation to have correct status in 9.025s eventually.go:104: Failed to get *v1beta1.NodePool: client rate limiter Wait returned an error: context deadline exceeded nodepool_test.go:350: Failed to wait for NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation to have correct status in 10m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation invalid at RV 49455 after 10m0s: eventually.go:227: - incorrect condition: wanted UpdatingConfig=False, got UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 58453630) eventually.go:227: - incorrect condition: wanted AllNodesHealthy=True, got AllNodesHealthy=False: NodeProvisioning(Machine node-pool-n7tkc-test-additional-trust-bundle-propagation-jt28vf: NodeProvisioning ) eventually.go:227: - incorrect condition: wanted ReachedIgnitionEndpoint=True, got ReachedIgnitionEndpoint=False: ignitionNotReached
TestNodePool/HostedCluster2/Main/TestAdditionalTrustBundlePropagation/AdditionalTrustBundlePropagationTest
20m10.15s
nodepool_additionalTrustBundlePropagation_test.go:70: Updating hosted cluster with additional trust bundle. Bundle: additional-trust-bundle nodepool_additionalTrustBundlePropagation_test.go:78: Successfully waited for Waiting for NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation to begin updating in 10.05s nodepool_additionalTrustBundlePropagation_test.go:92: Failed to wait for Waiting for NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation to stop updating in 20m0s: context deadline exceeded eventually.go:224: observed *v1beta1.NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation invalid at RV 32311 after 20m0s: eventually.go:227: - incorrect condition: wanted UpdatingConfig=False, got UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 58453630) eventually.go:227: - incorrect condition: wanted AllNodesHealthy=True, got AllNodesHealthy=False: WaitingForNodeRef(Machine node-pool-n7tkc-test-additional-trust-bundle-propagation-jt28vf: WaitingForNodeRef ) nodepool_additionalTrustBundlePropagation_test.go:92: *v1beta1.NodePool e2e-clusters-q4p65/node-pool-n7tkc-test-additional-trust-bundle-propagation conditions: nodepool_additionalTrustBundlePropagation_test.go:92: ValidGeneratedPayload=True: AsExpected(Payload generated successfully) nodepool_additionalTrustBundlePropagation_test.go:92: ReachedIgnitionEndpoint=False: ignitionNotReached nodepool_additionalTrustBundlePropagation_test.go:92: AutoscalingEnabled=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: UpdateManagementEnabled=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: ValidArchPlatform=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: UpdatingConfig=True: AsExpected(Updating config in progress. Target config: 58453630) nodepool_additionalTrustBundlePropagation_test.go:92: ValidReleaseImage=True: AsExpected(Using release image: registry.build01.ci.openshift.org/ci-op-z7rvrscy/release@sha256:074005975a550225223082a912ac9453a4f6bf91c7810a2f1bcf1c80dca91722) nodepool_additionalTrustBundlePropagation_test.go:92: ReconciliationActive=True: ReconciliationActive(Reconciliation active on resource) nodepool_additionalTrustBundlePropagation_test.go:92: ValidMachineConfig=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: UpdatingVersion=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: AllMachinesReady=False: Creating(1 of 2 machines are not ready Machine node-pool-n7tkc-test-additional-trust-bundle-propagation-jt28vf: Creating: virtualmachine creating or updating ) nodepool_additionalTrustBundlePropagation_test.go:92: AllNodesHealthy=False: WaitingForNodeRef(Machine node-pool-n7tkc-test-additional-trust-bundle-propagation-jt28vf: WaitingForNodeRef ) nodepool_additionalTrustBundlePropagation_test.go:92: ValidTuningConfig=True: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: UpdatingPlatformMachineTemplate=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: AutorepairEnabled=False: AsExpected nodepool_additionalTrustBundlePropagation_test.go:92: Ready=True: AsExpected