fix(kubernetes_manifest): preserve dynamicpseudotype in value convers…#2822
Closed
leefowlercu wants to merge 2 commits intohashicorp:mainfrom
Closed
fix(kubernetes_manifest): preserve dynamicpseudotype in value convers…#2822leefowlercu wants to merge 2 commits intohashicorp:mainfrom
leefowlercu wants to merge 2 commits intohashicorp:mainfrom
Conversation
…ion pipeline fixes "provider produced inconsistent result after apply" error when using kubernetes_manifest with crds that have x-kubernetes-preserve-unknown-fields the fix ensures type consistency between planned and actual state by: - preserving dynamicpseudotype from schema in type conversion functions - morphing apply output type structure to match deserialized planned state type
|
Found a problem during local testing. It causes a crash with the following configuration. Terraform config resource "kubernetes_manifest" "test" {
manifest = {
apiVersion = "company.local/v1alpha1"
kind = "Foo"
metadata = {
name = lower(data.manidae_parameter.name.value)
namespace = var.Kubernetes_namespace
}
spec = {
mapdata = {
test = {
datas = var.kubernetes_tunnel_extra_routes
}
# skipped data field on this object
test2 = {
}
}
}
}
}CRD define Error message |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(kubernetes_manifest): preserve DynamicPseudoType in value conversion pipeline
Summary
This PR fixes the "Provider produced inconsistent result after apply" error that occurs when using
kubernetes_manifestwith CRDs that havex-kubernetes-preserve-unknown-fields: true(represented asDynamicPseudoTypein the Terraform type system).The fix ensures type consistency between the planned state and the actual state by:
DynamicPseudoTypein the type conversion pipelineFixes
Fixes #2821
Root Cause
The bug has two interacting causes:
Cause 1: Type Conversion Used Actual Type Instead of Schema Type
The value conversion pipeline was using the actual value's type instead of the schema type when constructing output objects.
Cause 2: DynamicPseudoType Lost During Serialization
Even when the plan phase correctly produces types with
DynamicPseudoType, this type information is lost during Terraform protocol serialization/deserialization:DynamicPseudoTypein type structureObject[])DynamicPseudoType, type mismatch occursExample:
The Terraform SDK validates that planned and actual types match, causing the error:
Changes Made
manifest/payload/to_value.goModified
mapToTFObjectValueto preserveDynamicPseudoTypefrom the schema when constructing output types:Similar changes applied to:
sliceToTFListValuesliceToTFTupleValuesliceToTFSetValuemapToTFMapValuemanifest/morph/scaffold.goModified
DeepUnknownto preserveDynamicPseudoTypein both Object and Tuple cases:Object case:
Tuple expansion case:
manifest/morph/morph.goModified
morphObjectToTypeandmorphTupleIntoTypeto preserveDynamicPseudoTypefrom the target type schema.Added new function
MorphTypeStructure(~170 lines): This function recursively converts a value's type structure to match a target type while preserving the underlying data. It handles the case where serialization convertedDynamicPseudoTypeto concrete types.manifest/provider/apply.goAdded call to
MorphTypeStructureafter computing the apply result to ensure the output type structure matches the deserialized planned state's type:Testing
Unit Tests Added
6 new regression tests that verify output TYPE structure preserves
DynamicPseudoType:manifest/payload/to_value_test.go:empty-object-dynamic-pseudotype: Verifies empty object with DynamicPseudoType schema (KubeVirt scenario)nested-dynamic-pseudotype-leaf: Verifies DynamicPseudoType preserved in nested structuresmanifest/morph/scaffold_test.go:object-dynamic-pseudotype: Verifies DeepUnknown preserves DynamicPseudoType in Object schematuple-dynamic-expansion: Verifies DeepUnknown preserves DynamicPseudoType when tuples expandmanifest/morph/morph_test.go:object(empty-dynamic) -> object: Verifies morphObjectToType handles empty objects with DynamicPseudoType schemaTestMorphTypeStructure: 6 test cases verifyingMorphTypeStructurecorrectly morphs type structures:dynamic-to-concrete-empty-object: KubeVirt masquerade scenarionested-dynamic-to-concrete: Deeply nested DynamicPseudoType conversionlist-with-dynamic-elements: List element type conversionsame-type-passthrough: Passthrough when types already matchnull-value: Null value handlingunknown-value: Unknown value handlingAcceptance Test Added
manifest/test/acceptance/customresource_x_preserve_unknown_fields_test.go:TestKubernetesManifest_CustomResource_x_preserve_unknown_fields_empty_object: End-to-end regression test that creates a CR with an empty object (resources = {}) in a field marked withx-kubernetes-preserve-unknown-fields. Without the fix, this test would fail with "Provider produced inconsistent result after apply".Test fixture:
manifest/test/acceptance/testdata/x-kubernetes-preserve-unknown-fields/test-cr-empty.tfManual Testing
Manual testing was performed with a real KubeVirt VirtualMachine CRD to verify the fix works with production-grade complex CRDs.
Test Environment
Test Configuration
Test Results
Before Fix:
After Fix:
Verification:
Cleanup:
Testing Environments
Acceptance tests were run in two environments to ensure broad compatibility:
Environment 1: KinD (Recommended per CONTRIBUTING.md)
.github/config/acceptance_tests_kind_config.yamlEnvironment 2: Red Hat OpenShift (ROSA with HyperShift)
This environment was used for initial bug reproduction and verification with real-world KubeVirt VirtualMachine CRDs.
Cluster Details:
Node Configuration:
OpenShift Virtualization (CNV) Operator:
This environment provides access to the
virtualmachines.kubevirt.ioCRD which usesx-kubernetes-preserve-unknown-fields: trueextensively throughout its schema, making it an ideal real-world test case for the DynamicPseudoType preservation fix.Test Results
Unit Tests:
Acceptance Tests (KinD - Kubernetes v1.35.0):
Acceptance Tests (OpenShift - Kubernetes v1.33.5):
Related Issues
These issues report the same "wrong final value type: incorrect object attributes" error:
wrong final value type: incorrect object attributes.#1545 - Karpenter Provisioner CRDkubernetes_manifestthrows "Provider produced inconsistent result after apply" error when applying PrometheusRules CRD #1418 - PrometheusRules CRDChecklist
Rollback Plan
If a change needs to be reverted, we will publish an updated version of the library.
Changes to Security Controls
Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.
Release Note