Skip to content

Latest commit

 

History

History
362 lines (266 loc) · 12.9 KB

manifest.md

File metadata and controls

362 lines (266 loc) · 12.9 KB
subcategory page_title description
manifest
Kubernetes: kubernetes_manifest
The resource provides a way to create and manage custom resources

kubernetes_manifest

Represents one Kubernetes resource by supplying a manifest attribute. The manifest value is the HCL representation of a Kubernetes YAML manifest. To convert an existing manifest from YAML to HCL, you can use the Terraform built-in function yamldecode() or tfk8s.

Once applied, the object attribute contains the state of the resource as returned by the Kubernetes API, including all default values.

~> A minimum Terraform version of 0.14.8 is required to use this resource.

Schema

Required

  • manifest (Dynamic) A Kubernetes manifest describing the desired state of the resource in HCL format.

Optional

  • computed_fields (List of String) List of manifest fields whose values can be altered by the API server during 'apply'. Defaults to: ["metadata.annotations", "metadata.labels"]
  • field_manager (Block List, Max: 1) Configure field manager options. (see below for nested schema)
  • object (Dynamic) The resulting resource state, as returned by the API server after applying the desired state from manifest.
  • skip_validation (Boolean) When set to true, skips validation of the manifest against the Kubernetes API server during the plan phase. This can be useful when the cluster is not available or when you want to defer validation until apply time. Defaults to false.
  • timeouts (Block List, Max: 1) (see below for nested schema)
  • wait (Block List, Max: 1) Configure waiter options. (see below for nested schema)
  • wait_for (Object, Deprecated) A map of attribute paths and desired patterns to be matched. After each apply the provider will wait for all attributes listed here to reach a value that matches the desired pattern. (see below for nested schema)

Nested Schema for field_manager

Optional:

  • force_conflicts (Boolean) Force changes against conflicts.
  • name (String) The name to use for the field manager when creating and updating the resource.

Nested Schema for timeouts

Optional:

  • create (String) Timeout for the create operation.
  • delete (String) Timeout for the delete operation.
  • update (String) Timeout for the update operation.

Nested Schema for wait

Optional:

  • condition (Block List) (see below for nested schema)
  • fields (Map of String) A map of paths to fields to wait for a specific field value.
  • rollout (Boolean) Wait for rollout to complete on resources that support kubectl rollout status.

Nested Schema for wait.condition

Optional:

  • status (String) The condition status.
  • type (String) The type of condition.

Nested Schema for wait_for

Optional:

  • fields (Map of String)

Before you use this resource

  • This resource requires API access during planning time by default. This means the cluster has to be accessible at plan time and thus cannot be created in the same apply operation. However, you can use the skip_validation attribute to defer validation until apply time, which allows creating the cluster and its resources in the same Terraform run. We recommend only using this resource for custom resources or resources not yet fully supported by the provider.

  • This resource uses Server-side Apply to carry out apply operations. A minimum Kubernetes version of 1.16.x is required, but versions 1.17+ are strongly recommended as the SSA implementation in Kubernetes 1.16.x is incomplete and unstable.

Example: Create a Kubernetes ConfigMap

resource "kubernetes_manifest" "test-configmap" {
  manifest = {
    "apiVersion" = "v1"
    "kind"       = "ConfigMap"
    "metadata" = {
      "name"      = "test-config"
      "namespace" = "default"
    }
    "data" = {
      "foo" = "bar"
    }
  }
}

Example: Create a Kubernetes Custom Resource Definition

resource "kubernetes_manifest" "test-crd" {
  manifest = {
    apiVersion = "apiextensions.k8s.io/v1"
    kind       = "CustomResourceDefinition"

    metadata = {
      name = "testcrds.hashicorp.com"
    }

    spec = {
      group = "hashicorp.com"

      names = {
        kind   = "TestCrd"
        plural = "testcrds"
      }

      scope = "Namespaced"

      versions = [{
        name    = "v1"
        served  = true
        storage = true
        schema = {
          openAPIV3Schema = {
            type = "object"
            properties = {
              data = {
                type = "string"
              }
              refs = {
                type = "number"
              }
            }
          }
        }
      }]
    }
  }
}

Importing existing Kubernetes resources as kubernetes_manifest

Objects already present in a Kubernetes cluster can be imported into Terraform to be managed as kubernetes_manifest resources. Follow these steps to import a resource:

Extract the resource from Kubernetes and transform it into Terraform configuration

kubectl get secrets sample -o yaml | tfk8s --strip -o sample.tf

Import the resource state from the cluster

terraform import kubernetes_manifest.secret_sample "apiVersion=v1,kind=Secret,namespace=default,name=sample"

Note the import ID as the last argument to the import command. This ID points Terraform at which Kubernetes object to read when importing. It should be constructed with the following syntax: "apiVersion=<string>,kind=<string>,[namespace=<string>,]name=<string>". The namespace=<string> in the ID string is required only for Kubernetes namespaced objects and should be omitted for cluster-wide objects.

Using wait to block create and update calls

The kubernetes_manifest resource supports the ability to block create and update calls until a field is set or has a particular value by specifying the wait block. This is useful for when you create resources like Jobs and Services when you want to wait for something to happen after the resource is created by the API server before Terraform should consider the resource created.

wait supports supports a fields attribute which allows you specify a map of fields paths to regular expressions. You can also specify * if you just want to wait for a field to have any value.

resource "kubernetes_manifest" "test" {
  manifest = {
    // ...
  }

  wait {
    fields = {
      # Check the phase of a pod
      "status.phase" = "Running"

      # Check a container's status
      "status.containerStatuses[0].ready" = "true",

      # Check an ingress has an IP
      "status.loadBalancer.ingress[0].ip" = "^(\\d+(\\.|$)){4}"

      # Check the replica count of a Deployment
      "status.readyReplicas" = "2"
    }
  }

  timeouts {
    create = "10m"
    update = "10m"
    delete = "30s"
  }
}

The wait block also supports a rollout attribute which will wait for rollout to complete on Deployment, StatefulSet, and DaemonSet resources.

resource "kubernetes_manifest" "test" {
  manifest = {
    // ...
  }

  wait {
    rollout = true
  }
}

You can also wait for specified conditions to be met by specifying a condition block.

resource "kubernetes_manifest" "test" {
  manifest = {
    // ...
  }

  wait {
    condition {
      type   = "ContainersReady"
      status = "True"
    }
  }
}

Configuring field_manager

The kubernetes_manifest exposes configuration of the field manager through the optional field_manager block.

resource "kubernetes_manifest" "test" {
  provider = kubernetes-alpha

  manifest = {
    // ...
  }

  field_manager {
    # set the name of the field manager
    name = "myteam"

    # force field manager conflicts to be overridden
    force_conflicts = true
  }
}

Computed fields

When setting the value of an field in configuration, Terraform will check that the same value is returned after the apply operation. This ensures that the actual configuration requested by the user is successfully applied. In some cases, with the Kubernetes API this is not the desired behavior. Particularly when using mutating admission controllers, there is a chance that the values configured by the user will be modified by the API. This usually manifest as Error: Provider produced inconsistent result after apply and produced an unexpected new value: messages when applying.

To accommodate this, the kubernetes_manifest resources allows defining so-called "computed" fields. When an field is defined as "computed" Terraform will allow the final value stored in state after apply as returned by the API to be different than what the user requested.

The most common example of this is metadata.annotations. In some cases, the API will add extra annotations on top of the ones configured by the user. Unless the field is declared as "computed" Terraform will throw an error signaling that the state returned by the 'apply' operation is inconsistent with the value defined in the 'plan'.

To declare an field as "computed" add its full field path to the computed_fields field under the respective kubernetes_manifest resource. For example, to declare the "metadata.labels" field as "computed", add the following:

resource "kubernetes_manifest" "test-ns" {
  manifest = {
    ...
  }

  computed_fields = ["metadata.labels"]
 }

IMPORTANT: By default, metadata.labels and metadata.annotations are already included in the list. You don't have to set them explicitly in the computed_fields list. To turn off these defaults, set the value of computed_fields to an empty list or a concrete list of other fields. For example computed_fields = [].

The syntax for the field paths is the same as the one used in the wait block.

A field path is a string that describes the fully qualified address of a field within the resource, including its parent fields all the way up to "object". The syntax of a path string follows the rules below:

  • Fields of objects are addressed with .

  • Keys of a map field are addressed with ["<key-string>"]

  • Elements of a list or tuple field are addressed with [<index-numeral>]

    The following example waits for Kubernetes to create a ServiceAccount token in a Secret, where the data field of the Secret is a map.

    wait {
  fields = {
    "data[\"token\"]" = ".+"
  }
}

    You can use the type() Terraform function to determine the type of a field. With the resource created and present in state, run terraform console and then the following command:

    > type(kubernetes_manifest.my-secret.object.data)
  map(string)

Skipping validation during plan phase

By default, the kubernetes_manifest resource validates the manifest against the Kubernetes API server during the plan phase. This includes checking if the resource type exists, verifying CRD schemas, and validating namespace requirements. However, there are scenarios where you may want to skip this validation:

  • When the cluster is not available during plan time (e.g., creating the cluster and its resources in the same Terraform run)
  • When deploying CRDs and their custom resources in the same apply operation
  • When you want to defer validation until apply time

To skip validation during plan phase, set the skip_validation attribute to true:

resource "kubernetes_manifest" "test" {
  manifest = {
    apiVersion = "apps/v1"
    kind       = "Deployment"
    metadata = {
      name      = "test-deployment"
      namespace = "default"
    }
    spec = {
      replicas = 3
      selector = {
        matchLabels = {
          app = "test"
        }
      }
      template = {
        metadata = {
          labels = {
            app = "test"
          }
        }
        spec = {
          containers = [
            {
              name  = "nginx"
              image = "nginx:1.21"
            }
          ]
        }
      }
    }
  }

  skip_validation = true
}

Important notes:

  • When skip_validation is enabled, the provider will not connect to the Kubernetes cluster during plan phase, which means:

    • CRD validation will be skipped
    • Namespace validation will be skipped
    • OpenAPI schema validation will be skipped
    • The resource will be treated as requiring replacement on updates

  • Validation will still occur during the apply phase when the resource is actually created or updated in the cluster.

  • It is recommended to use this feature only when necessary, as it reduces the ability to catch configuration errors early during planning.