From 414d1d286ee6f0cac3efb23677ba39e688eb2725 Mon Sep 17 00:00:00 2001 From: Kenneth Bingham Date: Thu, 13 Mar 2025 18:01:37 -0400 Subject: [PATCH 1/3] upgrade ziti cli; set miniziti timeout --- .github/workflows/miniziti.yml | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/miniziti.yml b/.github/workflows/miniziti.yml index b92db0e32..ac4498e01 100644 --- a/.github/workflows/miniziti.yml +++ b/.github/workflows/miniziti.yml @@ -43,9 +43,9 @@ jobs: - name: install ziti cli uses: supplypike/setup-bin@v4 with: - uri: https://github.com/openziti/ziti/releases/download/v1.2.0/ziti-linux-amd64-1.2.0.tar.gz + uri: https://github.com/openziti/ziti/releases/download/v1.4.3/ziti-linux-amd64-1.4.3.tar.gz name: ziti - version: 1.2.0 + version: 1.4.3 # this is the kubernetes quickstart script from # https://openziti.io/docs/learn/quickstarts/network/local-kubernetes @@ -59,9 +59,13 @@ jobs: - name: Run miniziti with latest release charts if: vars.SKIP_MINIKUBE_LATEST_CHARTS != 'true' run: miniziti start --no-hosts --verbose + env: + MINIZITI_TIMEOUT_SECS: 300 - name: Upgrade miniziti with charts from this branch run: miniziti start --no-hosts --verbose --charts ./charts + env: + MINIZITI_TIMEOUT_SECS: 300 - name: Verify the Console is Available shell: bash From a48fcf0ae6a178e61becfb52a1b62c6920240d99 Mon Sep 17 00:00:00 2001 From: Kenneth Bingham Date: Thu, 13 Mar 2025 18:01:56 -0400 Subject: [PATCH 2/3] add migration script for breaking change --- charts/ziti-controller/README.md | 70 +++++++++++++++++-- charts/ziti-controller/README.md.gotmpl | 70 +++++++++++++++++-- .../files/chown-cert-manager.bash | 52 ++++++++++++++ 3 files changed, 180 insertions(+), 12 deletions(-) create mode 100755 charts/ziti-controller/files/chown-cert-manager.bash diff --git a/charts/ziti-controller/README.md b/charts/ziti-controller/README.md index d4c4af3b2..dfd31937d 100644 --- a/charts/ziti-controller/README.md +++ b/charts/ziti-controller/README.md @@ -16,14 +16,72 @@ helm repo add openziti https://docs.openziti.io/helm-charts/ This chart runs a Ziti controller in Kubernetes. -It uses the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). +### Mutual TLS -Ziti's TLS server ports must be published with a TLS passthrough. This may be done with a Traefik IngressRouteTCP, Gateway API TLSRoute, Ingress, NodePort, LoadBalancer, etc. The ctrl plane and management API share the client API's TLS listener by default, so there's one TCP port by default that must be published with TLS passthrough enabled. +Ziti's TLS server ports must be published with a TLS passthrough to allow the controller to validate the client certificates from routers and identities. This may be done with a Traefik IngressRouteTCP, Gateway API TLSRoute, Ingress, NodePort, LoadBalancer, etc. The ctrl plane and management API share the client API's TLS listener by default, so there's one TCP port by default that must be published with TLS passthrough enabled. + +### Certificates It is not normally necessary to obtain publicly trusted certificates for Ziti's TLS servers. Ziti manages the trust relationships between the controller and routers and clients independent of the web's root authorities. See the [Alternative Web Server Certificates](#alternative-web-server-certificates) section for more information. +### Deployment + The deployment must have exactly one replica. +### Custom Resources + +This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). + +```bash +helm upgrade --install cert-manager jetstack/cert-manager \ + --namespace cert-manager --create-namespace \ + --set crds.enabled=true + +kubectl create namespace ziti +helm upgrade --install trust-manager jetstack/trust-manager \ + --namespace cert-manager \ + --set crds.keep=false \ + --set app.trust.namespace=ziti +``` + +### Breaking Change + +Version 2 of this chart introduces a breaking change. + +**Symptom** + +> Error: Unable to continue with install: CustomResourceDefinition "certificaterequests.cert-manager.io" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "cert-manager"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "cert-manager" + +**Cause** + +Cert Manager and Trust Manager are no longer included as subcharts. If you have a v1 version of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. + +**Solution** + +Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in the "cert-manager" namespace, and your Ziti controller is installed in the "ziti" namespace, you can run the provided script to prep for installing the v2 chart. + +```bash +helm pull openziti/ziti-controller +tar -xvf ziti-controller-*.tgz +CM_NAMESPACE=cert-manager \ +CM_RELEASE_NAME=cert-manager \ +TM_NAMESPACE=cert-manager \ +ZITI_NAMESPACE=ziti \ +./ziti-controller/files/chown-cert-manager.bash +``` + +Now the cluster is ready for you to install cert-manager and trust-manager as separate releases before installing the v2 ziti-controller chart. + +The script performs an annotate and label operation on the cert-manager and trust-manager CRDs and the core resources created by trust-manager, each named for the release name (default `trust-manager`): `ClusterRole`, `ClusterRoleBinding`, `ValidatingWebhookConfiguration`, `Role`, `RoleBinding`. + +```bash +kubectl annotate crds bundles.trust.cert-manager.io --overwrite \ + meta.helm.sh/release-name="trust-manager" \ + meta.helm.sh/release-namespace="cert-manager" +kubectl label crds bundles.trust.cert-manager.io \ + app.kubernetes.io/managed-by=Helm +``` + ## NodePort Service Example | Value | Description | @@ -46,10 +104,10 @@ Here's the YAML representation of the same set of input values. ```yaml clientApi: - advertisedHost: ctrl1.ziti.example.com - advertisedPort: 32171 - service: - type: NodePort + advertisedHost: ctrl1.ziti.example.com + advertisedPort: 32171 + service: + type: NodePort ``` Visit the Ziti Administration Console (ZAC): https://ctrl1.ziti.example.com:32171/zac/ diff --git a/charts/ziti-controller/README.md.gotmpl b/charts/ziti-controller/README.md.gotmpl index 068eb7176..6c0dd93fa 100644 --- a/charts/ziti-controller/README.md.gotmpl +++ b/charts/ziti-controller/README.md.gotmpl @@ -25,14 +25,72 @@ helm repo add openziti https://docs.openziti.io/helm-charts/ This chart runs a Ziti controller in Kubernetes. -It uses the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). +### Mutual TLS -Ziti's TLS server ports must be published with a TLS passthrough. This may be done with a Traefik IngressRouteTCP, Gateway API TLSRoute, Ingress, NodePort, LoadBalancer, etc. The ctrl plane and management API share the client API's TLS listener by default, so there's one TCP port by default that must be published with TLS passthrough enabled. +Ziti's TLS server ports must be published with a TLS passthrough to allow the controller to validate the client certificates from routers and identities. This may be done with a Traefik IngressRouteTCP, Gateway API TLSRoute, Ingress, NodePort, LoadBalancer, etc. The ctrl plane and management API share the client API's TLS listener by default, so there's one TCP port by default that must be published with TLS passthrough enabled. + +### Certificates It is not normally necessary to obtain publicly trusted certificates for Ziti's TLS servers. Ziti manages the trust relationships between the controller and routers and clients independent of the web's root authorities. See the [Alternative Web Server Certificates](#alternative-web-server-certificates) section for more information. +### Deployment + The deployment must have exactly one replica. +### Custom Resources + +This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). + +```bash +helm upgrade --install cert-manager jetstack/cert-manager \ + --namespace cert-manager --create-namespace \ + --set crds.enabled=true + +kubectl create namespace ziti +helm upgrade --install trust-manager jetstack/trust-manager \ + --namespace cert-manager \ + --set crds.keep=false \ + --set app.trust.namespace=ziti +``` + +### Breaking Change + +Version 2 of this chart introduces a breaking change. + +**Symptom** + +> Error: Unable to continue with install: CustomResourceDefinition "certificaterequests.cert-manager.io" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "cert-manager"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "cert-manager" + +**Cause** + +Cert Manager and Trust Manager are no longer included as subcharts. If you have a v1 version of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. + +**Solution** + +Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in the "cert-manager" namespace, and your Ziti controller is installed in the "ziti" namespace, you can run the provided script to prep for installing the v2 chart. + +```bash +helm pull openziti/ziti-controller +tar -xvf ziti-controller-*.tgz +CM_NAMESPACE=cert-manager \ +CM_RELEASE_NAME=cert-manager \ +TM_NAMESPACE=cert-manager \ +ZITI_NAMESPACE=ziti \ +./ziti-controller/files/chown-cert-manager.bash +``` + +Now the cluster is ready for you to install cert-manager and trust-manager as separate releases before installing the v2 ziti-controller chart. + +The script performs an annotate and label operation on the cert-manager and trust-manager CRDs and the core resources created by trust-manager, each named for the release name (default `trust-manager`): `ClusterRole`, `ClusterRoleBinding`, `ValidatingWebhookConfiguration`, `Role`, `RoleBinding`. + +```bash +kubectl annotate crds bundles.trust.cert-manager.io --overwrite \ + meta.helm.sh/release-name="trust-manager" \ + meta.helm.sh/release-namespace="cert-manager" +kubectl label crds bundles.trust.cert-manager.io \ + app.kubernetes.io/managed-by=Helm +``` + ## NodePort Service Example | Value | Description | @@ -55,10 +113,10 @@ Here's the YAML representation of the same set of input values. ```yaml clientApi: - advertisedHost: ctrl1.ziti.example.com - advertisedPort: 32171 - service: - type: NodePort + advertisedHost: ctrl1.ziti.example.com + advertisedPort: 32171 + service: + type: NodePort ``` Visit the Ziti Administration Console (ZAC): https://ctrl1.ziti.example.com:32171/zac/ diff --git a/charts/ziti-controller/files/chown-cert-manager.bash b/charts/ziti-controller/files/chown-cert-manager.bash new file mode 100755 index 000000000..9ba2eade6 --- /dev/null +++ b/charts/ziti-controller/files/chown-cert-manager.bash @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# +# set owner labels and annotations for existing cert-manager and trust-manager CRDs and resources to allow a future +# cert-manager and trust-manager Helm releases to import them +# + +set -o errexit +set -o pipefail +set -o nounset + +: "${CM_NAMESPACE:=cert-manager}" +: "${CM_RELEASE_NAME:=cert-manager}" +: "${TM_NAMESPACE:=cert-manager}" +: "${TM_RELEASE_NAME:=trust-manager}" +: "${ZITI_NAMESPACE:=ziti}" + +# cert-manager CRDs, not trust-manager CRD +while read +do + kubectl annotate crds "$REPLY" --overwrite \ + meta.helm.sh/release-name="${CM_RELEASE_NAME}" \ + meta.helm.sh/release-namespace="${CM_NAMESPACE}" + kubectl label crds "$REPLY" \ + app.kubernetes.io/managed-by=Helm +done< <(kubectl get crds | grep -v 'bundles\.trust\.cert-manager\.io' | grep -w 'cert-manager\.io' | cut -f 1 -d ' ') + +# trust-manager CRD +kubectl annotate crds bundles.trust.cert-manager.io --overwrite \ + meta.helm.sh/release-name="${TM_RELEASE_NAME}" \ + meta.helm.sh/release-namespace="${CM_NAMESPACE}" +kubectl label crds bundles.trust.cert-manager.io \ + app.kubernetes.io/managed-by=Helm + +# cluster-wide core resources to be imported by trust-manager +for R in ClusterRole{,Binding} ValidatingWebhookConfiguration +do + kubectl annotate "$R" "${TM_RELEASE_NAME}" --overwrite \ + meta.helm.sh/release-name="${TM_RELEASE_NAME}" \ + meta.helm.sh/release-namespace="${CM_NAMESPACE}" + kubectl label "$R" "${TM_RELEASE_NAME}" \ + app.kubernetes.io/managed-by=Helm +done + +# namespaced core resources to be imported by trust-manager +for R in Role{,Binding} +do + kubectl annotate -n "${ZITI_NAMESPACE}" "$R" "${TM_RELEASE_NAME}" --overwrite \ + meta.helm.sh/release-name="${TM_RELEASE_NAME}" \ + meta.helm.sh/release-namespace="${CM_NAMESPACE}" + kubectl label -n "${ZITI_NAMESPACE}" "$R" "${TM_RELEASE_NAME}" \ + app.kubernetes.io/managed-by=Helm +done From 389acc8956a36eff230dd33b77032ad9c9924ff8 Mon Sep 17 00:00:00 2001 From: Kenneth Bingham Date: Thu, 13 Mar 2025 21:08:23 -0400 Subject: [PATCH 3/3] improve doc --- charts/ziti-controller/README.md | 22 +++++++--------------- charts/ziti-controller/README.md.gotmpl | 22 +++++++--------------- 2 files changed, 14 insertions(+), 30 deletions(-) diff --git a/charts/ziti-controller/README.md b/charts/ziti-controller/README.md index dfd31937d..e8e8af7a9 100644 --- a/charts/ziti-controller/README.md +++ b/charts/ziti-controller/README.md @@ -30,9 +30,11 @@ The deployment must have exactly one replica. ### Custom Resources -This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). +This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to compose a trust Bundle resource from Ziti's root CA cert(s). ```bash +helm repo add jetstack https://charts.jetstack.io + helm upgrade --install cert-manager jetstack/cert-manager \ --namespace cert-manager --create-namespace \ --set crds.enabled=true @@ -46,7 +48,7 @@ helm upgrade --install trust-manager jetstack/trust-manager \ ### Breaking Change -Version 2 of this chart introduces a breaking change. +Version 2 of this chart introduces a breaking change. You must decouple cert-manager and trust-manager from the Ziti controller chart if they were previously installed as subcharts. This allows them to be upgraded and configured independently of the Ziti controller chart. **Symptom** @@ -54,11 +56,11 @@ Version 2 of this chart introduces a breaking change. **Cause** -Cert Manager and Trust Manager are no longer included as subcharts. If you have a v1 version of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. +Cert Manager and Trust Manager are no longer included as subcharts. If you have version 1 of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. **Solution** -Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in the "cert-manager" namespace, and your Ziti controller is installed in the "ziti" namespace, you can run the provided script to prep for installing the v2 chart. +Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in namespace "cert-manager," and your Ziti controller is installed in namespace "ziti," you can run the provided script to pave the way to installing the version 2 ziti-controller chart. ```bash helm pull openziti/ziti-controller @@ -70,17 +72,7 @@ ZITI_NAMESPACE=ziti \ ./ziti-controller/files/chown-cert-manager.bash ``` -Now the cluster is ready for you to install cert-manager and trust-manager as separate releases before installing the v2 ziti-controller chart. - -The script performs an annotate and label operation on the cert-manager and trust-manager CRDs and the core resources created by trust-manager, each named for the release name (default `trust-manager`): `ClusterRole`, `ClusterRoleBinding`, `ValidatingWebhookConfiguration`, `Role`, `RoleBinding`. - -```bash -kubectl annotate crds bundles.trust.cert-manager.io --overwrite \ - meta.helm.sh/release-name="trust-manager" \ - meta.helm.sh/release-namespace="cert-manager" -kubectl label crds bundles.trust.cert-manager.io \ - app.kubernetes.io/managed-by=Helm -``` +You must install cert-manager and trust-manager Operators and CRDs before installing version 2 of the ziti-controller chart. See the [Custom Resources](#custom-resources) section above for an example. ## NodePort Service Example diff --git a/charts/ziti-controller/README.md.gotmpl b/charts/ziti-controller/README.md.gotmpl index 6c0dd93fa..70f6d631d 100644 --- a/charts/ziti-controller/README.md.gotmpl +++ b/charts/ziti-controller/README.md.gotmpl @@ -39,9 +39,11 @@ The deployment must have exactly one replica. ### Custom Resources -This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to build trust Bundle resources from Ziti's root CA cert(s). +This chart requires the custom resources provided by [cert-manager](https://cert-manager.io/docs/installation/) and [trust-manager](https://cert-manager.io/docs/projects/trust-manager/#installation), i.e., Issuer, Certificate, and Bundle. It is a limitation of Trust Manager to have one instance per cluster and one namespace from which trust Bundle inputs may be sourced, so a single Ziti controller may occupy the cluster unless your use case allows for controllers from multiple networks to share a namespace. You must set the Trust Manager's "trust namespace" to the namespace of the Ziti controller so that it will be able to compose a trust Bundle resource from Ziti's root CA cert(s). ```bash +helm repo add jetstack https://charts.jetstack.io + helm upgrade --install cert-manager jetstack/cert-manager \ --namespace cert-manager --create-namespace \ --set crds.enabled=true @@ -55,7 +57,7 @@ helm upgrade --install trust-manager jetstack/trust-manager \ ### Breaking Change -Version 2 of this chart introduces a breaking change. +Version 2 of this chart introduces a breaking change. You must decouple cert-manager and trust-manager from the Ziti controller chart if they were previously installed as subcharts. This allows them to be upgraded and configured independently of the Ziti controller chart. **Symptom** @@ -63,11 +65,11 @@ Version 2 of this chart introduces a breaking change. **Cause** -Cert Manager and Trust Manager are no longer included as subcharts. If you have a v1 version of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. +Cert Manager and Trust Manager are no longer included as subcharts. If you have version 1 of the chart installed and cert-manager or trust-manager were installed as subcharts in the same namespace as the Ziti controller, you can run `./files/chown-cert-manager.bash` to set the owner labels and annotations on existing cert-manager and trust-manager CRDs and resources, allowing them to be imported by a future cert-manager or trust-manager installation. **Solution** -Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in the "cert-manager" namespace, and your Ziti controller is installed in the "ziti" namespace, you can run the provided script to prep for installing the v2 chart. +Assuming your future CM release is named "cert-manager," your TM release is named "trust-manager," both are installed in namespace "cert-manager," and your Ziti controller is installed in namespace "ziti," you can run the provided script to pave the way to installing the version 2 ziti-controller chart. ```bash helm pull openziti/ziti-controller @@ -79,17 +81,7 @@ ZITI_NAMESPACE=ziti \ ./ziti-controller/files/chown-cert-manager.bash ``` -Now the cluster is ready for you to install cert-manager and trust-manager as separate releases before installing the v2 ziti-controller chart. - -The script performs an annotate and label operation on the cert-manager and trust-manager CRDs and the core resources created by trust-manager, each named for the release name (default `trust-manager`): `ClusterRole`, `ClusterRoleBinding`, `ValidatingWebhookConfiguration`, `Role`, `RoleBinding`. - -```bash -kubectl annotate crds bundles.trust.cert-manager.io --overwrite \ - meta.helm.sh/release-name="trust-manager" \ - meta.helm.sh/release-namespace="cert-manager" -kubectl label crds bundles.trust.cert-manager.io \ - app.kubernetes.io/managed-by=Helm -``` +You must install cert-manager and trust-manager Operators and CRDs before installing version 2 of the ziti-controller chart. See the [Custom Resources](#custom-resources) section above for an example. ## NodePort Service Example