Facilitate and document local test

- Add some Makefile variables for more granular construction of the
build image URI to facilitate simpler overriding for personal builds.
- Document the process to build, tag, push, and deploy a personal build
to a personal cluster.

Author: 2uasimojo

Makefile

@@ -5,7 +5,9 @@ GIT_HASH := $(shell git rev-parse --short=7 HEAD)
 IMAGETAG ?= ${GIT_HASH}
 
 BASE_IMG ?= managed-cluster-validating-webhooks
-IMG ?= quay.io/app-sre/${BASE_IMG}
+IMG_REGISTRY ?= quay.io
+IMG_ORG ?= app-sre
+IMG ?= $(IMG_REGISTRY)/$(IMG_ORG)/${BASE_IMG}
 
 # nb: registry.svc.ci.openshift.org/openshift/release:golang-1.14 doesn't work for this
 SYNCSET_GENERATOR_IMAGE := quay.io/app-sre/golang:1.14
@@ -105,6 +107,7 @@ skopeo-push:
 .PHONY: push-base
 push-base: build/Dockerfile
 	$(CONTAINER_ENGINE) push $(IMG):$(IMAGETAG)
+	$(CONTAINER_ENGINE) push $(IMG):latest
 
 coverage: coverage.txt
 coverage.txt: vet $(GO_SOURCES)

README.md

@@ -2,6 +2,25 @@
 
 A framework supporting validating webhooks for OpenShift.
 
+- [Managed Cluster Validating Webhooks](#managed-cluster-validating-webhooks)
+  - [Updating SelectorSyncSet Template](#updating-selectorsyncset-template)
+  - [Development](#development)
+    - [Adding New Webhooks](#adding-new-webhooks)
+    - [Helper Utils](#helper-utils)
+  - [Is The Request Valid and Authorized](#is-the-request-valid-and-authorized)
+    - [Building a Response](#building-a-response)
+    - [Sending Responses](#sending-responses)
+    - [Writing Unit Tests](#writing-unit-tests)
+    - [Local Live Testing](#local-live-testing)
+      - [Create a Repository](#create-a-repository)
+      - [Build and Push the Image](#build-and-push-the-image)
+      - [Pare Down Your Daemonset (Optional)](#pare-down-your-daemonset-optional)
+      - [Deploy the Image](#deploy-the-image)
+      - [Test Your Changes](#test-your-changes)
+    - [End to End Testing](#end-to-end-testing)
+  - [Disabling Webhooks](#disabling-webhooks)
+    - [Removing a Webhook](#removing-a-webhook)
+
 ## Updating SelectorSyncSet Template
 
 Ensure the git branch is current and run `make syncset`. The updated Template will be  [build/selectorsyncset.yaml](build/selectorsyncset.yaml) by default.
@@ -104,7 +123,7 @@ It is important to retain the UID from the incoming request with the outgoing re
 
 Once a [response is built](#building-a-response), it must be sent back to the HTTP client. This is done by returning the `admissionctl.Response` in the `Authorized` method. This structure can be [built up with the helpers mentioned above](#building-a-response).
 
-### Writing Tests
+### Writing Unit Tests
 
 Unit tests are important to ensure that webhooks behave as expected, and to help with that process, there is a [testutils](pkg/testutils/testutils.go) helper package designed to unify several processes. The package exports:
 
@@ -117,6 +136,117 @@ The first function, `CanCanNot`, is very simple and designed to make test failur
 
 The three helper functions are intended to provide for more integration style tests than true unit tests, as they assist in turning a specific set of test criteria a JSON representation and sending via `net/http/httptest` to the webhook's `Authorized`. When using `testutils.SendHTTPRequest`, the response is a `Response` object that can be used in the test suite to access the result of the webhook.
 
+### Local Live Testing
+
+Build and test your changes against your own cluster.
+Here is a [recorded demo](https://drive.google.com/file/d/1UaMz-siFDRaSKPVKxjLOneqGgbBuwpBF/view) of this process.
+
+#### Create a Repository
+
+Make sure you have a repository to host the image you are going to build.
+It must be **public** so your cluster is able to download the image.
+For subsequent steps, you will need the name of the registry, the organization, and the repository.
+For example, in the image URI `quay.io/my-user/managed-cluster-validating-webhooks:latest`:
+- `quay.io` is the *registry*
+- `my-user` is the *organization*
+- `managed-cluster-validating-webhooks` is the *repository*
+- `latest` is the *image tag*
+
+#### Build and Push the Image
+
+Use `make build-base` to build and tag the image; and `make push-base` to push it.
+In order to use your personal repository, you can override any of the components of this URI by setting the following variables:
+- `IMG_REGISTRY` overrides the *registry* (default: `quay.io`)
+- `IMG_ORG` overrides the *organization* (default: `app-sre`)
+- `BASE_IMG` overrides the *repository name* (default: `managed-cluster-validating-webhooks`)
+- `IMAGETAG` overrides the *image tag*. (By default this is the current commit hash of your local clone of the git repository; but `make build-base` will also always tag `latest`)
+
+For example, to build, tag, and push `quay.io/my-user/managed-cluster-validating-webhooks:latest`, you can run:
+
+```
+make IMG_ORG=my-user build-base push-base
+```
+
+#### Pare Down Your Daemonset (Optional)
+
+**Note:** Editing the daemonset requires elevated privileges.
+
+By default, the image will run on all master nodes.
+For testing purposes, you may find it easier to run on only one node.
+To facilitate this:
+1. Pick a master node to use. Any one will do. For example:
+   ```
+   $ oc get node | grep master | head -1
+   ip-10-0-147-186.ec2.internal   Ready    master         3h44m   v1.19.0+f173eb4
+   ```
+2. Give the node a unique label. For example:
+   ```
+   $ oc label node ip-10-0-147-186.ec2.internal mcvw-test=true
+   node/ip-10-0-147-186.ec2.internal labeled
+   ```
+3. Edit the `validation-webhook` daemonset's `.spec.template.spec.affinity.nodeAffinity`, replacing the `matchExpressions` entry for `node-role.kubernetes.io/master` to match your label instead.
+   For example, to match the label from step 2, run:
+   ```
+   oc edit -n openshift-validation-webhook daemonset validation-webhook
+   ```
+   and replace
+   ```
+           nodeAffinity:
+             requiredDuringSchedulingIgnoredDuringExecution:
+               nodeSelectorTerms:
+               - matchExpressions:
+                 - key: node-role.kubernetes.io/master
+                   operator: In
+                   values:
+                   - ""
+   ```
+   with
+   ```
+           nodeAffinity:
+             requiredDuringSchedulingIgnoredDuringExecution:
+               nodeSelectorTerms:
+               - matchExpressions:
+                 - key: mcvw-test   # <== your label's key
+                   operator: In
+                   values:
+                   - "true"         # <== your label's value
+   ```
+
+Once these changes are saved, you should see `validation-webhook-*` pods cycle.
+When they have settled, you can confirm that only one pod is running and that it is running on your desired node.
+For example:
+
+```
+$ oc describe pod -n openshift-validation-webhook | grep ^Node:
+Node:         ip-10-0-147-186.ec2.internal/10.0.147.186
+```
+
+#### Deploy the Image
+
+**Note:** Editing the daemonset requires elevated privileges.
+
+Edit the `validation-webhook` daemonset's `.spec.template.spec.containers[0].image`, replacing it with the URI of the image you built and pushed [above](#build-and-push-the-image).
+For example, if you created `quay.io/my-user/managed-cluster-validating-webhooks:latest`, replace
+```
+        image: quay.io/app-sre/managed-cluster-validating-webhooks:a324838
+```
+with
+```
+        image: quay.io/my-user/managed-cluster-validating-webhooks:latest
+```
+
+Once these changes are saved, you should see `validation-webhook-*` pods cycle.
+When they have settled, you can confirm that the pod(s) are running with your image.
+For example:
+```
+$ oc describe pod -n openshift-validation-webhook | grep '^ *Image:'
+    Image:         quay.io/my-user/managed-cluster-validating-webhooks:latest
+```
+
+#### Test Your Changes
+
+Now that your cluster is running your modified code, you can do whatever is necessary to validate your changes.
+
 ### End to End Testing
 
 End to End testing is managed by the [osde2e repo](https://github.com/openshift/osde2e/)