Skip to content

Commit c77b80c

Browse files
authored
Merge pull request #12 from shiftweek/move-ztwim-inside-oape
Move ztwim-test-generator inside oape plugin
2 parents 3c652fd + 48302fa commit c77b80c

File tree

12 files changed

+904
-2
lines changed

12 files changed

+904
-2
lines changed

README.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,9 @@ ln -s oape-ai-e2e ~/.cursor/commands/oape-ai-e2e
4343

4444
## Available Plugins
4545

46-
| Plugin | Description | Commands |
46+
| Plugin | Description | Commands |
4747
| ------------------------- | ---------------------------------------------- | --------------------------------------------------------------------------- |
48-
| **[oape](plugins/oape/)** | AI-driven OpenShift operator development tools | `/oape:api-generate`, `/oape:api-generate-tests`, `/oape:api-implement` |
48+
| **[oape](plugins/oape/)** | AI-driven OpenShift operator development tools (includes ZTWIM test generator) | `/oape:api-generate`, `/oape:api-generate-tests`, `/oape:api-implement`, `/oape:ztwim-generate-all`, `/oape:ztwim-generate-from-pr`, `/oape:ztwim-generate-execution-steps`, `/oape:ztwim-generate-e2e-from-pr` |
4949

5050
## Commands
5151

@@ -85,6 +85,20 @@ Reads an OpenShift enhancement proposal PR, extracts the required implementation
8585
/oape:api-implement https://github.com/openshift/enhancements/pull/1234
8686
```
8787

88+
### ZTWIM Test Generator (inside oape)
89+
90+
Generates test scenarios, step-by-step execution with `oc` commands, and e2e Go code for [openshift/zero-trust-workload-identity-manager](https://github.com/openshift/zero-trust-workload-identity-manager) PRs. See [plugins/oape/ztwim-test-generator/README.md](plugins/oape/ztwim-test-generator/README.md) for full docs.
91+
92+
**Single command (all artifacts):**
93+
94+
```shell
95+
/oape:ztwim-generate-all https://github.com/openshift/zero-trust-workload-identity-manager/pull/92
96+
```
97+
98+
Writes `test-cases.md`, `execution-steps.md`, `<prno>_test_e2e.go`, and `e2e-suggestions.md` into `output/ztwim_pr_<number>/`.
99+
100+
**Individual commands:** `/oape:ztwim-generate-from-pr`, `/oape:ztwim-generate-execution-steps`, `/oape:ztwim-generate-e2e-from-pr` (each with a PR URL).
101+
88102
### Adding a New Command
89103

90104
1. Add a new markdown file under `plugins/oape/commands/`
@@ -95,6 +109,7 @@ Reads an OpenShift enhancement proposal PR, extracts the required implementation
95109

96110
```text
97111
plugins/oape/
112+
├── ztwim-test-generator/ # ZTWIM fixtures, docs, skills (commands are in commands/)
98113
├── .claude-plugin/
99114
│ └── plugin.json # Required: plugin metadata
100115
├── commands/

plugins/oape/README.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,21 @@ Performs a "Principal Engineer" level code review that verifies code changes aga
8787
4. **Generates Report** -- Returns structured JSON with verdict, issues, and fix prompts
8888
5. **Applies Fixes Automatically** -- When issues are found, invokes `implement-review-fixes.md` to apply the suggested code changes in severity order (CRITICAL first), then verifies the build still passes
8989

90+
---
91+
92+
### ZTWIM Test Generator (ZTWIM operator PRs only)
93+
94+
Generates test scenarios, execution steps with `oc` commands, and e2e Go code for [openshift/zero-trust-workload-identity-manager](https://github.com/openshift/zero-trust-workload-identity-manager) PRs. Fixtures and docs live under `ztwim-test-generator/`; commands are exposed as `/oape:ztwim-*`.
95+
96+
| Command | Description |
97+
|---------|-------------|
98+
| **`/oape:ztwim-generate-all <pr-url>`** | Generate all artifacts in one run: `test-cases.md`, `execution-steps.md`, `<prno>_test_e2e.go`, `e2e-suggestions.md` in `output/ztwim_pr_<number>/`. |
99+
| `/oape:ztwim-generate-from-pr <pr-url>` | Generate only test scenarios (`test-cases.md`). |
100+
| `/oape:ztwim-generate-execution-steps <pr-url>` | Generate only execution steps (`execution-steps.md`). |
101+
| `/oape:ztwim-generate-e2e-from-pr <pr-url>` | Generate only e2e Go code and suggestions. |
102+
103+
See [ztwim-test-generator/README.md](ztwim-test-generator/README.md) for fixtures and usage.
104+
90105
## Prerequisites
91106

92107
- **gh** (GitHub CLI) -- installed and authenticated
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
description: Generate all ZTWIM PR test artifacts in one run (test scenarios, execution steps, e2e Go code)
3+
argument-hint: "<pr-url> [--output <path>] [--env <cluster-type>]"
4+
---
5+
6+
## Name
7+
oape:ztwim-generate-all
8+
9+
## Synopsis
10+
```
11+
/oape:ztwim-generate-all <pr-url> [--output <path>] [--env <cluster-type>]
12+
```
13+
14+
## Description
15+
16+
**Single command** that analyzes a ZTWIM operator Pull Request and generates **all three outputs** in one run:
17+
18+
1. **test-cases.md** — Test scenarios (what to test, PR-specific focus, verification, cleanup).
19+
2. **execution-steps.md** — Step-by-step procedure with executable `oc` commands (prerequisites, install, stack, verify, PR-specific steps, cleanup).
20+
3. **\<prno\>_test_e2e.go** — Go e2e test code (Ginkgo Describe/Context/It) for the ZTWIM repo `test/e2e/`, with operator/operand scenarios and PR-specific tests.
21+
22+
All files are written into **one output directory**: `<output-dir>/ztwim_pr_<number>/`. Default `<output-dir>` is `output` (create if missing). Use `--output <path>` to set a different base.
23+
24+
- **Repository**: openshift/zero-trust-workload-identity-manager only.
25+
- **Install/Stack**: Use plugin fixtures (`plugins/oape/ztwim-test-generator/fixtures/operator-install.yaml`, `plugins/oape/ztwim-test-generator/fixtures/ztwim-stack.yaml`). Do not discover from repo.
26+
- **E2E**: Follow upstream test/e2e structure; see plugin [docs/e2e-structure.md](../ztwim-test-generator/docs/e2e-structure.md) and [fixtures/e2e-important-scenarios.md](../ztwim-test-generator/fixtures/e2e-important-scenarios.md).
27+
28+
## Implementation
29+
30+
### Step 1: Validate PR and Analyze Changes (Once)
31+
32+
1. **Validate PR URL** is for ZTWIM: `https://github.com/openshift/zero-trust-workload-identity-manager/pull/<number>`. If not, inform the user this command is for ZTWIM PRs only.
33+
2. **Use browser tools**: Navigate to the PR URL, then to "Files changed" (append `/files`). Use **browser_snapshot** to read PR description and changed files. Do **not** use `gh` CLI.
34+
3. Extract PR number and (optionally) a short description from the title. Map changed files to:
35+
- Test focus (API types, CRD, controller, RBAC, samples, e2e).
36+
- E2E focus (operator/operand scenarios, PR-specific It blocks).
37+
38+
### Step 2: Output Directory
39+
40+
- **Path**: `<output-dir>/ztwim_pr_<number>/` (e.g. `output/ztwim_pr_72/`).
41+
- **Default `<output-dir>`**: `output` (relative to workspace root). Create the directory if it does not exist.
42+
- **With `--output <path>`**: Use `<path>` as the base; write into `<path>/ztwim_pr_<number>/`.
43+
- **With `--env`**: Use only when generating execution-steps content (e.g. env-specific notes); optional.
44+
45+
### Step 3: Generate test-cases.md
46+
47+
Write **test-cases.md** into the output directory. Content must include:
48+
49+
- Operator info (ZTWIM), repository.
50+
- Prerequisites (cluster, env vars: APP_DOMAIN, JWT_ISSUER_ENDPOINT, CLUSTER_NAME).
51+
- Install: fixture path, `oc apply -f .../operator-install.yaml`, wait for CSV and deployment.
52+
- Stack: fixture path, envsubst, `envsubst < .../ztwim-stack.yaml | oc apply -f -`.
53+
- PR-specific test cases derived from Files changed (field tests, controller tests, validation).
54+
- Verification (oc get CRs, oc wait, oc logs in zero-trust-workload-identity-manager).
55+
- Cleanup order: SpireOIDCDiscoveryProvider → SpiffeCSIDriver → SpireAgent → SpireServer → ZeroTrustWorkloadIdentityManager → subscription → CSV → OperatorGroup → namespace.
56+
57+
### Step 4: Generate execution-steps.md
58+
59+
Write **execution-steps.md** into the output directory. Content must include:
60+
61+
- Prerequisites: `which oc`, `oc version`, `oc whoami`, `oc get nodes`, `oc get clusterversion`, packagemanifests check, then APP_DOMAIN/JWT_ISSUER_ENDPOINT/CLUSTER_NAME and echo.
62+
- Install: `oc apply -f <path-to>/operator-install.yaml`, then oc wait for CSV and deployment, oc get pods.
63+
- Stack: envsubst and oc apply for ztwim-stack.yaml.
64+
- CR verification: oc get all ZTWIM CRs, oc wait for Ready.
65+
- PR-specific execution steps (from Files changed).
66+
- Cleanup: full oc delete sequence in fixed order.
67+
68+
Use plugin fixture paths; document `<path-to>` as `plugins/oape/ztwim-test-generator/fixtures`.
69+
70+
### Step 5: Generate \<prno\>_test_e2e.go and e2e-suggestions.md
71+
72+
**e2e-suggestions.md** (in the same output directory):
73+
74+
- Short list of which operator/operand e2e scenarios apply.
75+
- Which are highly recommended for this PR.
76+
- PR-specific It block suggestions.
77+
78+
**\<prno\>_test_e2e.go** (in the same output directory):
79+
80+
- Package `e2e`; same imports and style as upstream (see [docs/e2e-structure.md](../ztwim-test-generator/docs/e2e-structure.md), [fixtures/e2e-sample_test.go.example](../ztwim-test-generator/fixtures/e2e-sample_test.go.example)).
81+
- Use `k8sClient`, `clientset`, `testCtx`, `utils.*`; no BeforeSuite/TestE2E.
82+
- Describe/Context/It blocks with `By("…")`; comment each It (e.g. `// PR-suggested: ...`) for pick-and-choose.
83+
- Include important scenarios (install, recovery, operand conditions, ZTWIM aggregation, OperatorCondition Upgradeable, CR-driven config) and PR-specific tests from Files changed.
84+
85+
### Step 6: Confirm Output
86+
87+
Tell the user the output directory path and list the three (or four) generated files: test-cases.md, execution-steps.md, \<prno\>_test_e2e.go, and optionally e2e-suggestions.md.
88+
89+
## Arguments
90+
91+
- **$1 (pr-url)**: ZTWIM operator GitHub PR URL — `https://github.com/openshift/zero-trust-workload-identity-manager/pull/<number>`.
92+
- **--output**: Output base directory (optional). Default: `output`. All files go in `<output>/ztwim_pr_<number>/`.
93+
- **--env**: Target environment for execution-steps (optional): `aws`, `gcp`, `azure`, `vsphere`, `baremetal`.
94+
95+
## Examples
96+
97+
```
98+
/oape:ztwim-generate-all https://github.com/openshift/zero-trust-workload-identity-manager/pull/72
99+
# Writes: output/ztwim_pr_72/test-cases.md
100+
# output/ztwim_pr_72/execution-steps.md
101+
# output/ztwim_pr_72/72_test_e2e.go
102+
# output/ztwim_pr_72/e2e-suggestions.md
103+
104+
/oape:ztwim-generate-all https://github.com/openshift/zero-trust-workload-identity-manager/pull/72 --output .work
105+
# Writes: .work/ztwim_pr_72/test-cases.md, .work/ztwim_pr_72/execution-steps.md, .work/ztwim_pr_72/72_test_e2e.go, .work/ztwim_pr_72/e2e-suggestions.md
106+
```
107+
108+
## Notes
109+
110+
- **ZTWIM only**: For openshift/zero-trust-workload-identity-manager PRs only.
111+
- **Single PR analysis**: Browser is used once for the PR and Files changed; the same analysis drives all three outputs.
112+
- **Fixtures**: Install and stack content come from `plugins/oape/ztwim-test-generator/fixtures`; do not discover from repo.
Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
---
2+
description: Suggest e2e tests and generate Go code for ZTWIM operator PRs from upstream e2e structure
3+
argument-hint: "<pr-url> [--output <path>]"
4+
---
5+
6+
## Name
7+
oape:ztwim-generate-e2e-from-pr
8+
9+
## Synopsis
10+
```
11+
/oape:ztwim-generate-e2e-from-pr <pr-url> [--output <path>]
12+
```
13+
14+
## Description
15+
16+
Analyzes a **ZTWIM** operator Pull Request and (1) suggests which e2e tests to add based on the upstream [test/e2e](https://github.com/openshift/zero-trust-workload-identity-manager/tree/main/test/e2e) structure, and (2) generates **Go e2e test code** in a file **`<prno>_test_e2e.go`** (e.g. `123_test_e2e.go`) that you can copy from into the ZTWIM repo. Tests are written with **operator and operand context** and include **important scenarios** that should be highly checked.
17+
18+
- **Repository**: openshift/zero-trust-workload-identity-manager only.
19+
- **Upstream e2e**: test/e2e (e2e_suite_test.go, e2e_test.go, utils). See plugin [docs/e2e-structure.md](../ztwim-test-generator/docs/e2e-structure.md).
20+
21+
## Implementation
22+
23+
### Step 1: Validate PR and Analyze Changes
24+
25+
1. **Validate PR URL** is for ZTWIM: `https://github.com/openshift/zero-trust-workload-identity-manager/pull/<number>`. If not, inform the user this command is for ZTWIM PRs only.
26+
2. **Use browser tools**: Navigate to the PR, then to "Files changed" (append `/files` to PR URL). Use **browser_snapshot** to read changed files and paths.
27+
28+
Map file patterns to e2e focus:
29+
30+
| File Pattern | E2E Focus |
31+
|--------------|-----------|
32+
| `api/**/*_types.go` | New/updated CR fields; consider new or updated It specs for CR create/update and condition checks. |
33+
| `*controller*.go`, `*reconcile*.go` | Reconciliation and operand lifecycle; OperatorCondition; recovery tests. |
34+
| `config/crd/**` | Schema/validation; optional negative It (invalid CR). |
35+
| `config/samples/*.yaml` | Example CR usage; align generated test CRs with samples. |
36+
| `test/e2e/**` | Follow existing Describe/Context/It style; suggest similar or extended scenarios. |
37+
38+
### Step 2: Use Upstream E2E Structure
39+
40+
Read this plugin's [docs/e2e-structure.md](../ztwim-test-generator/docs/e2e-structure.md). Generated code must:
41+
42+
- **Package**: `e2e`.
43+
- **Imports**: Same as upstream (e.g. `context`, `fmt`, Ginkgo, `operatorv1alpha1`, `test/e2e/utils`, `corev1`, `metav1`, `client`, etc.). Do not add unused imports.
44+
- **Clients**: Use suite-level `k8sClient`, `clientset`, `apiextClient`, `configClient` and per-test `testCtx` (from `BeforeEach` with timeout). Assume these exist; do not redefine.
45+
- **Helpers**: Use `utils.*` (e.g. `utils.OperatorNamespace`, `utils.WaitForSpireServerConditions`, `utils.WaitForDeploymentAvailable`). Match names from upstream constants.go and utils.go.
46+
- **Style**: `Describe` / `Context` / `It`; `By("…")` for steps; `DeferCleanup` for teardown; `Eventually` with `WithTimeout`/`WithPolling` where appropriate. For a code-style reference, see plugin fixture [fixtures/e2e-sample_test.go.example](../ztwim-test-generator/fixtures/e2e-sample_test.go.example). For a scenario checklist, see [fixtures/e2e-important-scenarios.md](../ztwim-test-generator/fixtures/e2e-important-scenarios.md).
47+
48+
### Step 3: Suggest E2E Tests (Summary)
49+
50+
Produce a short **suggestion list** for the PR:
51+
52+
- Which **operator** scenarios apply (install, CRDs, pod recovery, log level via Subscription).
53+
- Which **operand** scenarios apply (SpireServer, SpireAgent, SpiffeCSIDriver, SpireOIDCDiscoveryProvider: create, conditions, Ready).
54+
- **ZTWIM** aggregation (operand status, Ready).
55+
- **OperatorCondition** Upgradeable (True when healthy; False when operand down; recovery).
56+
- **CR-driven config** (resources, nodeSelector, tolerations, affinity, log level) for any CR touched by the PR.
57+
- Any **PR-specific** It blocks (new fields, new validation, changed behavior).
58+
59+
Mark which scenarios are **highly recommended** for this PR given the Files changed.
60+
61+
### Step 4: Generate `<prno>_test_e2e.go`**
62+
63+
Generate a single Go file named **`<prno>_test_e2e.go`** (e.g. `123_test_e2e.go`). The file must be **self-contained** for the package `e2e` and assume the suite (e2e_suite_test.go) and utils are present.
64+
65+
- **Header**: Standard Apache-2.0 copyright and package `e2e`.
66+
- **Imports**: Only what is needed; match upstream style.
67+
- **Structure**: One or more `Describe` or `Context` blocks. Each `It` should be **commented** with a short line (e.g. `// PR-suggested: operator recovery`) so the user can easily pick and choose which tests to copy into `e2e_test.go` or keep in this file.
68+
- **Content**: Include both (a) **important scenarios** from [docs/e2e-structure.md](../ztwim-test-generator/docs/e2e-structure.md) that are relevant to the PR, and (b) **PR-specific** tests derived from Files changed. Prefer reusing existing utils and condition names; if the PR adds new API fields or conditions, generate plausible code and add a comment that the condition/field name may need to match the actual API.
69+
- **No duplicate suite logic**: Do not define `BeforeSuite`, `TestE2E`, or client setup; only test blocks that run inside the existing suite.
70+
71+
### Step 5: Important Scenarios to Include (when relevant)
72+
73+
Always consider including or suggesting tests for:
74+
75+
1. Operator installed; all managed CRDs Established; operator Deployment Available.
76+
2. Operator recovers from force pod deletion (new pod Running, deployment Available again).
77+
3. ZeroTrustWorkloadIdentityManager created with trust domain, cluster name, bundle ConfigMap.
78+
4. SpireServer / SpireAgent / SpiffeCSIDriver / SpireOIDCDiscoveryProvider created and respective conditions (e.g. StatefulSetAvailable, DaemonSetAvailable, DeploymentAvailable, Ready) True.
79+
5. ZeroTrustWorkloadIdentityManager aggregates 4 operands; each operand Ready in status.
80+
6. OperatorCondition Upgradeable: True when healthy; False when an operand pod is deleted; True again after recovery.
81+
7. CR-driven configuration: SpireServer (or other operand) resources, nodeSelector, tolerations, affinity, log level; operator log level via Subscription env.
82+
83+
Generate at least 2–3 **highly recommended** It blocks that directly exercise code or CRs touched by the PR.
84+
85+
### Step 6: Output
86+
87+
**All generated files go inside a single output directory.**
88+
89+
- **Output directory**: `<output-dir>/ztwim_pr_<number>/` (e.g. `output/ztwim_pr_123/`).
90+
- **Default `<output-dir>`**: `output` (relative to workspace root). Create it if it does not exist.
91+
- **With `--output <path>`**: Use `<path>` as the output base; write into `<path>/ztwim_pr_<number>/`.
92+
93+
- **Suggestion list**: Write **e2e-suggestions.md** inside that directory (and optionally show in reply). Content: which operator/operand scenarios apply, highly recommended tests, PR-specific It blocks.
94+
- **Go file**: Write **`<prno>_test_e2e.go`** inside that directory (e.g. `output/ztwim_pr_123/123_test_e2e.go`). Tell the user they can copy this file into the ZTWIM repo under `test/e2e/` or copy individual It blocks into `e2e_test.go`.
95+
96+
## Arguments
97+
98+
- **$1 (pr-url)**: ZTWIM operator GitHub PR URL — `https://github.com/openshift/zero-trust-workload-identity-manager/pull/<number>`.
99+
- **--output**: Output base directory (optional). Default: `output`. Generated files go in `<output>/ztwim_pr_<number>/`.
100+
101+
## Examples
102+
103+
```
104+
/oape:ztwim-generate-e2e-from-pr https://github.com/openshift/zero-trust-workload-identity-manager/pull/123
105+
# Writes: output/ztwim_pr_123/123_test_e2e.go, output/ztwim_pr_123/e2e-suggestions.md
106+
107+
/oape:ztwim-generate-e2e-from-pr https://github.com/openshift/zero-trust-workload-identity-manager/pull/123 --output .work
108+
# Writes: .work/ztwim_pr_123/123_test_e2e.go, .work/ztwim_pr_123/e2e-suggestions.md
109+
```
110+
111+
## Notes
112+
113+
- **ZTWIM only**: For openshift/zero-trust-workload-identity-manager PRs only.
114+
- **Browser**: Use browser tools for PR and Files changed; do not rely on gh CLI for file list.
115+
- **Pick-and-choose**: The generated Go file is meant for choosing which tests to add; comments in the file should make it easy to copy only the needed blocks.

0 commit comments

Comments
 (0)