Support for agentic coder cli (e.g. claude code, codex, gemini cli, open code)

[gltanaka]

github specify has a wrapper for these coders

this would be for the fix, crash and verify commands in case they can't fix the issue because it is outside a dev unit

[gltanaka]

https://github.com/github/spec-kit/tree/main/src/specify_cli

[jiaminc-cmu]

Hi! 👋 I’d like to take up this issue and work on adding support for the agentic coder CLI

[gltanaka]

Title: Integrate Agentic Coders as Fallback for crash, verify, and fix

Summary
Enable PDD to automatically invoke external “agentic coder” CLIs (via the Specify Python library) as a fallback when crash, verify, or fix fail within a single dev unit. This unlocks cross-dev-unit diagnosis and remediation when the root cause is outside the current dev unit.

Background

• Current behavior: PDD’s crash/verify/fix operate only within a single dev unit (prompt, code, example, test). About ~25% of failures require changes outside the dev unit, forcing manual use of external coders (e.g., Codex, Gemini CLI, Cloud Code).
• Goal: When built-in attempts exhaust (N retries) without resolving, PDD should automatically call an agentic coder (non-interactive) to:
  • Perform root cause analysis (RCA)
  • Propose and apply changes across files/dev units
  • Re-run the failing command loop and converge

Scope

• Commands affected: crash, verify, fix
• In-scope:
  • Add fallback pipeline triggered after internal attempts fail
  • Integrate Specify library to call multiple agent coders (Codex, Gemini CLI, Cloud Code, etc.)
  • Prompt design for RCA and multi-step remediation
  • Config, logging, and reproducibility
• Out-of-scope:
  • Changing behavior of other PDD commands
  • Interactive agent sessions (must be non-interactive)

User Stories

• As a contributor, when verify fails after internal retries, PDD automatically escalates to an agentic coder that can modify files outside the dev unit, then re-runs verify.
• As a maintainer, I can configure which agent(s) to use and their order, retry limits, and timeouts.
• As a developer, I can inspect logs/artifacts to understand what the agent changed and why.

Functional Requirements

1. Fallback Trigger

• Each command (crash/verify/fix) keeps current internal retry loop (configurable: internal.max_attempts).
• On failure, invoke fallback when:
  • Attempts exhausted OR a deterministic category is detected (e.g., missing dependency/file, cross-module import, version mismatch).
• Configurable bypass: allow users to disable or force fallback.

2. Agent Selection and Invocation

• Use Specify Python SDK to abstract agent CLIs.
• Selection order: config-driven list (e.g., ["specify.codex", "specify.gemini", "specify.claude"]) with per-agent enable flags.
• Per-agent config: API keys (via env), model, temperature, max_tokens, timeouts, max_rounds, cost caps.

3. Prompting Strategy

• Primary prompt goals:
  a) Summarize the failing context and suspected scope
  b) Perform RCA
  c) Propose minimal safe changes
  d) Apply changes (multi-file allowed), preserving project conventions
  e) Request re-run (PDD will re-run outside the agent)

4. Change Application

• PDD applies agent-proposed changes to workspace:
  • Use safe writes (atomic temp files), validate paths stay within repo, deny unsafe actions (e.g., editing secrets).
  • Optionally gate significant changes behind a confirm flag (default: auto-apply for non-interactive).
• After applying, PDD re-runs the original command loop from step 0.

5. Re-run and Convergence

• After agent changes, re-run crash/verify/fix from the beginning.
• Cap total fallback cycles (config: fallback.max_cycles).
• If success: exit 0, record success in logs.
• If still failing: either try next agent or exit with summarized failure.

6. Logging and Artifacts

• Log directory per run: .pdd/logs/{timestamp}/{command}/
  • internal_attempts/*.log
  • fallback/{agent}/prompt_step_{n}.txt
  • fallback/{agent}/output.json
  • applied_changes/*.diff
  • final_status.json
• Include agent name, version, and model for reproducibility.

7. Configuration (example, probably not correct)

• pdd.config.yaml additions:
  agent_fallback:
  enabled: true
  trigger_on: ["attempts_exhausted", "cross_unit_signals"]
  internal:
  max_attempts: 3
  fallback:
  max_cycles: 2
  agents:
  - name: specify.codex
  enabled: true
  model: "gpt-4o-mini" # example
  temperature: 0.2
  max_tokens: 4000
  timeout_s: 120
  - name: specify.gemini
  enabled: false
  safety:
  allow_file_create: true
  allow_file_delete: false
  path_allowlist: ["src/", "tests/", "examples/"]
  path_blocklist: [".git/", "secrets/"]

• Environment variables for credentials: documented mapping per agent via Specify.

• CLI flags:

  • --no-agent-fallback
  • --agent-fallback-only (debug)
  • --agent-order=codex,gemini
  • --agent-timeout=120
  • --agent-max-cycles=2

8. Security and Safety

• Path allow/block lists enforced.
• Max size for files provided to agent; large files summarized.
• Strip secrets and .env values from prompts.
• Cost/time guardrails with fail-fast messages.

9. Dev Unit and Prompt Updates

• For now, do not require updating proprietary prompts; rely on README-driven behavior.
• If agent changes alter behavior, generate/update unit tests and examples as needed to keep dev unit consistent.
• Ensure changes don’t silently drift from README; if behavior change detected, emit a notice to update README in subsequent PR.

Non-Functional Requirements

• Backward compatible: default behavior unchanged unless enabled.
• Performance: Fallback adds overhead only on failure paths; target < 2 minutes per fallback cycle for typical repos.
• Reliability: Deterministic re-runs when same codebase and config.
• Observability: Clear logs; exit codes unchanged (0 success, non-zero on failure).

Acceptance Criteria

• Given a cross-dev-unit failure in verify, when internal attempts exhaust, PDD invokes the first enabled agent via Specify, applies proposed changes safely, and verify completes successfully; logs include agent artifacts and applied diffs.
• Given crash failure due to missing dependency outside dev unit, fallback adds dependency config/code and crash passes on re-run.
• Given fix failure requiring edits in another module, fallback modifies multiple files and fix passes; unit tests updated if necessary.
• Config flags properly enable/disable fallback and select agents.
• Safety rules prevent edits in blocked paths and redact secrets from prompts.
• Comprehensive logs are produced for internal and fallback stages.

Implementation Plan

• Phase 1: Plumbing

  • Add fallback controller and config schema
  • Wrap internal loops for crash/verify/fix with fallback trigger
  • Introduce logging layout

• Phase 2: Specify Integration

  • Add Specify Python client abstraction
  • Implement agent selection and invocation
  • Define prompt templates and output schema

• Phase 3: Change Application + Re-run

  • Implement safe apply for create/modify/delete/move
  • Integrate re-run cycle with caps and agent rotation

• Phase 4: Hardening

  • Add safety checks, redaction, path guards
  • Add time/cost caps and telemetry

• Phase 5: Tests and Docs

  • E2E scenarios: cross-unit missing file, wrong import path, version mismatch
  • Unit tests for selection, prompting, apply, re-run
  • README updates and example configs

[jiaminc-cmu]

Hi,
I checked Specify‘s repo. It’s a CLI that scaffolds specs and drives interactive slash flows inside agent terminals; there’s no Python SDK for headless calls. We’d still need to spawn agent CLIs via subprocess/PTY to run non-interactive fixes, so adding Specify wouldn’t remove work—it would add another tool to install and maintain.

Plan: ship a small native fallback:

Spawn configured agent CLIs (Gemini/Claude/etc.), send a single non-interactive prompt, apply changes behind path allowlists, then re-run crash/verify/fix. Keep logs/diffs under .pdd/logs/… for auditability.

This gives us fewer moving parts, easier CI, and full control over safety/telemetry—without introducing a CLI layer we’d still have to script.

[gltanaka]

I know they don't have an API, but you can do a direct Python call or maybe just reference the code in the prompt so you don't have to start from scratch

[gltanaka]

almost done with fix

what's left is verify and crash target 10/16