Fix undeclared concept removal and prune unreachable specs#654
Fix undeclared concept removal and prune unreachable specs#654lchoquel wants to merge 3 commits intofeature/Build-PipeComposefrom
Conversation
Remove the flawed fixed_pipe_parallel_concepts optimization that prematurely removed concepts from the undeclared set even when referenced elsewhere (e.g., another pipe's input), wasting a fix-loop iteration. Add _prune_unreachable_specs method that walks the call graph from main_pipe to remove unreachable pipes and transitively unused concepts, cleaning up dead weight after all fixes are applied. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d28946ce9d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
After Step 3 nulls out combined_output and sets output to "Anything" on PipeParallel specs, the undeclared set was never reduced, causing Step 4 to trigger the LLM pipeline unnecessarily for concepts whose only references were just eliminated. Add _collect_local_bare_concept_codes helper that scans all pipe specs and concept definitions for locally-referenced bare concept codes, and use it after Step 3 to remove concepts that are no longer referenced from the undeclared set before the LLM call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Code reviewFound 1 issue:
Step C (no domain check): pipelex/pipelex/builder/builder_loop.py Lines 333 to 352 in daf1843
pipelex/pipelex/builder/builder_loop.py Lines 414 to 427 in daf1843 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
External domain references (e.g., "external.Document") were incorrectly marking local concepts with the same bare name as referenced, preventing them from being pruned. Extract a shared _extract_local_bare_code helper that filters by domain and apply it in Steps C and D. Also move reachable_pipes.add() after the existence check in Step A so non-existent pipes don't silently pollute the reachable set, and log a warning instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| bare_ref = self._extract_local_bare_code(concept_ref_or_code=field_spec.item_concept_ref, domain=domain) | ||
| if bare_ref is not None and bare_ref not in referenced_concepts and bare_ref in pipelex_bundle_spec.concept: | ||
| referenced_concepts.add(bare_ref) | ||
| changed = True |
There was a problem hiding this comment.
Duplicated concept-reference scanning across new methods
Low Severity
The concept-reference scanning logic (pipe outputs, inputs, PipeParallelSpec.combined_output, concept refines, structure concept_ref, and item_concept_ref) is duplicated nearly verbatim between _prune_unreachable_specs Steps C+D and _collect_local_bare_concept_codes. Both iterate the same fields and call _extract_local_bare_code identically. If a new concept-bearing field is added to any spec type, both locations need to be updated in lockstep, risking silent divergence.


Summary
fixed_pipe_parallel_conceptsoptimization in_fix_undeclared_concept_referencesthat prematurely removed concepts from the undeclared set even when referenced elsewhere, wasting a fix-loop iteration_prune_unreachable_specsmethod that walks the call graph frommain_pipeto remove unreachable pipes and transitively unused concepts after all fixes are appliedPipeSequence,PipeParallel,PipeBatch, andPipeCondition(filteringSpecialOutcomevalues)Test plan
make agent-check— all linting, formatting, pyright, and mypy passmake agent-test— full test suite passes🤖 Generated with Claude Code
Note
Medium Risk
Prunes pipes/concepts in-place based on reachability and reference scanning, which could remove specs that were previously retained if graph traversal or domain detection misses an edge case.
Overview
Improves the builder fix loop by recomputing the undeclared concept set after mutating
PipeParallelspecs, avoiding prematurely dropping concepts that may still be referenced elsewhere.Adds a post-fix pruning pass (
_prune_unreachable_specs) that walks frommain_pipeto delete unreachable pipes and then removes unused concepts (including transitiverefines/structure references), with domain-aware handling so external concept refs don’t keep local concepts alive; includes new unit tests covering pruning, domain filtering, and missing-pipe warnings.Written by Cursor Bugbot for commit 596c761. This will update automatically on new commits. Configure here.