Skip to content

Improve admin-approve workflow and AI fallback #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
felixwidjaja merged 1 commit into from
Feb 9, 2026
Merged

Improve admin-approve workflow and AI fallback #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 30 additions & 14 deletions .github/workflows/admin-approve-submission.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:

jobs:
approve:
# Only run on issues (not PRs) with submission label when admin comments /approve
# Only run on issues (not PRs) with submission label when admin comments /approve or approve
if: |
!github.event.issue.pull_request &&
contains(github.event.issue.labels.*.name, 'submission') &&
Expand Down Expand Up @@ -39,47 +39,48 @@ jobs:
issue_number: context.issue.number,
body: `❌ @${context.actor} You don't have permission to approve submissions.`
});
core.setOutput('allowed', 'false');
} else {
core.setOutput('allowed', 'true');
}

return allowed;
result-encoding: string

- name: Parse approval command
if: steps.check-permission.outputs.result == 'true'
if: steps.check-permission.outputs.allowed == 'true'
id: parse
uses: actions/github-script@v7
with:
script: |
const comment = context.payload.comment.body.trim();
// Match /approve or /approve 2 or /approve 3
const match = comment.match(/^\/approve(?:\s+(\d))?/);
// Match /approve, optionally with score (2 or 3)
const match = comment.match(/^\/approve(?:\s+(\d))?/i);

if (match) {
const scoreOverride = match[1] ? parseInt(match[1]) : null;
console.log(`Score override: ${scoreOverride}`);
core.setOutput('score_override', scoreOverride || '');
core.setOutput('valid', 'true');
} else {
console.log('No match found for approve command');
core.setOutput('valid', 'false');
}

- name: Checkout
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
uses: actions/checkout@v4

- name: Set up Python
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install dependencies
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
run: |
pip install requests beautifulsoup4 anthropic

- name: Re-evaluate with admin override
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
id: evaluate
env:
ISSUE_BODY: ${{ github.event.issue.body }}
Expand All @@ -88,23 +89,38 @@ jobs:
ADMIN_SCORE_OVERRIDE: ${{ steps.parse.outputs.score_override }}
ADMIN_APPROVED: 'true'
run: |
echo "=== Admin Approval Mode ==="
echo "Issue Number: $ISSUE_NUMBER"
echo "Score Override: $ADMIN_SCORE_OVERRIDE"
echo ""
python scripts/evaluate_submission.py
echo ""
echo "=== Evaluation Complete ==="
echo "Files created:"
ls -la *.json *.txt *.md 2>/dev/null || echo "No output files found"

- name: Create PR
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
run: |
echo "=== Debug: Checking for submission_data.json ==="
if [ -f submission_data.json ]; then
echo "Found submission_data.json:"
cat submission_data.json
echo ""
echo "=== Creating PR ==="
python scripts/create_submission_pr.py
else
echo "No submission data found"
echo "ERROR: No submission_data.json found"
echo "Files in current directory:"
ls -la
exit 1
fi

- name: Update labels and comment
if: steps.check-permission.outputs.result == 'true' && steps.parse.outputs.valid == 'true'
if: steps.check-permission.outputs.allowed == 'true' && steps.parse.outputs.valid == 'true'
uses: actions/github-script@v7
with:
script: |
Expand Down
25 changes: 24 additions & 1 deletion scripts/evaluate_submission.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,9 +287,12 @@ def generate_metadata_with_claude(url, page_content=None):
"""Use Claude API to generate name, description, and category"""
api_key = os.environ.get('ANTHROPIC_API_KEY')
if not api_key:
print("Warning: ANTHROPIC_API_KEY not set, skipping AI metadata generation")
print("ERROR: ANTHROPIC_API_KEY not set, cannot generate AI metadata")
print("Available env vars:", [k for k in os.environ.keys() if 'KEY' in k or 'TOKEN' in k or 'SECRET' in k])
return None

print(f"Calling Claude API for {url}...")

try:
import anthropic
client = anthropic.Anthropic(api_key=api_key)
Expand Down Expand Up @@ -579,6 +582,26 @@ def main():
'needs_manual_review': False if admin_approved else result.get('needs_manual_review', False),
'admin_approved': admin_approved,
})
elif admin_approved:
# Fallback for admin approval when Claude fails
# Use basic info extracted from URL
fallback_name = result['company_name']
print(f"Warning: Claude API failed, using fallback for {fallback_name}")
passing_services.append({
'name': fallback_name,
'url': url,
'description': f"Cloud service provider.",
'category': 'Infrastructure Clouds',
'score': result['score'],
'needs_manual_review': True,
'admin_approved': admin_approved,
})
result['ai_metadata'] = {
'name': fallback_name,
'description': 'Cloud service provider.',
'category': 'Infrastructure Clouds',
'fallback': True,
}

results_list.append(result)
print(f"Score: {result['score']}/3")
Expand Down