Skip to content

Anthropic scraper returns zero results - investigate if page structure changed #70

@leblancfg

Description

@leblancfg

Problem

The Anthropic scraper returns 0 deprecations:

✓ Scraped Anthropic: 0 deprecations

This could mean:

  1. Correct: Anthropic has no current deprecations
  2. Incorrect: Page structure changed and scraper can't find content

Investigation Needed

  1. Manually visit https://docs.anthropic.com/en/docs/about-claude/model-deprecations
  2. Check if there are any deprecation notices
  3. If yes, inspect the HTML structure and compare to scraper expectations
  4. Check browser console for any client-side rendering issues

Proposed Fix

If the page has changed:

  • Update selectors in src/scrapers/anthropic_scraper.py
  • Add logging to show what content is being found
  • Add test fixtures with sample HTML

If the page is correct:

  • Document that zero results is expected
  • Add comment explaining this is normal when Anthropic has no active deprecations

Test Case

Create a test with cached HTML from the Anthropic page to ensure scraper works correctly regardless of live page status.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions