City Scrapers Columbus

Web scrapers for public meetings in the Columbus, OH area.

Part of the City Scrapers project.

Spider prefix

All spiders in this repo use the coloh prefix (e.g. coloh_city_council).

Development

See the development documentation for more info on how to get started.

Setup

pipenv install --dev

Running spiders

pipenv run scrapy crawl coloh_city_council

Running all spiders

pipenv run scrapy list | xargs -I {} pipenv run scrapy crawl {}

Running tests

pipenv run pytest