Skip to content

Refactor job scraping logic to improve accuracy and remove unused city handling#655

Merged
lalalaurentiu merged 2 commits intopeviitor-ro:mainfrom
lalalaurentiu:main
Oct 28, 2025
Merged

Refactor job scraping logic to improve accuracy and remove unused city handling#655
lalalaurentiu merged 2 commits intopeviitor-ro:mainfrom
lalalaurentiu:main

Conversation

@lalalaurentiu
Copy link
Collaborator

This pull request refactors the job scraping logic in sites/veoneer.py to improve accuracy and maintainability. The main changes involve updating the HTML selectors to match the current site structure and removing unused or redundant code related to city and county mapping.

Scraping logic improvements:

  • Changed the job listing selector from searching for div elements with specific classes to searching for a ul element with the ID jobs_list_container, and updated the job item selector to target all li elements. This ensures the scraper matches the current structure of the target website.
  • Updated the job title extraction to use the text of the a element instead of a span with a specific class, reflecting changes in the site's markup.
  • Updated the county lookup logic to use only _counties.get_county(city), removing the previous use of the acurate_city_and_county mapping and related dictionary. This simplifies and clarifies how counties are determined for each city.

Code cleanup:

  • Removed the import of acurate_city_and_county from utils, as it is no longer used in the code.
  • Removed the creation and usage of the acurate_city dictionary, which previously mapped Iasi city and county, to streamline city and county handling

@lalalaurentiu lalalaurentiu merged commit cb9a34a into peviitor-ro:main Oct 28, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant