fix(site_generation): use shared URL util for site links

[anchildress1]

• build related post hrefs without duplicating the posts path segment
• compute HOME/ROOT_HOME via shared url_utils helper
• add URL utility module and unit tests for success/error/edge cases

Generated-by: GitHub Copilot copilot@github.com

---

This pull request introduces a new url_utils module to centralize and improve URL handling for the site, refactors related code to use these utilities, and adds comprehensive tests to ensure correctness. The changes also include a minor adjustment to the test coverage threshold and a stricter complexity check in the Makefile.

URL Handling Improvements:

• Added a new src/devto_mirror/core/url_utils.py module that provides centralized utilities for building site URLs, normalizing domains, and generating safe post page hrefs; this prevents duplicated path segments and path traversal issues.
• Refactored src/devto_mirror/ai_optimization/cross_reference.py, src/devto_mirror/site_generation/generator.py, and src/devto_mirror/site_generation/renderer.py to use the new URL utilities, ensuring consistent and correct URL construction throughout the codebase. [1] [2] [3] [4] [5] [6]

Testing and Validation:

• Added tests/test_url_utils.py with comprehensive unit tests for all new URL utility functions and for the related links feature, including regression tests to ensure no duplicated "posts" path segments appear in generated links.

Related Links Logic Refactor:

• Refactored related post link generation in cross_reference.py to use new helper functions for tag cleaning, scoring, and safe local link construction, improving code clarity and robustness. [1] [2] [3]

Build and Quality Checks:

• Increased the minimum test coverage threshold from 80% to 80.9% in the Makefile.
• Made the complexity check in the Makefile stricter by failing the build if complexity requirements are not met, rather than just warning.

[Copilot]

Pull request overview

This pull request introduces a new url_utils module to centralize URL handling and fixes a bug where related post links were duplicating the "posts" path segment (e.g., /posts/posts/foo.html). The changes include comprehensive unit tests and minor improvements to the build quality checks.

Changes:

• Introduced src/devto_mirror/core/url_utils.py with functions for normalizing domains, building site URLs, and generating safe post page hrefs
• Refactored generator.py, renderer.py, and cross_reference.py to use the new centralized URL utilities
• Added comprehensive unit tests in tests/test_url_utils.py including regression tests for the duplicate path bug
• Increased test coverage threshold from 80% to 80.9% and made complexity checks fail the build instead of just warning

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/devto_mirror/core/url_utils.py	New module providing centralized URL utilities with proper validation and security checks (path traversal prevention)
tests/test_url_utils.py	Comprehensive unit tests for URL utilities and regression test for the duplicate "posts" path segment bug
src/devto_mirror/site_generation/generator.py	Refactored to use build_site_urls() for computing HOME and ROOT_HOME instead of inline string formatting
src/devto_mirror/site_generation/renderer.py	Refactored to use build_site_urls() with proper exception handling
src/devto_mirror/ai_optimization/cross_reference.py	Refactored related links generation to use post_page_href(), fixing the duplicate path segment bug; extracted helper functions for better code organization
Makefile	Increased test coverage threshold to 80.9% and made complexity checks fail the build instead of warning