Relative links do not consider redirects in command-line inputs

[katrinafyi]

This page https://uq-pac.github.io/BASIL/api redirects to https://uq-pac.github.io/BASIL/api/ (with slash). The page has a relative link to basil which should become https://uq-pac.github.io/BASIL/api/basil.

If you use the non-slash URL as a lychee input, links will be flagged as broken

$ cargo run -- https://uq-pac.github.io/BASIL/api  
[https://uq-pac.github.io/BASIL/api]:
     [404] https://uq-pac.github.io/ | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/BASIL/basil | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/BASIL/basil-antlr | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/BASIL/basil-proto | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/BASIL/bnfc | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/BASIL/java-cup | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/docs/basil-il/BasilIR.html | Rejected status code (this depends on your "accept" configuration): Not Found
     [404] https://uq-pac.github.io/docs/basil-il/BasilIR.pdf | Rejected status code (this depends on your "accept" configuration): Not Found

🔍 9 Total (in 0s) ✅ 1 OK 🚫 8 Errors

Using the redirect target with slash checks links corretly:

$ cargo  run -- https://uq-pac.github.io/BASIL/api/ 
🔍 9 Total (in 1s) ✅ 4 OK 🚫 0 Errors 🔀 5 Redirects

This matches what you see when you go to the page in a browser: https://uq-pac.github.io/BASIL/api. The browser performs the redirect before resolving any relative links.

I think this happens because the inferred base for input sources does not consider redirects.

Related to: #1883

[thomas-zahner]

Thanks for the report. Seems like lychee does follow the redirect but does not keep track or update the "base" URL to resolve the relative URLs. If lychee didn't follow the redirect it would not find any URLs in the HTML.