Embedded doc mode using HTML comments to fix attribute order issues (fixes #5269)

[Rafa-Gu98]

This PR proposes an alternative implementation for the doc mode feature discussed in #5269 and prototyped in #5283.

Core Ideas:

• Instead of using separate #[pyo3_doc_mode] attributes (which can cause compiler errors if docs are placed above attributes in Rust style), embed mode switches directly in doc comments using HTML comments like <!-- pyo3_doc_mode: python -->.
  • HTML comments are ignored in Rustdoc (not rendered) and skipped during Python docstring extraction.
  • This eliminates evaluation order problems entirely.
• Supported modes: python, rust, both (default).
• Added a compile-time check: if the doc doesn't end in 'both' mode, raise an error to prevent unbalanced docs.

Advantages:

• No dependency on attribute order.
• All configuration stays within doc comments, making it more intuitive.
• Tested locally with a sample project (pyo3-doc-mode-test), verifying Rustdoc, Python docstrings, and error cases.

Changes are primarily in pyo3-macros-backend/src/utils.rs (updated get_doc function) and attributes.rs .

Full code diffs below. Feedback welcome—happy to iterate!

Fixes #5269
References #5283

[davidhewitt]

Thanks, the idea to use markdown comments here is nice! I guess the drawback is that this way around, unlike #5283, the comments can silently be wrong (e.g. typo) and the user will not be notified of their error. Is there any way we could improve that?

I think with #5283 we could solve the ordering issue by making the pyo3::doc_mode attribute automatically try to "upgrade" itself to parse the #[pyclass] or #[pyfunction] or whatever else it's attached to. It'd require a bit of refactoring of our macro entrypoints, however that may not be a bad thing.

Both approaches look viable, I guess we can keep refining and eventually we'll figure out what feels best 👍

[Rafa-Gu98]

Redesigning the macros is a good idea, but we need to handle the property chain order, which may require careful design🤔
For the HTML route, I've added a spell checker in the utils, which should help identify common mistakes :)
Ref: dbdf63f

Thanks, the idea to use markdown comments here is nice! I guess the drawback is that this way around, unlike #5283, the comments can silently be wrong (e.g. typo) and the user will not be notified of their error. Is there any way we could improve that?

I think with #5283 we could solve the ordering issue by making the pyo3::doc_mode attribute automatically try to "upgrade" itself to parse the #[pyclass] or #[pyfunction] or whatever else it's attached to. It'd require a bit of refactoring of our macro entrypoints, however that may not be a bad thing.

Both approaches look viable, I guess we can keep refining and eventually we'll figure out what feels best 👍