Documentation Upgrades

[VibrantClouds]

Partial update from #2858

• Adds Forking guide
• Extends documentation for creating crawlers with more information and examples
• Includes AGENTS.md documentation for users looking to utilize LLM assistance for sources
• Includes additional information on building docker containers for this project.
  • The makefile is likely sufficient for aliasing a lot of this for linux/cygwin users, this is likely only relevant to windows developers
• Update README.md pointing to the additional documents

---

Please note that some of the documentation is reliant on some pipeline adjustments which have not been made yet, it is likely worth deferring this merge until a direction is finalized for the git actions.

CICD workflows will currently not work like the FORKING guide suggests
ARM build failures are hinted at, but the reasoning is wrong until the Dockerfile is actually updated to handle ARM builds

[VibrantClouds]

There's some content in here which point to some older systems it appears you ran in this repository such as the Discord bot. Truthfully, I'm of the mind that it might be worth removing that section from the Docker.md file

[dipu-bd]

I have merged other 2 commits. I could not review everything yet in this commit. I will hold it until I get free time to check properly before merging.

I no longer plan to support discord or telegram bots. I think having a web is sufficient enough. Eventually I plan to support editing of sources from the web itself, and create auto commit in the repository. Before that I would need to separate out the sources folder.

I will do all these after I come back from vacation next week.