[Features]Automatic Question/Ground Truth Generation for Documents

[earayu]

We should add a feature to automatically generate question and ground truth pairs for documents within a collection.

Proposed Generation Methods:

1. Simple Questions: Generate questions by feeding document chunks directly to an LLM.
2. Complex Questions: Use the GraphIndex to create in-depth questions that require synthesizing information from different parts of a document or even across multiple documents.

Requirements:

• Store the generated question and ground truth pairs in the database.
• Add configuration options to the collection settings page to enable/disable this feature and set up a scheduled job for generation.

[github-actions]

This issue has been marked as stale because it has been open for 30 days with no activity