A page concept was added for each chunk. Priority settings (high, normal, low) are allowed, and the source is chosen based on the priority.

[SammiHungsm]

Description

This pull request introduces a page concept for each chunk in LightRAG. It also adds support for priority settings (high, normal, low), ensuring that sources are selected dynamically based on the assigned priority. These changes improve flexibility and control over how chunks are processed and retrieved.

Related Issues

• Addresses enhancement request for chunk prioritization and source selection logic.

Changes Made

• Added a page concept abstraction for every chunk.
• Implemented priority settings: high, normal, low.
• Updated source selection logic to respect priority levels.
• Refactored relevant functions to integrate priority-based decision-making.
• Improved documentation to reflect new chunk/page and priority features.

Checklist

• Changes tested locally
• Code reviewed
• Documentation updated (if necessary)
• Unit tests added (if applicable)

Additional Notes

• This change lays the groundwork for future enhancements such as weighted source selection and customizable priority rules.
• Reviewers may want to pay special attention to the new source selection logic to ensure backward compatibility.