Bootcamp/Euclid (Ephraim): Week 5 - Added a multi-provider rag system

[edfolmi]

I built a provider-agnostic (multiple provider), document-based QA system that:

• Works with OpenAI, OpenRouter, or Ollama
• Does not persist user data
• Answers questions strictly based on uploaded documents

Why This Design

• Keeps architecture simple and auditable
• Avoids unnecessary orchestration layers
• Allows flexible provider switching
• Keeps data privacy straightforward

The Architecture

The system follows a simple RAG (Retrieval-Augmented Generation) pattern:

1. Document Ingestion
   • File size validation
   • Text extraction
   • Recursive chunking (800/150 overlap)
2. Embedding Layer
   • Provider-specific embedding models
   • In-memory embedding cache to avoid duplicate computation
3. Vector Storage
   • In-memory Chroma instance
   • No disk persistence
4. Retrieval
   • Max Marginal Relevance (MMR)
   • Configurable top_k
   • Reduces redundancy in retrieved chunks
5. Generation
   • Provider-specific LLM abstraction
   • Streaming responses
   • Strict grounding instruction in system prompt
6. Transparency
   • Citation display (chunk source references)
   • Token estimation displayed to user

Limitations

• Designed for single-session usage
• In-memory vector store resets on restart
• Suitable for small-to-medium documents (size limit enforced)

Screenshot:

Screen recording:
https://drive.google.com/file/d/1uzKKAKaMjgCQKUsBn4jWgC_tGvOTmUNL/view?usp=sharing

@ranskills

[ranskills]

Thanks, @edfolmi

Open the PR