Welcome to my NLP Playground! This repository documents my self-taught journey in Natural Language Processing and Deep Learning. Here, I implement research papers, conduct curiosity-driven experiments, and deploy models following MLOps industry standards.
The ressources for these experiments are limited. My main goal here is to learn.
A comprehensive implementation of modern Language Models from the ground up, guided by Sebastian Raschka's "Building LLMs from Scratch". This project covers:
- Text preprocessing pipelines
- Attention mechanism implementation and visualization
- Complete transformer architecture
- GPT-2 model implementation
- Pretraining phase development
- Finetuning for:
- Text classification
- Instruction following
References:
- Attention Is All You Need
- InstructGPT: Training language models to follow instructions with human feedback
- LLMs-from-scratch
Implementation and experimentation with the Financial NER Open Research Dataset (FiNER-ORD), exploring various approaches to Named Entity Recognition in the financial domain.
Main notebook can be seen here
- BERT finetuning on FiNER-ORD
- Model deployment on AWS SageMaker using structured jobs (for reproducability and practice purposes)
- Closed-source LLM
- Base GLiNER model
- Document findings and compare performance
- Finetuned GLiNER model 🚧
References:
- FiNER-ORD: Building a High-Quality English Financial NER Open Research Dataset
- GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
Implementation and experimentation with retrieval augmented generation, using the huggingface documentation as a knowledge base, trying different approaches in the RAG pipeline.
- Embedding and Retrieval: Test different strategies for chunking, embedding, retrieval, and reranking
- Generator: Test different generator models, prompts, and verification strategies.
References:
- HuggingFace Guide: Advanced RAG on Hugging Face documentation using LangChain
- ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
- Creating a LLM-as-a-Judge That Drives Business Results
- A curated knowledge base of real-world LLMOps implementations, with detailed summaries and technical notes.
- Anthropic's Contextual Retrieval
- ✅ Completed
- 🚧 In Progress
- 📋 Planned