several types of attention modules written in PyTorch for learning purposes
Collection of different types of transformers for learning purposes
Examine cost-effective methods for optimizing GQA configurations, comparing the performance with its counterparts like Multi-Head Attention (MHA) and Multi-Query Attention (MQA).
This repository shows how to build a DeepSeek language model from scratch using PyTorch. It includes clean, well-structured implementations of advanced attention techniques such as key–value caching for fast decoding, multi-query attention, grouped-query attention, and multi-head latent attention.
CUDA implementation of Multi-Query Attention achieving 97% KV-cache memory reduction for LLM inference, enabling 32x larger batch sizes. Educational project demonstrating CUDA kernel development with PyTorch integration and Llama model benchmarks.
Add a description, image, and links to the multi-query-attention topic page so that developers can more easily learn about it.
To associate your repository with the multi-query-attention topic, visit your repo's landing page and select "manage topics."