Repositories list

• bud-runtime

  Public
  Bud AI Foundry - A comprehensive inference stack for compound AI deployment, optimization and scaling. Bud Stack provides intelligent infrastructure automation…

  aiinferenceagents+ 9observabilityguardrailsllm-inferenceai-runtimellm-evaluationcompound-ai-systemsai-foundry+ 2

  Python

  •

  GNU Affero General Public License v3.0

  •3•10•39•20•Updated Feb 16, 2026Feb 16, 2026

• bud-connect

  Public
  BudConnect is a cloud service that provides compatibility checking and update synchronization for Bud inference runtimes on customer infrastructure. It acts as …

  Python

  •

  Apache License 2.0

  •0•0•0•1•Updated Feb 13, 2026Feb 13, 2026

• BudAIFoundry-SDK

  Public
  Official Python SDK for the BudAI Foundry Platform.

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Feb 13, 2026Feb 13, 2026

• simulator

  Public
  Decoder only, Encoder only, Diffusion model simulation for SLO, Memory, Infra calculations. For model inference and training

  simulationllmopsllm-training+ 1llm-inference

  Python

  •22•0•0•1•Updated Feb 8, 2026Feb 8, 2026

• WaaV

  Public
  Bud WaaV is high performant, Scalable Audio AI Gateway written in Rust.

  audiogooglegateway+ 7ttsopenaivadsttdeepgramelevenlabsauidio-ai

  Rust

  •0•2•0•1•Updated Feb 4, 2026Feb 4, 2026

• LayerZero

  Public
  LayerZero is a GenAI kernel orchestration, and Dispatch system, that allow Inference engine developers to easily develop cross platform Inference engines.

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Feb 3, 2026Feb 3, 2026

• aibrix

  Public
  Cost-efficient and pluggable Infrastructure components for GenAI inference

  Go

  •

  Apache License 2.0

  •524•0•2•1•Updated Jan 26, 2026Jan 26, 2026

• scaler

  Public
  Bud Scaler is a Kubernetes components designed for GenAI workloads. It provides intelligent scaling, GPU virtualisation, routing strategies etc.

  Go

  •

  Apache License 2.0

  •0•0•0•1•Updated Jan 26, 2026Jan 26, 2026

• GenAIComps

  Public
  GenAI components at micro-service level; GenAI service composer to create mega-service

  Python

  •

  Apache License 2.0

  •218•0•2•0•Updated Jan 22, 2026Jan 22, 2026

• GenAIExamples

  Public
  Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterpris…

  Shell

  •

  Apache License 2.0

  •338•0•0•0•Updated Jan 21, 2026Jan 21, 2026

• bud-microframe

  Public
  Python

  •0•0•0•1•Updated Jan 16, 2026Jan 16, 2026

• budtiktok

  Public
  The fastest tokeniser ever, with SIMD & Algorithmic optimisations

  Python

  •0•0•0•0•Updated Jan 16, 2026Jan 16, 2026

• stove8s

  Public
  Preheats your containers for you

  Go

  •0•1•0•0•Updated Jan 13, 2026Jan 13, 2026

• vLLM-plugin-manager

  Public
  A manager to load vllm plugins without rebuilding image for each new plugin.

  vllmvllm-plugins

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Jan 10, 2026Jan 10, 2026

• Bud-Flow-Lang

  Public
  Bud Flow Lang is a domain-specific language embedded in Python that helps you easily write portable, high-performance SIMD programs.

  C++

  •

  Apache License 2.0

  •0•0•0•0•Updated Jan 7, 2026Jan 7, 2026

• simd-bench

  Public
  Profile SIMD code across multiple ISAs and hardware with ease. SIMD bench also comes with perf analyser that automatically recommends optimisations.

  armamdintel+ 5avxsimdamxprofilingkernels

  C++

  •0•0•0•0•Updated Dec 30, 2025Dec 30, 2025

• GPU-Virt-Bench

  Public
  A GPU Virtualisation benchmarking tool - For LLM like workloads, overhead evaluations and on resource isolation metrics

  serverlessgpugpu-computing+ 3gpu-virtualizationfcsphami-core

  Cuda

  •0•8•0•0•Updated Dec 30, 2025Dec 30, 2025

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  Python

  •

  Apache License 2.0

  •13k•0•0•0•Updated Dec 26, 2025Dec 26, 2025

• vllm-plugins

  Public
  vLLM Plugins for additional features like decoding strategies, monitoring, models etc

  vllmvllm-plugins

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Dec 24, 2025Dec 24, 2025

• llm-benchmark

  Public
  Python

  •

  GNU Affero General Public License v3.0

  •0•1•3•2•Updated Dec 15, 2025Dec 15, 2025

• Awesome-vLLM-plugins

  Public
  A curated list of plugins built on top of vLLM

  pluginsvllmvllm-operator+ 3vllm-servevllm-integrationvllm-plugins

  Apache License 2.0

  •0•6•0•0•Updated Dec 12, 2025Dec 12, 2025

• scid

  Public
  CI/CD/Stateless/Helm/Sops/ExecJobs/WatchPaths/Git/SlackIntegration

  Go

  •1•0•0•0•Updated Dec 4, 2025Dec 4, 2025

• Legal-Obligations-Agents

  Public
  An agent that can detect legal obligations with an SLM

  Python

  •0•0•0•0•Updated Dec 2, 2025Dec 2, 2025

• opencompass

  Public
  OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets…

  Python

  •

  Apache License 2.0

  •737•0•0•0•Updated Nov 26, 2025Nov 26, 2025

• .github

  Public
  0•0•0•0•Updated Nov 26, 2025Nov 26, 2025

• HAMi

  Public
  Heterogeneous AI Computing Virtualization Middleware(Project under CNCF)

  Go

  •

  Apache License 2.0

  •475•0•0•0•Updated Nov 24, 2025Nov 24, 2025

• minio-downloader

  Public
  A tool to download and upload models from MinIO to a local path.

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Oct 20, 2025Oct 20, 2025

• infra-debug-tools

  Public
  Nix

  •0•0•0•0•Updated Sep 30, 2025Sep 30, 2025

• d-UI

  Public
  A protocol layer for LLMs to create, interact with and respond to UI interactions by the end-user, allowing LLMs/Tools/APIs to also capture, respond and Interac…

  Python

  •0•0•0•0•Updated Sep 29, 2025Sep 29, 2025

• gpu-operator

  Public
  NVIDIA GPU Operator creates, configures, and manages GPUs in Kubernetes

  Go

  •

  Apache License 2.0

  •452•0•0•0•Updated Sep 13, 2025Sep 13, 2025