AI Observability & Evaluation

AI observability platform for production LLM and agent systems.

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.

Fiddler Auditor is a tool to evaluate language models.

Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.

A comprehensive solution for monitoring your AI models in production

A Python client to interact with Arize API

AI runtime inventory: discover shadow AI, trace LLM calls

Production operations framework for AI-powered SaaS. The architectural patterns, failure modes, and operational playbooks that determine whether your AI systems scale profitably or fail expensively.

Monitor, detect, and analyze security threats and Shadow AI in your LLM applications with comprehensive analytics and real-time alerting.

AxonFlow — Source-available AI control plane for production LLM systems

🎙️ Voice-native document intelligence using Gemini, ElevenLabs STT/TTS, and Datadog observability — turning text documents into spoken conversations.

AI Chat Watch (AICW) - free open-source tool for GEO marketers that track what & how AI mentions brands, products, companies.

Search for a holiday and get destination advice from an LLM. Observability by Dynatrace.

A report generator library for the ML models deployed on the Fiddler AI Observability platform

Open-source failure intelligence platform for LLM & agent systems. Adds failure memory, pre-flight warnings, pattern detection, and system health scoring.

A Java client to interact with Arize API