ml-observability

Free Open-source ML observability course for data scientists and ML engineers. Learn how to monitor and debug your ML models in production.

A comprehensive solution for monitoring your AI models in production

A Python client to interact with Arize API

A Java client to interact with Arize API

Example projects for Arthur Model Monitoring Platform

Enterprise-grade machine learning observability platform that detects data drift, concept drift, and performance degradation in production models. Features statistical drift detection (KS test, PSI), real-time alerting, Redis caching, and FastAPI backend.

Autonomous multi-agent system for ML infrastructure reliability. Detects, diagnoses, and remediates production ML incidents through a coordinated Sentry → Sleuth → Medic → Scribe agent pipeline. Built with Google ADK + Gemini, FastAPI, and a live failure-injection demo harness.

Autonomous ML supply chain guardian on the DataHub metadata graph, finds every model in the blast radius of an upstream change, files evidence-backed incidents, blocks bad deploys, and remembers through the graph.

Sentinel — Production ML Observability Platform MLOps · Model Drift Detection · Real-time Anomaly Monitoring · Full-stack A production-grade platform that monitors deployed ML models in real time — detecting data drift, concept drift, and prediction anomalies — visualised through a live dashboard with alerting. No hardware needed.

Your model didn't break. Your data did — three hops upstream, three days ago. An agent that root-causes silent ML failures through DataHub's lineage, and knows when not to blame the pipeline.

Autonomous AI agents on DataHub that catch silent data changes breaking production ML models — trace lineage, measure the AUC hit, open a fix PR, and write the warning back via the MCP Server.

Local-first ML observability — track runs, metrics, and environments without a cloud backend.

Detects ML features whose upstream warehouse columns were renamed, dropped, or changed type — using DataHub's schema and ML lineage metadata via the MCP Server.

Aporia — independent third-party profile of a public API surface, by API Evangelist. Aporia was an AI guardrails and observability platform for production LLM and ML applications, known for low-latency guardrails covering hallucination, prompt injection, PII, toxicity, off-topic responses, and custom policies, alongside full ML monitoring for drift

Evidence-based ML release-gate agent that catches cross-pipeline target and temporal leakage using DataHub column-level lineage