>_
ARSLAN VUZMAL LONEAI & Systems Engineer
Back to Selected Work Archive
Agentic SystemsSTATUS: Live 1 0

ModelSwitchyard AI Gateway

Multi-provider AI operations gateway with visual routing policies and failover

ModelSwitchyard acts as an enterprise proxy between client applications and commercial AI model providers. It enforces automated fallback cascades during provider outages, optimizes token costs by routing simple queries to lighter models, and tracks end-to-end request traces with virtual API key rate limiting.

Inspect GitHub Source Code View Live System
01 // THE BUSINESS PROBLEM & SITUATION

Operational Context

Applications relying on a single AI provider suffer catastrophic downtime during outages, face unpredictable billing spikes, and lack centralized token usage observability.

CLIENT / ENVIRONMENT: Created for SaaS startups and AI teams managing multiple models across staging and production environments.
02 // SYSTEM ARCHITECTURE & SOLUTION

What Was Built

Engineered a high-throughput proxy gateway with visual routing policies, circuit breakers, virtual key provisioning, and live analytics.

MODEL_SWITCHYARD // AI_GATEWAYMulti-Provider Routing Topology
INCOMING
Virtual API Key
POLICY ROUTER
Cost / Latency
PROVIDERS
Claude · GPT · Gemini
FALLBACK
Circuit Breaker
SYSTEMS_ARCHITECTURE_EXPLODER // LAYERED_STACKFULL-STACK SYSTEM DECOUPLING
Frontend Layer
Next.js 16 App Router, React 19, Tailwind CSS, TypeScript
API & Middleware
FastAPI, Pydantic Schema Validation, Rate Limiters
Orchestration
LangGraph Multi-Agent Supervisor & State Machines
Model Providers
OpenAI GPT-4o, Anthropic Claude 3.5, Vision AI
Data Stores
PostgreSQL (Prisma), Qdrant Vector Store, Redis Caching
Observability
LangSmith traces, Pydantic audit logs, Error retries
03 // ENGINEERING CONTRIBUTION & FEATURES

Exact Contribution

  • Designed the multi-provider abstraction layer with unified request/response schemas
  • Built automated failover circuit breakers with health-check probing
  • Developed virtual API key management with granular rate limits and budget caps
  • Implemented real-time request tracing and cost analytics dashboard

Key Capabilities Built

✓ Dynamic Latency & Cost-Optimized Routing
✓ Automated Multi-Provider Failover Cascades
✓ Virtual API Keys with Budget Caps
✓ Unified OpenAI-Compatible Gateway API
✓ Real-Time Request Tracing & Analytics
DATA_PROVENANCE_TIMELINE // AUDIT_TRACETRACE_ID: #TR-992041
14:22:01.002Source Ingestion
Received webhook payloadOK
14:22:01.045Schema Validation
Pydantic structured field checkOK
14:22:01.210LangGraph Reasoning
Evaluated criteria rulesOK
14:22:01.350Human Audit Check
Confidence threshold evaluationPASSED
14:22:01.480Database Commit
Created audit record in PostgreSQLCOMMITTED
*Illustrative system trace — demonstration data, not a client result.
04 // EXECUTION PIPELINE

Step-by-Step Workflow

STEP 01Client application sends standard chat completion request with virtual API key.
STEP 02Gateway authenticates key, validates budget limits, and inspects routing policy.
STEP 03Request dispatches to optimal provider based on real-time latency and token cost.
STEP 04If primary provider fails or times out, fallback circuit routes to secondary model instantly.
STEP 05Telemetry logs latency, token count, and cost to PostgreSQL database.
05 // ENGINEERING LESSONS & OUTCOMES

Technical Challenges Overcome

  • • Normalizing streaming response formats across disparate provider APIs.
  • • Sub-10ms gateway routing overhead under concurrent load.

Measurable Outcomes

→ 99.99% effective uptime achieved in testing via automatic fallback circuits.
→ Up to 40% reduction in average API inference costs through smart model tiering.
→ Complete visibility into per-feature token consumption and latency metrics.

Lessons & Engineering Rules

  • "Building for multi-provider resilience from day one prevents vendor lock-in and protects production workloads."
  • "Client applications should never hold raw master API keys—virtual scoped keys are mandatory."
NEXT RELATED CASE STUDY

Orchestrion Multi-Agent Studio

View Study
PROJECT METADATA
ARSLAN'S ROLEFull-Stack Systems Architect
YEAR & STATUS2025 // Live
CATEGORYAgentic Systems
TECHNOLOGY STACK
Next.js 16TypeScriptPrismaPostgreSQLSupabaseOpenAIAnthropicGeminiOpenRouter
GITHUB METRICS
Stars: 1
Forks: 0
Verified GitHub Update: 8/20/2026

Need a similar solution?

If your business faces similar data, automation, or software bottlenecks, let's discuss your requirements.

Start a Conversation