EventHubAiRouter/README.md

EventHub AI Router — Zed gateway + LiteLLM on IFT

OpenAI-compatible endpoint for Zed with automatic tier/lane routing (Novita AI), optional VLESS VPN, Grafana metrics, and LiteLLM Admin UI.

URLs (IFT, Calentiq TLS)

Service	URL
Zed API	https://ai-router.ift.calentiq.com/v1
LiteLLM Admin UI	https://litellm.ift.calentiq.com/ui (login: admin / LITELLM_MASTER_KEY)
Grafana	https://grafana.ift.calentiq.com
Legacy (self-signed)	https://ai-router.ift.eventhub.local/v1

Zed settings.json

{
  "openai": {
    "api_url": "https://ai-router.ift.calentiq.com/v1",
    "api_key": "<ROUTER_API_KEY>"
  },
  "assistant": {
    "default_model": { "provider": "openai", "model": "smart-router" }
  }
}

Architecture

• Gateway (router/) — vision/OCR split, tier classification, lanes A/B/C orchestration, Redis session
• LiteLLM — named models a-* / b-* / c-*, Auto Router v2 fallback (smart-router-internal)
• PostgreSQL — LiteLLM Admin UI, spend logs
• Redis — response cache + gateway session context
• VPN — vless-proxy replicas=0 by default; scripts/vpn-enable.sh

Config sources:

• config/routing_rules.yaml — tier keywords
• config/model_matrix.yaml — tier × lane → Novita models
• config/orchestration.yaml — start lanes, escalation, budget caps

Deploy on IFT

git clone git.sabilin.com/eventhub/EventHubAiRouter /opt/ai-router-stack
cd /opt/ai-router-stack
cp .env.example .env   # fill NOVITA_API_KEY, keys, POSTGRES_PASSWORD, LITELLM_SALT_KEY
bash scripts/deploy.sh
bash scripts/smoke-test.sh
sudo bash scripts/install-audit-cron.sh   # daily Novita audit 03:00 MSK

Prerequisites: Docker Swarm, external network eventhub-ift-net, Traefik routes in EventHubDevOps.

Swarm secrets

Created automatically by deploy.sh from .env: novita_api_key, litellm_master_key, litellm_salt_key, router_api_key, postgres_password, optional groq_api_key, gemini_api_key, vless_conf.

LITELLM_SALT_KEY — generate once, never change after first deploy.

VPN

bash scripts/vpn-enable.sh   # scale vless + watchdog, set HTTP_PROXY on litellm
bash scripts/vpn-disable.sh

Novita always direct (NO_PROXY=api.novita.ai).

Observability

• Prometheus scrape: litellm:4000/metrics, ai-router:8000/metrics (via eventhub-ift-net)
• Grafana: vendored LiteLLM v2 dashboard + custom gateway-tier-lane.json (EventHubDevOps)
• Uptime Kuma monitors — see table in plan / configure manually

Scripts

Script	Purpose
deploy.sh	secrets + build + stack deploy
smoke-test.sh	health, classify, chat max_tokens=16
sync-routing-config.sh	regen litellm_config.generated.yaml
audit-novita-pricing.sh	daily model catalog check
vendor-litellm-dashboards.sh	fetch Grafana JSON → EventHubDevOps
vpn-enable.sh / vpn-disable.sh	VPN toggle

Quality modes

Header X-AI-Quality: auto|economy|balanced|max or metadata.quality_mode in request body. Default: auto (start lane A/B by tier, escalate on retry/5xx).

LLM classifier (GigaChat, optional)

When CLASSIFIER_MODE=hybrid (default) and GIGACHAT_CREDENTIALS is set:

1. Heuristic classify first (0 cost)
2. If confidence < 0.6 → one call to GigaChat-2-Lite via LiteLLM (gigachat-classifier)
3. Lane orchestration unchanged (A/B/C, Redis, budget)

CLASSIFIER_MODE	Behavior
heuristic	Keywords only (no GigaChat)
hybrid	GigaChat only on low confidence
llm	Always GigaChat for text (except vision)

Freemium GigaChat — для личного некомmercial теста; prod — юр. тариф Сбера.

Setup: developers.sber.ru → Authorization key → .env GIGACHAT_CREDENTIALS.

Response field: x_router_meta.classifier_source = heuristic | gigachat | heuristic_fallback.

Response metadata

Each chat response includes x_router_meta:

{
  "tier": "MEDIUM_OPS",
  "lane": "A",
  "model": "a-medium-ops",
  "escalation_level": 0,
  "quality_mode": "auto"
}

Local build

docker build -f router/Dockerfile -t ai-router-gateway:local .

CI

Gitea Actions: .gitea/workflows/ci.yml — build gateway image, sync config check.