EventHub/EventHubAiRouter
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(agent): hierarchical executor with path resolve, runtime probe, quiet UI
CI / build-gateway (push) Failing after 16s
Details
CI / sync-config (push) Failing after 0s
Details

Browse Source 
Make Zed Agent closer to Cursor: deterministic DevOps path index, live Traefik
port probe before blind edits, stop-after-edit, and quieter Russian progress.
This commit is contained in:
Aleksey Sabilin
2026-08-13 11:41:27 +03:00
parent 6fd2f0e689
commit a2d238d92e
71 changed files with 10617 additions and 449 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+27 -7
View File
@@ -9,7 +9,7 @@ DEPLOY_HOST=https://ai-router.ift.calentiq.com
# --- LiteLLM ---
LITELLM_MASTER_KEY=sk-litellm-change-me
LITELLM_SALT_KEY=sk-salt-generate-once-never-change
LITELLM_IMAGE=ghcr.io/berriai/litellm:main-v1.96.0-stable
LITELLM_IMAGE=ghcr.io/berriai/litellm:v1.95.0
PROXY_BASE_URL=https://litellm.ift.calentiq.com
# Postgres (LiteLLM Admin UI + spend logs)
@@ -17,15 +17,27 @@ POSTGRES_USER=litellm
POSTGRES_DB=litellm
POSTGRES_PASSWORD=change-me-postgres
# Novita AI (direct from IFT, no VPN)
# --- Provider switch (lane backends) ---
# hybrid (default) = Novita workers + Novita plan/verify | openrouter = VPN reserve
PRIMARY_PROVIDER=hybrid
# Novita AI (direct from IFT, no VPN) — required for hybrid
NOVITA_API_KEY=
# Optional fallbacks
# Anthropic — optional (card/region often blocked); hierarchical uses Novita plan/verify
ANTHROPIC_API_KEY=
# Optional fixed fallbacks
GROQ_API_KEY=
GEMINI_API_KEY=
TOGETHER_API_KEY=
# OpenRouter reserve (VPN). Note: Anthropic/OpenAI/Google may be region-blocked on billing address.
OPENROUTER_API_KEY=
# GigaChat — optional LLM classifier (hybrid mode, freemium for физлица)
# Authorization key from https://developers.sber.ru/studio/ → GigaChat API
# xAI Grok — VPN only
XAI_API_KEY=
# GigaChat — classifier (hybrid mode)
GIGACHAT_CREDENTIALS=
CLASSIFIER_MODE=hybrid
CLASSIFIER_LLM_MODEL=gigachat-classifier
@@ -33,10 +45,18 @@ CLASSIFIER_LLM_MODEL=gigachat-classifier
# Budget (USD/month, also in litellm_config.yaml)
LITELLM_MAX_BUDGET=50
# --- VPN (off by default) ---
# --- VPN (OpenRouter + Grok + Anthropic → VPN; Novita/Groq/Gemini/GigaChat → NO_PROXY) ---
# Hierarchical plan/verify на Novita — VPN НЕ обязателен
# 1) vless/vless.conf или VLESS_SUB_URL
# 2) VPN_ENABLED=true → deploy + vpn-enable.sh (для OR/Grok/Anthropic)
# 3) api.anthropic.com, openrouter.ai, api.x.ai НЕ в NO_PROXY
VPN_ENABLED=false
VLESS_SUB_URL=
VLESS_HWID=
VLESS_UA=Happ/3.3.6/Windows/2607171516600
VLESS_PROXY_URL=http://vless-proxy:8080
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local
# См. scripts/no-proxy-default.sh
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local
DISABLE_AIOHTTP_TRANSPORT=True
USE_AIOHTTP_TRANSPORT=False
.gitea/workflows/ci.yml
+14 -1
View File
@@ -17,9 +17,22 @@ jobs:
- name: Lint Python syntax
run: python3 -m py_compile router/*.py
- name: Unit tests (hierarchical)
run: |
pip install httpx pyyaml
PYTHONPATH=router python3 -m unittest discover -s test/unit -v
sync-config:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Sync routing config
- name: Install PyYAML
run: pip install pyyaml
- name: Sync routing config (hybrid)
env:
PRIMARY_PROVIDER: hybrid
run: bash scripts/sync-routing-config.sh
- name: Sync routing config (openrouter reserve)
env:
PRIMARY_PROVIDER: openrouter
run: bash scripts/gen-litellm-config.py
.gitignore
+2
View File
@@ -6,3 +6,5 @@ __pycache__/
litellm_config.generated.yaml
/tmp/
*.log
.tmp-*
.tmp/
AGENTS.md
+38
View File
@@ -0,0 +1,38 @@
# AGENTS.md — EventHubAiRouter
Zed OpenAI-compatible gateway на IFT. Карта продукта: `../EventHubSpec/ZED-ARCHITECTURE.md`.
## Ключевые файлы
| Файл | Роль |
|------|------|
| `router/router.py` | `/v1/chat/completions`; ветка `tools` → agent hierarchical |
| `router/hierarchical.py` | Plan (Max) / workers / verify (DeepSeek) / `plan_confirm` |
| `router/agent_hier.py` | При tools: executor + tool_calls (Zed Write); без synthetic по умолчанию |
| `router/orchestrator.py` | Tiers / lanes, Redis session |
| `config/orchestration.yaml` | Роли моделей, таймауты, `plan_confirm`, synthetic=never |
| `config/providers.yaml` | Novita / VPN (`novita-planner` = Qwen3.8-Max) |
| `scripts/gen-litellm-config.py` | LiteLLM config |
## Модели (бюджет)
| Роль | Модель |
|------|--------|
| Planner | `novita-planner` → **qwen/qwen3.8-max** |
| Routine tools / medium_code | **qwen3-coder-30b** (`a-medium-code`) |
| Hard | DeepSeek V3.2 (`b-complex`); `quality=max` → Max (`c-complex`) |
| Verify | DeepSeek V3.2 |
| Text SIMPLE (без tools) | Llama 8B — **не** в Write/tools |
## Поведение для Zed
- **Primary клиент:** Zed Agent → `smart-router` (не Claude Code/Codex ACP).
- **Write** (`tools`): Max-план → утверждение → Coder-30B `tool_calls` (файлы правит Zed); escalate на DeepSeek после edit fails.
- **Без tools:** text hierarchical (plan → workers → synth).
- Endpoint: `https://ai-router.ift.calentiq.com/v1`.
## Не делать
- Не коммитить секреты (`.env`, ключи) и локальные `.tmp-*`.
- Не сажать Llama / `a-simple` на tool-loop.
- Deploy на IFT — только по просьбе; shell через WSL + `run-wsl-sh.sh`.
README.md
+111 -86
View File
@@ -1,6 +1,8 @@
# EventHub AI Router — Zed gateway + LiteLLM on IFT
OpenAI-compatible endpoint for [Zed](https://zed.dev) with automatic tier/lane routing (Novita AI), optional VLESS VPN, Grafana metrics, and LiteLLM Admin UI.
OpenAI-compatible endpoint for [Zed](https://zed.dev) with tier/lane routing, optional **hierarchical** orchestration (**Qwen3.8-Max** plan → cheap **Qwen3-Coder-30B** workers/executor → conditional **DeepSeek V3.2** verify), GigaChat classifier, VLESS VPN, Grafana, LiteLLM Admin UI.
Цель: качество правок близко к Cursor при экономии бюджета — дорогие модели только на plan/hard/verify, рутина на дешёвом coder.
## URLs (IFT, Calentiq TLS)
@@ -11,123 +13,146 @@ OpenAI-compatible endpoint for [Zed](https://zed.dev) with automatic tier/lane r
| **Grafana** | `https://grafana.ift.calentiq.com` |
| Legacy (self-signed) | `https://ai-router.ift.eventhub.local/v1` |
## Zed settings.json
## Клиент (канон): Zed Agent → AiRouter
**Primary:** встроенный Zed Agent (OpenAI-compatible), не Claude Code / Codex ACP — те обходят Novita-бюджет.
Пример `settings.json` (Zed 2026 openai-compatible provider):
```json
{
"openai": {
"api_url": "https://ai-router.ift.calentiq.com/v1",
"api_key": "<ROUTER_API_KEY>"
"language_models": {
"openai_compatible": {
"AiRouter": {
"api_url": "https://ai-router.ift.calentiq.com/v1",
"available_models": [
{
"name": "smart-router",
"display_name": "EventHub smart-router",
"max_tokens": 16384,
"max_output_tokens": 4096,
"capabilities": {
"tools": true,
"images": false,
"parallel_tool_calls": false
}
}
]
}
}
},
"assistant": {
"default_model": { "provider": "openai", "model": "smart-router" }
"agent": {
"default_model": {
"provider": "AiRouter",
"model": "smart-router"
}
}
}
```
API key: `ROUTER_API_KEY` в Agent Settings / provider UI.
Checklist:
- модель `smart-router`, `capabilities.tools: true`
- профиль **Write** (tools on) для правок файлов
- Ask без tools → text hierarchical / lane routing
Optional headers / metadata:
- `X-AI-Quality: auto|economy|balanced|max` — `max` поднимает hard на Qwen3.8-Max
- `X-AI-Orchestrate: off|auto|force` — hierarchical (`force` always; `auto` на COMPLEX/REASONING; `economy` quality пропускает hierarchical)
### Запасной клиент (тот же AiRouter)
Если native Zed Agent loop хрупкий: **OpenCode** или **Cline** с `base_url=https://ai-router.ift.calentiq.com/v1` и тем же ключом — иерархия бюджета сохраняется. Claude Code / Codex ACP в Zed — **не** primary (свой биллинг).
### Zed Agent Write + hierarchical
Если в запросе есть `tools` (профиль **Write**), gateway **не** подменяет agent-цикл текстовым синтезом:
1. **Max**-планировщик строит план (`paths` / `edit_goal` / `constraints`) → при `plan_confirm: true` ждёт **утвердить / правка / отмена**
2. После `ok` — **Qwen3-Coder-30B** executor получает `tools` и отвечает **`tool_calls`** (Zed сам правит файлы)
3. Пока в истории есть `role: tool` / `tool_calls` — только executor; при повторных fail edit → escalate на DeepSeek (без synthetic tool_calls)
Чат без tools (Ask / Minimal) по-прежнему может идти через полный text hierarchical.
### Progress в Zed (hierarchical)
При `stream: true` (дефолт Zed) gateway сразу стримит блок **«Ход выполнения»**: план → волны workers → verify → синтез, затем `---` и финальный ответ. Конфиг: `hierarchical.stream_progress` / `progress_in_content` в `config/orchestration.yaml`.
## Provider profiles (`PRIMARY_PROVIDER`)
| Profile | Keys | VPN | Role |
|---------|------|-----|------|
| **`hybrid`** (default) | `NOVITA_API_KEY` | workers direct | Novita workers + **Novita** plan/verify |
| **`openrouter`** | `OPENROUTER_API_KEY` | **VPN** | Reserve (`:free` / non-Anthropic); Claude/OpenAI/Google may be region-blocked |
GigaChat (`GIGACHAT_CREDENTIALS`) — classifier. Optional: Groq / Gemini / Grok / Anthropic (если биллинг заработает).
```bash
bash scripts/switch-provider.sh hybrid
bash scripts/deploy.sh
# VPN нужен только для OpenRouter / Grok / Anthropic — не для Novita hierarchical
```
## Hierarchical flow (COMPLEX / force)
1. **`novita-planner`** (**Qwen3.8-Max**) — JSON subtasks (`simple` | `medium_code` | `hard`) + `paths` / `edit_goal`
2. **Workers / agent executor**: `simple` → Llama (text only); `medium_code` → **Qwen3-Coder-30B**; `hard` → **DeepSeek V3.2**; `quality=max` hard → **Qwen3.8-Max**
3. Checks; **`novita-verifier`** (DeepSeek V3.2) только на fail / hard / `quality=max`
4. **Synthesize** (text path) на `a-medium-code`
> Anthropic / OpenRouter-Claude недоступны при RU billing — plan/verify идут через Novita.
Response `x_router_meta.mode=hierarchical` / `hierarchical_agent` with `worker_calls`, `executor_model`, `role_cost`, `verify_skipped`, etc.
## Architecture
- **Gateway** (`router/`) — vision/OCR split, tier classification, lanes A/B/C orchestration, Redis session
- **LiteLLM** — named models `a-*` / `b-*` / `c-*`, Auto Router v2 fallback (`smart-router-internal`)
- **PostgreSQL** — LiteLLM Admin UI, spend logs
- **Redis** — response cache + gateway session context
- **VPN** — `vless-proxy` replicas=0 by default; `scripts/vpn-enable.sh`
- **Gateway** (`router/`) — classify, lanes A/B/C, hierarchical runner, Redis session
- **LiteLLM** — named models + Auto Router (`smart-router-internal`)
- **PostgreSQL** — spend logs / Admin UI
- **Redis** — cache + session
- **VPN** — `vless-proxy` (Anthropic, OpenRouter, Grok)
Config sources:
- `config/routing_rules.yaml` — tier keywords
- `config/model_matrix.yaml` — tier × lane → Novita models
- `config/orchestration.yaml` — start lanes, escalation, budget caps
Config: `config/providers.yaml`, `orchestration.yaml`, `routing_rules.yaml`, `model_matrix.yaml`.
## Deploy on IFT
```bash
git clone git.sabilin.com/eventhub/EventHubAiRouter /opt/ai-router-stack
cd /opt/ai-router-stack
cp .env.example .env # fill NOVITA_API_KEY, keys, POSTGRES_PASSWORD, LITELLM_SALT_KEY
cp .env.example .env # NOVITA_API_KEY; PRIMARY_PROVIDER=hybrid
bash scripts/deploy.sh
bash scripts/smoke-test.sh
sudo bash scripts/install-audit-cron.sh # daily Novita audit 03:00 MSK
SMOKE_HIERARCHICAL=1 bash scripts/smoke-test.sh
```
Prerequisites: Docker Swarm, external network `eventhub-ift-net`, Traefik routes in EventHubDevOps.
### Swarm secrets
Created automatically by `deploy.sh` from `.env`: `novita_api_key`, `litellm_master_key`, `litellm_salt_key`, `router_api_key`, `postgres_password`, optional `groq_api_key`, `gemini_api_key`, `vless_conf`.
Swarm secrets: `novita_api_key`, optional `anthropic_api_key` / `openrouter_api_key`, `gigachat_credentials`, …
**`LITELLM_SALT_KEY`** — generate once, never change after first deploy.
## VPN
```bash
bash scripts/vpn-enable.sh # scale vless + watchdog, set HTTP_PROXY on litellm
bash scripts/vpn-enable.sh
bash scripts/vpn-disable.sh
```
Novita always direct (`NO_PROXY=api.novita.ai`).
| Provider | Host | Route |
|----------|------|-------|
| Novita | `api.novita.ai` | **direct** (NO_PROXY) |
| Groq / Gemini / GigaChat | … | **direct** |
| **Anthropic** | `api.anthropic.com` | **VPN** (App unavailable without) |
| **OpenRouter** | `openrouter.ai` | **VPN** |
| **xAI Grok** | `api.x.ai` | **VPN** |
## Observability
Canon: `scripts/no-proxy-default.sh`.
- Prometheus scrape: `litellm:4000/metrics`, `ai-router:8000/metrics` (via `eventhub-ift-net`)
- Grafana: vendored LiteLLM v2 dashboard + custom `gateway-tier-lane.json` (EventHubDevOps)
- Uptime Kuma monitors — see table in plan / configure manually
## LLM classifier (GigaChat)
`CLASSIFIER_MODE=hybrid|heuristic|llm` — see `config/orchestration.yaml`. Freemium GigaChat for personal test.
## Scripts
| Script | Purpose |
|--------|---------|
| `deploy.sh` | secrets + build + stack deploy |
| `smoke-test.sh` | health, classify, chat max_tokens=16 |
| `sync-routing-config.sh` | regen `litellm_config.generated.yaml` |
| `audit-novita-pricing.sh` | daily model catalog check |
| `vendor-litellm-dashboards.sh` | fetch Grafana JSON → EventHubDevOps |
| `vpn-enable.sh` / `vpn-disable.sh` | VPN toggle |
## Quality modes
Header `X-AI-Quality: auto|economy|balanced|max` or `metadata.quality_mode` in request body. Default: `auto` (start lane A/B by tier, escalate on retry/5xx).
## LLM classifier (GigaChat, optional)
When `CLASSIFIER_MODE=hybrid` (default) and `GIGACHAT_CREDENTIALS` is set:
1. **Heuristic** classify first (0 cost)
2. If `confidence < 0.6` → one call to **GigaChat-2-Lite** via LiteLLM (`gigachat-classifier`)
3. Lane orchestration unchanged (A/B/C, Redis, budget)
| `CLASSIFIER_MODE` | Behavior |
|-------------------|----------|
| `heuristic` | Keywords only (no GigaChat) |
| `hybrid` | GigaChat only on low confidence |
| `llm` | Always GigaChat for text (except vision) |
Freemium GigaChat — для личного некомmercial теста; prod — юр. тариф Сбера.
Setup: [developers.sber.ru](https://developers.sber.ru/docs/ru/gigachat/quickstart/ind-create-project) → Authorization key → `.env` `GIGACHAT_CREDENTIALS`.
Response field: `x_router_meta.classifier_source` = `heuristic` | `gigachat` | `heuristic_fallback`.
## Response metadata
Each chat response includes `x_router_meta`:
```json
{
"tier": "MEDIUM_OPS",
"lane": "A",
"model": "a-medium-ops",
"escalation_level": 0,
"quality_mode": "auto"
}
```
## Local build
```bash
docker build -f router/Dockerfile -t ai-router-gateway:local .
```
## CI
Gitea Actions: `.gitea/workflows/ci.yml` — build gateway image, sync config check.
See `scripts/` — `deploy.sh`, `smoke-test.sh`, `gen-litellm-config.py`, `switch-provider.sh`, VPN helpers.
config/litellm.base.yaml
+37
View File
@@ -0,0 +1,37 @@
# Static LiteLLM settings — model_list generated by scripts/gen-litellm-config.py
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
store_model_in_db: false
max_budget: 50
budget_duration: 30d
litellm_settings:
drop_params: true
set_verbose: false
request_timeout: 300
num_retries: 2
cache: true
cache_params:
type: redis
host: redis
port: 6379
ttl: 3600
callbacks: ["prometheus"]
require_auth_for_metrics_endpoint: false
environment_variables:
NOVITA_API_KEY: os.environ/NOVITA_API_KEY
GROQ_API_KEY: os.environ/GROQ_API_KEY
XAI_API_KEY: os.environ/XAI_API_KEY
GEMINI_API_KEY: os.environ/GEMINI_API_KEY
GIGACHAT_CREDENTIALS: os.environ/GIGACHAT_CREDENTIALS
TOGETHER_API_KEY: os.environ/TOGETHER_API_KEY
OPENROUTER_API_KEY: os.environ/OPENROUTER_API_KEY
router_settings:
routing_strategy: simple-shuffle
num_retries: 2
timeout: 300
allowed_fails: 5
cooldown_time: 15
config/model_matrix.yaml
+22 -50
View File
@@ -1,104 +1,76 @@
# tier × lane (A/B/C) → Novita model id + LiteLLM fallbacks
# Tier × lane orchestration metadata (model IDs → config/providers.yaml)
lanes:
A: economy
B: balanced
C: max
# LiteLLM per-lane RPM (proxy-side). Zed agent bursts tool calls — keep high for personal IFT.
# OpenRouter :free upstream may still 429; router fallbacks + cooldown_time apply separately.
models:
a-simple:
novita: novita/qwen/qwen3-4b-fp8
fallbacks: [b-simple, groq-llama-8b]
rpm: 60
rpm: 300
b-simple:
novita: novita/meta-llama/llama-3.1-8b-instruct
fallbacks: [a-simple, groq-llama-8b]
rpm: 60
rpm: 300
c-simple:
novita: novita/qwen/qwen3-8b-fp8
fallbacks: [b-simple]
rpm: 60
rpm: 300
a-medium-ops:
novita: novita/qwen/qwen3-8b-fp8
fallbacks: [a-medium-code, b-medium-ops]
rpm: 40
rpm: 240
b-medium-ops:
novita: novita/deepseek/deepseek-v3.2
fallbacks: [a-medium-code, c-medium-ops]
rpm: 40
rpm: 240
c-medium-ops:
novita: novita/deepseek/deepseek-v3.2
fallbacks: [b-medium-ops]
rpm: 40
rpm: 240
a-medium-code:
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
fallbacks: [a-medium-ops, groq-qwen-coder]
rpm: 40
rpm: 240
b-medium-code:
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
fallbacks: [b-medium-ops, c-medium-code]
rpm: 40
rpm: 240
c-medium-code:
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
fallbacks: [b-medium-code, c-medium-ops]
rpm: 30
rpm: 180
a-complex:
novita: novita/deepseek/deepseek-v3.2
fallbacks: [b-complex, a-reasoning]
rpm: 30
rpm: 180
b-complex:
novita: novita/deepseek/deepseek-r1-0528
fallbacks: [a-complex, c-complex]
rpm: 20
rpm: 120
c-complex:
novita: novita/deepseek/deepseek-r1-turbo
fallbacks: [b-complex]
rpm: 15
rpm: 120
a-reasoning:
novita: novita/deepseek/deepseek-r1-0528-qwen3-8b
fallbacks: [b-reasoning, gemini-flash]
rpm: 30
rpm: 120
b-reasoning:
novita: novita/deepseek/deepseek-r1-0528
fallbacks: [a-reasoning, c-reasoning]
rpm: 20
rpm: 120
c-reasoning:
novita: novita/deepseek/deepseek-r1-turbo
fallbacks: [b-reasoning, gemini-flash]
rpm: 15
rpm: 120
a-vision-ocr:
novita: novita/paddlepaddle/paddleocr-vl
fallbacks: [a-vision]
rpm: 30
rpm: 120
no_escalation: true
a-vision:
novita: novita/qwen/qwen3-vl-30b-a3b-instruct
fallbacks: [a-vision-ocr]
rpm: 20
rpm: 120
b-vision:
novita: novita/qwen/qwen2.5-vl-72b-instruct
fallbacks: [a-vision, c-vision]
rpm: 15
rpm: 90
c-vision:
novita: novita/qwen/qwen3-vl-235b-a22b-instruct
fallbacks: [b-vision]
rpm: 10
optional_providers:
groq-llama-8b:
model: groq/llama-3.1-8b-instant
api_key: os.environ/GROQ_API_KEY
groq-qwen-coder:
model: groq/qwen-qwen-2.5-coder-32b
api_key: os.environ/GROQ_API_KEY
gemini-flash:
model: gemini/gemini-2.0-flash
api_key: os.environ/GEMINI_API_KEY
rpm: 90
audit:
price_drift_threshold_pct: 10
config/orchestration.yaml
+81 -1
View File
@@ -32,9 +32,89 @@ redis:
ttl_sec: 1800
classifier:
# heuristic | hybrid (default) | llm
mode: hybrid
llm_model: gigachat-classifier
low_confidence_threshold: 0.6
timeout_sec: 15
max_tokens: 64
# Strong Max plan → cheap coder workers/executor → conditional DeepSeek verify
hierarchical:
enabled: true
trigger_tiers: [COMPLEX, REASONING]
header: X-AI-Orchestrate # off | auto | force
planner_model: novita-planner
verifier_model: novita-verifier
planner_fallback: b-complex
verifier_fallback: b-complex
verify_policy: on_fail_or_hard
verify_input: digests
synthesize_model: a-medium-code
plan_confirm: true
agent_executor_model: a-medium-code
agent_escalate_model: b-complex
agent_escalate_after_edit_failures: 2
skip_synthesize_if_single: true
max_subtasks: 3
max_verify_retries: 1
plan_max_tokens: 1024
verify_max_tokens: 384
synthesize_max_tokens: 1024
worker_max_tokens: 768
wall_clock_sec: 600
digest_chars: 300
call_timeout_sec: 300
# First turn (kickstart read) can stay longer; after read_file body Coder hangs —
# tool_loop uses short timeout + escalate_after_read → DeepSeek.
executor_timeout_sec: 90
executor_num_retries: 0
executor_input_chars: 16000
executor_tool_loop_chars: 12000
executor_tool_loop_timeout_sec: 45
executor_newest_tool_chars: 3500
executor_minimal_context: true
executor_max_tools: 6
executor_tool_desc_chars: 120
executor_tools_full_chars: 8000
executor_max_tokens: 1024
executor_use_stream: false
executor_synthetic_on_timeout: true
# First turn only: synthetic read_file on plan path if model returns prose.
# Mid-loop still stops (no fake edits).
executor_synthetic_mode: kickstart_only
executor_max_synthetic_continues: 1
executor_max_edit_failures: 4
executor_midloop_model_retry: true
executor_midloop_escalate: true
# After successful read_file skip DeepSeek (was 408→400 loop) → edit_file
executor_force_edit_after_read: true
# After a successful read_file, skip cheap coder — it stalls on fat tool results
executor_escalate_after_read: true
# Rewrite read_file on already-read paths → edit (breaks re-read loops)
executor_rewrite_reread: true
# When model returns 200 without tool_calls: one escalate retry before kickstart
executor_no_tools_escalate: true
# After plan approve: 1) deterministic path index 2) if gaps → cheap find_* model
path_resolve_enabled: true
path_resolve_model: a-simple
path_resolve_timeout_sec: 45
path_resolve_max_tools: 3
# DevOps (*.calentiq.com / Bad Gateway): probe live ports before edit
runtime_probe_enabled: true
runtime_probe_timeout_sec: 8
# Optional: ssh host for docker inspect (empty = TCP probe from gateway only)
runtime_probe_ssh_host: ""
# Quiet Zed: milestones only (no ctx bars / LiteLLM waits / subtask dumps)
progress_verbose: false
show_context_fill: false
progress_in_content: true
stream_progress: true
context_window_tokens: 32768
plan_input_chars: 48000
llm_cache_ttl_sec: 3600
worker_map:
simple: a-simple
medium_code: a-medium-code
hard: b-complex
worker_map_max:
hard: c-complex
config/providers.yaml
+133
View File
@@ -0,0 +1,133 @@
# Provider profiles — active: PRIMARY_PROVIDER env (default: hybrid)
# Regenerate: bash scripts/gen-litellm-config.py
profiles:
hybrid:
label: "Hybrid: Novita workers + Novita plan/verify (no Claude geo-billing)"
welcome_credit: false
requires: [NOVITA_API_KEY]
smart_router:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: NOVITA_API_KEY
fallbacks:
# No groq/gemini here — VPN off on IFT; those hang until client timeout.
a-simple: [b-simple, a-medium-ops]
a-medium-code: [b-medium-code, a-complex]
a-complex: [b-complex, a-reasoning]
b-complex: [a-complex, c-complex]
a-reasoning: [b-reasoning, a-complex]
a-vision: [a-vision-ocr]
smart-router-internal: [a-medium-ops, a-medium-code, a-complex]
# Planner is Max — fallback to DeepSeek then coder (never Llama)
novita-planner: [b-complex, a-medium-code]
novita-verifier: [b-complex, a-medium-code]
default_fallbacks: [a-medium-code, a-complex, a-medium-ops]
lanes:
# Prefer Llama for fast lanes — Qwen3 puts answers in reasoning_content
a-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
b-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
c-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
a-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
b-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
c-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
# medium_code → coder / Plus-class (disable thinking — else stream hangs)
a-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
b-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
c-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
# complex/reasoning A/B → coder/deepseek; C (hard) → Qwen3.8-Max only
a-complex: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
b-complex: { model: novita/deepseek/deepseek-v3.2, api_key: NOVITA_API_KEY }
c-complex: { model: novita/qwen/qwen3.8-max, api_key: NOVITA_API_KEY, disable_thinking: true }
a-reasoning: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
b-reasoning: { model: novita/deepseek/deepseek-v3.2, api_key: NOVITA_API_KEY }
c-reasoning: { model: novita/qwen/qwen3.8-max, api_key: NOVITA_API_KEY, disable_thinking: true }
a-vision-ocr: { model: novita/paddlepaddle/paddleocr-vl, api_key: NOVITA_API_KEY }
a-vision: { model: novita/qwen/qwen3-vl-30b-a3b-instruct, api_key: NOVITA_API_KEY }
b-vision: { model: novita/qwen/qwen2.5-vl-72b-instruct, api_key: NOVITA_API_KEY }
c-vision: { model: novita/qwen/qwen3-vl-235b-a22b-instruct, api_key: NOVITA_API_KEY }
openrouter:
label: OpenRouter reserve (VPN; free :free + pay-as-you-go)
welcome_credit: true
requires: [OPENROUTER_API_KEY]
requires_vpn: true
smart_router:
model: openrouter/openrouter/free
api_key: OPENROUTER_API_KEY
fallbacks:
a-simple: [b-simple, c-simple]
a-medium-code: [b-medium-code, c-medium-code]
a-complex: [b-complex, c-complex]
smart-router-internal: [a-medium-ops, a-medium-code]
default_fallbacks: [a-medium-code, a-medium-ops]
lanes:
a-simple: { model: openrouter/openai/gpt-oss-20b:free, api_key: OPENROUTER_API_KEY }
b-simple: { model: openrouter/openai/gpt-oss-20b:free, api_key: OPENROUTER_API_KEY }
c-simple: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
a-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
b-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
c-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
a-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
b-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
c-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
a-complex: { model: openrouter/google/gemma-4-26b-a4b-it:free, api_key: OPENROUTER_API_KEY }
b-complex: { model: openrouter/nvidia/nemotron-3-nano-30b-a3b:free, api_key: OPENROUTER_API_KEY }
c-complex: { model: openrouter/nvidia/nemotron-3-nano-30b-a3b:free, api_key: OPENROUTER_API_KEY }
a-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
b-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
c-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
a-vision-ocr: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
a-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
b-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
c-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
# Always registered when keys present (all profiles)
fixed_models:
# Hierarchical: strong Max plan (rare); DeepSeek verify (conditional)
novita-planner:
model: novita/qwen/qwen3.8-max
api_key: NOVITA_API_KEY
max_tokens: 1024
temperature: 0
disable_thinking: true
novita-verifier:
model: novita/deepseek/deepseek-v3.2
api_key: NOVITA_API_KEY
max_tokens: 1024
temperature: 0
# Optional Claude — only if ANTHROPIC_API_KEY works later (VPN + billing)
claude-haiku-planner:
model: anthropic/claude-haiku-4-5-20251001
api_key: ANTHROPIC_API_KEY
requires_vpn: true
max_tokens: 1024
temperature: 0
claude-sonnet-verifier:
model: anthropic/claude-sonnet-5
api_key: ANTHROPIC_API_KEY
requires_vpn: true
max_tokens: 2048
temperature: 0
grok-3:
model: xai/grok-3
api_key: XAI_API_KEY
requires_vpn: true
groq-llama-8b:
model: groq/llama-3.1-8b-instant
api_key: GROQ_API_KEY
groq-qwen-coder:
model: groq/qwen-qwen-2.5-coder-32b
api_key: GROQ_API_KEY
gemini-flash:
model: gemini/gemini-2.0-flash
api_key: GEMINI_API_KEY
gigachat-classifier:
model: gigachat/GigaChat-2-Lite
api_key: GIGACHAT_CREDENTIALS
ssl_verify: false
max_tokens: 64
temperature: 0
gigachat-pro:
model: gigachat/GigaChat-2-Pro
api_key: GIGACHAT_CREDENTIALS
ssl_verify: false
config/routing_rules.yaml
+14
View File
@@ -38,6 +38,18 @@ gateway:
- novita
- traefik
- mnesia
medium_code_keywords:
- поправь
- исправь
- fix
- edit
- patch
- bugfix
- typo
- rename
- add test
- unit test
- lint
ocr_keywords:
- ocr
- прочитай текст
@@ -64,6 +76,8 @@ litellm:
tier: SIMPLE
- keywords: ["bash", "docker", "swarm", "ci/cd", "gitea", "rebar", "erlang", "devops", "kubectl", "terraform", "playwright", "npm", "wsl"]
tier: MEDIUM
- keywords: ["поправь", "исправь", "fix", "edit", "patch", "bugfix", "typo", "rename", "add test", "lint"]
tier: MEDIUM
- keywords: ["рефакторинг", "refactor", "архитектура", "architecture", "спроектируй", "design system", "microservice", "distributed", "migration plan"]
tier: COMPLEX
- keywords: ["step by step", "reasoning", "prove", "analyze deeply", "think through"]
docker-stack.yml
+26 -9
View File
@@ -42,8 +42,16 @@ secrets:
external: true
groq_api_key:
external: true
xai_api_key:
external: true
gemini_api_key:
external: true
together_api_key:
external: true
openrouter_api_key:
external: true
anthropic_api_key:
external: true
gigachat_credentials:
external: true
vless_conf:
@@ -95,7 +103,7 @@ services:
retries: 3
litellm:
image: ${LITELLM_IMAGE:-ghcr.io/berriai/litellm:main-v1.96.0-stable}
image: ${LITELLM_IMAGE:-ghcr.io/berriai/litellm:v1.95.0}
entrypoint: ["/bin/sh", "/entrypoint/litellm-entrypoint.sh"]
configs:
- source: litellm_config
@@ -111,7 +119,7 @@ services:
USE_AIOHTTP_TRANSPORT: ${USE_AIOHTTP_TRANSPORT:-False}
HTTP_PROXY: ${HTTP_PROXY:-}
HTTPS_PROXY: ${HTTPS_PROXY:-}
NO_PROXY: ${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}
NO_PROXY: ${NO_PROXY:-localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local}
POSTGRES_USER: ${POSTGRES_USER:-litellm}
POSTGRES_DB: ${POSTGRES_DB:-litellm}
secrets:
@@ -120,7 +128,11 @@ services:
- litellm_salt_key
- postgres_password
- groq_api_key
- xai_api_key
- gemini_api_key
- together_api_key
- openrouter_api_key
- anthropic_api_key
- gigachat_credentials
networks:
ai-internal:
@@ -136,16 +148,16 @@ services:
delay: 5s
update_config:
parallelism: 1
failure_action: rollback
failure_action: pause
placement:
constraints:
- node.role == manager
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:4000/health/liveliness || exit 1"]
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:4000/health/liveliness', timeout=5)\" || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
timeout: 15s
retries: 5
start_period: 120s
router:
image: ${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}
@@ -195,7 +207,12 @@ services:
- source: vless_conf
target: /app/vless.conf
networks:
- ai-internal
ai-internal:
aliases:
- vless-proxy
eventhub-ift:
aliases:
- vless-proxy
deploy:
replicas: 0
restart_policy:
@@ -206,7 +223,7 @@ services:
constraints:
- node.role == manager
healthcheck:
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:8080 || exit 1"]
test: ["CMD-SHELL", "pgrep Xray >/dev/null || exit 1"]
interval: 15s
timeout: 5s
retries: 5
litellm_config.yaml
+348 -183
View File
@@ -1,16 +1,16 @@
# LiteLLM Proxy — Novita lanes A/B/C + Auto Router v2 fallback
# Regenerate model_list tail: bash scripts/sync-routing-config.sh
# LiteLLM — generated for PRIMARY_PROVIDER=hybrid
# Profile: Hybrid: Novita workers + Novita plan/verify (no Claude geo-billing)
# Regenerate: PRIMARY_PROVIDER=hybrid bash scripts/gen-litellm-config.py
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
store_model_in_db: false
max_budget: 50
budget_duration: 30d
litellm_settings:
drop_params: true
set_verbose: false
request_timeout: 120
request_timeout: 300
num_retries: 2
cache: true
cache_params:
@@ -18,193 +18,358 @@ litellm_settings:
host: redis
port: 6379
ttl: 3600
callbacks: ["prometheus"]
callbacks:
- prometheus
require_auth_for_metrics_endpoint: false
environment_variables:
NOVITA_API_KEY: os.environ/NOVITA_API_KEY
GROQ_API_KEY: os.environ/GROQ_API_KEY
XAI_API_KEY: os.environ/XAI_API_KEY
GEMINI_API_KEY: os.environ/GEMINI_API_KEY
GIGACHAT_CREDENTIALS: os.environ/GIGACHAT_CREDENTIALS
TOGETHER_API_KEY: os.environ/TOGETHER_API_KEY
OPENROUTER_API_KEY: os.environ/OPENROUTER_API_KEY
router_settings:
routing_strategy: simple-shuffle
num_retries: 2
timeout: 120
allowed_fails: 2
cooldown_time: 30
timeout: 300
allowed_fails: 5
cooldown_time: 15
fallbacks:
- a-simple: ["b-simple", "groq-llama-8b"]
- a-medium-code: ["a-medium-ops", "groq-qwen-coder"]
- a-complex: ["b-complex", "a-reasoning"]
- b-complex: ["a-complex", "c-complex"]
- a-reasoning: ["b-reasoning", "gemini-flash"]
- a-vision: ["a-vision-ocr"]
- smart-router-internal: ["a-medium-ops", "a-medium-code", "a-complex"]
default_fallbacks: ["a-medium-code", "a-complex", "groq-qwen-coder"]
- a-simple:
- b-simple
- a-medium-ops
- a-medium-code:
- b-medium-code
- a-complex
- a-complex:
- b-complex
- a-reasoning
- b-complex:
- a-complex
- c-complex
- a-reasoning:
- b-reasoning
- a-complex
- a-vision:
- a-vision-ocr
- smart-router-internal:
- a-medium-ops
- a-medium-code
- a-complex
- novita-planner:
- b-complex
- a-medium-code
- novita-verifier:
- b-complex
- a-medium-code
default_fallbacks:
- a-medium-code
- a-complex
- a-medium-ops
model_list:
# --- LLM tier classifier (GigaChat freemium, hybrid mode only) ---
- model_name: gigachat-classifier
litellm_params:
model: gigachat/GigaChat-2-Lite
api_key: os.environ/GIGACHAT_CREDENTIALS
ssl_verify: false
max_tokens: 64
temperature: 0
# --- Lane models (generated from config/model_matrix.yaml) ---
- model_name: a-simple
litellm_params:
model: novita/qwen/qwen3-4b-fp8
api_key: os.environ/NOVITA_API_KEY
rpm: 60
- model_name: b-simple
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 60
- model_name: c-simple
litellm_params:
model: novita/qwen/qwen3-8b-fp8
api_key: os.environ/NOVITA_API_KEY
rpm: 60
- model_name: a-medium-ops
litellm_params:
model: novita/qwen/qwen3-8b-fp8
api_key: os.environ/NOVITA_API_KEY
rpm: 40
- model_name: b-medium-ops
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
rpm: 40
- model_name: c-medium-ops
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
rpm: 40
- model_name: a-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 40
- model_name: b-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 40
- model_name: c-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 30
- model_name: a-complex
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
rpm: 30
- model_name: b-complex
litellm_params:
model: novita/deepseek/deepseek-r1-0528
api_key: os.environ/NOVITA_API_KEY
rpm: 20
- model_name: c-complex
litellm_params:
model: novita/deepseek/deepseek-r1-turbo
api_key: os.environ/NOVITA_API_KEY
rpm: 15
- model_name: a-reasoning
litellm_params:
model: novita/deepseek/deepseek-r1-0528-qwen3-8b
api_key: os.environ/NOVITA_API_KEY
rpm: 30
- model_name: b-reasoning
litellm_params:
model: novita/deepseek/deepseek-r1-0528
api_key: os.environ/NOVITA_API_KEY
rpm: 20
- model_name: c-reasoning
litellm_params:
model: novita/deepseek/deepseek-r1-turbo
api_key: os.environ/NOVITA_API_KEY
rpm: 15
- model_name: a-vision-ocr
litellm_params:
model: novita/paddlepaddle/paddleocr-vl
api_key: os.environ/NOVITA_API_KEY
rpm: 30
- model_name: a-vision
litellm_params:
model: novita/qwen/qwen3-vl-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 20
- model_name: b-vision
litellm_params:
model: novita/qwen/qwen2.5-vl-72b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 15
- model_name: c-vision
litellm_params:
model: novita/qwen/qwen3-vl-235b-a22b-instruct
api_key: os.environ/NOVITA_API_KEY
rpm: 10
# --- Optional fallbacks ---
- model_name: groq-llama-8b
litellm_params:
model: groq/llama-3.1-8b-instant
api_key: os.environ/GROQ_API_KEY
- model_name: groq-qwen-coder
litellm_params:
model: groq/qwen-qwen-2.5-coder-32b
api_key: os.environ/GROQ_API_KEY
- model_name: gemini-flash
litellm_params:
model: gemini/gemini-2.0-flash
api_key: os.environ/GEMINI_API_KEY
# --- Auto Router v2 fallback when gateway confidence low ---
- model_name: smart-router-internal
litellm_params:
model: auto_router/complexity_router
drop_params: true
complexity_router_default_model: a-medium-ops
complexity_router_config:
tiers:
SIMPLE: a-simple
MEDIUM: a-medium-ops
MEDIUM_CODE: a-medium-code
COMPLEX: a-complex
REASONING: a-reasoning
classifier_fallback: heuristic
keyword_tier_rules:
- keywords: ["привет", "hello", "hi", "thanks", "спасибо", "что такое", "what is", "define"]
tier: SIMPLE
- keywords: ["bash", "docker", "swarm", "ci/cd", "gitea", "rebar", "erlang", "devops", "kubectl", "terraform"]
tier: MEDIUM
- keywords: ["рефакторинг", "refactor", "архитектура", "architecture", "спроектируй", "migration plan"]
tier: COMPLEX
- keywords: ["step by step", "prove", "analyze deeply", "think through"]
tier: REASONING
custom_technical_keywords: [erlang, rebar3, traefik, mnesia, litellm, novita]
token_thresholds:
simple: 20
complex: 500
tier_boundaries:
simple_medium: 0.18
medium_complex: 0.38
complex_reasoning: 0.62
session_affinity: true
session_affinity_ttl_seconds: 1800
# Alias for Zed default model name (gateway resolves before LiteLLM)
- model_name: smart-router
litellm_params:
model: novita/qwen/qwen3-8b-fp8
api_key: os.environ/NOVITA_API_KEY
- model_name: gigachat-classifier
litellm_params:
model: gigachat/GigaChat-2-Lite
api_key: os.environ/GIGACHAT_CREDENTIALS
timeout: 180
ssl_verify: false
max_tokens: 64
temperature: 0
- model_name: a-simple
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 300
- model_name: b-simple
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 300
- model_name: c-simple
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 300
- model_name: a-medium-ops
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 240
- model_name: b-medium-ops
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 240
- model_name: c-medium-ops
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 240
- model_name: a-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 240
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: b-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 240
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: c-medium-code
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 180
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: a-complex
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 180
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: b-complex
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
- model_name: c-complex
litellm_params:
model: novita/qwen/qwen3.8-max
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: a-reasoning
litellm_params:
model: novita/qwen/qwen3-coder-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: b-reasoning
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
- model_name: c-reasoning
litellm_params:
model: novita/qwen/qwen3.8-max
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: a-vision-ocr
litellm_params:
model: novita/paddlepaddle/paddleocr-vl
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
- model_name: a-vision
litellm_params:
model: novita/qwen/qwen3-vl-30b-a3b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 120
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: b-vision
litellm_params:
model: novita/qwen/qwen2.5-vl-72b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 90
- model_name: c-vision
litellm_params:
model: novita/qwen/qwen3-vl-235b-a22b-instruct
api_key: os.environ/NOVITA_API_KEY
timeout: 180
rpm: 90
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: novita-planner
litellm_params:
model: novita/qwen/qwen3.8-max
api_key: os.environ/NOVITA_API_KEY
timeout: 180
max_tokens: 1024
temperature: 0
extra_body:
chat_template_kwargs:
enable_thinking: false
enable_thinking: false
- model_name: novita-verifier
litellm_params:
model: novita/deepseek/deepseek-v3.2
api_key: os.environ/NOVITA_API_KEY
timeout: 180
max_tokens: 1024
temperature: 0
- model_name: claude-haiku-planner
litellm_params:
model: anthropic/claude-haiku-4-5-20251001
api_key: os.environ/ANTHROPIC_API_KEY
timeout: 180
max_tokens: 1024
temperature: 0
- model_name: claude-sonnet-verifier
litellm_params:
model: anthropic/claude-sonnet-5
api_key: os.environ/ANTHROPIC_API_KEY
timeout: 180
max_tokens: 2048
temperature: 0
- model_name: groq-llama-8b
litellm_params:
model: groq/llama-3.1-8b-instant
api_key: os.environ/GROQ_API_KEY
timeout: 180
- model_name: groq-qwen-coder
litellm_params:
model: groq/qwen-qwen-2.5-coder-32b
api_key: os.environ/GROQ_API_KEY
timeout: 180
- model_name: gemini-flash
litellm_params:
model: gemini/gemini-2.0-flash
api_key: os.environ/GEMINI_API_KEY
timeout: 180
- model_name: grok-3
litellm_params:
model: xai/grok-3
api_key: os.environ/XAI_API_KEY
timeout: 180
- model_name: gigachat-pro
litellm_params:
model: gigachat/GigaChat-2-Pro
api_key: os.environ/GIGACHAT_CREDENTIALS
timeout: 180
ssl_verify: false
- model_name: smart-router-internal
litellm_params:
model: auto_router/complexity_router
drop_params: true
complexity_router_default_model: a-medium-ops
complexity_router_config:
tiers:
SIMPLE: a-simple
MEDIUM: a-medium-ops
MEDIUM_CODE: a-medium-code
COMPLEX: a-complex
REASONING: a-reasoning
classifier_fallback: heuristic
keyword_tier_rules:
- keywords:
- привет
- hello
- hi
- thanks
- спасибо
- что такое
- what is
- define
tier: SIMPLE
- keywords:
- bash
- docker
- swarm
- ci/cd
- gitea
- rebar
- erlang
- devops
- kubectl
- terraform
- playwright
- npm
- wsl
tier: MEDIUM
- keywords:
- поправь
- исправь
- fix
- edit
- patch
- bugfix
- typo
- rename
- add test
- lint
tier: MEDIUM
- keywords:
- рефакторинг
- refactor
- архитектура
- architecture
- спроектируй
- design system
- microservice
- distributed
- migration plan
tier: COMPLEX
- keywords:
- step by step
- reasoning
- prove
- analyze deeply
- think through
tier: REASONING
custom_technical_keywords:
- erlang
- rebar3
- traefik
- mnesia
- litellm
- novita
token_thresholds:
simple: 20
complex: 500
tier_boundaries:
simple_medium: 0.18
medium_complex: 0.38
complex_reasoning: 0.62
session_affinity: true
session_affinity_ttl_seconds: 1800
- model_name: smart-router
litellm_params:
model: novita/meta-llama/llama-3.1-8b-instruct
api_key: os.environ/NOVITA_API_KEY
router/agent_hier.py
+2546
View File
File diff suppressed because it is too large Load Diff
router/agent_stream.py
+587
View File
@@ -0,0 +1,587 @@
"""Streaming agent-hierarchical path — never block Zed on a silent await."""
from __future__ import annotations
import asyncio
import json
import logging
import time
from typing import Any
import httpx
from fastapi.responses import StreamingResponse
log = logging.getLogger("ai-router.agent_stream")
def sse_chunk(
*,
cid: str,
model: str,
delta: dict[str, Any],
finish_reason: str | None = None,
) -> bytes:
payload = {
"id": cid,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": model,
"choices": [
{
"index": 0,
"delta": delta,
"finish_reason": finish_reason,
}
],
}
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n".encode("utf-8")
def format_progress_block(lines: list[str]) -> str:
from progress_ui import format_progress_block as _fmt
return _fmt(lines)
async def stream_agent_plan_then_act(
*,
runner: Any,
last_user: str,
quality_mode: str | None,
session_id: str,
messages: list[dict[str, Any]],
body: dict[str, Any],
meta: dict[str, Any],
decision: Any,
prompt_hash: str,
litellm_url: str,
litellm_key: str,
cfg: dict[str, Any],
find_pending_plan: Any,
plan_payload_from_meta: Any,
inject_plan_context: Any,
pick_agent_executor_model: Any,
prepare_agent_executor_forward: Any,
completion_to_sse_chunks: Any,
synthetic_first_tool_completion: Any,
meta_header: Any,
orchestrator: Any,
requests_metric: Any,
executor_fallback_completion: Any = None,
) -> StreamingResponse:
"""Immediately SSE-stream progress; plan in background; never silent-hang."""
cid = f"hier-agent-{int(time.time() * 1000)}"
queue: asyncio.Queue[tuple[str, Any]] = asyncio.Queue()
async def on_progress(msg: str) -> None:
await queue.put(("progress", msg))
async def work() -> None:
try:
hier = await runner.run(
last_user,
quality_mode=quality_mode,
session_id=session_id,
messages=messages,
on_progress=on_progress,
agent_mode=True,
)
await queue.put(("done", hier))
except Exception as exc: # noqa: BLE001
log.exception("agent plan failed: %s", exc)
await queue.put(("error", exc))
async def gen():
# First bytes ASAP — Zed must leave spinner for content
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"role": "assistant",
"content": "**Agent**\n· Планирую задачу\n",
},
)
task = asyncio.create_task(work())
hier = None
failed: Exception | None = None
progress_lines: list[str] = ["Планирую задачу"]
while True:
try:
kind, payload = await asyncio.wait_for(queue.get(), timeout=12.0)
except asyncio.TimeoutError:
# SSE comment keepalive (proxies / Zed idle timeout)
yield b": keepalive\n\n"
if task.done() and queue.empty():
# drain outcome if any
if not task.cancelled():
try:
_ = task.result()
except Exception as exc: # noqa: BLE001
failed = exc
break
continue
if kind == "progress":
from progress_ui import humanize_line, stream_step
line = humanize_line(str(payload))
if not line:
continue
progress_lines.append(line)
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={"content": stream_step(line)},
)
continue
if kind == "error":
failed = payload # type: ignore[assignment]
break
hier = payload
break
if not task.done():
await task
if failed is not None:
err = (str(failed) or type(failed).__name__)[:200]
meta["hierarchical_error"] = err
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": (
f"\n---\nОшибка плана (роутер, не LiteLLM UI): `{err}`.\n"
"Повтори запрос или напиши проще. Спиннер без шагов = баг "
"роутера — мы должны стримить progress; если снова тишина, "
"проверь деплой.\n"
)
},
)
yield sse_chunk(
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
)
yield b"data: [DONE]\n\n"
return
if hier is None:
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={"content": "\n---\nПустой ответ плана.\n"},
)
yield sse_chunk(
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
)
yield b"data: [DONE]\n\n"
return
hier_meta = {**meta, **hier.meta}
if hier.meta.get("agent_execute"):
plan = plan_payload_from_meta(hier.meta) or find_pending_plan(messages)
orchestrator.after_request(
session_id, prompt_hash=prompt_hash, success=True, escalate=False
)
from path_resolve import (
apply_deterministic_path_index,
path_resolve_needed,
pick_path_resolve_model,
)
if cfg.get("path_resolve_enabled", True):
plan, unresolved = apply_deterministic_path_index(plan)
if unresolved:
log.info("path_index miss (stream) → %s", unresolved)
if cfg.get("runtime_probe_enabled", True) and isinstance(plan, dict):
try:
from runtime_probe import attach_runtime_probe, runtime_facts_line
user_bits = []
for m in reversed(messages or []):
if isinstance(m, dict) and m.get("role") == "user":
c = m.get("content")
user_bits.append(c if isinstance(c, str) else str(c or ""))
break
plan = attach_runtime_probe(
plan, user_text=user_bits[0] if user_bits else "", cfg=cfg
)
line = runtime_facts_line(plan)
if line:
log.info("%s", line)
except Exception as exc: # noqa: BLE001
log.warning("runtime_probe skipped (stream): %s", exc)
path_mode = bool(cfg.get("path_resolve_enabled", True)) and path_resolve_needed(
plan
)
if path_mode:
executor = pick_path_resolve_model(cfg)
else:
executor = pick_agent_executor_model(cfg)
timeout = float(
cfg.get("path_resolve_timeout_sec")
if path_mode
else (
cfg.get("executor_timeout_sec")
or cfg.get("call_timeout_sec")
or 90
)
)
max_ctx = int(cfg.get("executor_input_chars", 12000))
use_stream = bool(cfg.get("executor_use_stream", False))
verbose = bool(cfg.get("progress_verbose", False))
from progress_ui import execution_banner
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": execution_banner(
path_mode=path_mode, plan=plan, model=executor
)
},
)
if verbose:
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": f"· модель `{executor}` · таймаут {int(timeout)}с\n"
},
)
_ = inject_plan_context
forward = prepare_agent_executor_forward(
body,
messages,
plan,
cfg,
executor=executor,
stream=use_stream,
minimal=True,
path_resolve=path_mode,
)
n_tools = len(forward.get("tools") or [])
approx = len(json.dumps(forward, ensure_ascii=False, default=str))
from agent_hier import context_fill_for_forward
ctx_line, ctx_meta = context_fill_for_forward(
forward, cfg, budget_chars=max_ctx
)
log.info(
"executor forward model=%s tools=%s bytes≈%s stream=%s %s",
executor,
n_tools,
approx,
use_stream,
ctx_line,
)
if verbose or cfg.get("show_context_fill", False):
payload_line = f"- tools={n_tools}\n"
if cfg.get("show_context_fill", False):
payload_line += f"- {ctx_line}\n"
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={"content": payload_line},
)
_ = ctx_meta
try:
body_timeout = float(forward.get("timeout") or timeout)
client_timeout = body_timeout + 30.0
if use_stream:
async with httpx.AsyncClient(
timeout=httpx.Timeout(client_timeout, connect=15.0)
) as client:
async with client.stream(
"POST",
f"{litellm_url.rstrip('/')}/v1/chat/completions",
headers={
"Authorization": f"Bearer {litellm_key}",
"Content-Type": "application/json",
},
json=forward,
) as resp:
if resp.status_code >= 400:
err_body = (await resp.aread())[:300]
raise RuntimeError(
f"litellm HTTP {resp.status_code}: "
f"{err_body.decode('utf-8', 'replace')}"
)
buf = b""
aiter = resp.aiter_bytes().__aiter__()
deadline = time.perf_counter() + client_timeout
got_data = False
while time.perf_counter() < deadline:
try:
piece = await asyncio.wait_for(
aiter.__anext__(), timeout=12.0
)
except StopAsyncIteration:
break
except asyncio.TimeoutError:
yield b": keepalive\n\n"
if not got_data:
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": "- executor: ждём LiteLLM…\n"
},
)
continue
got_data = True
buf += piece
while True:
nl = buf.find(b"\n")
if nl < 0:
break
line = buf[:nl].rstrip(b"\r")
buf = buf[nl + 1 :]
if not line:
yield b"\n"
continue
yield line + b"\n"
if buf.strip():
yield buf.rstrip(b"\r") + b"\n"
yield b"data: [DONE]\n\n"
else:
# Non-stream: keepalive while awaiting Novita (headers otherwise block)
async def _post() -> httpx.Response:
async with httpx.AsyncClient(
timeout=httpx.Timeout(client_timeout, connect=15.0)
) as client:
return await client.post(
f"{litellm_url.rstrip('/')}/v1/chat/completions",
headers={
"Authorization": f"Bearer {litellm_key}",
"Content-Type": "application/json",
},
json=forward,
)
task = asyncio.create_task(_post())
while not task.done():
try:
await asyncio.wait_for(asyncio.shield(task), timeout=8.0)
except asyncio.TimeoutError:
yield b": keepalive\n\n"
yield (
b": executor waiting LiteLLM non-stream\n\n"
)
resp = task.result()
if resp.status_code >= 400:
raise RuntimeError(
f"litellm HTTP {resp.status_code}: "
f"{resp.text[:300]}"
)
data = resp.json()
msg = ((data.get("choices") or [{}])[0].get("message") or {})
has_tools = bool(msg.get("tool_calls"))
if not has_tools:
from agent_hier import (
completion_preview,
force_kickstart_read,
pick_agent_escalate_model,
completion_has_tool_calls as _has_tc,
)
log.warning(
"executor no tool_calls after plan: %s",
completion_preview(data),
)
# 1) Escalate to DeepSeek with hard tool_choice
if cfg.get("executor_no_tools_escalate", True):
esc = pick_agent_escalate_model(cfg)
esc_fwd = dict(forward)
esc_fwd["model"] = esc
esc_fwd["tool_choice"] = "required"
path_hint = None
try:
from agent_hier import _first_path_hint
path_hint = _first_path_hint(
plan, forward.get("tools") or [], messages
)
except Exception: # noqa: BLE001
path_hint = None
nudge = (
"Call a tool now. Start with read_file"
+ (f" on `{path_hint}`." if path_hint else ".")
+ " No prose."
)
esc_fwd["messages"] = list(esc_fwd.get("messages") or []) + [
{"role": "user", "content": nudge}
]
yield (
f": no tool_calls → escalate `{esc}`\n\n".encode(
"utf-8"
)
)
async with httpx.AsyncClient(
timeout=httpx.Timeout(client_timeout, connect=15.0)
) as client2:
resp2 = await client2.post(
f"{litellm_url.rstrip('/')}/v1/chat/completions",
headers={
"Authorization": f"Bearer {litellm_key}",
"Content-Type": "application/json",
},
json=esc_fwd,
)
if resp2.status_code < 400:
data2 = resp2.json()
if isinstance(data2, dict) and _has_tc(data2):
data = data2
has_tools = True
log.info("escalate no_tools → tool_calls OK")
# 2) Kickstart read_file on plan path (keeps Zed loop alive)
if not has_tools and cfg.get(
"executor_synthetic_on_timeout", True
):
yield b": no tool_calls -> kickstart read_file\n\n"
if str(cfg.get("executor_synthetic_mode") or "").lower() in (
"never",
):
# still kickstart once on first turn — otherwise dead end
data = force_kickstart_read(
plan=plan,
tools=forward.get("tools") or [],
model=executor,
messages=messages,
)
elif executor_fallback_completion:
data = executor_fallback_completion(
plan=plan,
tools=forward.get("tools") or [],
model=executor,
messages=messages,
cfg=cfg,
reason="no_tool_calls",
)
else:
data = force_kickstart_read(
plan=plan,
tools=forward.get("tools") or [],
model=executor,
messages=messages,
)
has_tools = bool(
(
(data.get("choices") or [{}])[0].get("message") or {}
).get("tool_calls")
)
# Visible status for Zed (short), details stay in SSE comments
from progress_ui import plan_paths, tool_status
tc_names: list[str] = []
msg0 = ((data.get("choices") or [{}])[0].get("message") or {})
for tc in msg0.get("tool_calls") or []:
if isinstance(tc, dict):
tc_names.append(
str(((tc.get("function") or {}).get("name")) or "")
)
paths = plan_paths(plan)
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": tool_status(
tc_names, path=paths[0] if paths else None
)
},
)
yield (
f": executor OK tool_calls={has_tools}\n\n".encode("utf-8")
)
for chunk in completion_to_sse_chunks(
data, cid=cid, model=str(data.get("model") or executor)
):
yield chunk
except Exception as exc: # noqa: BLE001
err = (str(exc) or type(exc).__name__)[:220]
log.exception("executor after plan failed: %s", err)
if cfg.get("executor_synthetic_on_timeout", True):
yield (
f": timeout → executor fallback ({err[:80]})\n\n".encode(
"utf-8"
)
)
if executor_fallback_completion:
data = executor_fallback_completion(
plan=plan,
tools=forward.get("tools") or [],
model=executor,
messages=messages,
cfg=cfg,
reason=f"exception:{err[:80]}",
)
else:
data = synthetic_first_tool_completion(
plan=plan,
tools=forward.get("tools") or [],
model=executor,
messages=messages,
max_synthetic=int(
cfg.get("executor_max_synthetic_continues", 1) or 1
),
max_edit_failures=int(
cfg.get("executor_max_edit_failures", 4) or 4
),
)
for chunk in completion_to_sse_chunks(
data, cid=cid, model=executor
):
yield chunk
else:
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={
"content": (
f"\nExecutor не ответил: `{err}`. "
"Роутер жив — увеличь timeout / урежь контекст; "
"повтор после ok обычно быстрее (cache).\n"
)
},
)
yield sse_chunk(
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
)
yield b"data: [DONE]\n\n"
return
# Confirm / cancel / amend — text plan for user
content = hier.content or ""
yield sse_chunk(
cid=cid,
model="hierarchical",
delta={"content": f"\n---\n\n{content}"},
)
try:
requests_metric.labels(
tier=decision.tier.value,
lane=decision.lane,
model="hierarchical_agent",
status="200",
).inc()
except Exception: # noqa: BLE001
pass
orchestrator.after_request(
session_id, prompt_hash=prompt_hash, success=True, escalate=False
)
yield sse_chunk(
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
)
yield b"data: [DONE]\n\n"
return StreamingResponse(
gen(),
media_type="text/event-stream",
headers={
"X-Router-Meta": meta_header({**meta, "mode": "hierarchical_agent"}),
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
},
)
router/hierarchical.py
+1137
View File
File diff suppressed because it is too large Load Diff
router/llm_cache.py
+134
View File
@@ -0,0 +1,134 @@
"""Redis cache + in-flight coalescing for identical LiteLLM chat calls."""
from __future__ import annotations
import asyncio
import hashlib
import json
import logging
import os
from collections.abc import Awaitable, Callable
from typing import Any
log = logging.getLogger("llm_cache")
_redis = None
_redis_tried = False
_inflight: dict[str, asyncio.Future[str]] = {}
_inflight_lock = asyncio.Lock()
def _get_redis():
global _redis, _redis_tried
if _redis_tried:
return _redis
_redis_tried = True
url = os.environ.get("REDIS_URL", "").strip()
if not url:
return None
try:
import redis
client = redis.from_url(url, decode_responses=True)
client.ping()
_redis = client
except Exception as exc: # noqa: BLE001
log.warning("llm cache redis unavailable: %s", exc)
_redis = None
return _redis
def cache_key(
*,
model: str,
messages: list[dict[str, Any]],
max_tokens: int,
temperature: float,
) -> str:
blob = json.dumps(
{
"model": model,
"messages": messages,
"max_tokens": max_tokens,
"temperature": temperature,
},
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
digest = hashlib.sha256(blob.encode("utf-8")).hexdigest()
return f"ai-router:llm-cache:{digest}"
def get_cached(key: str) -> str | None:
r = _get_redis()
if not r:
return None
try:
val = r.get(key)
return str(val) if val is not None else None
except Exception as exc: # noqa: BLE001
log.warning("llm cache get failed: %s", exc)
return None
def set_cached(key: str, value: str, ttl_sec: int) -> None:
r = _get_redis()
if not r or ttl_sec <= 0:
return
try:
r.setex(key, ttl_sec, value)
except Exception as exc: # noqa: BLE001
log.warning("llm cache set failed: %s", exc)
async def run_cached(
key: str,
factory: Callable[[], Awaitable[str]],
*,
ttl_sec: int,
on_hit: Callable[[str], Awaitable[None] | None] | None = None,
) -> tuple[str, str]:
"""
Return (text, source) where source is cache|coalesce|live.
Coalesce concurrent identical calls onto one upstream request.
"""
cached = get_cached(key)
if cached is not None:
if on_hit:
maybe = on_hit("cache")
if asyncio.iscoroutine(maybe):
await maybe
return cached, "cache"
async with _inflight_lock:
existing = _inflight.get(key)
if existing is not None:
fut: asyncio.Future[str] = existing
mine = False
else:
fut = asyncio.get_running_loop().create_future()
_inflight[key] = fut
mine = True
if not mine:
if on_hit:
maybe = on_hit("coalesce")
if asyncio.iscoroutine(maybe):
await maybe
return await fut, "coalesce"
try:
text = await factory()
set_cached(key, text, ttl_sec)
if not fut.done():
fut.set_result(text)
return text, "live"
except Exception as exc:
if not fut.done():
fut.set_exception(exc)
raise
finally:
async with _inflight_lock:
if _inflight.get(key) is fut:
_inflight.pop(key, None)
router/orchestrator.py
+2 -1
View File
@@ -115,6 +115,7 @@ class Classifier:
self._complex = self._kw_re(gw.get("complex_keywords", []))
self._reasoning = self._kw_re(gw.get("reasoning_keywords", []))
self._medium_ops = self._kw_re(gw.get("medium_ops_keywords", []))
self._medium_code = self._kw_re(gw.get("medium_code_keywords", []))
self._ocr = self._kw_re(gw.get("ocr_keywords", []))
self._escalation = self._kw_re(gw.get("escalation_keywords", []))
wt = gw.get("word_thresholds", {})
@@ -159,7 +160,7 @@ class Classifier:
return Tier.REASONING, 0.88
if self._complex.search(text):
return Tier.COMPLEX, 0.88
if self._code_block.search(text):
if self._code_block.search(text) or self._medium_code.search(text):
return Tier.MEDIUM_CODE, 0.85
if self._medium_ops.search(text):
return Tier.MEDIUM_OPS, 0.82
router/path_resolve.py
+430
View File
@@ -0,0 +1,430 @@
"""Path resolve: deterministic index first, cheap find_* tools if gaps remain."""
from __future__ import annotations
import json
import logging
import re
from typing import Any
log = logging.getLogger("ai-router")
# Absolute Windows root for EventHubDevOps (Zed multi-root).
_DEFAULT_DEVOPS_ROOT = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps"
# keyword groups → relative paths under DevOps root (first match wins per group).
# Order matters: more specific rules first.
_INDEX_RULES: list[tuple[tuple[str, ...], tuple[str, ...]]] = [
(
("loadtest", "load-test", "load_test"),
(r"ift\traefik\dynamic_conf.loadtest.yml",),
),
(
(
"traefik",
"dynamic_conf",
"router",
"middleware",
"calentiq",
"observer",
"host(",
"ift.calentiq",
"stage.calentiq",
),
(r"ift\traefik\dynamic_conf.yml",),
),
(
("portainer",),
(r"ift\docker-compose.portainer.yml",),
),
(
("grafana", "dashboard"),
(
r"ift\observability\grafana\provisioning\dashboards\dashboard.yml",
r"ift\observability\grafana\provisioning\datasources\prometheus.yml",
),
),
(
("prometheus", "observability"),
(r"ift\observability\prometheus.yml",),
),
(
("admin", "compose.admin", "docker-compose.admin"),
(r"ift\docker-compose.admin.yml",),
),
(
("client", "compose.client", "docker-compose.client"),
(r"ift\docker-compose.client.yml",),
),
(
("compose", "swarm", "stack", "docker-compose", "service"),
(r"ift\docker-compose.core.yml",),
),
]
def _join_under(root: str, rel: str) -> str:
root = root.rstrip("\\/")
rel = rel.replace("/", "\\").lstrip("\\")
return root + "\\" + rel
def _norm(path: str) -> str:
return path.replace("/", "\\").rstrip("\\").lower()
def _blob(plan: dict[str, Any] | None) -> str:
return json.dumps(plan or {}, ensure_ascii=False).lower()
def devops_root_from_plan(plan: dict[str, Any] | None) -> str:
blob = json.dumps(plan or {}, ensure_ascii=False)
m = re.search(r"([A-Za-z]:\\[^\"'\n\r]*?EventHubDevOps)", blob)
if m:
return m.group(1)
return _DEFAULT_DEVOPS_ROOT
def looks_devops(plan: dict[str, Any] | None) -> bool:
b = _blob(plan)
return any(
k in b
for k in (
"traefik",
"calentiq",
"eventhubdevops",
"docker-compose",
"swarm",
"observer",
"ift.",
"devops",
)
)
def match_index_paths(plan: dict[str, Any] | None) -> list[str]:
"""Return absolute paths from keyword index (may be empty)."""
if not looks_devops(plan):
return []
root = devops_root_from_plan(plan)
text = _blob(plan)
found: list[str] = []
seen: set[str] = set()
for keys, rels in _INDEX_RULES:
if not any(k in text for k in keys):
continue
for rel in rels:
abs_p = _join_under(root, rel)
nk = _norm(abs_p)
if nk in seen:
continue
seen.add(nk)
found.append(abs_p)
# one rule group is usually enough for a focused plan
if found:
break
return found
def apply_deterministic_path_index(
plan: dict[str, Any] | None,
) -> tuple[dict[str, Any] | None, list[str]]:
"""Fill plan.paths from index. Returns (plan, unresolved_queries).
unresolved_queries empty ⇒ index covered the goal; else cheap find_* needed.
"""
if not isinstance(plan, dict):
return plan, []
out = dict(plan)
tasks = out.get("subtasks")
if not isinstance(tasks, list) or not tasks:
# still try goal-level match
indexed = match_index_paths(out)
if indexed:
out["subtasks"] = [
{
"id": "1",
"prompt": str(out.get("user_goal") or "edit")[:200],
"worker_tier": "medium_code",
"paths": indexed,
"edit_goal": "",
"constraints": [],
}
]
out["path_resolve"] = "index"
return out, []
if looks_devops(out):
q = _queries_from_text(_blob(out))
out["path_resolve_queries"] = q
return out, q
return out, []
indexed = match_index_paths(out)
unresolved: list[str] = []
fixed: list[Any] = []
for item in tasks:
if not isinstance(item, dict):
fixed.append(item)
continue
it = dict(item)
paths = [
p
for p in (it.get("paths") or [])
if isinstance(p, str) and p.strip()
]
# Prefer index hits when devops; replace empty/weak paths
if indexed:
it["paths"] = indexed
it["path_source"] = "index"
elif not paths:
q = _queries_from_text(
str(it.get("prompt") or "")
+ " "
+ str(it.get("edit_goal") or "")
+ " "
+ _blob(out)
)
unresolved.extend(q)
it["paths"] = []
it["path_source"] = "missing"
else:
it["path_source"] = "planner"
fixed.append(it)
out["subtasks"] = fixed
if indexed:
out["path_resolve"] = "index"
out["path_resolve_queries"] = []
log.info("path_index hit → %s", indexed)
return out, []
# Planner had paths but index missed — if devops, still verify via find
if looks_devops(out) and not indexed:
unresolved = unresolved or _queries_from_text(_blob(out))
out["path_resolve_queries"] = unresolved
out["path_resolve"] = "needed"
return out, unresolved
out["path_resolve_queries"] = unresolved
if unresolved:
out["path_resolve"] = "needed"
return out, unresolved
def _queries_from_text(text: str) -> list[str]:
low = text.lower()
qs: list[str] = []
if any(k in low for k in ("traefik", "router", "calentiq", "observer", "dynamic")):
qs.append("dynamic_conf.yml")
if any(k in low for k in ("compose", "swarm", "docker")):
qs.append("docker-compose.core.yml")
if "portainer" in low:
qs.append("docker-compose.portainer.yml")
if "grafana" in low:
qs.append("grafana")
if "prometheus" in low:
qs.append("prometheus.yml")
if not qs:
qs.append("dynamic_conf.yml")
# unique preserve order
out: list[str] = []
for q in qs:
if q not in out:
out.append(q)
return out[:3]
def enrich_plan_from_discovery_tools(
plan: dict[str, Any] | None,
messages: list[dict[str, Any]] | None,
) -> dict[str, Any] | None:
"""Pull absolute paths from find_path / list_directory tool results into plan."""
if not isinstance(plan, dict):
return plan
# Never overwrite a good deterministic index hit with noisy find parses
if str(plan.get("path_resolve") or "") == "index":
return plan
tasks = plan.get("subtasks") or []
if (
isinstance(tasks, list)
and tasks
and isinstance(tasks[0], dict)
and str(tasks[0].get("path_source") or "") == "index"
):
return plan
found = _paths_from_discovery_messages(messages)
if not found:
return plan
out = dict(plan)
tasks = list(out.get("subtasks") or [])
if not tasks:
out["subtasks"] = [
{
"id": "1",
"prompt": "edit",
"worker_tier": "medium_code",
"paths": found[:3],
"path_source": "find",
}
]
else:
fixed = []
for item in tasks:
if not isinstance(item, dict):
fixed.append(item)
continue
it = dict(item)
it["paths"] = found[:3]
it["path_source"] = "find"
fixed.append(it)
out["subtasks"] = fixed
out["path_resolve"] = "find"
out["path_resolve_queries"] = []
log.info("path_resolve find → %s", found[:3])
return out
def _is_workspace_path(path: str) -> bool:
"""Accept only real EventHub project paths — never /etc or null:/etc artifacts."""
key = _norm(path)
if "eventhubdevops" not in key and "eventhub" not in key:
return False
if "\\etc\\" in key or key.startswith("etc\\"):
return False
if "\\nginx\\" in key:
return False
# Reject 1-letter drive artifacts from YAML like null:/etc → l:\etc
if re.match(r"^[a-z]:\\etc\\", key):
return False
base = key.rsplit("\\", 1)[-1]
if base in ("traefik.yml", "traefik.yaml", "docker-compose.yml"):
if "\\ift\\" not in key and "\\stage\\" not in key:
return False
return True
def _paths_from_discovery_messages(
messages: list[dict[str, Any]] | None,
) -> list[str]:
"""Extract existing file paths from find/list tool results only."""
call_ids: set[str] = set()
for msg in messages or []:
if not isinstance(msg, dict) or msg.get("role") != "assistant":
continue
for tc in msg.get("tool_calls") or []:
if not isinstance(tc, dict):
continue
name = str((tc.get("function") or {}).get("name") or "").lower()
if not any(k in name for k in ("find", "list", "search", "grep", "glob")):
continue
# Do not treat read_file / edit_file as discovery
if any(k in name for k in ("read", "edit", "write", "create", "delete")):
continue
cid = str(tc.get("id") or "")
if cid:
call_ids.add(cid)
# No discovery tool calls → do not scrape read_file bodies (YAML has /etc/… paths)
if not call_ids:
return []
paths: list[str] = []
seen: set[str] = set()
for msg in messages or []:
if not isinstance(msg, dict) or msg.get("role") != "tool":
continue
if str(msg.get("tool_call_id") or "") not in call_ids:
continue
content = str(msg.get("content") or "")
low = content.lower()
if "not found" in low and len(content) < 200:
continue
for m in re.finditer(
r"([A-Za-z]:[\\/][^\s\"'<>\]\n\r]+\.(?:yml|yaml|toml|md|json|conf))",
content,
):
p = m.group(1).replace("/", "\\")
if not _is_workspace_path(p):
continue
nk = _norm(p)
if nk in seen:
continue
seen.add(nk)
paths.append(p)
# Also accept plain relative hits that Zed returns under EventHubDevOps
for m in re.finditer(
r"((?:EventHubDevOps[\\/](?:ift|stage)[\\/][^\s\"'<>\]\n\r]+\.(?:yml|yaml)))",
content,
re.I,
):
p = _join_under(
_DEFAULT_DEVOPS_ROOT,
m.group(1).split("EventHubDevOps", 1)[-1].lstrip("\\/"),
)
if not _is_workspace_path(p):
continue
nk = _norm(p)
if nk in seen:
continue
seen.add(nk)
paths.append(p)
return paths
def discovery_tools_only(tools: list[Any], *, max_tools: int = 3) -> list[Any]:
prefer_keys = ("find_path", "find", "list_directory", "list_dir", "grep", "search", "glob")
ranked: list[Any] = []
for key in prefer_keys:
for t in tools or []:
if not isinstance(t, dict):
continue
fn = t.get("function") if isinstance(t.get("function"), dict) else {}
name = str(fn.get("name") or t.get("name") or "").lower()
if key in name and t not in ranked:
ranked.append(t)
if len(ranked) >= max_tools:
break
return ranked[:max_tools]
def pick_path_resolve_model(cfg: dict[str, Any]) -> str:
"""Cheap model for find_* only — allow a-simple (unlike edit executor)."""
return str(cfg.get("path_resolve_model") or "a-simple")
def path_resolve_needed(plan: dict[str, Any] | None) -> bool:
if not isinstance(plan, dict):
return False
qs = plan.get("path_resolve_queries") or []
if qs:
return True
return str(plan.get("path_resolve") or "") == "needed"
def force_find_path_kickstart(
*,
model: str,
tools: list[Any],
queries: list[str],
) -> dict[str, Any]:
"""Synthetic find_path so Zed searches instead of inventing read_file."""
from agent_hier import _synthetic_tool_completion
tool_name = "find_path"
for t in tools or []:
if not isinstance(t, dict):
continue
fn = t.get("function") if isinstance(t.get("function"), dict) else {}
name = str(fn.get("name") or "")
low = name.lower()
if "find" in low or "glob" in low or ("list" in low and "dir" in low):
tool_name = name
break
q = (queries[0] if queries else "dynamic_conf.yml").strip()
args: dict[str, Any]
if "list" in tool_name.lower():
args = {
"path": _join_under(_DEFAULT_DEVOPS_ROOT, r"ift\traefik"),
}
else:
args = {"query": q, "path": _DEFAULT_DEVOPS_ROOT}
# Zed variants
args["glob"] = f"**/{q}" if "." in q else f"**/*{q}*"
return _synthetic_tool_completion(
model=model, tool_name=tool_name, arguments=args
)
router/progress_ui.py
+201
View File
@@ -0,0 +1,201 @@
"""Compact, readable progress lines for Zed agent chat."""
from __future__ import annotations
import re
from typing import Any
_MODEL_LABEL = {
"novita-planner": "Max",
"novita-verifier": "verify",
"a-medium-code": "Coder",
"a-simple": "Llama",
"b-complex": "DeepSeek",
"c-complex": "Max",
"approved-plan": "план",
}
_NOISE = (
"ждём LiteLLM",
"payload:",
"ctx ",
"ctx-",
"worker #",
"волна ",
"параллельно",
"bytes≈",
"upstream_stream",
"non-stream",
)
def model_label(model: str | None) -> str:
if not model:
return "модель"
m = str(model)
return _MODEL_LABEL.get(m, m.split("/")[-1][:18])
def short_path(path: str | None, *, max_parts: int = 3) -> str:
if not path:
return ""
p = str(path).replace("/", "\\").rstrip("\\")
parts = [x for x in p.split("\\") if x]
if len(parts) <= max_parts:
return "\\".join(parts)
return "\\".join(parts[-max_parts:])
def plan_paths(plan: dict[str, Any] | None) -> list[str]:
out: list[str] = []
for item in (plan or {}).get("subtasks") or []:
if not isinstance(item, dict):
continue
for p in item.get("paths") or []:
if isinstance(p, str) and p.strip() and p not in out:
out.append(p.strip())
return out
def is_noise(line: str) -> bool:
s = (line or "").strip()
if not s:
return True
low = s.lower()
return any(n.lower() in low for n in _NOISE)
def humanize_line(line: str) -> str | None:
"""Map internal progress → short Russian status. None = drop."""
s = (line or "").strip()
if not s or is_noise(s):
return None
low = s.lower()
if s in ("план…", "план...", "план"):
return "Планирую задачу"
if "план через" in low or low.startswith("hierarchical:"):
return "Планирую задачу"
if "утверждён" in low or "утвержден" in low:
m = re.search(r"(\d+)\s*подзадач", s)
n = m.group(1) if m else ""
return f"План утверждён{f' · {n} шаг(а)' if n else ''}"
if "отменён" in low or "отменен" in low:
return "План отменён"
if "правки от пользователя" in low or "пересборк" in low:
return "Пересобираю план по правкам"
if "ожидает утверждения" in low or "ждём утверждения" in low or "awaiting" in low:
return "План готов — подтверди («ok»)"
if "agent: план готов" in low or "executor с tools" in low:
return "Перехожу к выполнению"
if "готово за" in low:
return "Готово"
if "json битый" in low or "repair" in low:
return "Чиню формат плана"
if low.startswith("path_index") or "path_index hit" in low:
return None # shown via dedicated path line
if "path_resolve" in low or "поиск файл" in low:
return "Ищу файлы в проекте"
if "runtime_probe" in low or "runtime:" in low:
return "Проверяю runtime"
if "force edit" in low:
return "Правлю файл"
if "kickstart" in low:
return "Читаю файл"
if "escalate" in low or "mid-loop" in low:
return None
if "408" in s or "timeout" in low:
return "Таймаут модели — продолжаю иначе"
if "400" in s and "bad" in low:
return None
# strip technical prefixes
s = re.sub(r"^план:\s*", "", s, flags=re.I)
s = re.sub(r"^agent:\s*", "", s, flags=re.I)
if len(s) > 90:
s = s[:87] + "…"
return s
def format_progress_block(lines: list[str]) -> str:
"""Final/static progress block — numbered, no junk."""
steps: list[str] = []
seen: set[str] = set()
for raw in lines or []:
h = humanize_line(str(raw))
if not h or h in seen:
continue
seen.add(h)
steps.append(h)
if not steps:
return ""
body = "\n".join(f"{i}. {t}" for i, t in enumerate(steps, 1))
return f"**Ход**\n{body}\n\n---\n\n"
def stream_header() -> str:
return "**Agent**\n"
def stream_step(text: str) -> str:
return f"· {text}\n"
def execution_banner(
*,
path_mode: bool,
plan: dict[str, Any] | None,
model: str | None,
) -> str:
"""Shown once after plan approve — what happens next."""
paths = plan_paths(plan)
lines = ["", "---", "", "**Выполнение**"]
facts = (plan or {}).get("runtime_facts") if isinstance(plan, dict) else None
step = 1
if isinstance(facts, dict) and (facts.get("host") or facts.get("hint")):
host = facts.get("host") or "?"
status = facts.get("http_status")
suggest = facts.get("suggested_backend_url")
bit = f"{host}"
if status is not None:
bit += f" → HTTP {status}"
if suggest:
bit += f" · fix `{suggest}`"
lines.append(f"{step}. Runtime: {bit}")
step += 1
if paths:
shown = ", ".join(f"`{short_path(p)}`" for p in paths[:2])
src = ""
tasks = (plan or {}).get("subtasks") or []
if tasks and isinstance(tasks[0], dict):
ps = str(tasks[0].get("path_source") or "")
if ps == "index":
src = " · индекс"
elif ps == "find":
src = " · поиск"
lines.append(f"{step}. Файл{src}: {shown}")
step += 1
if path_mode:
lines.append(f"{step}. Уточняю путь в репозитории…")
else:
lines.append(f"{step}. Читаю → правлю")
elif path_mode:
lines.append(f"{step}. Ищу нужные файлы…")
else:
lines.append(f"{step}. Запуск ({model_label(model)})")
lines.append("")
return "\n".join(lines) + "\n"
def tool_status(tool_names: list[str] | None, *, path: str | None = None) -> str:
names = [str(n).lower() for n in (tool_names or []) if n]
if any("edit" in n or "write" in n for n in names):
base = "Правлю"
elif any("read" in n for n in names):
base = "Читаю"
elif any("find" in n or "list" in n or "grep" in n for n in names):
base = "Ищу"
else:
base = "Инструмент"
if path:
return stream_step(f"{base} `{short_path(path)}`")
return stream_step(base)
router/router.py
+1317 -20
View File
File diff suppressed because it is too large Load Diff
router/runtime_probe.py
+348
View File
@@ -0,0 +1,348 @@
"""DevOps runtime probe: gather live facts before editing Traefik/compose."""
from __future__ import annotations
import json
import logging
import re
import socket
import subprocess
import urllib.error
import urllib.request
from typing import Any
log = logging.getLogger("ai-router")
_HOST_RE = re.compile(
r"\b([a-z0-9-]+(?:\.(?:ift|stage))?\.calentiq\.com)\b",
re.I,
)
# hostname pattern → docker DNS name + candidate ports
_SERVICE_MAP: list[tuple[str, str, tuple[int, ...]]] = [
(r"^observer\.", "observer_web", (80, 4000, 8080, 3000)),
(r"^grafana\.", "grafana", (3000, 80)),
(r"^prometheus\.", "prometheus", (9090, 80)),
(r"^portainer\.", "portainer", (9000, 9443, 80)),
(r"^ai-router\.", "ai-router", (8000, 80)),
(r"^litellm\.", "litellm", (4000, 80)),
(r"^logs\.|^loglynx\.", "loglynx", (6123, 80)),
(r"^kuma\.", "uptime-kuma", (3001, 80)),
]
def extract_hosts(*texts: str) -> list[str]:
found: list[str] = []
seen: set[str] = set()
for text in texts:
if not text:
continue
for m in _HOST_RE.finditer(text):
h = m.group(1).lower()
if h not in seen:
seen.add(h)
found.append(h)
return found
def is_devops_request(plan: dict[str, Any] | None, *extra: str) -> bool:
blob = json.dumps(plan or {}, ensure_ascii=False).lower()
for t in extra:
blob += "\n" + str(t or "").lower()
keys = (
"calentiq",
"traefik",
"bad gateway",
"502",
"504",
"eventhubdevops",
"docker-compose",
"swarm",
"observer",
"ift.",
"stage.",
"devops",
"gateway",
)
if any(k in blob for k in keys):
return True
return bool(extract_hosts(blob))
def service_for_host(host: str) -> tuple[str, tuple[int, ...]]:
h = host.lower()
for pat, name, ports in _SERVICE_MAP:
if re.search(pat, h):
return name, ports
# fallback: first label as service guess
label = h.split(".")[0]
return label.replace("-", "_"), (80, 443, 8080, 3000, 4000)
def _tcp_open(host: str, port: int, timeout: float = 2.0) -> bool:
try:
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
def _http_status(url: str, timeout: float = 8.0) -> int | None:
try:
req = urllib.request.Request(url, method="GET")
with urllib.request.urlopen(req, timeout=timeout) as resp: # noqa: S310
return int(getattr(resp, "status", 200) or 200)
except urllib.error.HTTPError as exc:
return int(exc.code)
except Exception: # noqa: BLE001
return None
def probe_host(host: str, *, cfg: dict[str, Any] | None = None) -> dict[str, Any]:
"""Live probe from AiRouter container (same Docker network as services)."""
cfg = cfg or {}
timeout = float(cfg.get("runtime_probe_timeout_sec") or 8)
service, ports = service_for_host(host)
https = f"https://{host}/"
http = f"http://{host}/"
status = _http_status(https, timeout=timeout)
if status is None:
status = _http_status(http, timeout=timeout)
port_hits: dict[str, bool] = {}
open_ports: list[int] = []
for p in ports:
ok = _tcp_open(service, p, timeout=min(2.0, timeout))
port_hits[f"{service}:{p}"] = ok
if ok:
open_ports.append(p)
closed_preferred = not port_hits.get(f"{service}:80", True) and bool(open_ports)
hint = ""
suggested_url = None
if status in (502, 503, 504) and open_ports:
# Classic misconfig: Traefik → :80 while app on :4000
if 80 not in open_ports and open_ports:
suggested_url = f"http://{service}:{open_ports[0]}"
hint = (
f"HTTP {status}: Traefik likely targets a closed port. "
f"{service}:80 open={port_hits.get(f'{service}:80')}; "
f"open ports={open_ports}. Prefer {suggested_url} in dynamic_conf.yml."
)
else:
hint = (
f"HTTP {status} but {service} has open ports {open_ports or 'none'}. "
"Check container health / upstream path."
)
elif status in (502, 503, 504):
hint = (
f"HTTP {status}: no open TCP on guessed service `{service}` "
f"ports {list(ports)}. Service down or wrong DNS name."
)
elif status and status < 400:
hint = f"HTTP {status}: host responds; issue may be app-level."
else:
hint = f"HTTP status={status}; ports={port_hits}"
facts: dict[str, Any] = {
"host": host,
"http_status": status,
"service": service,
"ports": port_hits,
"open_ports": open_ports,
"suggested_backend_url": suggested_url,
"hint": hint,
"source": "gateway_tcp",
"closed_port_80_but_alt_open": closed_preferred,
}
log.info(
"runtime_probe host=%s status=%s service=%s open=%s suggest=%s",
host,
status,
service,
open_ports,
suggested_url,
)
return facts
def attach_runtime_probe(
plan: dict[str, Any] | None,
*,
user_text: str = "",
cfg: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
"""If DevOps task — probe hosts and attach runtime_facts to plan."""
if not isinstance(plan, dict):
return plan
cfg = cfg or {}
if not cfg.get("runtime_probe_enabled", True):
return plan
if plan.get("runtime_facts"):
return plan
if not is_devops_request(plan, user_text):
return plan
out = dict(plan)
hosts = extract_hosts(
user_text,
json.dumps(plan, ensure_ascii=False),
str(plan.get("user_goal") or ""),
)
if not hosts:
# DevOps without host — still mark needed for terminal phase
out["runtime_probe"] = "needed"
out["runtime_facts"] = {
"hint": "DevOps task without clear host — curl/docker inspect before edit.",
"source": "none",
}
return out
facts_list = [probe_host(h, cfg=cfg) for h in hosts[:3]]
primary = facts_list[0]
out["runtime_facts"] = primary
out["runtime_facts_all"] = facts_list
out["runtime_probe"] = "gateway"
# Strengthen paths toward Traefik dynamic conf for ift hosts
if "ift." in primary.get("host", "") or primary.get("suggested_backend_url"):
root = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps"
yml = root + r"\ift\traefik\dynamic_conf.yml"
tasks = list(out.get("subtasks") or [])
if tasks and isinstance(tasks[0], dict):
t0 = dict(tasks[0])
paths = list(t0.get("paths") or [])
if yml not in paths:
paths = [yml] + paths
t0["paths"] = paths[:4]
if primary.get("suggested_backend_url"):
t0["edit_goal"] = (
f"Set Traefik service backend to {primary['suggested_backend_url']} "
f"(HTTP {primary.get('http_status')}: wrong upstream port)."
)
tasks[0] = t0
out["subtasks"] = tasks
return out
def runtime_facts_line(plan: dict[str, Any] | None) -> str:
facts = (plan or {}).get("runtime_facts") if isinstance(plan, dict) else None
if not isinstance(facts, dict) or not facts:
return ""
parts = [
f"host={facts.get('host')}",
f"http={facts.get('http_status')}",
f"service={facts.get('service')}",
]
if facts.get("open_ports"):
parts.append(f"open={facts.get('open_ports')}")
if facts.get("suggested_backend_url"):
parts.append(f"fix→{facts.get('suggested_backend_url')}")
hint = str(facts.get("hint") or "")[:180]
return "RUNTIME: " + "; ".join(str(p) for p in parts) + (f" | {hint}" if hint else "")
def devops_blocks_blind_force_edit(plan: dict[str, Any] | None) -> bool:
"""Do not blind-edit DevOps YAML until runtime facts exist."""
if not isinstance(plan, dict):
return False
if not is_devops_request(plan):
return False
facts = plan.get("runtime_facts")
if not facts:
return True
# Facts present — allow targeted edit (including force with goal from facts)
return False
def suggested_traefik_edit(
plan: dict[str, Any] | None,
) -> dict[str, str] | None:
"""If probe found closed:80 + open:alt — return old/new URL snippets."""
if not isinstance(plan, dict):
return None
facts = plan.get("runtime_facts")
if not isinstance(facts, dict):
return None
url = facts.get("suggested_backend_url")
service = facts.get("service")
if not url or not service:
return None
# Common misconfig in our dynamic_conf
old = f'url: "http://{service}:80"'
new = f'url: "{url}"'
if old == new:
return None
return {
"path": r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml",
"old_text": old,
"new_text": new,
"reason": str(facts.get("hint") or "port mismatch"),
}
def force_runtime_fix_edit(
*,
plan: dict[str, Any] | None,
tools: list[Any],
model: str,
) -> dict[str, Any] | None:
"""Synthetic edit_file for clear Traefik port mismatch — skip blind marker edits."""
fix = suggested_traefik_edit(plan)
if not fix:
return None
# late import to avoid cycles
from agent_hier import _pick_edit_tool, _synthetic_tool_completion
edit_name = _pick_edit_tool(tools)
if not edit_name:
return None
log.info(
"runtime_probe → targeted edit %s => %s",
fix["old_text"],
fix["new_text"],
)
return _synthetic_tool_completion(
model=model,
tool_name=edit_name,
arguments={
"path": fix["path"],
"edits": [{"old_text": fix["old_text"], "new_text": fix["new_text"]}],
},
)
def ssh_probe_fallback(host: str, cfg: dict[str, Any]) -> dict[str, Any] | None:
"""Optional: ssh to host and docker inspect (if gateway has keys)."""
ssh_host = str(cfg.get("runtime_probe_ssh_host") or "").strip()
if not ssh_host:
return None
service, _ports = service_for_host(host)
script = (
f"echo HTTP=$(curl -sk -o /dev/null -w '%{{http_code}}' --max-time 5 https://{host}/ || echo err); "
f"CID=$(docker ps -q --filter name={service} | head -1); "
f"echo CID=$CID; "
f"if [ -n \"$CID\" ]; then docker inspect \"$CID\" --format 'PORTS={{{{json .NetworkSettings.Ports}}}}'; fi"
)
try:
proc = subprocess.run(
[
"ssh",
"-o",
"BatchMode=yes",
"-o",
"ConnectTimeout=5",
ssh_host,
"bash",
"-lc",
script,
],
capture_output=True,
text=True,
timeout=20,
check=False,
)
out = (proc.stdout or "") + "\n" + (proc.stderr or "")
return {"host": host, "ssh_raw": out[:1500], "source": "ssh", "service": service}
except Exception as exc: # noqa: BLE001
log.warning("runtime ssh probe failed: %s", exc)
return None
scripts/check-litellm-master.sh
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
echo "== Tokens/users now =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT count(*) AS tokens FROM \"LiteLLM_VerificationToken\";
SELECT count(*) AS users FROM \"LiteLLM_UserTable\";
SELECT \"user_id\", \"user_role\" FROM \"LiteLLM_UserTable\" LIMIT 5;
"
echo "== Master key models probe =="
code=$(curl -sS -o /tmp/m.json -w "%{http_code}" --max-time 20 \
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
https://litellm.ift.calentiq.com/v1/models)
echo "models HTTP $code"
head -c 120 /tmp/m.json; echo
echo "== Flush redis key cache (best effort) =="
RID=$(docker ps -q -f name=ai-router_redis | head -1)
if [[ -n "$RID" ]]; then
docker exec "$RID" redis-cli KEYS '*token*' 2>/dev/null | head -20 || true
docker exec "$RID" redis-cli KEYS '*litellm*' 2>/dev/null | head -20 || true
# do not FLUSHALL — may kill router session cache; only clear litellm-ish if safe
fi
echo DONE
REMOTE
scripts/check-observer-live.sh
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
echo "== curl observer =="
curl -sS -o /tmp/obs.body -w "http=%{http_code} time=%{time_total}\n" \
--max-time 15 -k https://observer.ift.calentiq.com/ || true
head -c 400 /tmp/obs.body; echo
echo "== swarm observer_web =="
docker service ls 2>/dev/null | grep -i observer || true
docker service ps $(docker service ls -q --filter name=observer 2>/dev/null | head -1) --no-trunc 2>/dev/null | head -8 || \
docker ps -a --filter name=observer --format '{{.Names}} {{.Status}}' | head -10
echo "== network aliases =="
docker network inspect eventhub-ift-net --format '{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}' 2>/dev/null | grep -i observer || true
REMOTE
scripts/clean-spurious-litellm-fails.sh
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Explain + delete spurious LiteLLM failure logs from restart disconnects
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
USER="${POSTGRES_USER:-litellm}"
DB="${POSTGRES_DB:-litellm}"
echo "== Error message sample =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -t -A -c "
SELECT left(metadata->'error_information'->>'error_message', 200)
FROM \"LiteLLM_SpendLogs\"
WHERE status='failure'
LIMIT 3;
"
echo "== Delete empty-model restart failures =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "
DELETE FROM \"LiteLLM_SpendLogs\"
WHERE status = 'failure'
AND (model IS NULL OR model = '')
AND (call_type IS NULL OR call_type = '');
"
echo "== Remaining =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "
SELECT status, count(*) FROM \"LiteLLM_SpendLogs\" GROUP BY 1;
"
echo CLEANED_SPURIOUS_FAILS_OK
REMOTE
scripts/clear-litellm-db-ift.sh
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env bash
# Clear LiteLLM Postgres spend/logs on IFT (keep schema + keys if present)
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
if [[ -z "$PG" ]]; then
echo "ERROR: postgres container not running" >&2
exit 1
fi
USER="${POSTGRES_USER:-litellm}"
DB="${POSTGRES_DB:-litellm}"
echo "== Tables before =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "\dt"
# LiteLLM Prisma tables — truncate data, keep schema
# Prefer spend/usage; also clear invite/audit-ish if present
SQL=$(cat <<'EOS'
DO $$
DECLARE
r RECORD;
n bigint;
BEGIN
FOR r IN
SELECT tablename
FROM pg_tables
WHERE schemaname = 'public'
AND tablename NOT IN ('_prisma_migrations')
LOOP
EXECUTE format('SELECT count(*) FROM %I', r.tablename) INTO n;
RAISE NOTICE 'truncate % (% rows)', r.tablename, n;
EXECUTE format('TRUNCATE TABLE %I RESTART IDENTITY CASCADE', r.tablename);
END LOOP;
END $$;
EOS
)
echo "== Truncate all public tables (except _prisma_migrations) =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" -i "$PG" \
psql -U "$USER" -d "$DB" -v ON_ERROR_STOP=1 <<< "$SQL"
echo "== Counts after =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "
SELECT relname AS table, n_live_tup AS approx_rows
FROM pg_stat_user_tables
ORDER BY relname;
"
echo "== Restart litellm (refresh Admin UI caches) =="
docker service update --force ai-router_litellm >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null && break
sleep 2
done
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness && echo
echo LITELLM_DB_CLEARED_OK
REMOTE
scripts/deploy-cursor-parity-ift.sh
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env bash
# Deploy Cursor-parity AiRouter (Max plan / Coder routine / DeepSeek hard) to IFT
set -euo pipefail
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
cd "$AIR"
echo "== Local unit + compile =="
python3 -m py_compile \
router/agent_hier.py router/router.py router/hierarchical.py router/orchestrator.py
python3 -m unittest discover -s test/unit -q
PRIMARY_PROVIDER=hybrid python3 scripts/gen-litellm-config.py
echo "== Sync to eventhub-ift =="
scp \
router/agent_hier.py \
router/router.py \
router/hierarchical.py \
router/orchestrator.py \
eventhub-ift:/opt/ai-router-stack/router/
scp \
config/orchestration.yaml \
config/providers.yaml \
config/routing_rules.yaml \
eventhub-ift:/opt/ai-router-stack/config/
scp litellm_config.yaml eventhub-ift:/opt/ai-router-stack/litellm_config.yaml
scp README.md AGENTS.md eventhub-ift:/opt/ai-router-stack/ 2>/dev/null || true
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
export PRIMARY_PROVIDER="${PRIMARY_PROVIDER:-hybrid}"
sed -i 's/\r$//' \
router/agent_hier.py router/router.py router/hierarchical.py router/orchestrator.py \
config/orchestration.yaml config/providers.yaml config/routing_rules.yaml \
litellm_config.yaml || true
echo "== Build gateway image =="
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
LITC="litellm_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
docker config create "$LITC" ./litellm_config.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
OLD_LIT=$(docker service inspect ai-router_litellm --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep litellm_config | grep -v entrypoint | tail -1 || true)
echo "== Update ai-router_router =="
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
if [ -n "${OLD_ORCH:-}" ]; then
RARGS+=(--config-rm "$OLD_ORCH")
fi
docker service update "${RARGS[@]}" ai-router_router >/dev/null
echo "== Update ai-router_litellm =="
LARGS=(--force)
LARGS+=(--config-add "source=${LITC},target=/app/config.yaml")
if [ -n "${OLD_LIT:-}" ]; then
LARGS+=(--config-rm "$OLD_LIT")
fi
docker service update "${LARGS[@]}" ai-router_litellm >/dev/null
echo "== Wait health =="
ok=0
for i in $(seq 1 60); do
if curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null \
&& curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null; then
ok=1
break
fi
sleep 3
done
if [ "$ok" != 1 ]; then
echo "ERROR: health not ready" >&2
docker service ps ai-router_router --no-trunc | head -8
docker service ps ai-router_litellm --no-trunc | head -8
exit 1
fi
RID=$(docker ps -q -f name=ai-router_router | head -1)
LID=$(docker ps -q -f name=ai-router_litellm | head -1)
echo "== Verify orchestration =="
docker exec "$RID" grep -E 'planner_model:|verifier_model:|agent_executor_model:|executor_synthetic_mode:|worker_map:|hard:' /app/config/orchestration.yaml
echo "== Verify litellm novita-planner =="
docker exec "$LID" sh -c 'grep -A6 "model_name: novita-planner" /app/config.yaml | head -8'
echo "== Smoke classify + chat =="
curl -sf --max-time 30 -H "Authorization: Bearer ${ROUTER_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"text":"поправь timeout в router.py"}' \
https://ai-router.ift.calentiq.com/classify | head -c 400
echo
curl -sf --max-time 90 -H "Authorization: Bearer ${ROUTER_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model":"smart-router","messages":[{"role":"user","content":"ping"}],"max_tokens":8}' \
https://ai-router.ift.calentiq.com/v1/chat/completions | head -c 500
echo
echo CURSOR_PARITY_DEPLOY_OK
REMOTE
scripts/deploy-fast-after-read.sh
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
PYTHONPATH=router python3 -m py_compile router/agent_hier.py router/router.py
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
scp router/agent_hier.py router/router.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/agent_hier.py router/router.py config/orchestration.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
sleep 2
done
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
grep -E 'executor_tool_loop_timeout_sec|executor_escalate_after_read|executor_rewrite_reread|executor_newest_tool' \
/app/config/orchestration.yaml
echo FAST_AFTER_READ_OK
REMOTE
scripts/deploy-kickstart-no-tools.sh
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
set -euo pipefail
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
cd "$AIR"
python3 -m py_compile router/agent_hier.py router/agent_stream.py router/router.py
python3 -m unittest discover -s test/unit -q
scp router/agent_hier.py router/agent_stream.py router/router.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/agent_hier.py router/agent_stream.py router/router.py config/orchestration.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
sleep 2
done
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
grep -E 'executor_synthetic_mode|executor_no_tools_escalate' /app/config/orchestration.yaml
echo KICKSTART_NO_TOOLS_OK
REMOTE
scripts/deploy-path-resolve.sh
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
PYTHONPATH=router python3 -m py_compile \
router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
scp router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
sleep 2
done
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
grep -E 'path_resolve_' /app/config/orchestration.yaml
test -f /opt/ai-router-stack/router/path_resolve.py
echo PATH_RESOLVE_OK
REMOTE
scripts/deploy-progress-ui.sh
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
PYTHONPATH=router python3 -m py_compile \
router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py router/progress_ui.py
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
scp router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
sleep 2
done
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
grep -E 'executor_force_edit_after_read|path_resolve_enabled' /app/config/orchestration.yaml
test -f /opt/ai-router-stack/router/progress_ui.py
echo PROGRESS_UI_OK
REMOTE
scripts/deploy-quiet-progress-ift.sh
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Hotfix: quiet progress + executor timeout/escalate
set -euo pipefail
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
cd "$AIR"
python3 -m py_compile router/hierarchical.py router/agent_stream.py router/router.py router/agent_hier.py
python3 -m unittest discover -s test/unit -q
PRIMARY_PROVIDER=hybrid LITELLM_MODEL_TIMEOUT=180 python3 scripts/gen-litellm-config.py
scp router/hierarchical.py router/agent_stream.py router/router.py router/agent_hier.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
scp litellm_config.yaml eventhub-ift:/opt/ai-router-stack/litellm_config.yaml
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/*.py config/orchestration.yaml litellm_config.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
LITC="litellm_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
docker config create "$LITC" ./litellm_config.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
OLD_LIT=$(docker service inspect ai-router_litellm --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep litellm_config | grep -v entrypoint | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
LARGS=(--force --config-add "source=${LITC},target=/app/config.yaml")
[[ -n "${OLD_LIT:-}" ]] && LARGS+=(--config-rm "$OLD_LIT")
docker service update "${LARGS[@]}" ai-router_litellm >/dev/null
for i in $(seq 1 50); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && \
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null && break
sleep 3
done
RID=$(docker ps -q -f name=ai-router_router | head -1)
LID=$(docker ps -q -f name=ai-router_litellm | head -1)
docker exec "$RID" grep -E 'progress_verbose|executor_timeout|executor_midloop_escalate|show_context_fill' /app/config/orchestration.yaml
docker exec "$LID" sh -c 'grep -A5 "model_name: a-medium-code" /app/config.yaml | head -8'
echo QUIET_PROGRESS_TIMEOUT_OK
REMOTE
scripts/deploy-runtime-probe.sh
+37
View File
@@ -0,0 +1,37 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
PYTHONPATH=router python3 -m py_compile \
router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
router/runtime_probe.py
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
scp router/agent_hier.py router/router.py router/agent_stream.py \
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
router/runtime_probe.py \
eventhub-ift:/opt/ai-router-stack/router/
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
TS=$(date +%Y%m%d%H%M%S)
ORCH="orchestration_config_${TS}"
docker config create "$ORCH" ./config/orchestration.yaml
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
docker service update "${RARGS[@]}" ai-router_router >/dev/null
for i in $(seq 1 40); do
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
sleep 2
done
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
grep -E 'runtime_probe_enabled|path_resolve_enabled' /app/config/orchestration.yaml
test -f /opt/ai-router-stack/router/runtime_probe.py
echo RUNTIME_PROBE_OK
REMOTE
scripts/deploy.sh
+38 -5
View File
@@ -23,21 +23,48 @@ ensure_secret() {
if docker secret inspect "$name" >/dev/null 2>&1; then
echo "secret exists: $name"
else
if [[ -z "$value" ]]; then
value="_"
fi
echo -n "$value" | docker secret create "$name" -
echo "created secret: $name"
fi
}
echo "== Ensure Swarm secrets =="
ensure_secret novita_api_key "${NOVITA_API_KEY:?NOVITA_API_KEY required}"
PRIMARY="${PRIMARY_PROVIDER:-hybrid}"
if [[ "$PRIMARY" == "hybrid" || "$PRIMARY" == "novita" ]]; then
ensure_secret novita_api_key "${NOVITA_API_KEY:?NOVITA_API_KEY required for PRIMARY_PROVIDER=${PRIMARY}}"
else
ensure_secret novita_api_key "${NOVITA_API_KEY:-}"
fi
ensure_secret anthropic_api_key "${ANTHROPIC_API_KEY:-}"
ensure_secret litellm_master_key "${LITELLM_MASTER_KEY:?LITELLM_MASTER_KEY required}"
ensure_secret litellm_salt_key "${LITELLM_SALT_KEY:?LITELLM_SALT_KEY required}"
ensure_secret router_api_key "${ROUTER_API_KEY:?ROUTER_API_KEY required}"
ensure_secret postgres_password "${POSTGRES_PASSWORD:?POSTGRES_PASSWORD required}"
ensure_secret groq_api_key "${GROQ_API_KEY:-}"
ensure_secret xai_api_key "${XAI_API_KEY:-}"
ensure_secret gemini_api_key "${GEMINI_API_KEY:-}"
ensure_secret together_api_key "${TOGETHER_API_KEY:-}"
ensure_secret openrouter_api_key "${OPENROUTER_API_KEY:-}"
ensure_secret gigachat_credentials "${GIGACHAT_CREDENTIALS:-}"
vless_conf_is_stub() {
local f="${1:-vless/vless.conf}"
[[ ! -f "$f" ]] && return 0
grep -qE "0\.0\.0\.0|example\.com|UUID@host|^#" "$f" && return 0
grep -qE "^vless://" "$f" || return 0
return 1
}
if [[ "${VPN_ENABLED:-false}" == "true" ]] && [[ -n "${VLESS_SUB_URL:-}" ]]; then
if vless_conf_is_stub "vless/vless.conf"; then
echo "== Fetch vless.conf from subscription =="
bash scripts/fetch-vless-subscription.sh vless/vless.conf
fi
fi
if [[ ! -f vless/vless.conf ]]; then
echo "WARN: vless/vless.conf missing — stub for secret (VPN off until configured)"
cp vless/vless.conf.example vless/vless.conf 2>/dev/null || echo "# stub" > vless/vless.conf
@@ -46,11 +73,17 @@ ensure_secret vless_conf "$(cat vless/vless.conf)"
echo "== Build images =="
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
docker build -t ai-router/vless-proxy:local ./vless
docker build -t ai-router/vpn-watchdog:local ./watchdog
if [[ "${VPN_ENABLED:-false}" == "true" ]]; then
docker build -t ai-router/vless-proxy:local ./vless
docker build -t ai-router/vpn-watchdog:local ./watchdog
else
echo "VPN off — stub images (replicas=0)"
docker build -f vless/Dockerfile.stub -t ai-router/vless-proxy:local ./vless
docker build -t ai-router/vpn-watchdog:local ./watchdog
fi
echo "== Sync routing config (optional regen) =="
bash scripts/sync-routing-config.sh || true
echo "== Sync routing config (PRIMARY_PROVIDER=${PRIMARY_PROVIDER:-hybrid}) =="
python3 scripts/gen-litellm-config.py
echo "== Deploy stack: ${STACK_NAME} =="
docker stack deploy -c docker-stack.yml --with-registry-auth "${STACK_NAME}"
scripts/diag-400-body.sh
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== litellm 400/408 detail =="
docker service logs --since 40m --raw ai-router_litellm 2>&1 \
| grep -iE 'BadRequest|invalid request|400|deepseek|tool|error|trace_id' \
| tail -80
echo
echo "== router 400 bodies =="
docker service logs --since 40m --raw ai-router_router 2>&1 \
| grep -iE '400|BadRequest|invalid|executor failed|mid-loop' \
| tail -40
REMOTE
scripts/diag-408-after-index.sh
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend 15m =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
coalesce(\"request_duration_ms\",0) AS ms,
left(coalesce(metadata->'error_information'->>'error_message',''), 100) AS er
FROM \"LiteLLM_SpendLogs\"
WHERE \"startTime\" >= now() - interval '20 minutes'
ORDER BY \"startTime\" DESC LIMIT 25;
"
echo "== Router =="
docker service logs --since 20m --raw ai-router_router 2>&1 \
| grep -iE 'path_index|path_resolve|hierarchical_agent phase|executor result|408|escalate|mid-loop|rewrite|kickstart|loadtest|dynamic_conf' \
| tail -60
REMOTE
scripts/diag-compose-path.sh
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
docker service logs --since 20m --raw ai-router_router 2>&1 \
| grep -iE 'docker-compose|kickstart|tool_calls|executor result|rewrite|hierarchical_agent phase|not found' \
| tail -50
REMOTE
scripts/diag-edit-loop.sh
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend 20m =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT to_char(\"startTime\", 'HH24:MI:SS') AS t, status,
CASE WHEN model LIKE '%qwen3.8-max%' THEN 'Max'
WHEN model LIKE '%qwen3-coder%' THEN 'Coder'
WHEN model LIKE '%deepseek%' THEN 'DeepSeek'
ELSE left(model,28) END AS role,
total_tokens AS tok, completion_tokens AS out,
coalesce(\"request_duration_ms\",0) AS ms
FROM \"LiteLLM_SpendLogs\"
WHERE \"startTime\" >= now() - interval '20 minutes'
ORDER BY \"startTime\" ASC;
"
echo
echo "== Router (edit/force/loop) =="
docker service logs --since 20m --raw ai-router_router 2>&1 \
| grep -iE 'force edit|edit_file|rewrite|path_index|hierarchical_agent phase|executor result|synth|sticky|Стоп|fallback|tool_calls|calentiq-ift' \
| grep -vE 'metrics|health|Waiting|Uvicorn|Started|Finished|Shutting' \
| tail -80
REMOTE
scripts/diag-executor-fail.sh
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
echo "== Spend last 30m =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, call_type, total_tokens,
left(coalesce(metadata->'error_information'->>'error_message',''), 180) AS er
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC
LIMIT 25;
"
echo "== Router logs (executor/plan) =="
docker service logs --since 25m ai-router_router 2>&1 \
| grep -iE 'hierarchical|executor|novita-planner|a-medium-code|timeout|408|error|fail|tool' \
| tail -50
echo "== LiteLLM logs errors =="
docker service logs --since 25m ai-router_litellm 2>&1 \
| grep -iE 'ERROR|Exception|timeout|408|502|qwen3-coder|novita-planner|a-medium-code|Give Feedback' \
| tail -40
REMOTE
scripts/diag-fake-paths.sh
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
docker service logs --since 25m --raw ai-router_router 2>&1 \
| grep -iE 'traefik\.yml|rewrite|bogus|kickstart|executor result|hierarchical_agent phase|paths:|Plan approved|tool_calls' \
| tail -60
REMOTE
scripts/diag-hang2.sh
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
echo "== live router logs (last 5m, full) =="
docker service logs --since 8m --raw ai-router_router 2>&1 | tail -80
echo
echo "== litellm last =="
docker service logs --since 8m --raw ai-router_litellm 2>&1 | grep -iE 'qwen|deepseek|timeout|408|error|POST' | tail -40
echo
echo "== in-flight? =="
docker service ps ai-router_router --no-trunc | head -5
REMOTE
scripts/diag-hang3.sh
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend after 19:06 =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
coalesce(\"request_duration_ms\",0) AS ms,
left(coalesce(metadata->'error_information'->>'error_message',''), 120) AS er
FROM \"LiteLLM_SpendLogs\"
WHERE \"startTime\" >= '2026-08-12 19:06:00'
ORDER BY \"startTime\" DESC;
"
echo "== router since 19:06 full agent lines =="
docker service logs --since 15m --raw ai-router_router 2>&1 \
| grep -iE '19:0[6-9]|19:1|executor|tool_loop|408|HTTP Request|kickstart|escalate|fallback|synth|no tool|POST /v1' \
| tail -60
REMOTE
scripts/diag-litellm-fails.sh
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Services =="
docker stack services ai-router --format '{{.Name}} {{.Replicas}} {{.Image}}' | grep -E 'litellm|postgres|router' || true
echo "== Health =="
curl -sS -o /tmp/llh.txt -w "liveliness:%{http_code}\n" --max-time 10 https://litellm.ift.calentiq.com/health/liveliness || true
curl -sS -o /tmp/llr.txt -w "readiness:%{http_code}\n" --max-time 10 https://litellm.ift.calentiq.com/health/readiness || true
head -c 300 /tmp/llh.txt; echo; head -c 500 /tmp/llr.txt; echo
echo "== Recent litellm logs (errors) =="
docker service logs --tail 120 ai-router_litellm 2>&1 \
| grep -iE 'error|exception|fail|traceback|500|401|prisma|database|budget|key|warning' \
| tail -60 || true
echo "== Last 40 lines raw =="
docker service logs --tail 40 ai-router_litellm 2>&1 | tail -40
echo "== Postgres quick =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c \
"SELECT count(*) AS spend FROM \"LiteLLM_SpendLogs\"; SELECT count(*) AS users FROM \"LiteLLM_UserTable\"; SELECT count(*) AS tokens FROM \"LiteLLM_VerificationToken\";"
echo "== UI/home probe =="
curl -sS -o /tmp/ui.txt -w "ui:%{http_code}\n" --max-time 15 https://litellm.ift.calentiq.com/ui/ || true
curl -sS -o /tmp/models.txt -w "models:%{http_code}\n" --max-time 20 \
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
https://litellm.ift.calentiq.com/v1/models || true
head -c 200 /tmp/models.txt; echo
REMOTE
scripts/diag-litellm-fails2.sh
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
echo "== SpendLogs status breakdown =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT status, count(*) FROM \"LiteLLM_SpendLogs\" GROUP BY 1 ORDER BY 2 DESC;
"
echo "== Recent spend rows =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, \"api_key\",
left(coalesce(\"error_information\"::text, ''), 120) AS err,
left(coalesce(request_tags::text, ''), 60) AS tags
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC
LIMIT 20;
"
echo "== ErrorLogs =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c \
"SELECT count(*) FROM \"LiteLLM_ErrorLogs\";"
echo "== Body-read errors around restart =="
docker service logs --since 15m ai-router_litellm 2>&1 \
| grep -iE 'Unexpected error reading request body|ClientDisconnect|ConnectionReset|502|504|failed' \
| tail -30 || true
echo "== Uptime-kuma / probes hitting litellm =="
docker service logs --tail 30 ai-router_uptime-kuma 2>&1 | tail -20 || true
REMOTE
scripts/diag-litellm-fails3.sh
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
USER="${POSTGRES_USER:-litellm}"
DB="${POSTGRES_DB:-litellm}"
echo "== SpendLogs columns =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "\d+ \"LiteLLM_SpendLogs\"" | head -80
echo "== Recent failures =="
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "$USER" -d "$DB" -c "
SELECT \"startTime\", status, model,
left(coalesce(messages::text,''), 80) AS messages,
left(coalesce(metadata::text,''), 200) AS metadata,
left(coalesce(\"proxyServerRequest\"::text,''), 120) AS req,
left(coalesce(response::text,''), 200) AS response
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC
LIMIT 15;
"
REMOTE
scripts/diag-litellm-fails4.sh
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, call_type,
left(coalesce(metadata::text,''), 400) AS metadata,
left(coalesce(response::text,''), 300) AS response
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC
LIMIT 15;
"
REMOTE
scripts/diag-no-tool-calls.sh
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend last 15m =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, call_type, total_tokens, completion_tokens,
left(coalesce(metadata->'error_information'->>'error_message',''), 120) AS er
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC LIMIT 15;
"
echo "== Router =="
docker service logs --since 15m ai-router_router 2>&1 \
| grep -iE 'executor|tool_calls|finish|no_tool|408|escalate|hierarchical_agent|a-medium-code|b-complex' \
| tail -40
REMOTE
scripts/diag-observer-502.sh
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
echo "== curl inside from traefik net =="
# find traefik containe
TR=$(docker ps --filter name=traefik -q | head -1)
OW=$(docker ps --filter name=observer_web -q | head -1)
echo "traefik=$TR observer=$OW"
if [[ -n "$OW" ]]; then
echo "-- observer logs --"
docker logs --tail 30 "$OW" 2>&1 | tail -30
echo "-- wget localhost from observer --"
docker exec "$OW" sh -c 'wget -qO- --timeout=3 http://127.0.0.1/ 2>&1 | head -c 200; echo; wget -qO- --timeout=3 http://127.0.0.1:80/ 2>&1 | head -c 200; echo' || true
echo "-- inspect ports --"
docker inspect "$OW" --format 'IP={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} Ports={{json .NetworkSettings.Ports}}'
fi
if [[ -n "$TR" && -n "$OW" ]]; then
echo "-- traefik -> observer_web:80 --"
docker exec "$TR" wget -qO- --timeout=5 http://observer_web:80/ 2>&1 | head -c 300 || \
docker exec "$TR" wget -qO- --timeout=5 http://eventhub-ift-core_observer_web:80/ 2>&1 | head -c 300 || true
echo
# DNS from traefik
docker exec "$TR" getent hosts observer_web 2>&1 || docker exec "$TR" nslookup observer_web 2>&1 | head -10 || true
fi
echo "== dynamic_conf observer snippet =="
grep -n -A6 'observer' /opt/ai-router-stack/../EventHubDevOps/ift/traefik/dynamic_conf.yml 2>/dev/null | head -5 || \
grep -n -A5 'observer:' /home/*/IdeaProjects/eventHub/EventHubDevOps/ift/traefik/dynamic_conf.yml 2>/dev/null | head || true
# try find compose on host
find /opt -name 'dynamic_conf.yml' 2>/dev/null | head -5
REMOTE
scripts/diag-slow-read.sh
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend last 20m =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
coalesce(\"request_duration_ms\",0) AS ms,
left(coalesce(metadata->'error_information'->>'error_message',''), 100) AS er
FROM \"LiteLLM_SpendLogs\"
ORDER BY \"startTime\" DESC LIMIT 20;
"
echo "== Router recent =="
docker service logs --since 20m ai-router_router 2>&1 \
| grep -iE 'executor|tool_loop|kickstart|escalate|408|no tool|hierarchical_agent|phase=|waiting|timeout' \
| tail -50
REMOTE
scripts/explain-logs.sh
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
echo "== Spend last 25m =="
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
SELECT to_char(\"startTime\", 'HH24:MI:SS') AS t, status,
CASE
WHEN model LIKE '%qwen3.8-max%' THEN 'planner-Max'
WHEN model LIKE '%qwen3-coder%' THEN 'coder-30b'
WHEN model LIKE '%deepseek%' THEN 'DeepSeek'
WHEN model LIKE '%llama%' THEN 'Llama'
ELSE left(model, 40)
END AS role,
total_tokens AS tok, completion_tokens AS out,
coalesce(\"request_duration_ms\",0) AS ms,
left(coalesce(metadata->'error_information'->>'error_message',''), 80) AS er
FROM \"LiteLLM_SpendLogs\"
WHERE \"startTime\" >= now() - interval '25 minutes'
ORDER BY \"startTime\" ASC;
"
echo
echo "== Router agent timeline =="
docker service logs --since 25m --raw ai-router_router 2>&1 \
| grep -iE 'path_index|path_resolve|hierarchical_agent phase|executor forward|executor result|escalate|408|400|rewrite|kickstart|no tool|mid-loop|synth|Выполн|plan' \
| grep -vE 'metrics|health|Waiting for|Application|Uvicorn|Finished server|Started server|Shutting' \
| tail -80
REMOTE
scripts/fetch-vless-subscription.sh
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
# Fetch vless.conf from Happ/mireon subscription (JSON or plain vless:// lines).
# Env: VLESS_SUB_URL, VLESS_UA, VLESS_HWID (Happ → Settings / app id)
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
if [[ -f "${ROOT}/.env" ]]; then
set -a
# shellcheck disable=SC1091
source "${ROOT}/.env"
set +a
fi
SUB_URL="${VLESS_SUB_URL:?set VLESS_SUB_URL}"
OUT="${1:-${ROOT}/vless/vless.conf}"
UA="${VLESS_UA:-Happ/3.3.6/Windows/2607171516600}"
HWID="${VLESS_HWID:-}"
TMP="$(mktemp)"
trap 'rm -f "${TMP}"' EXIT
CURL=(curl -fsSL -A "$UA")
[[ -n "$HWID" ]] && CURL+=(-H "x-hwid: ${HWID}")
"${CURL[@]}" "$SUB_URL" | tr -d '\n\r' | base64 -d >"${TMP}" 2>/dev/null || "${CURL[@]}" "$SUB_URL" >"${TMP}"
pick="$(grep -E '^vless://' "${TMP}" | grep -v '@0\.0\.0\.0:1' | head -1 || true)"
if [[ -z "$pick" ]]; then
pick="$(python3 "${SCRIPT_DIR}/vless-from-subscription-json.py" "${TMP}")" || true
fi
if [[ -z "$pick" ]]; then
echo "fetch-vless-subscription: no valid vless URI" >&2
head -c 400 "${TMP}" >&2
exit 1
fi
mkdir -p "$(dirname "$OUT")"
printf '%s\n' "$pick" >"$OUT"
chmod 600 "$OUT"
echo "Wrote $(wc -c <"$OUT") bytes -> $OUT"
scripts/gen-litellm-config.py
+178
View File
@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""Generate litellm_config.yaml from PRIMARY_PROVIDER + config/*.yaml."""
from __future__ import annotations
import os
import sys
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parent.parent
PROFILE = os.environ.get("PRIMARY_PROVIDER", "hybrid").strip().lower()
def load(path: Path) -> dict:
with path.open(encoding="utf-8") as f:
return yaml.safe_load(f) or {}
def api_key_ref(name: str) -> str:
return f"os.environ/{name}"
def _apply_thinking(params: dict, spec: dict) -> None:
"""Qwen3 defaults to thinking; empty content + hang unless disabled."""
model = str(params.get("model") or spec.get("model") or "").lower()
force = bool(spec.get("disable_thinking"))
auto = ("qwen3" in model) and not spec.get("enable_thinking")
if force or auto:
params["extra_body"] = {
"chat_template_kwargs": {"enable_thinking": False},
"enable_thinking": False,
}
def lane_entry(name: str, spec: dict, rpm: int | None) -> dict:
params: dict = {
"model": spec["model"],
"api_key": api_key_ref(spec["api_key"]),
# Per-deployment timeout: litellm_settings.request_timeout often shows as
# Deployment Info timeout: None on stream/tool hangs.
"timeout": int(os.environ.get("LITELLM_MODEL_TIMEOUT", "180")),
}
if rpm:
params["rpm"] = rpm
_apply_thinking(params, spec)
return {"model_name": name, "litellm_params": params}
def fixed_entry(name: str, spec: dict) -> dict:
params: dict = {
"model": spec["model"],
"api_key": api_key_ref(spec["api_key"]),
"timeout": int(os.environ.get("LITELLM_MODEL_TIMEOUT", "180")),
}
for key in ("ssl_verify", "max_tokens", "temperature"):
if key in spec:
params[key] = spec[key]
_apply_thinking(params, spec)
return {"model_name": name, "litellm_params": params}
def smart_router_internal(rules: dict) -> dict:
litellm_rules = rules.get("litellm", {})
return {
"model_name": "smart-router-internal",
"litellm_params": {
"model": "auto_router/complexity_router",
"drop_params": True,
"complexity_router_default_model": "a-medium-ops",
"complexity_router_config": {
"tiers": {
"SIMPLE": "a-simple",
"MEDIUM": "a-medium-ops",
"MEDIUM_CODE": "a-medium-code",
"COMPLEX": "a-complex",
"REASONING": "a-reasoning",
},
"classifier_fallback": "heuristic",
"keyword_tier_rules": litellm_rules.get("keyword_tier_rules", []),
"custom_technical_keywords": litellm_rules.get(
"custom_technical_keywords", []
),
"token_thresholds": {"simple": 20, "complex": 500},
"tier_boundaries": {
"simple_medium": 0.18,
"medium_complex": 0.38,
"complex_reasoning": 0.62,
},
"session_affinity": True,
"session_affinity_ttl_seconds": 1800,
},
},
}
def main() -> int:
providers = load(ROOT / "config/providers.yaml")
profiles = providers.get("profiles", {})
if PROFILE not in profiles:
print(f"ERROR: unknown PRIMARY_PROVIDER={PROFILE!r}", file=sys.stderr)
print(f"Available: {', '.join(sorted(profiles))}", file=sys.stderr)
return 1
profile = profiles[PROFILE]
base = load(ROOT / "config/litellm.base.yaml")
matrix = load(ROOT / "config/model_matrix.yaml")
rules = load(ROOT / "config/routing_rules.yaml")
meta = matrix.get("models", {})
model_list: list[dict] = []
# Classifier first
fixed = providers.get("fixed_models", {})
if "gigachat-classifier" in fixed:
model_list.append(fixed_entry("gigachat-classifier", fixed["gigachat-classifier"]))
# Lane models from active profile
lanes = profile.get("lanes", {})
for name, spec in lanes.items():
rpm = meta.get(name, {}).get("rpm")
model_list.append(lane_entry(name, spec, rpm))
# Fixed models (Claude / OR fallbacks / GigaChat / optional free tiers)
for name in (
"novita-planner",
"novita-verifier",
"claude-haiku-planner",
"claude-sonnet-verifier",
"groq-llama-8b",
"groq-qwen-coder",
"gemini-flash",
"grok-3",
"gigachat-pro",
):
if name in fixed:
model_list.append(fixed_entry(name, fixed[name]))
model_list.append(smart_router_internal(rules))
# smart-router alias (Zed default)
sr = profile.get("smart_router", {})
if sr:
sr_params: dict = {
"model": sr["model"],
"api_key": api_key_ref(sr["api_key"]),
}
_apply_thinking(sr_params, sr)
model_list.append({"model_name": "smart-router", "litellm_params": sr_params})
router = base.setdefault("router_settings", {})
router["fallbacks"] = [
{k: v for k, v in row.items()}
for row in _fallback_list(profile.get("fallbacks", {}))
]
router["default_fallbacks"] = profile.get(
"default_fallbacks", ["a-medium-code", "groq-qwen-coder"]
)
out_cfg = {**base, "model_list": model_list}
out_path = ROOT / "litellm_config.yaml"
header = (
f"# LiteLLM — generated for PRIMARY_PROVIDER={PROFILE}\n"
f"# Profile: {profile.get('label', PROFILE)}\n"
f"# Regenerate: PRIMARY_PROVIDER={PROFILE} bash scripts/gen-litellm-config.py\n\n"
)
body = yaml.dump(out_cfg, allow_unicode=True, sort_keys=False)
out_path.write_text(header + body, encoding="utf-8")
print(f"Wrote {out_path} (profile={PROFILE}, {len(model_list)} models)")
return 0
def _fallback_list(fallbacks: dict) -> list[dict]:
return [{model: targets} for model, targets in fallbacks.items()]
if __name__ == "__main__":
sys.exit(main())
scripts/litellm-entrypoint.sh
+12
View File
@@ -20,9 +20,21 @@ fi
if [ -f /run/secrets/groq_api_key ]; then
export GROQ_API_KEY="$(read_secret /run/secrets/groq_api_key)"
fi
if [ -f /run/secrets/xai_api_key ]; then
export XAI_API_KEY="$(read_secret /run/secrets/xai_api_key)"
fi
if [ -f /run/secrets/gemini_api_key ]; then
export GEMINI_API_KEY="$(read_secret /run/secrets/gemini_api_key)"
fi
if [ -f /run/secrets/together_api_key ]; then
export TOGETHER_API_KEY="$(read_secret /run/secrets/together_api_key)"
fi
if [ -f /run/secrets/openrouter_api_key ]; then
export OPENROUTER_API_KEY="$(read_secret /run/secrets/openrouter_api_key)"
fi
if [ -f /run/secrets/anthropic_api_key ]; then
export ANTHROPIC_API_KEY="$(read_secret /run/secrets/anthropic_api_key)"
fi
if [ -f /run/secrets/gigachat_credentials ]; then
export GIGACHAT_CREDENTIALS="$(read_secret /run/secrets/gigachat_credentials)"
fi
scripts/no-proxy-default.sh
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Canonical NO_PROXY when litellm uses HTTP_PROXY (VLESS VPN).
#
# Direct (bypass VPN): Novita, Groq, Gemini, GigaChat, internal/Calentiq.
# VPN-only (NOT listed): Anthropic → api.anthropic.com; OpenRouter → openrouter.ai;
# xAI Grok → api.x.ai; Together → api.together.xyz
#
# shellcheck disable=SC2034
NO_PROXY_DEFAULT='localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local'
scripts/refresh-vless-subscription.sh
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Refresh vless.conf from subscription and reload vless-proxy on Swarm (IFT).
set -euo pipefail
STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
STACK_NAME="${STACK_NAME:-ai-router}"
cd "$STACK_DIR"
set -a
# shellcheck disable=SC1091
source .env
set +a
bash "${STACK_DIR}/scripts/fetch-vless-subscription.sh" "${STACK_DIR}/vless/vless.conf"
CONF="$(cat "${STACK_DIR}/vless/vless.conf")"
SVC="${STACK_NAME}_vless-proxy"
rotate_vless_secret() {
if docker secret inspect vless_conf >/dev/null 2>&1; then
echo "== Detach old vless_conf secret =="
docker service scale "${SVC}=0" >/dev/null || true
sleep 3
docker service update --secret-rm vless_conf "${SVC}" >/dev/null 2>&1 || true
sleep 2
docker secret rm vless_conf >/dev/null 2>&1 || true
fi
echo -n "$CONF" | docker secret create vless_conf -
}
if docker service inspect "${SVC}" >/dev/null 2>&1; then
rotate_vless_secret
docker service update \
--secret-add "source=vless_conf,target=/app/vless.conf,mode=0444" \
"${SVC}" >/dev/null
docker service update --force "${SVC}" >/dev/null
TARGET="${VPN_ENABLED:-false}"
if [[ "$TARGET" == "true" ]]; then
docker service scale "${SVC}=1" >/dev/null
fi
echo "refresh-vless: vless-proxy reloaded"
else
rotate_vless_secret
echo "refresh-vless: secret vless_conf updated (service ${SVC} missing — run deploy.sh)"
fi
scripts/setup-vless-refresh-cron.sh
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Cron: refresh VPN subscription every 6h (config может меняться на стороне провайдера).
set -euo pipefail
STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CRON_FILE="/etc/cron.d/ai-router-vless-refresh"
LOG="/var/log/ai-router-vless-refresh.log"
sudo tee "$CRON_FILE" >/dev/null <<EOF
# AI Router: refresh Happ/mireon vless subscription
0 */6 * * * deploy cd ${STACK_DIR} && set -a && . ./.env && set +a && bash scripts/refresh-vless-subscription.sh >>${LOG} 2>&1
EOF
sudo chmod 644 "$CRON_FILE"
echo "Installed ${CRON_FILE} (every 6h)"
scripts/smoke-observer-approve.sh
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
PLAN=$(python3 - <<'PY'
import json
d=json.load(open("/tmp/agent-plan.json"))
msg=(d.get("choices") or [{}])[0].get("message") or {}
print(msg.get("content") or "")
PY
)
# Approve with tools like Zed agent
python3 - <<'PY' > /tmp/agent-approve-req.json
import json
plan=open("/tmp/agent-plan.json").read()
d=json.loads(plan)
content=((d.get("choices") or [{}])[0].get("message") or {}).get("content") or ""
tools=[{
"type":"function",
"function":{
"name":"read_file",
"description":"Read a file",
"parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}
}
},{
"type":"function",
"function":{
"name":"edit_file",
"description":"Edit a file",
"parameters":{"type":"object","properties":{
"path":{"type":"string"},
"edits":{"type":"array","items":{"type":"object","properties":{
"old_text":{"type":"string"},"new_text":{"type":"string"}
}}}
},"required":["path","edits"]}
}
}]
body={
"model":"smart-router",
"stream": False,
"tools": tools,
"tool_choice":"auto",
"messages":[
{"role":"user","content":"observer.ift.calentiq.com не работает - Bad Gateway почини"},
{"role":"assistant","content":content},
{"role":"user","content":"ok"}
]
}
json.dump(body, open("/tmp/agent-approve-req.json","w"), ensure_ascii=False)
print("req bytes", len(json.dumps(body)))
PY
echo "== approve+tools =="
curl -sS --max-time 180 "$BASE/v1/chat/completions" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-H "X-AI-Orchestrate: force" \
-H "X-AI-Quality: balanced" \
-d @/tmp/agent-approve-req.json | tee /tmp/agent-exec1.json | python3 -c '
import json,sys
d=json.load(sys.stdin)
ch=(d.get("choices") or [{}])[0]
msg=ch.get("message") or {}
print("finish", ch.get("finish_reason"))
print("content", (msg.get("content") or "")[:1200])
tcs=msg.get("tool_calls") or []
print("tool_calls", len(tcs))
for tc in tcs:
fn=tc.get("function") or {}
print(" TOOL", fn.get("name"), (fn.get("arguments") or "")[:300])
meta=d.get("x_router_meta") or {}
print("meta phase", meta.get("agent_phase"), "force_edit", meta.get("executor_force_edit_after_read"), "model", meta.get("executor_model") or meta.get("selected_model"))
'
echo "== router log since approve =="
docker service logs --since 3m --raw ai-router_router 2>&1 \
| grep -iE 'path_index|force edit|stop after|hierarchical_agent|executor result|kickstart|phase=' \
| tail -30
REMOTE
scripts/smoke-observer-approve2.sh
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift 'bash -s' <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
test -f /tmp/agent-plan.json && echo "plan_ok $(wc -c </tmp/agent-plan.json)" || echo "no plan"
python3 <<'PY'
import json
d=json.load(open("/tmp/agent-plan.json"))
content=((d.get("choices") or [{}])[0].get("message") or {}).get("content") or ""
tools=[{
"type":"function",
"function":{
"name":"read_file",
"description":"Read a file",
"parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}
}
},{
"type":"function",
"function":{
"name":"edit_file",
"description":"Edit a file",
"parameters":{"type":"object","properties":{
"path":{"type":"string"},
"edits":{"type":"array","items":{"type":"object","properties":{
"old_text":{"type":"string"},"new_text":{"type":"string"}
}}}
},"required":["path","edits"]}
}
}]
body={
"model":"smart-router",
"stream": False,
"tools": tools,
"messages":[
{"role":"user","content":"observer.ift.calentiq.com не работает - Bad Gateway почини"},
{"role":"assistant","content":content},
{"role":"user","content":"ok"}
]
}
json.dump(body, open("/tmp/agent-approve-req.json","w"), ensure_ascii=False)
print("req_ok", len(content), "tools", len(tools))
PY
code=$(curl -sS --max-time 180 -o /tmp/agent-exec1.json -w "%{http_code}" \
"$BASE/v1/chat/completions" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-H "X-AI-Orchestrate: force" \
-H "X-AI-Quality: balanced" \
-d @/tmp/agent-approve-req.json || true)
echo "http=$code bytes=$(wc -c </tmp/agent-exec1.json)"
head -c 500 /tmp/agent-exec1.json; echo
python3 <<'PY'
import json
raw=open("/tmp/agent-exec1.json").read().strip()
if not raw:
print("EMPTY"); raise SystemExit
d=json.loads(raw)
if d.get("error"):
print("ERROR", d["error"]); raise SystemExit
ch=(d.get("choices") or [{}])[0]
msg=ch.get("message") or {}
print("finish", ch.get("finish_reason"))
print("content_head:")
print((msg.get("content") or "")[:1500])
tcs=msg.get("tool_calls") or []
print("n_tools", len(tcs))
for tc in tcs:
fn=tc.get("function") or {}
print("TOOL", fn.get("name"))
print("ARGS", (fn.get("arguments") or "")[:400])
meta=d.get("x_router_meta") or {}
print("phase", meta.get("agent_phase"), "exec", meta.get("executor_model") or meta.get("selected_model"),
"force", meta.get("executor_force_edit_after_read"), "stop", meta.get("executor_stop_after_edit"))
PY
docker service logs --since 4m --raw ai-router_router 2>&1 \
| grep -iE 'path_index|force edit|stop after|phase=|executor result|kickstart|Выполн' \
| tail -25
REMOTE
scripts/smoke-observer-request.sh
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
set -euo pipefail
cd /opt/ai-router-stack
set -a; source .env; set +a
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
echo "BASE=$BASE"
# 1) Plan request (no tools) — hierarchical plan confirm
curl -sS --max-time 120 "$BASE/v1/chat/completions" \
-H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-H "X-AI-Orchestrate: force" \
-H "X-AI-Quality: balanced" \
-d '{
"model": "smart-router",
"stream": false,
"messages": [
{"role": "user", "content": "observer.ift.calentiq.com не работает - Bad Gateway почини"}
]
}' | tee /tmp/agent-plan.json | python3 -c '
import json,sys
d=json.load(sys.stdin)
msg=(d.get("choices") or [{}])[0].get("message") or {}
print("=== PLAN CONTENT ===")
print((msg.get("content") or "")[:2500])
meta=d.get("x_router_meta") or {}
print("=== META keys ===", sorted(meta.keys())[:30])
print("mode", meta.get("mode"), "awaiting", meta.get("awaiting_plan_confirm"), "planner", meta.get("planner_model"))
'
REMOTE
scripts/smoke-test.sh
+36 -6
View File
@@ -14,9 +14,28 @@ BASE="${TEST_BASE_URL:-http://127.0.0.1:8000}"
LITELLM_BASE="${TEST_LITELLM_URL:-http://127.0.0.1:4000}"
ROUTER_KEY="${ROUTER_API_KEY:?ROUTER_API_KEY required}"
CURL=(curl -sf)
if [[ "$BASE" == https:* ]]; then
CURL+=( -k )
curl_common() {
local url="$1"
CURL=(curl -sf)
if [[ "$url" == https:* ]]; then
CURL+=( -k )
fi
if [[ -n "${TEST_RESOLVE_IP:-}" ]]; then
local host="${url#*://}"
host="${host%%/*}"
host="${host%%:*}"
CURL+=( --resolve "${host}:443:${TEST_RESOLVE_IP}" --resolve "${host}:80:${TEST_RESOLVE_IP}" )
fi
}
curl_common "${BASE}"
if [[ "$LITELLM_BASE" == https:* && -n "${TEST_RESOLVE_IP:-}" ]]; then
LITELLM_CURL=(curl -sf -k)
lh="${LITELLM_BASE#*://}"; lh="${lh%%/*}"; lh="${lh%%:*}"
LITELLM_CURL+=( --resolve "${lh}:443:${TEST_RESOLVE_IP}" --resolve "${lh}:80:${TEST_RESOLVE_IP}" )
else
LITELLM_CURL=(curl -sf)
[[ "$LITELLM_BASE" == https:* ]] && LITELLM_CURL+=( -k )
fi
echo "== Router health =="
@@ -34,20 +53,31 @@ echo "== Classify COMPLEX =="
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Спроектируй архитектуру microservices"}]}' | jq .
echo "== Chat smart-router max_tokens=16 =="
echo "== Chat smart-router max_tokens=16 (single-shot / no force hierarchical) =="
"${CURL[@]}" "${BASE}/v1/chat/completions" \
-H "Authorization: Bearer ${ROUTER_KEY}" \
-H "Content-Type: application/json" \
-H "X-AI-Orchestrate: off" \
-d '{"model":"smart-router","max_tokens":16,"messages":[{"role":"user","content":"bash docker service ls"}]}' \
| jq '.choices[0].message.content, .x_router_meta // empty'
if [[ "${SMOKE_HIERARCHICAL:-}" == "1" ]]; then
echo "== Hierarchical force (Novita plan/verify + workers; VPN not required) =="
"${CURL[@]}" "${BASE}/v1/chat/completions" \
-H "Authorization: Bearer ${ROUTER_KEY}" \
-H "Content-Type: application/json" \
-H "X-AI-Orchestrate: force" \
-d '{"model":"smart-router","max_tokens":256,"messages":[{"role":"user","content":"Разбей на 2 шага: 1) что такое Docker 2) одна команда docker ps. Кратко."}]}' \
| jq '.x_router_meta.mode, .x_router_meta.planner_model, .x_router_meta.worker_calls, .x_router_meta.verify_skipped, (.choices[0].message.content|.[0:200])'
fi
if [[ -n "${SKIP_LITELLM_SMOKE:-}" ]]; then
echo "SKIP_LITELLM_SMOKE set — skipping LiteLLM checks"
else
echo "== LiteLLM liveliness =="
"${CURL[@]}" "${LITELLM_BASE}/health/liveliness" && echo
"${LITELLM_CURL[@]}" "${LITELLM_BASE}/health/liveliness" && echo
echo "== LiteLLM UI =="
curl -sfI -k "${LITELLM_BASE}/ui" | head -3 || true
"${LITELLM_CURL[@]}" -I "${LITELLM_BASE}/ui" | head -3 || true
fi
echo "All smoke checks passed."
scripts/switch-provider.sh
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
# Switch PRIMARY_PROVIDER and regenerate litellm_config.yaml
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
PROFILE="${1:-}"
if [[ -z "$PROFILE" ]]; then
echo "Usage: bash scripts/switch-provider.sh <profile>" >&2
echo "Profiles:" >&2
python3 -c "
import yaml
from pathlib import Path
p = yaml.safe_load(Path('config/providers.yaml').read_text(encoding='utf-8'))
for k, v in sorted(p.get('profiles', {}).items()):
wc = ' [welcome credit]' if v.get('welcome_credit') else ''
print(f' {k:12} {v.get(\"label\", \"\")}{wc}')
"
exit 1
fi
if [[ ! -f .env ]]; then
echo "ERROR: .env missing — copy from .env.example" >&2
exit 1
fi
if grep -q '^PRIMARY_PROVIDER=' .env; then
sed -i "s/^PRIMARY_PROVIDER=.*/PRIMARY_PROVIDER=${PROFILE}/" .env
else
echo "PRIMARY_PROVIDER=${PROFILE}" >> .env
fi
set -a
# shellcheck disable=SC1091
source .env
set +a
python3 scripts/gen-litellm-config.py
echo ""
echo "Switched to PRIMARY_PROVIDER=${PROFILE}"
echo "Redeploy on IFT: bash scripts/deploy.sh (Swarm configs are immutable — stack rm if deploy fails)"
scripts/sync-routing-config.sh
+9 -75
View File
@@ -1,78 +1,12 @@
#!/usr/bin/env bash
# Sync model_list fragment from config/*.yaml
# Regenerate litellm_config.yaml from PRIMARY_PROVIDER + config/providers.yaml
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
export ROOT
python3 <<'PY'
import os
import yaml
from pathlib import Path
root = Path(os.environ["ROOT"])
rules = yaml.safe_load((root / "config/routing_rules.yaml").read_text(encoding="utf-8"))
matrix = yaml.safe_load((root / "config/model_matrix.yaml").read_text(encoding="utf-8"))
entries = []
for name, cfg in matrix.get("models", {}).items():
novita = cfg.get("novita")
if not novita:
continue
entry = {
"model_name": name,
"litellm_params": {
"model": novita,
"api_key": "os.environ/NOVITA_API_KEY",
},
}
if cfg.get("rpm"):
entry["litellm_params"]["rpm"] = cfg["rpm"]
entries.append(entry)
for name, cfg in matrix.get("optional_providers", {}).items():
entries.append({
"model_name": name,
"litellm_params": {
"model": cfg["model"],
"api_key": cfg.get("api_key", "os.environ/GROQ_API_KEY"),
},
})
litellm_rules = rules.get("litellm", {})
entries.append({
"model_name": "smart-router-internal",
"litellm_params": {
"model": "auto_router/complexity_router",
"drop_params": True,
"complexity_router_default_model": "a-medium-ops",
"complexity_router_config": {
"tiers": {
"SIMPLE": "a-simple",
"MEDIUM": "a-medium-ops",
"MEDIUM_CODE": "a-medium-code",
"COMPLEX": "a-complex",
"REASONING": "a-reasoning",
},
"classifier_fallback": "heuristic",
"keyword_tier_rules": litellm_rules.get("keyword_tier_rules", []),
"custom_technical_keywords": litellm_rules.get("custom_technical_keywords", []),
"token_thresholds": {"simple": 20, "complex": 500},
"tier_boundaries": {
"simple_medium": 0.18,
"medium_complex": 0.38,
"complex_reasoning": 0.62,
},
"session_affinity": True,
"session_affinity_ttl_seconds": 1800,
},
},
})
out = root / "litellm_config.generated.yaml"
out.write_text(
yaml.dump({"model_list": entries}, allow_unicode=True, sort_keys=False),
encoding="utf-8",
)
print(f"Wrote {out} ({len(entries)} models)")
PY
cd "$ROOT"
if [[ -f .env ]]; then
set -a
# shellcheck disable=SC1091
source .env
set +a
fi
python3 scripts/gen-litellm-config.py
scripts/vless-from-subscription-json.py
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""Extract first working vless:// URI from Happ/mireon subscription JSON."""
import json
import sys
import urllib.parse
path = sys.argv[1]
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
configs = data if isinstance(data, list) else [data]
for j in configs:
if not isinstance(j, dict):
continue
for ob in j.get("outbounds", []):
if ob.get("protocol") != "vless":
continue
tag = ob.get("tag") or "proxy"
st = ob.get("settings") or {}
vnext = (st.get("vnext") or [{}])[0]
addr, port = vnext.get("address"), vnext.get("port")
users = (vnext.get("users") or [{}])[0]
uid = users.get("id")
if not all([addr, port, uid]) or addr in ("0.0.0.0", "127.0.0.1"):
continue
flow = users.get("flow") or ""
stream = ob.get("streamSettings") or {}
net = stream.get("network") or "tcp"
sec = stream.get("security") or "none"
rs = stream.get("realitySettings") or {}
ts = stream.get("tlsSettings") or {}
sni = rs.get("serverName") or ts.get("serverName") or ""
params = {"encryption": "none", "security": sec, "type": net}
if sni:
params["sni"] = sni
if rs.get("publicKey"):
params["pbk"] = rs["publicKey"]
if rs.get("shortId"):
params["sid"] = rs["shortId"]
if rs.get("fingerprint"):
params["fp"] = rs["fingerprint"]
if flow:
params["flow"] = flow
q = urllib.parse.urlencode(params)
print(f"vless://{uid}@{addr}:{port}?{q}#{tag}")
sys.exit(0)
sys.exit(1)
scripts/vpn-disable.sh
+3 -1
View File
@@ -10,7 +10,9 @@ set -a
source .env
set +a
NO_PROXY_VAL="${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}"
# shellcheck disable=SC1091
source "${STACK_DIR}/scripts/no-proxy-default.sh"
NO_PROXY_VAL="${NO_PROXY:-${NO_PROXY_DEFAULT}}"
echo "== Scale vless-proxy and vpn-watchdog to 0 =="
docker service scale "${STACK_NAME}_vless-proxy=0" "${STACK_NAME}_vpn-watchdog=0" || true
scripts/vpn-enable.sh
+4 -2
View File
@@ -11,7 +11,9 @@ source .env
set +a
VLESS_URL="${VLESS_PROXY_URL:-http://vless-proxy:8080}"
NO_PROXY_VAL="${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}"
# shellcheck disable=SC1091
source "${STACK_DIR}/scripts/no-proxy-default.sh"
NO_PROXY_VAL="${NO_PROXY:-${NO_PROXY_DEFAULT}}"
echo "== Scale vless-proxy and vpn-watchdog to 1 =="
docker service scale "${STACK_NAME}_vless-proxy=1" "${STACK_NAME}_vpn-watchdog=1"
@@ -42,4 +44,4 @@ docker service update \
--env-add "NO_PROXY=${NO_PROXY_VAL}" \
"${STACK_NAME}_litellm"
echo "VPN enabled. Novita bypasses via NO_PROXY."
echo "VPN enabled. Direct: Novita/Groq/Gemini/GigaChat. VPN: Anthropic/OpenRouter/Grok/Together."
test/unit/test_hierarchical.py
+1044
View File
File diff suppressed because it is too large Load Diff
test/unit/test_path_resolve.py
+154
View File
@@ -0,0 +1,154 @@
"""Unit tests for deterministic path index + resolve gating."""
from __future__ import annotations
import json
import unittest
from path_resolve import (
apply_deterministic_path_index,
enrich_plan_from_discovery_tools,
match_index_paths,
path_resolve_needed,
)
class TestPathResolve(unittest.TestCase):
def test_index_hits_traefik_not_fake_yml(self) -> None:
plan = {
"user_goal": "fix observer.ift.calentiq.com traefik route",
"subtasks": [
{
"id": "1",
"prompt": "edit traefik for observer",
"paths": [
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\traefik.yml"
],
}
],
}
# sanitize would drop fake; index should still hit via keywords
indexed = match_index_paths(plan)
self.assertTrue(indexed)
self.assertTrue(
indexed[0].lower().replace("/", "\\").endswith(
r"ift\traefik\dynamic_conf.yml"
)
)
fixed, unresolved = apply_deterministic_path_index(plan)
self.assertEqual(unresolved, [])
self.assertFalse(path_resolve_needed(fixed))
self.assertTrue(
fixed["subtasks"][0]["paths"][0]
.lower()
.replace("/", "\\")
.endswith(r"ift\traefik\dynamic_conf.yml")
)
def test_index_compose_core(self) -> None:
plan = {
"user_goal": "bump timeout in docker compose swarm",
"subtasks": [{"id": "1", "prompt": "edit compose", "paths": []}],
}
fixed, unresolved = apply_deterministic_path_index(plan)
self.assertEqual(unresolved, [])
self.assertIn("docker-compose.core.yml", fixed["subtasks"][0]["paths"][0])
def test_miss_triggers_queries(self) -> None:
plan = {
"user_goal": "EventHubDevOps mystery widget xyz",
"subtasks": [{"id": "1", "prompt": "widget", "paths": []}],
}
fixed, unresolved = apply_deterministic_path_index(plan)
self.assertTrue(unresolved)
self.assertTrue(path_resolve_needed(fixed))
def test_enrich_from_find_tool(self) -> None:
plan = {
"path_resolve_queries": ["dynamic_conf.yml"],
"path_resolve": "needed",
"subtasks": [{"id": "1", "prompt": "x", "paths": []}],
}
yml = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
messages = [
{
"role": "assistant",
"tool_calls": [
{
"id": "f1",
"type": "function",
"function": {
"name": "find_path",
"arguments": json.dumps({"query": "dynamic_conf"}),
},
}
],
},
{"role": "tool", "tool_call_id": "f1", "content": yml + "\n"},
]
enriched = enrich_plan_from_discovery_tools(plan, messages)
self.assertEqual(enriched["path_resolve"], "find")
self.assertFalse(path_resolve_needed(enriched))
self.assertEqual(enriched["subtasks"][0]["paths"][0], yml)
def test_enrich_ignores_yaml_etc_artifacts(self) -> None:
plan = {
"path_resolve": "index",
"subtasks": [
{
"id": "1",
"prompt": "traefik",
"paths": [
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
],
"path_source": "index",
}
],
}
# read_file body mentions container paths — must NOT poison plan
yml_body = (
"http:\n routers:\n"
" # volume: /etc/traefik/dynamic_conf.yml\n"
" # also null:/etc/nginx/conf.d/default.conf\n"
)
messages = [
{
"role": "assistant",
"tool_calls": [
{
"id": "r1",
"type": "function",
"function": {
"name": "read_file",
"arguments": json.dumps(
{
"path": r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
}
),
},
}
],
},
{"role": "tool", "tool_call_id": "r1", "content": yml_body},
]
enriched = enrich_plan_from_discovery_tools(plan, messages)
self.assertEqual(enriched["path_resolve"], "index")
self.assertIn(
"dynamic_conf.yml",
enriched["subtasks"][0]["paths"][0],
)
self.assertNotIn("nginx", json.dumps(enriched).lower())
# Without index, still ignore read bodies (no find_* call ids)
plan2 = {
"path_resolve_queries": ["x"],
"path_resolve": "needed",
"subtasks": [{"id": "1", "prompt": "x", "paths": []}],
}
enriched2 = enrich_plan_from_discovery_tools(plan2, messages)
self.assertEqual(enriched2.get("path_resolve"), "needed")
self.assertEqual(enriched2["subtasks"][0]["paths"], [])
if __name__ == "__main__":
unittest.main()
test/unit/test_progress_ui.py
+67
View File
@@ -0,0 +1,67 @@
"""Tests for Zed-facing progress UI."""
from __future__ import annotations
import unittest
from progress_ui import (
execution_banner,
format_progress_block,
humanize_line,
short_path,
tool_status,
)
class TestProgressUi(unittest.TestCase):
def test_humanize_drops_noise(self) -> None:
self.assertIsNone(humanize_line("ждём LiteLLM `b-complex` (до 45с)…"))
self.assertIsNone(humanize_line("ctx 7358/12000"))
self.assertEqual(humanize_line("план…"), "Планирую задачу")
self.assertIn(
"утверждён",
humanize_line("план: утверждён пользователем (3 подзадач)") or "",
)
def test_format_block_numbered(self) -> None:
block = format_progress_block(
[
"план…",
"ждём LiteLLM x",
"план: утверждён пользователем (3 подзадач)",
"agent: план готов → executor с tools (Zed)",
]
)
self.assertIn("**Ход**", block)
self.assertIn("1. Планирую задачу", block)
self.assertIn("2. План утверждён", block)
self.assertNotIn("LiteLLM", block)
def test_execution_banner_shows_short_path(self) -> None:
plan = {
"subtasks": [
{
"paths": [
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
],
"path_source": "index",
}
]
}
text = execution_banner(path_mode=False, plan=plan, model="a-medium-code")
self.assertIn("**Выполнение**", text)
self.assertIn("dynamic_conf.yml", text)
self.assertIn("индекс", text)
self.assertNotIn("a-medium-code", text)
def test_short_path_and_tool_status(self) -> None:
self.assertEqual(
short_path(r"C:\x\EventHubDevOps\ift\traefik\dynamic_conf.yml"),
r"ift\traefik\dynamic_conf.yml",
)
self.assertIn("Читаю", tool_status(["read_file"], path=r"a\b\c.yml"))
self.assertIn("Правлю", tool_status(["edit_file"]))
if __name__ == "__main__":
unittest.main()
test/unit/test_runtime_probe.py
+98
View File
@@ -0,0 +1,98 @@
"""Unit tests for DevOps runtime_probe."""
from __future__ import annotations
import unittest
from unittest.mock import patch
from runtime_probe import (
attach_runtime_probe,
devops_blocks_blind_force_edit,
extract_hosts,
is_devops_request,
suggested_traefik_edit,
)
class TestRuntimeProbe(unittest.TestCase):
def test_extract_hosts(self) -> None:
hosts = extract_hosts(
"observer.ift.calentiq.com не работает — Bad Gateway"
)
self.assertEqual(hosts, ["observer.ift.calentiq.com"])
def test_is_devops_request(self) -> None:
self.assertTrue(
is_devops_request(
{"user_goal": "fix traefik"},
"observer.ift.calentiq.com Bad Gateway",
)
)
self.assertFalse(is_devops_request({"user_goal": "rename button"}))
def test_attach_probe_sets_facts(self) -> None:
plan = {
"user_goal": "observer.ift.calentiq.com Bad Gateway",
"subtasks": [
{
"id": "1",
"prompt": "fix traefik",
"paths": [],
}
],
}
fake = {
"host": "observer.ift.calentiq.com",
"http_status": 502,
"service": "observer_web",
"ports": {"observer_web:80": False, "observer_web:4000": True},
"open_ports": [4000],
"suggested_backend_url": "http://observer_web:4000",
"hint": "port mismatch",
"source": "gateway_tcp",
"closed_port_80_but_alt_open": True,
}
with patch("runtime_probe.probe_host", return_value=fake):
out = attach_runtime_probe(
plan,
user_text="observer.ift.calentiq.com не работает",
cfg={"runtime_probe_enabled": True},
)
self.assertEqual(out["runtime_probe"], "gateway")
self.assertEqual(
out["runtime_facts"]["suggested_backend_url"],
"http://observer_web:4000",
)
self.assertIn("4000", out["subtasks"][0]["edit_goal"])
path0 = out["subtasks"][0]["paths"][0]
self.assertTrue(path0.endswith("dynamic_conf.yml"))
def test_suggested_traefik_edit(self) -> None:
plan = {
"runtime_facts": {
"service": "observer_web",
"suggested_backend_url": "http://observer_web:4000",
"hint": "x",
}
}
fix = suggested_traefik_edit(plan)
assert fix is not None
self.assertEqual(fix["old_text"], 'url: "http://observer_web:80"')
self.assertEqual(fix["new_text"], 'url: "http://observer_web:4000"')
def test_blocks_blind_without_facts(self) -> None:
plan = {"user_goal": "Bad Gateway traefik observer.ift.calentiq.com"}
self.assertTrue(devops_blocks_blind_force_edit(plan))
plan["runtime_facts"] = {"host": "x", "http_status": 502}
self.assertFalse(devops_blocks_blind_force_edit(plan))
def test_disabled(self) -> None:
plan = {"user_goal": "observer.ift.calentiq.com"}
out = attach_runtime_probe(
plan, user_text="x", cfg={"runtime_probe_enabled": False}
)
self.assertIsNone(out.get("runtime_facts"))
if __name__ == "__main__":
unittest.main()
vless/Dockerfile
+8 -2
View File
@@ -1,3 +1,9 @@
FROM thejohnd0e/vless-to-http:latest
FROM alpine:3.20
# vless.conf mounted via Swarm secret at /app/vless.conf
WORKDIR /app
RUN apk add --no-cache curl unzip ca-certificates wget procps && curl -L -o /tmp/xray.zip https://github.com/XTLS/Xray-core/releases/latest/download/Xray-linux-64.zip && unzip /tmp/xray.zip -d /tmp/xray && install -m755 /tmp/xray/xray /usr/local/bin/Xray && mkdir -p /usr/local/share/xray && mv /tmp/xray/geoip.dat /tmp/xray/geosite.dat /usr/local/share/xray/ && rm -rf /tmp/xray /tmp/xray.zip
COPY entrypoint.sh /app/entrypoint.sh
RUN chmod +x /app/entrypoint.sh
EXPOSE 8080
CMD ["/app/entrypoint.sh"]
vless/Dockerfile.stub
+5
View File
@@ -0,0 +1,5 @@
# Stub when VPN_ENABLED=false (vless replicas=0). Replace with Dockerfile when enabling VPN.
FROM alpine:3.20
RUN apk add --no-cache wget
EXPOSE 8080
CMD ["sleep", "infinity"]
vless/entrypoint.sh
+184
View File
@@ -0,0 +1,184 @@
#!/bin/sh
set -eu
CFG=/app/config.json
# ---- read vless.conf (compose mount or swarm secret rotate) ----
CONF=""
for p in /app/vless.conf /run/secrets/vless_conf; do
if [ -s "$p" ]; then CONF="$p"; break; fi
done
if [ -z "$CONF" ]; then
echo "vless.conf not found or empty" >&2
exit 23
fi
VLESS_URL="$(tr -d '\r\n' < "$CONF")"
VLESS_URL="${VLESS_URL%%#*}"
# ---- extract fields from URL ----
USER_ID="$(printf '%s' "$VLESS_URL" | sed -n 's#^vless://\([^@/]*\).*#\1#p')"
SERVER="$( printf '%s' "$VLESS_URL" | sed -n 's#.*@\([^:/?]*\).*#\1#p')"
PORT_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*:\([0-9][0-9]*\).*#\1#p')"
PORT="$(printf '%s' "$PORT_RAW" | tr -cd '0-9')"
PUBKEY="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]pbk=\([^&]*\).*#\1#p')"
SNI="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]sni=\([^&]*\).*#\1#p')"
FP="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]fp=\([^&]*\).*#\1#p')"
SID="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]sid=\([^&]*\).*#\1#p')"
SPX="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]spx=\([^&]*\).*#\1#p' | sed 's/%2F/\//g')"
FLOW="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]flow=\([^&]*\).*#\1#p')"
TYPE="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]type=\([^&]*\).*#\1#p')"
PATH_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]path=\([^&]*\).*#\1#p')"
HOST_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]host=\([^&]*\).*#\1#p')"
MODE="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]mode=\([^&]*\).*#\1#p')"
[ -z "${FP:-}" ] && FP="firefox"
[ -z "${SPX:-}" ] && SPX="/"
[ -z "${TYPE:-}" ] && TYPE="tcp"
XHTTP_PATH="$(printf '%s' "${PATH_RAW:-}" | sed 's/%2[Ff]/\//g')"
HOST="$(printf '%s' "${HOST_RAW:-}" | sed 's/%2[Cc]/,/g')"
# ---- minimal validation ----
[ -n "${USER_ID:-}" ] || { echo "ERR: empty USER_ID"; exit 23; }
[ -n "${SERVER:-}" ] || { echo "ERR: empty SERVER"; exit 23; }
[ -n "${PORT:-}" ] || { echo "ERR: empty PORT"; exit 23; }
[ -n "${PUBKEY:-}" ] || { echo "ERR: empty PUBKEY"; exit 23; }
[ -n "${SNI:-}" ] || { echo "ERR: empty SNI"; exit 23; }
[ -n "${SID:-}" ] || { echo "ERR: empty SID"; exit 23; }
case "$TYPE" in
tcp|xhttp) ;;
*)
echo "ERR: unsupported transport type '$TYPE' (supported: tcp, xhttp)" >&2
exit 23
;;
esac
if [ "$TYPE" = "xhttp" ] && [ -z "${XHTTP_PATH:-}" ]; then
echo "ERR: empty PATH for xhttp transport" >&2
exit 23
fi
# ---- build user block safely (with/without flow) ----
if [ -n "${FLOW:-}" ]; then
USER_BLOCK=$(cat <<JSON
{
"id": "${USER_ID}",
"encryption": "none",
"level": 0,
"flow": "${FLOW}"
}
JSON
)
else
USER_BLOCK=$(cat <<JSON
{
"id": "${USER_ID}",
"encryption": "none",
"level": 0
}
JSON
)
fi
if [ "$TYPE" = "xhttp" ]; then
XHTTP_HOST_LINE=""
XHTTP_MODE_LINE=""
if [ -n "${HOST:-}" ]; then
XHTTP_HOST_LINE=$(cat <<JSON
,
"host": "${HOST}"
JSON
)
fi
if [ -n "${MODE:-}" ]; then
XHTTP_MODE_LINE=$(cat <<JSON
,
"mode": "${MODE}"
JSON
)
fi
STREAM_SETTINGS_BLOCK=$(cat <<JSON
"streamSettings": {
"network": "xhttp",
"security": "reality",
"realitySettings": {
"show": false,
"publicKey": "${PUBKEY}",
"shortId": "${SID}",
"spiderX": "${SPX}",
"fingerprint": "${FP}",
"serverName": "${SNI}"
},
"xhttpSettings": {
"path": "${XHTTP_PATH}"${XHTTP_HOST_LINE}${XHTTP_MODE_LINE}
}
},
JSON
)
else
STREAM_SETTINGS_BLOCK=$(cat <<JSON
"streamSettings": {
"network": "tcp",
"security": "reality",
"realitySettings": {
"show": false,
"publicKey": "${PUBKEY}",
"shortId": "${SID}",
"spiderX": "${SPX}",
"fingerprint": "${FP}",
"serverName": "${SNI}"
}
},
JSON
)
fi
# ---- generate config.json ----
cat > "$CFG" <<EOF
{
"log": { "loglevel": "debug" },
"inbounds": [
{
"port": 8080,
"protocol": "http",
"listen": "0.0.0.0",
"settings": { "allowTransparent": true, "timeout": 300 },
"sniffing": { "enabled": true, "destOverride": ["http","tls"] }
},
{
"port": 1080,
"protocol": "socks",
"listen": "0.0.0.0",
"settings": { "auth": "noauth", "udp": true }
}
],
"outbounds": [
{
"protocol": "vless",
"settings": {
"vnext": [
{
"address": "${SERVER}",
"port": ${PORT},
"users": [
${USER_BLOCK}
]
}
]
},
${STREAM_SETTINGS_BLOCK}
"tag": "proxy"
},
{ "protocol": "freedom", "settings": {}, "tag": "direct" }
]
}
EOF
# ---- print generated config for debugging ----
echo "===== GENERATED CONFIG ====="
cat "$CFG"
echo "============================"
# ---- start Xray ----
exec /usr/local/bin/Xray run -config "$CFG"