feat(agent): hierarchical executor with path resolve, runtime probe, quiet UI
Make Zed Agent closer to Cursor: deterministic DevOps path index, live Traefik port probe before blind edits, stop-after-edit, and quieter Russian progress.
This commit is contained in:
+27
-7
@@ -9,7 +9,7 @@ DEPLOY_HOST=https://ai-router.ift.calentiq.com
|
||||
# --- LiteLLM ---
|
||||
LITELLM_MASTER_KEY=sk-litellm-change-me
|
||||
LITELLM_SALT_KEY=sk-salt-generate-once-never-change
|
||||
LITELLM_IMAGE=ghcr.io/berriai/litellm:main-v1.96.0-stable
|
||||
LITELLM_IMAGE=ghcr.io/berriai/litellm:v1.95.0
|
||||
PROXY_BASE_URL=https://litellm.ift.calentiq.com
|
||||
|
||||
# Postgres (LiteLLM Admin UI + spend logs)
|
||||
@@ -17,15 +17,27 @@ POSTGRES_USER=litellm
|
||||
POSTGRES_DB=litellm
|
||||
POSTGRES_PASSWORD=change-me-postgres
|
||||
|
||||
# Novita AI (direct from IFT, no VPN)
|
||||
# --- Provider switch (lane backends) ---
|
||||
# hybrid (default) = Novita workers + Novita plan/verify | openrouter = VPN reserve
|
||||
PRIMARY_PROVIDER=hybrid
|
||||
|
||||
# Novita AI (direct from IFT, no VPN) — required for hybrid
|
||||
NOVITA_API_KEY=
|
||||
|
||||
# Optional fallbacks
|
||||
# Anthropic — optional (card/region often blocked); hierarchical uses Novita plan/verify
|
||||
ANTHROPIC_API_KEY=
|
||||
|
||||
# Optional fixed fallbacks
|
||||
GROQ_API_KEY=
|
||||
GEMINI_API_KEY=
|
||||
TOGETHER_API_KEY=
|
||||
# OpenRouter reserve (VPN). Note: Anthropic/OpenAI/Google may be region-blocked on billing address.
|
||||
OPENROUTER_API_KEY=
|
||||
|
||||
# GigaChat — optional LLM classifier (hybrid mode, freemium for физлица)
|
||||
# Authorization key from https://developers.sber.ru/studio/ → GigaChat API
|
||||
# xAI Grok — VPN only
|
||||
XAI_API_KEY=
|
||||
|
||||
# GigaChat — classifier (hybrid mode)
|
||||
GIGACHAT_CREDENTIALS=
|
||||
CLASSIFIER_MODE=hybrid
|
||||
CLASSIFIER_LLM_MODEL=gigachat-classifier
|
||||
@@ -33,10 +45,18 @@ CLASSIFIER_LLM_MODEL=gigachat-classifier
|
||||
# Budget (USD/month, also in litellm_config.yaml)
|
||||
LITELLM_MAX_BUDGET=50
|
||||
|
||||
# --- VPN (off by default) ---
|
||||
# --- VPN (OpenRouter + Grok + Anthropic → VPN; Novita/Groq/Gemini/GigaChat → NO_PROXY) ---
|
||||
# Hierarchical plan/verify на Novita — VPN НЕ обязателен
|
||||
# 1) vless/vless.conf или VLESS_SUB_URL
|
||||
# 2) VPN_ENABLED=true → deploy + vpn-enable.sh (для OR/Grok/Anthropic)
|
||||
# 3) api.anthropic.com, openrouter.ai, api.x.ai НЕ в NO_PROXY
|
||||
VPN_ENABLED=false
|
||||
VLESS_SUB_URL=
|
||||
VLESS_HWID=
|
||||
VLESS_UA=Happ/3.3.6/Windows/2607171516600
|
||||
VLESS_PROXY_URL=http://vless-proxy:8080
|
||||
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local
|
||||
# См. scripts/no-proxy-default.sh
|
||||
NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local
|
||||
|
||||
DISABLE_AIOHTTP_TRANSPORT=True
|
||||
USE_AIOHTTP_TRANSPORT=False
|
||||
|
||||
+14
-1
@@ -17,9 +17,22 @@ jobs:
|
||||
- name: Lint Python syntax
|
||||
run: python3 -m py_compile router/*.py
|
||||
|
||||
- name: Unit tests (hierarchical)
|
||||
run: |
|
||||
pip install httpx pyyaml
|
||||
PYTHONPATH=router python3 -m unittest discover -s test/unit -v
|
||||
|
||||
sync-config:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Sync routing config
|
||||
- name: Install PyYAML
|
||||
run: pip install pyyaml
|
||||
- name: Sync routing config (hybrid)
|
||||
env:
|
||||
PRIMARY_PROVIDER: hybrid
|
||||
run: bash scripts/sync-routing-config.sh
|
||||
- name: Sync routing config (openrouter reserve)
|
||||
env:
|
||||
PRIMARY_PROVIDER: openrouter
|
||||
run: bash scripts/gen-litellm-config.py
|
||||
|
||||
@@ -6,3 +6,5 @@ __pycache__/
|
||||
litellm_config.generated.yaml
|
||||
/tmp/
|
||||
*.log
|
||||
.tmp-*
|
||||
.tmp/
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# AGENTS.md — EventHubAiRouter
|
||||
|
||||
Zed OpenAI-compatible gateway на IFT. Карта продукта: `../EventHubSpec/ZED-ARCHITECTURE.md`.
|
||||
|
||||
## Ключевые файлы
|
||||
|
||||
| Файл | Роль |
|
||||
|------|------|
|
||||
| `router/router.py` | `/v1/chat/completions`; ветка `tools` → agent hierarchical |
|
||||
| `router/hierarchical.py` | Plan (Max) / workers / verify (DeepSeek) / `plan_confirm` |
|
||||
| `router/agent_hier.py` | При tools: executor + tool_calls (Zed Write); без synthetic по умолчанию |
|
||||
| `router/orchestrator.py` | Tiers / lanes, Redis session |
|
||||
| `config/orchestration.yaml` | Роли моделей, таймауты, `plan_confirm`, synthetic=never |
|
||||
| `config/providers.yaml` | Novita / VPN (`novita-planner` = Qwen3.8-Max) |
|
||||
| `scripts/gen-litellm-config.py` | LiteLLM config |
|
||||
|
||||
## Модели (бюджет)
|
||||
|
||||
| Роль | Модель |
|
||||
|------|--------|
|
||||
| Planner | `novita-planner` → **qwen/qwen3.8-max** |
|
||||
| Routine tools / medium_code | **qwen3-coder-30b** (`a-medium-code`) |
|
||||
| Hard | DeepSeek V3.2 (`b-complex`); `quality=max` → Max (`c-complex`) |
|
||||
| Verify | DeepSeek V3.2 |
|
||||
| Text SIMPLE (без tools) | Llama 8B — **не** в Write/tools |
|
||||
|
||||
## Поведение для Zed
|
||||
|
||||
- **Primary клиент:** Zed Agent → `smart-router` (не Claude Code/Codex ACP).
|
||||
- **Write** (`tools`): Max-план → утверждение → Coder-30B `tool_calls` (файлы правит Zed); escalate на DeepSeek после edit fails.
|
||||
- **Без tools:** text hierarchical (plan → workers → synth).
|
||||
- Endpoint: `https://ai-router.ift.calentiq.com/v1`.
|
||||
|
||||
## Не делать
|
||||
|
||||
- Не коммитить секреты (`.env`, ключи) и локальные `.tmp-*`.
|
||||
- Не сажать Llama / `a-simple` на tool-loop.
|
||||
- Deploy на IFT — только по просьбе; shell через WSL + `run-wsl-sh.sh`.
|
||||
@@ -1,6 +1,8 @@
|
||||
# EventHub AI Router — Zed gateway + LiteLLM on IFT
|
||||
|
||||
OpenAI-compatible endpoint for [Zed](https://zed.dev) with automatic tier/lane routing (Novita AI), optional VLESS VPN, Grafana metrics, and LiteLLM Admin UI.
|
||||
OpenAI-compatible endpoint for [Zed](https://zed.dev) with tier/lane routing, optional **hierarchical** orchestration (**Qwen3.8-Max** plan → cheap **Qwen3-Coder-30B** workers/executor → conditional **DeepSeek V3.2** verify), GigaChat classifier, VLESS VPN, Grafana, LiteLLM Admin UI.
|
||||
|
||||
Цель: качество правок близко к Cursor при экономии бюджета — дорогие модели только на plan/hard/verify, рутина на дешёвом coder.
|
||||
|
||||
## URLs (IFT, Calentiq TLS)
|
||||
|
||||
@@ -11,123 +13,146 @@ OpenAI-compatible endpoint for [Zed](https://zed.dev) with automatic tier/lane r
|
||||
| **Grafana** | `https://grafana.ift.calentiq.com` |
|
||||
| Legacy (self-signed) | `https://ai-router.ift.eventhub.local/v1` |
|
||||
|
||||
## Zed settings.json
|
||||
## Клиент (канон): Zed Agent → AiRouter
|
||||
|
||||
**Primary:** встроенный Zed Agent (OpenAI-compatible), не Claude Code / Codex ACP — те обходят Novita-бюджет.
|
||||
|
||||
Пример `settings.json` (Zed 2026 openai-compatible provider):
|
||||
|
||||
```json
|
||||
{
|
||||
"openai": {
|
||||
"language_models": {
|
||||
"openai_compatible": {
|
||||
"AiRouter": {
|
||||
"api_url": "https://ai-router.ift.calentiq.com/v1",
|
||||
"api_key": "<ROUTER_API_KEY>"
|
||||
"available_models": [
|
||||
{
|
||||
"name": "smart-router",
|
||||
"display_name": "EventHub smart-router",
|
||||
"max_tokens": 16384,
|
||||
"max_output_tokens": 4096,
|
||||
"capabilities": {
|
||||
"tools": true,
|
||||
"images": false,
|
||||
"parallel_tool_calls": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"assistant": {
|
||||
"default_model": { "provider": "openai", "model": "smart-router" }
|
||||
"agent": {
|
||||
"default_model": {
|
||||
"provider": "AiRouter",
|
||||
"model": "smart-router"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
API key: `ROUTER_API_KEY` в Agent Settings / provider UI.
|
||||
|
||||
Checklist:
|
||||
|
||||
- модель `smart-router`, `capabilities.tools: true`
|
||||
- профиль **Write** (tools on) для правок файлов
|
||||
- Ask без tools → text hierarchical / lane routing
|
||||
|
||||
Optional headers / metadata:
|
||||
|
||||
- `X-AI-Quality: auto|economy|balanced|max` — `max` поднимает hard на Qwen3.8-Max
|
||||
- `X-AI-Orchestrate: off|auto|force` — hierarchical (`force` always; `auto` на COMPLEX/REASONING; `economy` quality пропускает hierarchical)
|
||||
|
||||
### Запасной клиент (тот же AiRouter)
|
||||
|
||||
Если native Zed Agent loop хрупкий: **OpenCode** или **Cline** с `base_url=https://ai-router.ift.calentiq.com/v1` и тем же ключом — иерархия бюджета сохраняется. Claude Code / Codex ACP в Zed — **не** primary (свой биллинг).
|
||||
|
||||
### Zed Agent Write + hierarchical
|
||||
|
||||
Если в запросе есть `tools` (профиль **Write**), gateway **не** подменяет agent-цикл текстовым синтезом:
|
||||
|
||||
1. **Max**-планировщик строит план (`paths` / `edit_goal` / `constraints`) → при `plan_confirm: true` ждёт **утвердить / правка / отмена**
|
||||
2. После `ok` — **Qwen3-Coder-30B** executor получает `tools` и отвечает **`tool_calls`** (Zed сам правит файлы)
|
||||
3. Пока в истории есть `role: tool` / `tool_calls` — только executor; при повторных fail edit → escalate на DeepSeek (без synthetic tool_calls)
|
||||
|
||||
Чат без tools (Ask / Minimal) по-прежнему может идти через полный text hierarchical.
|
||||
|
||||
### Progress в Zed (hierarchical)
|
||||
|
||||
При `stream: true` (дефолт Zed) gateway сразу стримит блок **«Ход выполнения»**: план → волны workers → verify → синтез, затем `---` и финальный ответ. Конфиг: `hierarchical.stream_progress` / `progress_in_content` в `config/orchestration.yaml`.
|
||||
|
||||
## Provider profiles (`PRIMARY_PROVIDER`)
|
||||
|
||||
| Profile | Keys | VPN | Role |
|
||||
|---------|------|-----|------|
|
||||
| **`hybrid`** (default) | `NOVITA_API_KEY` | workers direct | Novita workers + **Novita** plan/verify |
|
||||
| **`openrouter`** | `OPENROUTER_API_KEY` | **VPN** | Reserve (`:free` / non-Anthropic); Claude/OpenAI/Google may be region-blocked |
|
||||
|
||||
GigaChat (`GIGACHAT_CREDENTIALS`) — classifier. Optional: Groq / Gemini / Grok / Anthropic (если биллинг заработает).
|
||||
|
||||
```bash
|
||||
bash scripts/switch-provider.sh hybrid
|
||||
bash scripts/deploy.sh
|
||||
# VPN нужен только для OpenRouter / Grok / Anthropic — не для Novita hierarchical
|
||||
```
|
||||
|
||||
## Hierarchical flow (COMPLEX / force)
|
||||
|
||||
1. **`novita-planner`** (**Qwen3.8-Max**) — JSON subtasks (`simple` | `medium_code` | `hard`) + `paths` / `edit_goal`
|
||||
2. **Workers / agent executor**: `simple` → Llama (text only); `medium_code` → **Qwen3-Coder-30B**; `hard` → **DeepSeek V3.2**; `quality=max` hard → **Qwen3.8-Max**
|
||||
3. Checks; **`novita-verifier`** (DeepSeek V3.2) только на fail / hard / `quality=max`
|
||||
4. **Synthesize** (text path) на `a-medium-code`
|
||||
|
||||
> Anthropic / OpenRouter-Claude недоступны при RU billing — plan/verify идут через Novita.
|
||||
|
||||
Response `x_router_meta.mode=hierarchical` / `hierarchical_agent` with `worker_calls`, `executor_model`, `role_cost`, `verify_skipped`, etc.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Gateway** (`router/`) — vision/OCR split, tier classification, lanes A/B/C orchestration, Redis session
|
||||
- **LiteLLM** — named models `a-*` / `b-*` / `c-*`, Auto Router v2 fallback (`smart-router-internal`)
|
||||
- **PostgreSQL** — LiteLLM Admin UI, spend logs
|
||||
- **Redis** — response cache + gateway session context
|
||||
- **VPN** — `vless-proxy` replicas=0 by default; `scripts/vpn-enable.sh`
|
||||
- **Gateway** (`router/`) — classify, lanes A/B/C, hierarchical runner, Redis session
|
||||
- **LiteLLM** — named models + Auto Router (`smart-router-internal`)
|
||||
- **PostgreSQL** — spend logs / Admin UI
|
||||
- **Redis** — cache + session
|
||||
- **VPN** — `vless-proxy` (Anthropic, OpenRouter, Grok)
|
||||
|
||||
Config sources:
|
||||
|
||||
- `config/routing_rules.yaml` — tier keywords
|
||||
- `config/model_matrix.yaml` — tier × lane → Novita models
|
||||
- `config/orchestration.yaml` — start lanes, escalation, budget caps
|
||||
Config: `config/providers.yaml`, `orchestration.yaml`, `routing_rules.yaml`, `model_matrix.yaml`.
|
||||
|
||||
## Deploy on IFT
|
||||
|
||||
```bash
|
||||
git clone git.sabilin.com/eventhub/EventHubAiRouter /opt/ai-router-stack
|
||||
cd /opt/ai-router-stack
|
||||
cp .env.example .env # fill NOVITA_API_KEY, keys, POSTGRES_PASSWORD, LITELLM_SALT_KEY
|
||||
cp .env.example .env # NOVITA_API_KEY; PRIMARY_PROVIDER=hybrid
|
||||
bash scripts/deploy.sh
|
||||
bash scripts/smoke-test.sh
|
||||
sudo bash scripts/install-audit-cron.sh # daily Novita audit 03:00 MSK
|
||||
SMOKE_HIERARCHICAL=1 bash scripts/smoke-test.sh
|
||||
```
|
||||
|
||||
Prerequisites: Docker Swarm, external network `eventhub-ift-net`, Traefik routes in EventHubDevOps.
|
||||
|
||||
### Swarm secrets
|
||||
|
||||
Created automatically by `deploy.sh` from `.env`: `novita_api_key`, `litellm_master_key`, `litellm_salt_key`, `router_api_key`, `postgres_password`, optional `groq_api_key`, `gemini_api_key`, `vless_conf`.
|
||||
Swarm secrets: `novita_api_key`, optional `anthropic_api_key` / `openrouter_api_key`, `gigachat_credentials`, …
|
||||
|
||||
**`LITELLM_SALT_KEY`** — generate once, never change after first deploy.
|
||||
|
||||
## VPN
|
||||
|
||||
```bash
|
||||
bash scripts/vpn-enable.sh # scale vless + watchdog, set HTTP_PROXY on litellm
|
||||
bash scripts/vpn-enable.sh
|
||||
bash scripts/vpn-disable.sh
|
||||
```
|
||||
|
||||
Novita always direct (`NO_PROXY=api.novita.ai`).
|
||||
| Provider | Host | Route |
|
||||
|----------|------|-------|
|
||||
| Novita | `api.novita.ai` | **direct** (NO_PROXY) |
|
||||
| Groq / Gemini / GigaChat | … | **direct** |
|
||||
| **Anthropic** | `api.anthropic.com` | **VPN** (App unavailable without) |
|
||||
| **OpenRouter** | `openrouter.ai` | **VPN** |
|
||||
| **xAI Grok** | `api.x.ai` | **VPN** |
|
||||
|
||||
## Observability
|
||||
Canon: `scripts/no-proxy-default.sh`.
|
||||
|
||||
- Prometheus scrape: `litellm:4000/metrics`, `ai-router:8000/metrics` (via `eventhub-ift-net`)
|
||||
- Grafana: vendored LiteLLM v2 dashboard + custom `gateway-tier-lane.json` (EventHubDevOps)
|
||||
- Uptime Kuma monitors — see table in plan / configure manually
|
||||
## LLM classifier (GigaChat)
|
||||
|
||||
`CLASSIFIER_MODE=hybrid|heuristic|llm` — see `config/orchestration.yaml`. Freemium GigaChat for personal test.
|
||||
|
||||
## Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `deploy.sh` | secrets + build + stack deploy |
|
||||
| `smoke-test.sh` | health, classify, chat max_tokens=16 |
|
||||
| `sync-routing-config.sh` | regen `litellm_config.generated.yaml` |
|
||||
| `audit-novita-pricing.sh` | daily model catalog check |
|
||||
| `vendor-litellm-dashboards.sh` | fetch Grafana JSON → EventHubDevOps |
|
||||
| `vpn-enable.sh` / `vpn-disable.sh` | VPN toggle |
|
||||
|
||||
## Quality modes
|
||||
|
||||
Header `X-AI-Quality: auto|economy|balanced|max` or `metadata.quality_mode` in request body. Default: `auto` (start lane A/B by tier, escalate on retry/5xx).
|
||||
|
||||
## LLM classifier (GigaChat, optional)
|
||||
|
||||
When `CLASSIFIER_MODE=hybrid` (default) and `GIGACHAT_CREDENTIALS` is set:
|
||||
|
||||
1. **Heuristic** classify first (0 cost)
|
||||
2. If `confidence < 0.6` → one call to **GigaChat-2-Lite** via LiteLLM (`gigachat-classifier`)
|
||||
3. Lane orchestration unchanged (A/B/C, Redis, budget)
|
||||
|
||||
| `CLASSIFIER_MODE` | Behavior |
|
||||
|-------------------|----------|
|
||||
| `heuristic` | Keywords only (no GigaChat) |
|
||||
| `hybrid` | GigaChat only on low confidence |
|
||||
| `llm` | Always GigaChat for text (except vision) |
|
||||
|
||||
Freemium GigaChat — для личного некомmercial теста; prod — юр. тариф Сбера.
|
||||
|
||||
Setup: [developers.sber.ru](https://developers.sber.ru/docs/ru/gigachat/quickstart/ind-create-project) → Authorization key → `.env` `GIGACHAT_CREDENTIALS`.
|
||||
|
||||
Response field: `x_router_meta.classifier_source` = `heuristic` | `gigachat` | `heuristic_fallback`.
|
||||
|
||||
## Response metadata
|
||||
|
||||
Each chat response includes `x_router_meta`:
|
||||
|
||||
```json
|
||||
{
|
||||
"tier": "MEDIUM_OPS",
|
||||
"lane": "A",
|
||||
"model": "a-medium-ops",
|
||||
"escalation_level": 0,
|
||||
"quality_mode": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## Local build
|
||||
|
||||
```bash
|
||||
docker build -f router/Dockerfile -t ai-router-gateway:local .
|
||||
```
|
||||
|
||||
## CI
|
||||
|
||||
Gitea Actions: `.gitea/workflows/ci.yml` — build gateway image, sync config check.
|
||||
See `scripts/` — `deploy.sh`, `smoke-test.sh`, `gen-litellm-config.py`, `switch-provider.sh`, VPN helpers.
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Static LiteLLM settings — model_list generated by scripts/gen-litellm-config.py
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
store_model_in_db: false
|
||||
max_budget: 50
|
||||
budget_duration: 30d
|
||||
|
||||
litellm_settings:
|
||||
drop_params: true
|
||||
set_verbose: false
|
||||
request_timeout: 300
|
||||
num_retries: 2
|
||||
cache: true
|
||||
cache_params:
|
||||
type: redis
|
||||
host: redis
|
||||
port: 6379
|
||||
ttl: 3600
|
||||
callbacks: ["prometheus"]
|
||||
require_auth_for_metrics_endpoint: false
|
||||
|
||||
environment_variables:
|
||||
NOVITA_API_KEY: os.environ/NOVITA_API_KEY
|
||||
GROQ_API_KEY: os.environ/GROQ_API_KEY
|
||||
XAI_API_KEY: os.environ/XAI_API_KEY
|
||||
GEMINI_API_KEY: os.environ/GEMINI_API_KEY
|
||||
GIGACHAT_CREDENTIALS: os.environ/GIGACHAT_CREDENTIALS
|
||||
TOGETHER_API_KEY: os.environ/TOGETHER_API_KEY
|
||||
OPENROUTER_API_KEY: os.environ/OPENROUTER_API_KEY
|
||||
|
||||
router_settings:
|
||||
routing_strategy: simple-shuffle
|
||||
num_retries: 2
|
||||
timeout: 300
|
||||
allowed_fails: 5
|
||||
cooldown_time: 15
|
||||
+22
-50
@@ -1,104 +1,76 @@
|
||||
# tier × lane (A/B/C) → Novita model id + LiteLLM fallbacks
|
||||
# Tier × lane orchestration metadata (model IDs → config/providers.yaml)
|
||||
|
||||
lanes:
|
||||
A: economy
|
||||
B: balanced
|
||||
C: max
|
||||
|
||||
# LiteLLM per-lane RPM (proxy-side). Zed agent bursts tool calls — keep high for personal IFT.
|
||||
# OpenRouter :free upstream may still 429; router fallbacks + cooldown_time apply separately.
|
||||
models:
|
||||
a-simple:
|
||||
novita: novita/qwen/qwen3-4b-fp8
|
||||
fallbacks: [b-simple, groq-llama-8b]
|
||||
rpm: 60
|
||||
rpm: 300
|
||||
b-simple:
|
||||
novita: novita/meta-llama/llama-3.1-8b-instruct
|
||||
fallbacks: [a-simple, groq-llama-8b]
|
||||
rpm: 60
|
||||
rpm: 300
|
||||
c-simple:
|
||||
novita: novita/qwen/qwen3-8b-fp8
|
||||
fallbacks: [b-simple]
|
||||
rpm: 60
|
||||
rpm: 300
|
||||
|
||||
a-medium-ops:
|
||||
novita: novita/qwen/qwen3-8b-fp8
|
||||
fallbacks: [a-medium-code, b-medium-ops]
|
||||
rpm: 40
|
||||
rpm: 240
|
||||
b-medium-ops:
|
||||
novita: novita/deepseek/deepseek-v3.2
|
||||
fallbacks: [a-medium-code, c-medium-ops]
|
||||
rpm: 40
|
||||
rpm: 240
|
||||
c-medium-ops:
|
||||
novita: novita/deepseek/deepseek-v3.2
|
||||
fallbacks: [b-medium-ops]
|
||||
rpm: 40
|
||||
rpm: 240
|
||||
|
||||
a-medium-code:
|
||||
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
fallbacks: [a-medium-ops, groq-qwen-coder]
|
||||
rpm: 40
|
||||
rpm: 240
|
||||
b-medium-code:
|
||||
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
fallbacks: [b-medium-ops, c-medium-code]
|
||||
rpm: 40
|
||||
rpm: 240
|
||||
c-medium-code:
|
||||
novita: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
fallbacks: [b-medium-code, c-medium-ops]
|
||||
rpm: 30
|
||||
rpm: 180
|
||||
|
||||
a-complex:
|
||||
novita: novita/deepseek/deepseek-v3.2
|
||||
fallbacks: [b-complex, a-reasoning]
|
||||
rpm: 30
|
||||
rpm: 180
|
||||
b-complex:
|
||||
novita: novita/deepseek/deepseek-r1-0528
|
||||
fallbacks: [a-complex, c-complex]
|
||||
rpm: 20
|
||||
rpm: 120
|
||||
c-complex:
|
||||
novita: novita/deepseek/deepseek-r1-turbo
|
||||
fallbacks: [b-complex]
|
||||
rpm: 15
|
||||
rpm: 120
|
||||
|
||||
a-reasoning:
|
||||
novita: novita/deepseek/deepseek-r1-0528-qwen3-8b
|
||||
fallbacks: [b-reasoning, gemini-flash]
|
||||
rpm: 30
|
||||
rpm: 120
|
||||
b-reasoning:
|
||||
novita: novita/deepseek/deepseek-r1-0528
|
||||
fallbacks: [a-reasoning, c-reasoning]
|
||||
rpm: 20
|
||||
rpm: 120
|
||||
c-reasoning:
|
||||
novita: novita/deepseek/deepseek-r1-turbo
|
||||
fallbacks: [b-reasoning, gemini-flash]
|
||||
rpm: 15
|
||||
rpm: 120
|
||||
|
||||
a-vision-ocr:
|
||||
novita: novita/paddlepaddle/paddleocr-vl
|
||||
fallbacks: [a-vision]
|
||||
rpm: 30
|
||||
rpm: 120
|
||||
no_escalation: true
|
||||
a-vision:
|
||||
novita: novita/qwen/qwen3-vl-30b-a3b-instruct
|
||||
fallbacks: [a-vision-ocr]
|
||||
rpm: 20
|
||||
rpm: 120
|
||||
b-vision:
|
||||
novita: novita/qwen/qwen2.5-vl-72b-instruct
|
||||
fallbacks: [a-vision, c-vision]
|
||||
rpm: 15
|
||||
rpm: 90
|
||||
c-vision:
|
||||
novita: novita/qwen/qwen3-vl-235b-a22b-instruct
|
||||
fallbacks: [b-vision]
|
||||
rpm: 10
|
||||
|
||||
optional_providers:
|
||||
groq-llama-8b:
|
||||
model: groq/llama-3.1-8b-instant
|
||||
api_key: os.environ/GROQ_API_KEY
|
||||
groq-qwen-coder:
|
||||
model: groq/qwen-qwen-2.5-coder-32b
|
||||
api_key: os.environ/GROQ_API_KEY
|
||||
gemini-flash:
|
||||
model: gemini/gemini-2.0-flash
|
||||
api_key: os.environ/GEMINI_API_KEY
|
||||
rpm: 90
|
||||
|
||||
audit:
|
||||
price_drift_threshold_pct: 10
|
||||
|
||||
@@ -32,9 +32,89 @@ redis:
|
||||
ttl_sec: 1800
|
||||
|
||||
classifier:
|
||||
# heuristic | hybrid (default) | llm
|
||||
mode: hybrid
|
||||
llm_model: gigachat-classifier
|
||||
low_confidence_threshold: 0.6
|
||||
timeout_sec: 15
|
||||
max_tokens: 64
|
||||
|
||||
# Strong Max plan → cheap coder workers/executor → conditional DeepSeek verify
|
||||
hierarchical:
|
||||
enabled: true
|
||||
trigger_tiers: [COMPLEX, REASONING]
|
||||
header: X-AI-Orchestrate # off | auto | force
|
||||
planner_model: novita-planner
|
||||
verifier_model: novita-verifier
|
||||
planner_fallback: b-complex
|
||||
verifier_fallback: b-complex
|
||||
verify_policy: on_fail_or_hard
|
||||
verify_input: digests
|
||||
synthesize_model: a-medium-code
|
||||
plan_confirm: true
|
||||
agent_executor_model: a-medium-code
|
||||
agent_escalate_model: b-complex
|
||||
agent_escalate_after_edit_failures: 2
|
||||
skip_synthesize_if_single: true
|
||||
max_subtasks: 3
|
||||
max_verify_retries: 1
|
||||
plan_max_tokens: 1024
|
||||
verify_max_tokens: 384
|
||||
synthesize_max_tokens: 1024
|
||||
worker_max_tokens: 768
|
||||
wall_clock_sec: 600
|
||||
digest_chars: 300
|
||||
call_timeout_sec: 300
|
||||
# First turn (kickstart read) can stay longer; after read_file body Coder hangs —
|
||||
# tool_loop uses short timeout + escalate_after_read → DeepSeek.
|
||||
executor_timeout_sec: 90
|
||||
executor_num_retries: 0
|
||||
executor_input_chars: 16000
|
||||
executor_tool_loop_chars: 12000
|
||||
executor_tool_loop_timeout_sec: 45
|
||||
executor_newest_tool_chars: 3500
|
||||
executor_minimal_context: true
|
||||
executor_max_tools: 6
|
||||
executor_tool_desc_chars: 120
|
||||
executor_tools_full_chars: 8000
|
||||
executor_max_tokens: 1024
|
||||
executor_use_stream: false
|
||||
executor_synthetic_on_timeout: true
|
||||
# First turn only: synthetic read_file on plan path if model returns prose.
|
||||
# Mid-loop still stops (no fake edits).
|
||||
executor_synthetic_mode: kickstart_only
|
||||
executor_max_synthetic_continues: 1
|
||||
executor_max_edit_failures: 4
|
||||
executor_midloop_model_retry: true
|
||||
executor_midloop_escalate: true
|
||||
# After successful read_file skip DeepSeek (was 408→400 loop) → edit_file
|
||||
executor_force_edit_after_read: true
|
||||
# After a successful read_file, skip cheap coder — it stalls on fat tool results
|
||||
executor_escalate_after_read: true
|
||||
# Rewrite read_file on already-read paths → edit (breaks re-read loops)
|
||||
executor_rewrite_reread: true
|
||||
# When model returns 200 without tool_calls: one escalate retry before kickstart
|
||||
executor_no_tools_escalate: true
|
||||
# After plan approve: 1) deterministic path index 2) if gaps → cheap find_* model
|
||||
path_resolve_enabled: true
|
||||
path_resolve_model: a-simple
|
||||
path_resolve_timeout_sec: 45
|
||||
path_resolve_max_tools: 3
|
||||
# DevOps (*.calentiq.com / Bad Gateway): probe live ports before edit
|
||||
runtime_probe_enabled: true
|
||||
runtime_probe_timeout_sec: 8
|
||||
# Optional: ssh host for docker inspect (empty = TCP probe from gateway only)
|
||||
runtime_probe_ssh_host: ""
|
||||
# Quiet Zed: milestones only (no ctx bars / LiteLLM waits / subtask dumps)
|
||||
progress_verbose: false
|
||||
show_context_fill: false
|
||||
progress_in_content: true
|
||||
stream_progress: true
|
||||
context_window_tokens: 32768
|
||||
plan_input_chars: 48000
|
||||
llm_cache_ttl_sec: 3600
|
||||
worker_map:
|
||||
simple: a-simple
|
||||
medium_code: a-medium-code
|
||||
hard: b-complex
|
||||
worker_map_max:
|
||||
hard: c-complex
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
# Provider profiles — active: PRIMARY_PROVIDER env (default: hybrid)
|
||||
# Regenerate: bash scripts/gen-litellm-config.py
|
||||
|
||||
profiles:
|
||||
hybrid:
|
||||
label: "Hybrid: Novita workers + Novita plan/verify (no Claude geo-billing)"
|
||||
welcome_credit: false
|
||||
requires: [NOVITA_API_KEY]
|
||||
smart_router:
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: NOVITA_API_KEY
|
||||
fallbacks:
|
||||
# No groq/gemini here — VPN off on IFT; those hang until client timeout.
|
||||
a-simple: [b-simple, a-medium-ops]
|
||||
a-medium-code: [b-medium-code, a-complex]
|
||||
a-complex: [b-complex, a-reasoning]
|
||||
b-complex: [a-complex, c-complex]
|
||||
a-reasoning: [b-reasoning, a-complex]
|
||||
a-vision: [a-vision-ocr]
|
||||
smart-router-internal: [a-medium-ops, a-medium-code, a-complex]
|
||||
# Planner is Max — fallback to DeepSeek then coder (never Llama)
|
||||
novita-planner: [b-complex, a-medium-code]
|
||||
novita-verifier: [b-complex, a-medium-code]
|
||||
default_fallbacks: [a-medium-code, a-complex, a-medium-ops]
|
||||
lanes:
|
||||
# Prefer Llama for fast lanes — Qwen3 puts answers in reasoning_content
|
||||
a-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
b-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
c-simple: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
a-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
b-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
c-medium-ops: { model: novita/meta-llama/llama-3.1-8b-instruct, api_key: NOVITA_API_KEY }
|
||||
# medium_code → coder / Plus-class (disable thinking — else stream hangs)
|
||||
a-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
b-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
c-medium-code: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
# complex/reasoning A/B → coder/deepseek; C (hard) → Qwen3.8-Max only
|
||||
a-complex: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
b-complex: { model: novita/deepseek/deepseek-v3.2, api_key: NOVITA_API_KEY }
|
||||
c-complex: { model: novita/qwen/qwen3.8-max, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
a-reasoning: { model: novita/qwen/qwen3-coder-30b-a3b-instruct, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
b-reasoning: { model: novita/deepseek/deepseek-v3.2, api_key: NOVITA_API_KEY }
|
||||
c-reasoning: { model: novita/qwen/qwen3.8-max, api_key: NOVITA_API_KEY, disable_thinking: true }
|
||||
a-vision-ocr: { model: novita/paddlepaddle/paddleocr-vl, api_key: NOVITA_API_KEY }
|
||||
a-vision: { model: novita/qwen/qwen3-vl-30b-a3b-instruct, api_key: NOVITA_API_KEY }
|
||||
b-vision: { model: novita/qwen/qwen2.5-vl-72b-instruct, api_key: NOVITA_API_KEY }
|
||||
c-vision: { model: novita/qwen/qwen3-vl-235b-a22b-instruct, api_key: NOVITA_API_KEY }
|
||||
|
||||
openrouter:
|
||||
label: OpenRouter reserve (VPN; free :free + pay-as-you-go)
|
||||
welcome_credit: true
|
||||
requires: [OPENROUTER_API_KEY]
|
||||
requires_vpn: true
|
||||
smart_router:
|
||||
model: openrouter/openrouter/free
|
||||
api_key: OPENROUTER_API_KEY
|
||||
fallbacks:
|
||||
a-simple: [b-simple, c-simple]
|
||||
a-medium-code: [b-medium-code, c-medium-code]
|
||||
a-complex: [b-complex, c-complex]
|
||||
smart-router-internal: [a-medium-ops, a-medium-code]
|
||||
default_fallbacks: [a-medium-code, a-medium-ops]
|
||||
lanes:
|
||||
a-simple: { model: openrouter/openai/gpt-oss-20b:free, api_key: OPENROUTER_API_KEY }
|
||||
b-simple: { model: openrouter/openai/gpt-oss-20b:free, api_key: OPENROUTER_API_KEY }
|
||||
c-simple: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
a-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
b-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
c-medium-ops: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
a-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
|
||||
b-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
|
||||
c-medium-code: { model: openrouter/cohere/north-mini-code:free, api_key: OPENROUTER_API_KEY }
|
||||
a-complex: { model: openrouter/google/gemma-4-26b-a4b-it:free, api_key: OPENROUTER_API_KEY }
|
||||
b-complex: { model: openrouter/nvidia/nemotron-3-nano-30b-a3b:free, api_key: OPENROUTER_API_KEY }
|
||||
c-complex: { model: openrouter/nvidia/nemotron-3-nano-30b-a3b:free, api_key: OPENROUTER_API_KEY }
|
||||
a-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
|
||||
b-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
|
||||
c-reasoning: { model: openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free, api_key: OPENROUTER_API_KEY }
|
||||
a-vision-ocr: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
a-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
b-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
c-vision: { model: openrouter/openrouter/free, api_key: OPENROUTER_API_KEY }
|
||||
|
||||
# Always registered when keys present (all profiles)
|
||||
fixed_models:
|
||||
# Hierarchical: strong Max plan (rare); DeepSeek verify (conditional)
|
||||
novita-planner:
|
||||
model: novita/qwen/qwen3.8-max
|
||||
api_key: NOVITA_API_KEY
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
disable_thinking: true
|
||||
novita-verifier:
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
api_key: NOVITA_API_KEY
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
# Optional Claude — only if ANTHROPIC_API_KEY works later (VPN + billing)
|
||||
claude-haiku-planner:
|
||||
model: anthropic/claude-haiku-4-5-20251001
|
||||
api_key: ANTHROPIC_API_KEY
|
||||
requires_vpn: true
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
claude-sonnet-verifier:
|
||||
model: anthropic/claude-sonnet-5
|
||||
api_key: ANTHROPIC_API_KEY
|
||||
requires_vpn: true
|
||||
max_tokens: 2048
|
||||
temperature: 0
|
||||
grok-3:
|
||||
model: xai/grok-3
|
||||
api_key: XAI_API_KEY
|
||||
requires_vpn: true
|
||||
groq-llama-8b:
|
||||
model: groq/llama-3.1-8b-instant
|
||||
api_key: GROQ_API_KEY
|
||||
groq-qwen-coder:
|
||||
model: groq/qwen-qwen-2.5-coder-32b
|
||||
api_key: GROQ_API_KEY
|
||||
gemini-flash:
|
||||
model: gemini/gemini-2.0-flash
|
||||
api_key: GEMINI_API_KEY
|
||||
gigachat-classifier:
|
||||
model: gigachat/GigaChat-2-Lite
|
||||
api_key: GIGACHAT_CREDENTIALS
|
||||
ssl_verify: false
|
||||
max_tokens: 64
|
||||
temperature: 0
|
||||
gigachat-pro:
|
||||
model: gigachat/GigaChat-2-Pro
|
||||
api_key: GIGACHAT_CREDENTIALS
|
||||
ssl_verify: false
|
||||
@@ -38,6 +38,18 @@ gateway:
|
||||
- novita
|
||||
- traefik
|
||||
- mnesia
|
||||
medium_code_keywords:
|
||||
- поправь
|
||||
- исправь
|
||||
- fix
|
||||
- edit
|
||||
- patch
|
||||
- bugfix
|
||||
- typo
|
||||
- rename
|
||||
- add test
|
||||
- unit test
|
||||
- lint
|
||||
ocr_keywords:
|
||||
- ocr
|
||||
- прочитай текст
|
||||
@@ -64,6 +76,8 @@ litellm:
|
||||
tier: SIMPLE
|
||||
- keywords: ["bash", "docker", "swarm", "ci/cd", "gitea", "rebar", "erlang", "devops", "kubectl", "terraform", "playwright", "npm", "wsl"]
|
||||
tier: MEDIUM
|
||||
- keywords: ["поправь", "исправь", "fix", "edit", "patch", "bugfix", "typo", "rename", "add test", "lint"]
|
||||
tier: MEDIUM
|
||||
- keywords: ["рефакторинг", "refactor", "архитектура", "architecture", "спроектируй", "design system", "microservice", "distributed", "migration plan"]
|
||||
tier: COMPLEX
|
||||
- keywords: ["step by step", "reasoning", "prove", "analyze deeply", "think through"]
|
||||
|
||||
+26
-9
@@ -42,8 +42,16 @@ secrets:
|
||||
external: true
|
||||
groq_api_key:
|
||||
external: true
|
||||
xai_api_key:
|
||||
external: true
|
||||
gemini_api_key:
|
||||
external: true
|
||||
together_api_key:
|
||||
external: true
|
||||
openrouter_api_key:
|
||||
external: true
|
||||
anthropic_api_key:
|
||||
external: true
|
||||
gigachat_credentials:
|
||||
external: true
|
||||
vless_conf:
|
||||
@@ -95,7 +103,7 @@ services:
|
||||
retries: 3
|
||||
|
||||
litellm:
|
||||
image: ${LITELLM_IMAGE:-ghcr.io/berriai/litellm:main-v1.96.0-stable}
|
||||
image: ${LITELLM_IMAGE:-ghcr.io/berriai/litellm:v1.95.0}
|
||||
entrypoint: ["/bin/sh", "/entrypoint/litellm-entrypoint.sh"]
|
||||
configs:
|
||||
- source: litellm_config
|
||||
@@ -111,7 +119,7 @@ services:
|
||||
USE_AIOHTTP_TRANSPORT: ${USE_AIOHTTP_TRANSPORT:-False}
|
||||
HTTP_PROXY: ${HTTP_PROXY:-}
|
||||
HTTPS_PROXY: ${HTTPS_PROXY:-}
|
||||
NO_PROXY: ${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}
|
||||
NO_PROXY: ${NO_PROXY:-localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-litellm}
|
||||
POSTGRES_DB: ${POSTGRES_DB:-litellm}
|
||||
secrets:
|
||||
@@ -120,7 +128,11 @@ services:
|
||||
- litellm_salt_key
|
||||
- postgres_password
|
||||
- groq_api_key
|
||||
- xai_api_key
|
||||
- gemini_api_key
|
||||
- together_api_key
|
||||
- openrouter_api_key
|
||||
- anthropic_api_key
|
||||
- gigachat_credentials
|
||||
networks:
|
||||
ai-internal:
|
||||
@@ -136,16 +148,16 @@ services:
|
||||
delay: 5s
|
||||
update_config:
|
||||
parallelism: 1
|
||||
failure_action: rollback
|
||||
failure_action: pause
|
||||
placement:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:4000/health/liveliness || exit 1"]
|
||||
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:4000/health/liveliness', timeout=5)\" || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
timeout: 15s
|
||||
retries: 5
|
||||
start_period: 120s
|
||||
|
||||
router:
|
||||
image: ${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}
|
||||
@@ -195,7 +207,12 @@ services:
|
||||
- source: vless_conf
|
||||
target: /app/vless.conf
|
||||
networks:
|
||||
- ai-internal
|
||||
ai-internal:
|
||||
aliases:
|
||||
- vless-proxy
|
||||
eventhub-ift:
|
||||
aliases:
|
||||
- vless-proxy
|
||||
deploy:
|
||||
replicas: 0
|
||||
restart_policy:
|
||||
@@ -206,7 +223,7 @@ services:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -q -O- http://127.0.0.1:8080 || exit 1"]
|
||||
test: ["CMD-SHELL", "pgrep Xray >/dev/null || exit 1"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
+236
-71
@@ -1,16 +1,16 @@
|
||||
# LiteLLM Proxy — Novita lanes A/B/C + Auto Router v2 fallback
|
||||
# Regenerate model_list tail: bash scripts/sync-routing-config.sh
|
||||
# LiteLLM — generated for PRIMARY_PROVIDER=hybrid
|
||||
# Profile: Hybrid: Novita workers + Novita plan/verify (no Claude geo-billing)
|
||||
# Regenerate: PRIMARY_PROVIDER=hybrid bash scripts/gen-litellm-config.py
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
store_model_in_db: false
|
||||
max_budget: 50
|
||||
budget_duration: 30d
|
||||
|
||||
litellm_settings:
|
||||
drop_params: true
|
||||
set_verbose: false
|
||||
request_timeout: 120
|
||||
request_timeout: 300
|
||||
num_retries: 2
|
||||
cache: true
|
||||
cache_params:
|
||||
@@ -18,158 +18,272 @@ litellm_settings:
|
||||
host: redis
|
||||
port: 6379
|
||||
ttl: 3600
|
||||
callbacks: ["prometheus"]
|
||||
callbacks:
|
||||
- prometheus
|
||||
require_auth_for_metrics_endpoint: false
|
||||
|
||||
environment_variables:
|
||||
NOVITA_API_KEY: os.environ/NOVITA_API_KEY
|
||||
GROQ_API_KEY: os.environ/GROQ_API_KEY
|
||||
XAI_API_KEY: os.environ/XAI_API_KEY
|
||||
GEMINI_API_KEY: os.environ/GEMINI_API_KEY
|
||||
GIGACHAT_CREDENTIALS: os.environ/GIGACHAT_CREDENTIALS
|
||||
|
||||
TOGETHER_API_KEY: os.environ/TOGETHER_API_KEY
|
||||
OPENROUTER_API_KEY: os.environ/OPENROUTER_API_KEY
|
||||
router_settings:
|
||||
routing_strategy: simple-shuffle
|
||||
num_retries: 2
|
||||
timeout: 120
|
||||
allowed_fails: 2
|
||||
cooldown_time: 30
|
||||
timeout: 300
|
||||
allowed_fails: 5
|
||||
cooldown_time: 15
|
||||
fallbacks:
|
||||
- a-simple: ["b-simple", "groq-llama-8b"]
|
||||
- a-medium-code: ["a-medium-ops", "groq-qwen-coder"]
|
||||
- a-complex: ["b-complex", "a-reasoning"]
|
||||
- b-complex: ["a-complex", "c-complex"]
|
||||
- a-reasoning: ["b-reasoning", "gemini-flash"]
|
||||
- a-vision: ["a-vision-ocr"]
|
||||
- smart-router-internal: ["a-medium-ops", "a-medium-code", "a-complex"]
|
||||
default_fallbacks: ["a-medium-code", "a-complex", "groq-qwen-coder"]
|
||||
|
||||
- a-simple:
|
||||
- b-simple
|
||||
- a-medium-ops
|
||||
- a-medium-code:
|
||||
- b-medium-code
|
||||
- a-complex
|
||||
- a-complex:
|
||||
- b-complex
|
||||
- a-reasoning
|
||||
- b-complex:
|
||||
- a-complex
|
||||
- c-complex
|
||||
- a-reasoning:
|
||||
- b-reasoning
|
||||
- a-complex
|
||||
- a-vision:
|
||||
- a-vision-ocr
|
||||
- smart-router-internal:
|
||||
- a-medium-ops
|
||||
- a-medium-code
|
||||
- a-complex
|
||||
- novita-planner:
|
||||
- b-complex
|
||||
- a-medium-code
|
||||
- novita-verifier:
|
||||
- b-complex
|
||||
- a-medium-code
|
||||
default_fallbacks:
|
||||
- a-medium-code
|
||||
- a-complex
|
||||
- a-medium-ops
|
||||
model_list:
|
||||
# --- LLM tier classifier (GigaChat freemium, hybrid mode only) ---
|
||||
- model_name: gigachat-classifier
|
||||
litellm_params:
|
||||
model: gigachat/GigaChat-2-Lite
|
||||
api_key: os.environ/GIGACHAT_CREDENTIALS
|
||||
timeout: 180
|
||||
ssl_verify: false
|
||||
max_tokens: 64
|
||||
temperature: 0
|
||||
|
||||
# --- Lane models (generated from config/model_matrix.yaml) ---
|
||||
- model_name: a-simple
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-4b-fp8
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 60
|
||||
timeout: 180
|
||||
rpm: 300
|
||||
- model_name: b-simple
|
||||
litellm_params:
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 60
|
||||
timeout: 180
|
||||
rpm: 300
|
||||
- model_name: c-simple
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-8b-fp8
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 60
|
||||
|
||||
timeout: 180
|
||||
rpm: 300
|
||||
- model_name: a-medium-ops
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-8b-fp8
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 40
|
||||
timeout: 180
|
||||
rpm: 240
|
||||
- model_name: b-medium-ops
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 40
|
||||
timeout: 180
|
||||
rpm: 240
|
||||
- model_name: c-medium-ops
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 40
|
||||
|
||||
timeout: 180
|
||||
rpm: 240
|
||||
- model_name: a-medium-code
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 40
|
||||
timeout: 180
|
||||
rpm: 240
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: b-medium-code
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 40
|
||||
timeout: 180
|
||||
rpm: 240
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: c-medium-code
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 30
|
||||
|
||||
timeout: 180
|
||||
rpm: 180
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: a-complex
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
timeout: 180
|
||||
rpm: 180
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: b-complex
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 30
|
||||
- model_name: b-complex
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-r1-0528
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 20
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
- model_name: c-complex
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-r1-turbo
|
||||
model: novita/qwen/qwen3.8-max
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 15
|
||||
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: a-reasoning
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-r1-0528-qwen3-8b
|
||||
model: novita/qwen/qwen3-coder-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 30
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: b-reasoning
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-r1-0528
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 20
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
- model_name: c-reasoning
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-r1-turbo
|
||||
model: novita/qwen/qwen3.8-max
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 15
|
||||
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: a-vision-ocr
|
||||
litellm_params:
|
||||
model: novita/paddlepaddle/paddleocr-vl
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 30
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
- model_name: a-vision
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-vl-30b-a3b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 20
|
||||
timeout: 180
|
||||
rpm: 120
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: b-vision
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen2.5-vl-72b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 15
|
||||
timeout: 180
|
||||
rpm: 90
|
||||
- model_name: c-vision
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-vl-235b-a22b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
rpm: 10
|
||||
|
||||
# --- Optional fallbacks ---
|
||||
timeout: 180
|
||||
rpm: 90
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: novita-planner
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3.8-max
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
timeout: 180
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
extra_body:
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
enable_thinking: false
|
||||
- model_name: novita-verifier
|
||||
litellm_params:
|
||||
model: novita/deepseek/deepseek-v3.2
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
timeout: 180
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
- model_name: claude-haiku-planner
|
||||
litellm_params:
|
||||
model: anthropic/claude-haiku-4-5-20251001
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
timeout: 180
|
||||
max_tokens: 1024
|
||||
temperature: 0
|
||||
- model_name: claude-sonnet-verifier
|
||||
litellm_params:
|
||||
model: anthropic/claude-sonnet-5
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
timeout: 180
|
||||
max_tokens: 2048
|
||||
temperature: 0
|
||||
- model_name: groq-llama-8b
|
||||
litellm_params:
|
||||
model: groq/llama-3.1-8b-instant
|
||||
api_key: os.environ/GROQ_API_KEY
|
||||
timeout: 180
|
||||
- model_name: groq-qwen-coder
|
||||
litellm_params:
|
||||
model: groq/qwen-qwen-2.5-coder-32b
|
||||
api_key: os.environ/GROQ_API_KEY
|
||||
timeout: 180
|
||||
- model_name: gemini-flash
|
||||
litellm_params:
|
||||
model: gemini/gemini-2.0-flash
|
||||
api_key: os.environ/GEMINI_API_KEY
|
||||
|
||||
# --- Auto Router v2 fallback when gateway confidence low ---
|
||||
timeout: 180
|
||||
- model_name: grok-3
|
||||
litellm_params:
|
||||
model: xai/grok-3
|
||||
api_key: os.environ/XAI_API_KEY
|
||||
timeout: 180
|
||||
- model_name: gigachat-pro
|
||||
litellm_params:
|
||||
model: gigachat/GigaChat-2-Pro
|
||||
api_key: os.environ/GIGACHAT_CREDENTIALS
|
||||
timeout: 180
|
||||
ssl_verify: false
|
||||
- model_name: smart-router-internal
|
||||
litellm_params:
|
||||
model: auto_router/complexity_router
|
||||
@@ -184,15 +298,68 @@ model_list:
|
||||
REASONING: a-reasoning
|
||||
classifier_fallback: heuristic
|
||||
keyword_tier_rules:
|
||||
- keywords: ["привет", "hello", "hi", "thanks", "спасибо", "что такое", "what is", "define"]
|
||||
- keywords:
|
||||
- привет
|
||||
- hello
|
||||
- hi
|
||||
- thanks
|
||||
- спасибо
|
||||
- что такое
|
||||
- what is
|
||||
- define
|
||||
tier: SIMPLE
|
||||
- keywords: ["bash", "docker", "swarm", "ci/cd", "gitea", "rebar", "erlang", "devops", "kubectl", "terraform"]
|
||||
- keywords:
|
||||
- bash
|
||||
- docker
|
||||
- swarm
|
||||
- ci/cd
|
||||
- gitea
|
||||
- rebar
|
||||
- erlang
|
||||
- devops
|
||||
- kubectl
|
||||
- terraform
|
||||
- playwright
|
||||
- npm
|
||||
- wsl
|
||||
tier: MEDIUM
|
||||
- keywords: ["рефакторинг", "refactor", "архитектура", "architecture", "спроектируй", "migration plan"]
|
||||
- keywords:
|
||||
- поправь
|
||||
- исправь
|
||||
- fix
|
||||
- edit
|
||||
- patch
|
||||
- bugfix
|
||||
- typo
|
||||
- rename
|
||||
- add test
|
||||
- lint
|
||||
tier: MEDIUM
|
||||
- keywords:
|
||||
- рефакторинг
|
||||
- refactor
|
||||
- архитектура
|
||||
- architecture
|
||||
- спроектируй
|
||||
- design system
|
||||
- microservice
|
||||
- distributed
|
||||
- migration plan
|
||||
tier: COMPLEX
|
||||
- keywords: ["step by step", "prove", "analyze deeply", "think through"]
|
||||
- keywords:
|
||||
- step by step
|
||||
- reasoning
|
||||
- prove
|
||||
- analyze deeply
|
||||
- think through
|
||||
tier: REASONING
|
||||
custom_technical_keywords: [erlang, rebar3, traefik, mnesia, litellm, novita]
|
||||
custom_technical_keywords:
|
||||
- erlang
|
||||
- rebar3
|
||||
- traefik
|
||||
- mnesia
|
||||
- litellm
|
||||
- novita
|
||||
token_thresholds:
|
||||
simple: 20
|
||||
complex: 500
|
||||
@@ -202,9 +369,7 @@ model_list:
|
||||
complex_reasoning: 0.62
|
||||
session_affinity: true
|
||||
session_affinity_ttl_seconds: 1800
|
||||
|
||||
# Alias for Zed default model name (gateway resolves before LiteLLM)
|
||||
- model_name: smart-router
|
||||
litellm_params:
|
||||
model: novita/qwen/qwen3-8b-fp8
|
||||
model: novita/meta-llama/llama-3.1-8b-instruct
|
||||
api_key: os.environ/NOVITA_API_KEY
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,587 @@
|
||||
"""Streaming agent-hierarchical path — never block Zed on a silent await."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
log = logging.getLogger("ai-router.agent_stream")
|
||||
|
||||
|
||||
def sse_chunk(
|
||||
*,
|
||||
cid: str,
|
||||
model: str,
|
||||
delta: dict[str, Any],
|
||||
finish_reason: str | None = None,
|
||||
) -> bytes:
|
||||
payload = {
|
||||
"id": cid,
|
||||
"object": "chat.completion.chunk",
|
||||
"created": int(time.time()),
|
||||
"model": model,
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"delta": delta,
|
||||
"finish_reason": finish_reason,
|
||||
}
|
||||
],
|
||||
}
|
||||
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n".encode("utf-8")
|
||||
|
||||
|
||||
def format_progress_block(lines: list[str]) -> str:
|
||||
from progress_ui import format_progress_block as _fmt
|
||||
|
||||
return _fmt(lines)
|
||||
|
||||
|
||||
async def stream_agent_plan_then_act(
|
||||
*,
|
||||
runner: Any,
|
||||
last_user: str,
|
||||
quality_mode: str | None,
|
||||
session_id: str,
|
||||
messages: list[dict[str, Any]],
|
||||
body: dict[str, Any],
|
||||
meta: dict[str, Any],
|
||||
decision: Any,
|
||||
prompt_hash: str,
|
||||
litellm_url: str,
|
||||
litellm_key: str,
|
||||
cfg: dict[str, Any],
|
||||
find_pending_plan: Any,
|
||||
plan_payload_from_meta: Any,
|
||||
inject_plan_context: Any,
|
||||
pick_agent_executor_model: Any,
|
||||
prepare_agent_executor_forward: Any,
|
||||
completion_to_sse_chunks: Any,
|
||||
synthetic_first_tool_completion: Any,
|
||||
meta_header: Any,
|
||||
orchestrator: Any,
|
||||
requests_metric: Any,
|
||||
executor_fallback_completion: Any = None,
|
||||
) -> StreamingResponse:
|
||||
"""Immediately SSE-stream progress; plan in background; never silent-hang."""
|
||||
|
||||
cid = f"hier-agent-{int(time.time() * 1000)}"
|
||||
queue: asyncio.Queue[tuple[str, Any]] = asyncio.Queue()
|
||||
|
||||
async def on_progress(msg: str) -> None:
|
||||
await queue.put(("progress", msg))
|
||||
|
||||
async def work() -> None:
|
||||
try:
|
||||
hier = await runner.run(
|
||||
last_user,
|
||||
quality_mode=quality_mode,
|
||||
session_id=session_id,
|
||||
messages=messages,
|
||||
on_progress=on_progress,
|
||||
agent_mode=True,
|
||||
)
|
||||
await queue.put(("done", hier))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.exception("agent plan failed: %s", exc)
|
||||
await queue.put(("error", exc))
|
||||
|
||||
async def gen():
|
||||
# First bytes ASAP — Zed must leave spinner for content
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"role": "assistant",
|
||||
"content": "**Agent**\n· Планирую задачу\n",
|
||||
},
|
||||
)
|
||||
task = asyncio.create_task(work())
|
||||
hier = None
|
||||
failed: Exception | None = None
|
||||
progress_lines: list[str] = ["Планирую задачу"]
|
||||
|
||||
while True:
|
||||
try:
|
||||
kind, payload = await asyncio.wait_for(queue.get(), timeout=12.0)
|
||||
except asyncio.TimeoutError:
|
||||
# SSE comment keepalive (proxies / Zed idle timeout)
|
||||
yield b": keepalive\n\n"
|
||||
if task.done() and queue.empty():
|
||||
# drain outcome if any
|
||||
if not task.cancelled():
|
||||
try:
|
||||
_ = task.result()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
failed = exc
|
||||
break
|
||||
continue
|
||||
|
||||
if kind == "progress":
|
||||
from progress_ui import humanize_line, stream_step
|
||||
|
||||
line = humanize_line(str(payload))
|
||||
if not line:
|
||||
continue
|
||||
progress_lines.append(line)
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={"content": stream_step(line)},
|
||||
)
|
||||
continue
|
||||
if kind == "error":
|
||||
failed = payload # type: ignore[assignment]
|
||||
break
|
||||
hier = payload
|
||||
break
|
||||
|
||||
if not task.done():
|
||||
await task
|
||||
|
||||
if failed is not None:
|
||||
err = (str(failed) or type(failed).__name__)[:200]
|
||||
meta["hierarchical_error"] = err
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": (
|
||||
f"\n---\nОшибка плана (роутер, не LiteLLM UI): `{err}`.\n"
|
||||
"Повтори запрос или напиши проще. Спиннер без шагов = баг "
|
||||
"роутера — мы должны стримить progress; если снова тишина, "
|
||||
"проверь деплой.\n"
|
||||
)
|
||||
},
|
||||
)
|
||||
yield sse_chunk(
|
||||
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
|
||||
)
|
||||
yield b"data: [DONE]\n\n"
|
||||
return
|
||||
|
||||
if hier is None:
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={"content": "\n---\nПустой ответ плана.\n"},
|
||||
)
|
||||
yield sse_chunk(
|
||||
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
|
||||
)
|
||||
yield b"data: [DONE]\n\n"
|
||||
return
|
||||
|
||||
hier_meta = {**meta, **hier.meta}
|
||||
|
||||
if hier.meta.get("agent_execute"):
|
||||
plan = plan_payload_from_meta(hier.meta) or find_pending_plan(messages)
|
||||
orchestrator.after_request(
|
||||
session_id, prompt_hash=prompt_hash, success=True, escalate=False
|
||||
)
|
||||
from path_resolve import (
|
||||
apply_deterministic_path_index,
|
||||
path_resolve_needed,
|
||||
pick_path_resolve_model,
|
||||
)
|
||||
|
||||
if cfg.get("path_resolve_enabled", True):
|
||||
plan, unresolved = apply_deterministic_path_index(plan)
|
||||
if unresolved:
|
||||
log.info("path_index miss (stream) → %s", unresolved)
|
||||
if cfg.get("runtime_probe_enabled", True) and isinstance(plan, dict):
|
||||
try:
|
||||
from runtime_probe import attach_runtime_probe, runtime_facts_line
|
||||
|
||||
user_bits = []
|
||||
for m in reversed(messages or []):
|
||||
if isinstance(m, dict) and m.get("role") == "user":
|
||||
c = m.get("content")
|
||||
user_bits.append(c if isinstance(c, str) else str(c or ""))
|
||||
break
|
||||
plan = attach_runtime_probe(
|
||||
plan, user_text=user_bits[0] if user_bits else "", cfg=cfg
|
||||
)
|
||||
line = runtime_facts_line(plan)
|
||||
if line:
|
||||
log.info("%s", line)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("runtime_probe skipped (stream): %s", exc)
|
||||
path_mode = bool(cfg.get("path_resolve_enabled", True)) and path_resolve_needed(
|
||||
plan
|
||||
)
|
||||
if path_mode:
|
||||
executor = pick_path_resolve_model(cfg)
|
||||
else:
|
||||
executor = pick_agent_executor_model(cfg)
|
||||
timeout = float(
|
||||
cfg.get("path_resolve_timeout_sec")
|
||||
if path_mode
|
||||
else (
|
||||
cfg.get("executor_timeout_sec")
|
||||
or cfg.get("call_timeout_sec")
|
||||
or 90
|
||||
)
|
||||
)
|
||||
max_ctx = int(cfg.get("executor_input_chars", 12000))
|
||||
use_stream = bool(cfg.get("executor_use_stream", False))
|
||||
verbose = bool(cfg.get("progress_verbose", False))
|
||||
from progress_ui import execution_banner
|
||||
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": execution_banner(
|
||||
path_mode=path_mode, plan=plan, model=executor
|
||||
)
|
||||
},
|
||||
)
|
||||
if verbose:
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": f"· модель `{executor}` · таймаут {int(timeout)}с\n"
|
||||
},
|
||||
)
|
||||
_ = inject_plan_context
|
||||
forward = prepare_agent_executor_forward(
|
||||
body,
|
||||
messages,
|
||||
plan,
|
||||
cfg,
|
||||
executor=executor,
|
||||
stream=use_stream,
|
||||
minimal=True,
|
||||
path_resolve=path_mode,
|
||||
)
|
||||
n_tools = len(forward.get("tools") or [])
|
||||
approx = len(json.dumps(forward, ensure_ascii=False, default=str))
|
||||
from agent_hier import context_fill_for_forward
|
||||
|
||||
ctx_line, ctx_meta = context_fill_for_forward(
|
||||
forward, cfg, budget_chars=max_ctx
|
||||
)
|
||||
log.info(
|
||||
"executor forward model=%s tools=%s bytes≈%s stream=%s %s",
|
||||
executor,
|
||||
n_tools,
|
||||
approx,
|
||||
use_stream,
|
||||
ctx_line,
|
||||
)
|
||||
if verbose or cfg.get("show_context_fill", False):
|
||||
payload_line = f"- tools={n_tools}\n"
|
||||
if cfg.get("show_context_fill", False):
|
||||
payload_line += f"- {ctx_line}\n"
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={"content": payload_line},
|
||||
)
|
||||
_ = ctx_meta
|
||||
try:
|
||||
body_timeout = float(forward.get("timeout") or timeout)
|
||||
client_timeout = body_timeout + 30.0
|
||||
if use_stream:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(client_timeout, connect=15.0)
|
||||
) as client:
|
||||
async with client.stream(
|
||||
"POST",
|
||||
f"{litellm_url.rstrip('/')}/v1/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {litellm_key}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json=forward,
|
||||
) as resp:
|
||||
if resp.status_code >= 400:
|
||||
err_body = (await resp.aread())[:300]
|
||||
raise RuntimeError(
|
||||
f"litellm HTTP {resp.status_code}: "
|
||||
f"{err_body.decode('utf-8', 'replace')}"
|
||||
)
|
||||
buf = b""
|
||||
aiter = resp.aiter_bytes().__aiter__()
|
||||
deadline = time.perf_counter() + client_timeout
|
||||
got_data = False
|
||||
while time.perf_counter() < deadline:
|
||||
try:
|
||||
piece = await asyncio.wait_for(
|
||||
aiter.__anext__(), timeout=12.0
|
||||
)
|
||||
except StopAsyncIteration:
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
yield b": keepalive\n\n"
|
||||
if not got_data:
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": "- executor: ждём LiteLLM…\n"
|
||||
},
|
||||
)
|
||||
continue
|
||||
got_data = True
|
||||
buf += piece
|
||||
while True:
|
||||
nl = buf.find(b"\n")
|
||||
if nl < 0:
|
||||
break
|
||||
line = buf[:nl].rstrip(b"\r")
|
||||
buf = buf[nl + 1 :]
|
||||
if not line:
|
||||
yield b"\n"
|
||||
continue
|
||||
yield line + b"\n"
|
||||
if buf.strip():
|
||||
yield buf.rstrip(b"\r") + b"\n"
|
||||
yield b"data: [DONE]\n\n"
|
||||
else:
|
||||
# Non-stream: keepalive while awaiting Novita (headers otherwise block)
|
||||
async def _post() -> httpx.Response:
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(client_timeout, connect=15.0)
|
||||
) as client:
|
||||
return await client.post(
|
||||
f"{litellm_url.rstrip('/')}/v1/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {litellm_key}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json=forward,
|
||||
)
|
||||
|
||||
task = asyncio.create_task(_post())
|
||||
while not task.done():
|
||||
try:
|
||||
await asyncio.wait_for(asyncio.shield(task), timeout=8.0)
|
||||
except asyncio.TimeoutError:
|
||||
yield b": keepalive\n\n"
|
||||
yield (
|
||||
b": executor waiting LiteLLM non-stream\n\n"
|
||||
)
|
||||
resp = task.result()
|
||||
if resp.status_code >= 400:
|
||||
raise RuntimeError(
|
||||
f"litellm HTTP {resp.status_code}: "
|
||||
f"{resp.text[:300]}"
|
||||
)
|
||||
data = resp.json()
|
||||
msg = ((data.get("choices") or [{}])[0].get("message") or {})
|
||||
has_tools = bool(msg.get("tool_calls"))
|
||||
if not has_tools:
|
||||
from agent_hier import (
|
||||
completion_preview,
|
||||
force_kickstart_read,
|
||||
pick_agent_escalate_model,
|
||||
completion_has_tool_calls as _has_tc,
|
||||
)
|
||||
|
||||
log.warning(
|
||||
"executor no tool_calls after plan: %s",
|
||||
completion_preview(data),
|
||||
)
|
||||
# 1) Escalate to DeepSeek with hard tool_choice
|
||||
if cfg.get("executor_no_tools_escalate", True):
|
||||
esc = pick_agent_escalate_model(cfg)
|
||||
esc_fwd = dict(forward)
|
||||
esc_fwd["model"] = esc
|
||||
esc_fwd["tool_choice"] = "required"
|
||||
path_hint = None
|
||||
try:
|
||||
from agent_hier import _first_path_hint
|
||||
|
||||
path_hint = _first_path_hint(
|
||||
plan, forward.get("tools") or [], messages
|
||||
)
|
||||
except Exception: # noqa: BLE001
|
||||
path_hint = None
|
||||
nudge = (
|
||||
"Call a tool now. Start with read_file"
|
||||
+ (f" on `{path_hint}`." if path_hint else ".")
|
||||
+ " No prose."
|
||||
)
|
||||
esc_fwd["messages"] = list(esc_fwd.get("messages") or []) + [
|
||||
{"role": "user", "content": nudge}
|
||||
]
|
||||
yield (
|
||||
f": no tool_calls → escalate `{esc}`\n\n".encode(
|
||||
"utf-8"
|
||||
)
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(client_timeout, connect=15.0)
|
||||
) as client2:
|
||||
resp2 = await client2.post(
|
||||
f"{litellm_url.rstrip('/')}/v1/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {litellm_key}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json=esc_fwd,
|
||||
)
|
||||
if resp2.status_code < 400:
|
||||
data2 = resp2.json()
|
||||
if isinstance(data2, dict) and _has_tc(data2):
|
||||
data = data2
|
||||
has_tools = True
|
||||
log.info("escalate no_tools → tool_calls OK")
|
||||
# 2) Kickstart read_file on plan path (keeps Zed loop alive)
|
||||
if not has_tools and cfg.get(
|
||||
"executor_synthetic_on_timeout", True
|
||||
):
|
||||
yield b": no tool_calls -> kickstart read_file\n\n"
|
||||
if str(cfg.get("executor_synthetic_mode") or "").lower() in (
|
||||
"never",
|
||||
):
|
||||
# still kickstart once on first turn — otherwise dead end
|
||||
data = force_kickstart_read(
|
||||
plan=plan,
|
||||
tools=forward.get("tools") or [],
|
||||
model=executor,
|
||||
messages=messages,
|
||||
)
|
||||
elif executor_fallback_completion:
|
||||
data = executor_fallback_completion(
|
||||
plan=plan,
|
||||
tools=forward.get("tools") or [],
|
||||
model=executor,
|
||||
messages=messages,
|
||||
cfg=cfg,
|
||||
reason="no_tool_calls",
|
||||
)
|
||||
else:
|
||||
data = force_kickstart_read(
|
||||
plan=plan,
|
||||
tools=forward.get("tools") or [],
|
||||
model=executor,
|
||||
messages=messages,
|
||||
)
|
||||
has_tools = bool(
|
||||
(
|
||||
(data.get("choices") or [{}])[0].get("message") or {}
|
||||
).get("tool_calls")
|
||||
)
|
||||
# Visible status for Zed (short), details stay in SSE comments
|
||||
from progress_ui import plan_paths, tool_status
|
||||
|
||||
tc_names: list[str] = []
|
||||
msg0 = ((data.get("choices") or [{}])[0].get("message") or {})
|
||||
for tc in msg0.get("tool_calls") or []:
|
||||
if isinstance(tc, dict):
|
||||
tc_names.append(
|
||||
str(((tc.get("function") or {}).get("name")) or "")
|
||||
)
|
||||
paths = plan_paths(plan)
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": tool_status(
|
||||
tc_names, path=paths[0] if paths else None
|
||||
)
|
||||
},
|
||||
)
|
||||
yield (
|
||||
f": executor OK tool_calls={has_tools}\n\n".encode("utf-8")
|
||||
)
|
||||
for chunk in completion_to_sse_chunks(
|
||||
data, cid=cid, model=str(data.get("model") or executor)
|
||||
):
|
||||
yield chunk
|
||||
except Exception as exc: # noqa: BLE001
|
||||
err = (str(exc) or type(exc).__name__)[:220]
|
||||
log.exception("executor after plan failed: %s", err)
|
||||
if cfg.get("executor_synthetic_on_timeout", True):
|
||||
yield (
|
||||
f": timeout → executor fallback ({err[:80]})\n\n".encode(
|
||||
"utf-8"
|
||||
)
|
||||
)
|
||||
if executor_fallback_completion:
|
||||
data = executor_fallback_completion(
|
||||
plan=plan,
|
||||
tools=forward.get("tools") or [],
|
||||
model=executor,
|
||||
messages=messages,
|
||||
cfg=cfg,
|
||||
reason=f"exception:{err[:80]}",
|
||||
)
|
||||
else:
|
||||
data = synthetic_first_tool_completion(
|
||||
plan=plan,
|
||||
tools=forward.get("tools") or [],
|
||||
model=executor,
|
||||
messages=messages,
|
||||
max_synthetic=int(
|
||||
cfg.get("executor_max_synthetic_continues", 1) or 1
|
||||
),
|
||||
max_edit_failures=int(
|
||||
cfg.get("executor_max_edit_failures", 4) or 4
|
||||
),
|
||||
)
|
||||
for chunk in completion_to_sse_chunks(
|
||||
data, cid=cid, model=executor
|
||||
):
|
||||
yield chunk
|
||||
else:
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={
|
||||
"content": (
|
||||
f"\nExecutor не ответил: `{err}`. "
|
||||
"Роутер жив — увеличь timeout / урежь контекст; "
|
||||
"повтор после ok обычно быстрее (cache).\n"
|
||||
)
|
||||
},
|
||||
)
|
||||
yield sse_chunk(
|
||||
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
|
||||
)
|
||||
yield b"data: [DONE]\n\n"
|
||||
return
|
||||
|
||||
# Confirm / cancel / amend — text plan for user
|
||||
content = hier.content or ""
|
||||
yield sse_chunk(
|
||||
cid=cid,
|
||||
model="hierarchical",
|
||||
delta={"content": f"\n---\n\n{content}"},
|
||||
)
|
||||
try:
|
||||
requests_metric.labels(
|
||||
tier=decision.tier.value,
|
||||
lane=decision.lane,
|
||||
model="hierarchical_agent",
|
||||
status="200",
|
||||
).inc()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
orchestrator.after_request(
|
||||
session_id, prompt_hash=prompt_hash, success=True, escalate=False
|
||||
)
|
||||
yield sse_chunk(
|
||||
cid=cid, model="hierarchical", delta={}, finish_reason="stop"
|
||||
)
|
||||
yield b"data: [DONE]\n\n"
|
||||
|
||||
return StreamingResponse(
|
||||
gen(),
|
||||
media_type="text/event-stream",
|
||||
headers={
|
||||
"X-Router-Meta": meta_header({**meta, "mode": "hierarchical_agent"}),
|
||||
"Cache-Control": "no-cache",
|
||||
"X-Accel-Buffering": "no",
|
||||
},
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,134 @@
|
||||
"""Redis cache + in-flight coalescing for identical LiteLLM chat calls."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Any
|
||||
|
||||
log = logging.getLogger("llm_cache")
|
||||
|
||||
_redis = None
|
||||
_redis_tried = False
|
||||
_inflight: dict[str, asyncio.Future[str]] = {}
|
||||
_inflight_lock = asyncio.Lock()
|
||||
|
||||
|
||||
def _get_redis():
|
||||
global _redis, _redis_tried
|
||||
if _redis_tried:
|
||||
return _redis
|
||||
_redis_tried = True
|
||||
url = os.environ.get("REDIS_URL", "").strip()
|
||||
if not url:
|
||||
return None
|
||||
try:
|
||||
import redis
|
||||
|
||||
client = redis.from_url(url, decode_responses=True)
|
||||
client.ping()
|
||||
_redis = client
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("llm cache redis unavailable: %s", exc)
|
||||
_redis = None
|
||||
return _redis
|
||||
|
||||
|
||||
def cache_key(
|
||||
*,
|
||||
model: str,
|
||||
messages: list[dict[str, Any]],
|
||||
max_tokens: int,
|
||||
temperature: float,
|
||||
) -> str:
|
||||
blob = json.dumps(
|
||||
{
|
||||
"model": model,
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
"temperature": temperature,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
digest = hashlib.sha256(blob.encode("utf-8")).hexdigest()
|
||||
return f"ai-router:llm-cache:{digest}"
|
||||
|
||||
|
||||
def get_cached(key: str) -> str | None:
|
||||
r = _get_redis()
|
||||
if not r:
|
||||
return None
|
||||
try:
|
||||
val = r.get(key)
|
||||
return str(val) if val is not None else None
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("llm cache get failed: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def set_cached(key: str, value: str, ttl_sec: int) -> None:
|
||||
r = _get_redis()
|
||||
if not r or ttl_sec <= 0:
|
||||
return
|
||||
try:
|
||||
r.setex(key, ttl_sec, value)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("llm cache set failed: %s", exc)
|
||||
|
||||
|
||||
async def run_cached(
|
||||
key: str,
|
||||
factory: Callable[[], Awaitable[str]],
|
||||
*,
|
||||
ttl_sec: int,
|
||||
on_hit: Callable[[str], Awaitable[None] | None] | None = None,
|
||||
) -> tuple[str, str]:
|
||||
"""
|
||||
Return (text, source) where source is cache|coalesce|live.
|
||||
Coalesce concurrent identical calls onto one upstream request.
|
||||
"""
|
||||
cached = get_cached(key)
|
||||
if cached is not None:
|
||||
if on_hit:
|
||||
maybe = on_hit("cache")
|
||||
if asyncio.iscoroutine(maybe):
|
||||
await maybe
|
||||
return cached, "cache"
|
||||
|
||||
async with _inflight_lock:
|
||||
existing = _inflight.get(key)
|
||||
if existing is not None:
|
||||
fut: asyncio.Future[str] = existing
|
||||
mine = False
|
||||
else:
|
||||
fut = asyncio.get_running_loop().create_future()
|
||||
_inflight[key] = fut
|
||||
mine = True
|
||||
|
||||
if not mine:
|
||||
if on_hit:
|
||||
maybe = on_hit("coalesce")
|
||||
if asyncio.iscoroutine(maybe):
|
||||
await maybe
|
||||
return await fut, "coalesce"
|
||||
|
||||
try:
|
||||
text = await factory()
|
||||
set_cached(key, text, ttl_sec)
|
||||
if not fut.done():
|
||||
fut.set_result(text)
|
||||
return text, "live"
|
||||
except Exception as exc:
|
||||
if not fut.done():
|
||||
fut.set_exception(exc)
|
||||
raise
|
||||
finally:
|
||||
async with _inflight_lock:
|
||||
if _inflight.get(key) is fut:
|
||||
_inflight.pop(key, None)
|
||||
@@ -115,6 +115,7 @@ class Classifier:
|
||||
self._complex = self._kw_re(gw.get("complex_keywords", []))
|
||||
self._reasoning = self._kw_re(gw.get("reasoning_keywords", []))
|
||||
self._medium_ops = self._kw_re(gw.get("medium_ops_keywords", []))
|
||||
self._medium_code = self._kw_re(gw.get("medium_code_keywords", []))
|
||||
self._ocr = self._kw_re(gw.get("ocr_keywords", []))
|
||||
self._escalation = self._kw_re(gw.get("escalation_keywords", []))
|
||||
wt = gw.get("word_thresholds", {})
|
||||
@@ -159,7 +160,7 @@ class Classifier:
|
||||
return Tier.REASONING, 0.88
|
||||
if self._complex.search(text):
|
||||
return Tier.COMPLEX, 0.88
|
||||
if self._code_block.search(text):
|
||||
if self._code_block.search(text) or self._medium_code.search(text):
|
||||
return Tier.MEDIUM_CODE, 0.85
|
||||
if self._medium_ops.search(text):
|
||||
return Tier.MEDIUM_OPS, 0.82
|
||||
|
||||
@@ -0,0 +1,430 @@
|
||||
"""Path resolve: deterministic index first, cheap find_* tools if gaps remain."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
log = logging.getLogger("ai-router")
|
||||
|
||||
# Absolute Windows root for EventHubDevOps (Zed multi-root).
|
||||
_DEFAULT_DEVOPS_ROOT = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps"
|
||||
|
||||
# keyword groups → relative paths under DevOps root (first match wins per group).
|
||||
# Order matters: more specific rules first.
|
||||
_INDEX_RULES: list[tuple[tuple[str, ...], tuple[str, ...]]] = [
|
||||
(
|
||||
("loadtest", "load-test", "load_test"),
|
||||
(r"ift\traefik\dynamic_conf.loadtest.yml",),
|
||||
),
|
||||
(
|
||||
(
|
||||
"traefik",
|
||||
"dynamic_conf",
|
||||
"router",
|
||||
"middleware",
|
||||
"calentiq",
|
||||
"observer",
|
||||
"host(",
|
||||
"ift.calentiq",
|
||||
"stage.calentiq",
|
||||
),
|
||||
(r"ift\traefik\dynamic_conf.yml",),
|
||||
),
|
||||
(
|
||||
("portainer",),
|
||||
(r"ift\docker-compose.portainer.yml",),
|
||||
),
|
||||
(
|
||||
("grafana", "dashboard"),
|
||||
(
|
||||
r"ift\observability\grafana\provisioning\dashboards\dashboard.yml",
|
||||
r"ift\observability\grafana\provisioning\datasources\prometheus.yml",
|
||||
),
|
||||
),
|
||||
(
|
||||
("prometheus", "observability"),
|
||||
(r"ift\observability\prometheus.yml",),
|
||||
),
|
||||
(
|
||||
("admin", "compose.admin", "docker-compose.admin"),
|
||||
(r"ift\docker-compose.admin.yml",),
|
||||
),
|
||||
(
|
||||
("client", "compose.client", "docker-compose.client"),
|
||||
(r"ift\docker-compose.client.yml",),
|
||||
),
|
||||
(
|
||||
("compose", "swarm", "stack", "docker-compose", "service"),
|
||||
(r"ift\docker-compose.core.yml",),
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def _join_under(root: str, rel: str) -> str:
|
||||
root = root.rstrip("\\/")
|
||||
rel = rel.replace("/", "\\").lstrip("\\")
|
||||
return root + "\\" + rel
|
||||
|
||||
|
||||
def _norm(path: str) -> str:
|
||||
return path.replace("/", "\\").rstrip("\\").lower()
|
||||
|
||||
|
||||
def _blob(plan: dict[str, Any] | None) -> str:
|
||||
return json.dumps(plan or {}, ensure_ascii=False).lower()
|
||||
|
||||
|
||||
def devops_root_from_plan(plan: dict[str, Any] | None) -> str:
|
||||
blob = json.dumps(plan or {}, ensure_ascii=False)
|
||||
m = re.search(r"([A-Za-z]:\\[^\"'\n\r]*?EventHubDevOps)", blob)
|
||||
if m:
|
||||
return m.group(1)
|
||||
return _DEFAULT_DEVOPS_ROOT
|
||||
|
||||
|
||||
def looks_devops(plan: dict[str, Any] | None) -> bool:
|
||||
b = _blob(plan)
|
||||
return any(
|
||||
k in b
|
||||
for k in (
|
||||
"traefik",
|
||||
"calentiq",
|
||||
"eventhubdevops",
|
||||
"docker-compose",
|
||||
"swarm",
|
||||
"observer",
|
||||
"ift.",
|
||||
"devops",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def match_index_paths(plan: dict[str, Any] | None) -> list[str]:
|
||||
"""Return absolute paths from keyword index (may be empty)."""
|
||||
if not looks_devops(plan):
|
||||
return []
|
||||
root = devops_root_from_plan(plan)
|
||||
text = _blob(plan)
|
||||
found: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for keys, rels in _INDEX_RULES:
|
||||
if not any(k in text for k in keys):
|
||||
continue
|
||||
for rel in rels:
|
||||
abs_p = _join_under(root, rel)
|
||||
nk = _norm(abs_p)
|
||||
if nk in seen:
|
||||
continue
|
||||
seen.add(nk)
|
||||
found.append(abs_p)
|
||||
# one rule group is usually enough for a focused plan
|
||||
if found:
|
||||
break
|
||||
return found
|
||||
|
||||
|
||||
def apply_deterministic_path_index(
|
||||
plan: dict[str, Any] | None,
|
||||
) -> tuple[dict[str, Any] | None, list[str]]:
|
||||
"""Fill plan.paths from index. Returns (plan, unresolved_queries).
|
||||
|
||||
unresolved_queries empty ⇒ index covered the goal; else cheap find_* needed.
|
||||
"""
|
||||
if not isinstance(plan, dict):
|
||||
return plan, []
|
||||
out = dict(plan)
|
||||
tasks = out.get("subtasks")
|
||||
if not isinstance(tasks, list) or not tasks:
|
||||
# still try goal-level match
|
||||
indexed = match_index_paths(out)
|
||||
if indexed:
|
||||
out["subtasks"] = [
|
||||
{
|
||||
"id": "1",
|
||||
"prompt": str(out.get("user_goal") or "edit")[:200],
|
||||
"worker_tier": "medium_code",
|
||||
"paths": indexed,
|
||||
"edit_goal": "",
|
||||
"constraints": [],
|
||||
}
|
||||
]
|
||||
out["path_resolve"] = "index"
|
||||
return out, []
|
||||
if looks_devops(out):
|
||||
q = _queries_from_text(_blob(out))
|
||||
out["path_resolve_queries"] = q
|
||||
return out, q
|
||||
return out, []
|
||||
|
||||
indexed = match_index_paths(out)
|
||||
unresolved: list[str] = []
|
||||
fixed: list[Any] = []
|
||||
for item in tasks:
|
||||
if not isinstance(item, dict):
|
||||
fixed.append(item)
|
||||
continue
|
||||
it = dict(item)
|
||||
paths = [
|
||||
p
|
||||
for p in (it.get("paths") or [])
|
||||
if isinstance(p, str) and p.strip()
|
||||
]
|
||||
# Prefer index hits when devops; replace empty/weak paths
|
||||
if indexed:
|
||||
it["paths"] = indexed
|
||||
it["path_source"] = "index"
|
||||
elif not paths:
|
||||
q = _queries_from_text(
|
||||
str(it.get("prompt") or "")
|
||||
+ " "
|
||||
+ str(it.get("edit_goal") or "")
|
||||
+ " "
|
||||
+ _blob(out)
|
||||
)
|
||||
unresolved.extend(q)
|
||||
it["paths"] = []
|
||||
it["path_source"] = "missing"
|
||||
else:
|
||||
it["path_source"] = "planner"
|
||||
fixed.append(it)
|
||||
out["subtasks"] = fixed
|
||||
if indexed:
|
||||
out["path_resolve"] = "index"
|
||||
out["path_resolve_queries"] = []
|
||||
log.info("path_index hit → %s", indexed)
|
||||
return out, []
|
||||
# Planner had paths but index missed — if devops, still verify via find
|
||||
if looks_devops(out) and not indexed:
|
||||
unresolved = unresolved or _queries_from_text(_blob(out))
|
||||
out["path_resolve_queries"] = unresolved
|
||||
out["path_resolve"] = "needed"
|
||||
return out, unresolved
|
||||
out["path_resolve_queries"] = unresolved
|
||||
if unresolved:
|
||||
out["path_resolve"] = "needed"
|
||||
return out, unresolved
|
||||
|
||||
|
||||
def _queries_from_text(text: str) -> list[str]:
|
||||
low = text.lower()
|
||||
qs: list[str] = []
|
||||
if any(k in low for k in ("traefik", "router", "calentiq", "observer", "dynamic")):
|
||||
qs.append("dynamic_conf.yml")
|
||||
if any(k in low for k in ("compose", "swarm", "docker")):
|
||||
qs.append("docker-compose.core.yml")
|
||||
if "portainer" in low:
|
||||
qs.append("docker-compose.portainer.yml")
|
||||
if "grafana" in low:
|
||||
qs.append("grafana")
|
||||
if "prometheus" in low:
|
||||
qs.append("prometheus.yml")
|
||||
if not qs:
|
||||
qs.append("dynamic_conf.yml")
|
||||
# unique preserve order
|
||||
out: list[str] = []
|
||||
for q in qs:
|
||||
if q not in out:
|
||||
out.append(q)
|
||||
return out[:3]
|
||||
|
||||
|
||||
def enrich_plan_from_discovery_tools(
|
||||
plan: dict[str, Any] | None,
|
||||
messages: list[dict[str, Any]] | None,
|
||||
) -> dict[str, Any] | None:
|
||||
"""Pull absolute paths from find_path / list_directory tool results into plan."""
|
||||
if not isinstance(plan, dict):
|
||||
return plan
|
||||
# Never overwrite a good deterministic index hit with noisy find parses
|
||||
if str(plan.get("path_resolve") or "") == "index":
|
||||
return plan
|
||||
tasks = plan.get("subtasks") or []
|
||||
if (
|
||||
isinstance(tasks, list)
|
||||
and tasks
|
||||
and isinstance(tasks[0], dict)
|
||||
and str(tasks[0].get("path_source") or "") == "index"
|
||||
):
|
||||
return plan
|
||||
found = _paths_from_discovery_messages(messages)
|
||||
if not found:
|
||||
return plan
|
||||
out = dict(plan)
|
||||
tasks = list(out.get("subtasks") or [])
|
||||
if not tasks:
|
||||
out["subtasks"] = [
|
||||
{
|
||||
"id": "1",
|
||||
"prompt": "edit",
|
||||
"worker_tier": "medium_code",
|
||||
"paths": found[:3],
|
||||
"path_source": "find",
|
||||
}
|
||||
]
|
||||
else:
|
||||
fixed = []
|
||||
for item in tasks:
|
||||
if not isinstance(item, dict):
|
||||
fixed.append(item)
|
||||
continue
|
||||
it = dict(item)
|
||||
it["paths"] = found[:3]
|
||||
it["path_source"] = "find"
|
||||
fixed.append(it)
|
||||
out["subtasks"] = fixed
|
||||
out["path_resolve"] = "find"
|
||||
out["path_resolve_queries"] = []
|
||||
log.info("path_resolve find → %s", found[:3])
|
||||
return out
|
||||
|
||||
|
||||
def _is_workspace_path(path: str) -> bool:
|
||||
"""Accept only real EventHub project paths — never /etc or null:/etc artifacts."""
|
||||
key = _norm(path)
|
||||
if "eventhubdevops" not in key and "eventhub" not in key:
|
||||
return False
|
||||
if "\\etc\\" in key or key.startswith("etc\\"):
|
||||
return False
|
||||
if "\\nginx\\" in key:
|
||||
return False
|
||||
# Reject 1-letter drive artifacts from YAML like null:/etc → l:\etc
|
||||
if re.match(r"^[a-z]:\\etc\\", key):
|
||||
return False
|
||||
base = key.rsplit("\\", 1)[-1]
|
||||
if base in ("traefik.yml", "traefik.yaml", "docker-compose.yml"):
|
||||
if "\\ift\\" not in key and "\\stage\\" not in key:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _paths_from_discovery_messages(
|
||||
messages: list[dict[str, Any]] | None,
|
||||
) -> list[str]:
|
||||
"""Extract existing file paths from find/list tool results only."""
|
||||
call_ids: set[str] = set()
|
||||
for msg in messages or []:
|
||||
if not isinstance(msg, dict) or msg.get("role") != "assistant":
|
||||
continue
|
||||
for tc in msg.get("tool_calls") or []:
|
||||
if not isinstance(tc, dict):
|
||||
continue
|
||||
name = str((tc.get("function") or {}).get("name") or "").lower()
|
||||
if not any(k in name for k in ("find", "list", "search", "grep", "glob")):
|
||||
continue
|
||||
# Do not treat read_file / edit_file as discovery
|
||||
if any(k in name for k in ("read", "edit", "write", "create", "delete")):
|
||||
continue
|
||||
cid = str(tc.get("id") or "")
|
||||
if cid:
|
||||
call_ids.add(cid)
|
||||
# No discovery tool calls → do not scrape read_file bodies (YAML has /etc/… paths)
|
||||
if not call_ids:
|
||||
return []
|
||||
paths: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for msg in messages or []:
|
||||
if not isinstance(msg, dict) or msg.get("role") != "tool":
|
||||
continue
|
||||
if str(msg.get("tool_call_id") or "") not in call_ids:
|
||||
continue
|
||||
content = str(msg.get("content") or "")
|
||||
low = content.lower()
|
||||
if "not found" in low and len(content) < 200:
|
||||
continue
|
||||
for m in re.finditer(
|
||||
r"([A-Za-z]:[\\/][^\s\"'<>\]\n\r]+\.(?:yml|yaml|toml|md|json|conf))",
|
||||
content,
|
||||
):
|
||||
p = m.group(1).replace("/", "\\")
|
||||
if not _is_workspace_path(p):
|
||||
continue
|
||||
nk = _norm(p)
|
||||
if nk in seen:
|
||||
continue
|
||||
seen.add(nk)
|
||||
paths.append(p)
|
||||
# Also accept plain relative hits that Zed returns under EventHubDevOps
|
||||
for m in re.finditer(
|
||||
r"((?:EventHubDevOps[\\/](?:ift|stage)[\\/][^\s\"'<>\]\n\r]+\.(?:yml|yaml)))",
|
||||
content,
|
||||
re.I,
|
||||
):
|
||||
p = _join_under(
|
||||
_DEFAULT_DEVOPS_ROOT,
|
||||
m.group(1).split("EventHubDevOps", 1)[-1].lstrip("\\/"),
|
||||
)
|
||||
if not _is_workspace_path(p):
|
||||
continue
|
||||
nk = _norm(p)
|
||||
if nk in seen:
|
||||
continue
|
||||
seen.add(nk)
|
||||
paths.append(p)
|
||||
return paths
|
||||
|
||||
|
||||
def discovery_tools_only(tools: list[Any], *, max_tools: int = 3) -> list[Any]:
|
||||
prefer_keys = ("find_path", "find", "list_directory", "list_dir", "grep", "search", "glob")
|
||||
ranked: list[Any] = []
|
||||
for key in prefer_keys:
|
||||
for t in tools or []:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
fn = t.get("function") if isinstance(t.get("function"), dict) else {}
|
||||
name = str(fn.get("name") or t.get("name") or "").lower()
|
||||
if key in name and t not in ranked:
|
||||
ranked.append(t)
|
||||
if len(ranked) >= max_tools:
|
||||
break
|
||||
return ranked[:max_tools]
|
||||
|
||||
|
||||
def pick_path_resolve_model(cfg: dict[str, Any]) -> str:
|
||||
"""Cheap model for find_* only — allow a-simple (unlike edit executor)."""
|
||||
return str(cfg.get("path_resolve_model") or "a-simple")
|
||||
|
||||
|
||||
def path_resolve_needed(plan: dict[str, Any] | None) -> bool:
|
||||
if not isinstance(plan, dict):
|
||||
return False
|
||||
qs = plan.get("path_resolve_queries") or []
|
||||
if qs:
|
||||
return True
|
||||
return str(plan.get("path_resolve") or "") == "needed"
|
||||
|
||||
|
||||
def force_find_path_kickstart(
|
||||
*,
|
||||
model: str,
|
||||
tools: list[Any],
|
||||
queries: list[str],
|
||||
) -> dict[str, Any]:
|
||||
"""Synthetic find_path so Zed searches instead of inventing read_file."""
|
||||
from agent_hier import _synthetic_tool_completion
|
||||
|
||||
tool_name = "find_path"
|
||||
for t in tools or []:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
fn = t.get("function") if isinstance(t.get("function"), dict) else {}
|
||||
name = str(fn.get("name") or "")
|
||||
low = name.lower()
|
||||
if "find" in low or "glob" in low or ("list" in low and "dir" in low):
|
||||
tool_name = name
|
||||
break
|
||||
q = (queries[0] if queries else "dynamic_conf.yml").strip()
|
||||
args: dict[str, Any]
|
||||
if "list" in tool_name.lower():
|
||||
args = {
|
||||
"path": _join_under(_DEFAULT_DEVOPS_ROOT, r"ift\traefik"),
|
||||
}
|
||||
else:
|
||||
args = {"query": q, "path": _DEFAULT_DEVOPS_ROOT}
|
||||
# Zed variants
|
||||
args["glob"] = f"**/{q}" if "." in q else f"**/*{q}*"
|
||||
return _synthetic_tool_completion(
|
||||
model=model, tool_name=tool_name, arguments=args
|
||||
)
|
||||
@@ -0,0 +1,201 @@
|
||||
"""Compact, readable progress lines for Zed agent chat."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
_MODEL_LABEL = {
|
||||
"novita-planner": "Max",
|
||||
"novita-verifier": "verify",
|
||||
"a-medium-code": "Coder",
|
||||
"a-simple": "Llama",
|
||||
"b-complex": "DeepSeek",
|
||||
"c-complex": "Max",
|
||||
"approved-plan": "план",
|
||||
}
|
||||
|
||||
_NOISE = (
|
||||
"ждём LiteLLM",
|
||||
"payload:",
|
||||
"ctx ",
|
||||
"ctx-",
|
||||
"worker #",
|
||||
"волна ",
|
||||
"параллельно",
|
||||
"bytes≈",
|
||||
"upstream_stream",
|
||||
"non-stream",
|
||||
)
|
||||
|
||||
|
||||
def model_label(model: str | None) -> str:
|
||||
if not model:
|
||||
return "модель"
|
||||
m = str(model)
|
||||
return _MODEL_LABEL.get(m, m.split("/")[-1][:18])
|
||||
|
||||
|
||||
def short_path(path: str | None, *, max_parts: int = 3) -> str:
|
||||
if not path:
|
||||
return ""
|
||||
p = str(path).replace("/", "\\").rstrip("\\")
|
||||
parts = [x for x in p.split("\\") if x]
|
||||
if len(parts) <= max_parts:
|
||||
return "\\".join(parts)
|
||||
return "\\".join(parts[-max_parts:])
|
||||
|
||||
|
||||
def plan_paths(plan: dict[str, Any] | None) -> list[str]:
|
||||
out: list[str] = []
|
||||
for item in (plan or {}).get("subtasks") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
for p in item.get("paths") or []:
|
||||
if isinstance(p, str) and p.strip() and p not in out:
|
||||
out.append(p.strip())
|
||||
return out
|
||||
|
||||
|
||||
def is_noise(line: str) -> bool:
|
||||
s = (line or "").strip()
|
||||
if not s:
|
||||
return True
|
||||
low = s.lower()
|
||||
return any(n.lower() in low for n in _NOISE)
|
||||
|
||||
|
||||
def humanize_line(line: str) -> str | None:
|
||||
"""Map internal progress → short Russian status. None = drop."""
|
||||
s = (line or "").strip()
|
||||
if not s or is_noise(s):
|
||||
return None
|
||||
low = s.lower()
|
||||
|
||||
if s in ("план…", "план...", "план"):
|
||||
return "Планирую задачу"
|
||||
if "план через" in low or low.startswith("hierarchical:"):
|
||||
return "Планирую задачу"
|
||||
if "утверждён" in low or "утвержден" in low:
|
||||
m = re.search(r"(\d+)\s*подзадач", s)
|
||||
n = m.group(1) if m else ""
|
||||
return f"План утверждён{f' · {n} шаг(а)' if n else ''}"
|
||||
if "отменён" in low or "отменен" in low:
|
||||
return "План отменён"
|
||||
if "правки от пользователя" in low or "пересборк" in low:
|
||||
return "Пересобираю план по правкам"
|
||||
if "ожидает утверждения" in low or "ждём утверждения" in low or "awaiting" in low:
|
||||
return "План готов — подтверди («ok»)"
|
||||
if "agent: план готов" in low or "executor с tools" in low:
|
||||
return "Перехожу к выполнению"
|
||||
if "готово за" in low:
|
||||
return "Готово"
|
||||
if "json битый" in low or "repair" in low:
|
||||
return "Чиню формат плана"
|
||||
if low.startswith("path_index") or "path_index hit" in low:
|
||||
return None # shown via dedicated path line
|
||||
if "path_resolve" in low or "поиск файл" in low:
|
||||
return "Ищу файлы в проекте"
|
||||
if "runtime_probe" in low or "runtime:" in low:
|
||||
return "Проверяю runtime"
|
||||
if "force edit" in low:
|
||||
return "Правлю файл"
|
||||
if "kickstart" in low:
|
||||
return "Читаю файл"
|
||||
if "escalate" in low or "mid-loop" in low:
|
||||
return None
|
||||
if "408" in s or "timeout" in low:
|
||||
return "Таймаут модели — продолжаю иначе"
|
||||
if "400" in s and "bad" in low:
|
||||
return None
|
||||
# strip technical prefixes
|
||||
s = re.sub(r"^план:\s*", "", s, flags=re.I)
|
||||
s = re.sub(r"^agent:\s*", "", s, flags=re.I)
|
||||
if len(s) > 90:
|
||||
s = s[:87] + "…"
|
||||
return s
|
||||
|
||||
|
||||
def format_progress_block(lines: list[str]) -> str:
|
||||
"""Final/static progress block — numbered, no junk."""
|
||||
steps: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for raw in lines or []:
|
||||
h = humanize_line(str(raw))
|
||||
if not h or h in seen:
|
||||
continue
|
||||
seen.add(h)
|
||||
steps.append(h)
|
||||
if not steps:
|
||||
return ""
|
||||
body = "\n".join(f"{i}. {t}" for i, t in enumerate(steps, 1))
|
||||
return f"**Ход**\n{body}\n\n---\n\n"
|
||||
|
||||
|
||||
def stream_header() -> str:
|
||||
return "**Agent**\n"
|
||||
|
||||
|
||||
def stream_step(text: str) -> str:
|
||||
return f"· {text}\n"
|
||||
|
||||
|
||||
def execution_banner(
|
||||
*,
|
||||
path_mode: bool,
|
||||
plan: dict[str, Any] | None,
|
||||
model: str | None,
|
||||
) -> str:
|
||||
"""Shown once after plan approve — what happens next."""
|
||||
paths = plan_paths(plan)
|
||||
lines = ["", "---", "", "**Выполнение**"]
|
||||
facts = (plan or {}).get("runtime_facts") if isinstance(plan, dict) else None
|
||||
step = 1
|
||||
if isinstance(facts, dict) and (facts.get("host") or facts.get("hint")):
|
||||
host = facts.get("host") or "?"
|
||||
status = facts.get("http_status")
|
||||
suggest = facts.get("suggested_backend_url")
|
||||
bit = f"{host}"
|
||||
if status is not None:
|
||||
bit += f" → HTTP {status}"
|
||||
if suggest:
|
||||
bit += f" · fix `{suggest}`"
|
||||
lines.append(f"{step}. Runtime: {bit}")
|
||||
step += 1
|
||||
if paths:
|
||||
shown = ", ".join(f"`{short_path(p)}`" for p in paths[:2])
|
||||
src = ""
|
||||
tasks = (plan or {}).get("subtasks") or []
|
||||
if tasks and isinstance(tasks[0], dict):
|
||||
ps = str(tasks[0].get("path_source") or "")
|
||||
if ps == "index":
|
||||
src = " · индекс"
|
||||
elif ps == "find":
|
||||
src = " · поиск"
|
||||
lines.append(f"{step}. Файл{src}: {shown}")
|
||||
step += 1
|
||||
if path_mode:
|
||||
lines.append(f"{step}. Уточняю путь в репозитории…")
|
||||
else:
|
||||
lines.append(f"{step}. Читаю → правлю")
|
||||
elif path_mode:
|
||||
lines.append(f"{step}. Ищу нужные файлы…")
|
||||
else:
|
||||
lines.append(f"{step}. Запуск ({model_label(model)})")
|
||||
lines.append("")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def tool_status(tool_names: list[str] | None, *, path: str | None = None) -> str:
|
||||
names = [str(n).lower() for n in (tool_names or []) if n]
|
||||
if any("edit" in n or "write" in n for n in names):
|
||||
base = "Правлю"
|
||||
elif any("read" in n for n in names):
|
||||
base = "Читаю"
|
||||
elif any("find" in n or "list" in n or "grep" in n for n in names):
|
||||
base = "Ищу"
|
||||
else:
|
||||
base = "Инструмент"
|
||||
if path:
|
||||
return stream_step(f"{base} `{short_path(path)}`")
|
||||
return stream_step(base)
|
||||
+1315
-18
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,348 @@
|
||||
"""DevOps runtime probe: gather live facts before editing Traefik/compose."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import socket
|
||||
import subprocess
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from typing import Any
|
||||
|
||||
log = logging.getLogger("ai-router")
|
||||
|
||||
_HOST_RE = re.compile(
|
||||
r"\b([a-z0-9-]+(?:\.(?:ift|stage))?\.calentiq\.com)\b",
|
||||
re.I,
|
||||
)
|
||||
|
||||
# hostname pattern → docker DNS name + candidate ports
|
||||
_SERVICE_MAP: list[tuple[str, str, tuple[int, ...]]] = [
|
||||
(r"^observer\.", "observer_web", (80, 4000, 8080, 3000)),
|
||||
(r"^grafana\.", "grafana", (3000, 80)),
|
||||
(r"^prometheus\.", "prometheus", (9090, 80)),
|
||||
(r"^portainer\.", "portainer", (9000, 9443, 80)),
|
||||
(r"^ai-router\.", "ai-router", (8000, 80)),
|
||||
(r"^litellm\.", "litellm", (4000, 80)),
|
||||
(r"^logs\.|^loglynx\.", "loglynx", (6123, 80)),
|
||||
(r"^kuma\.", "uptime-kuma", (3001, 80)),
|
||||
]
|
||||
|
||||
|
||||
def extract_hosts(*texts: str) -> list[str]:
|
||||
found: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for text in texts:
|
||||
if not text:
|
||||
continue
|
||||
for m in _HOST_RE.finditer(text):
|
||||
h = m.group(1).lower()
|
||||
if h not in seen:
|
||||
seen.add(h)
|
||||
found.append(h)
|
||||
return found
|
||||
|
||||
|
||||
def is_devops_request(plan: dict[str, Any] | None, *extra: str) -> bool:
|
||||
blob = json.dumps(plan or {}, ensure_ascii=False).lower()
|
||||
for t in extra:
|
||||
blob += "\n" + str(t or "").lower()
|
||||
keys = (
|
||||
"calentiq",
|
||||
"traefik",
|
||||
"bad gateway",
|
||||
"502",
|
||||
"504",
|
||||
"eventhubdevops",
|
||||
"docker-compose",
|
||||
"swarm",
|
||||
"observer",
|
||||
"ift.",
|
||||
"stage.",
|
||||
"devops",
|
||||
"gateway",
|
||||
)
|
||||
if any(k in blob for k in keys):
|
||||
return True
|
||||
return bool(extract_hosts(blob))
|
||||
|
||||
|
||||
def service_for_host(host: str) -> tuple[str, tuple[int, ...]]:
|
||||
h = host.lower()
|
||||
for pat, name, ports in _SERVICE_MAP:
|
||||
if re.search(pat, h):
|
||||
return name, ports
|
||||
# fallback: first label as service guess
|
||||
label = h.split(".")[0]
|
||||
return label.replace("-", "_"), (80, 443, 8080, 3000, 4000)
|
||||
|
||||
|
||||
def _tcp_open(host: str, port: int, timeout: float = 2.0) -> bool:
|
||||
try:
|
||||
with socket.create_connection((host, port), timeout=timeout):
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _http_status(url: str, timeout: float = 8.0) -> int | None:
|
||||
try:
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp: # noqa: S310
|
||||
return int(getattr(resp, "status", 200) or 200)
|
||||
except urllib.error.HTTPError as exc:
|
||||
return int(exc.code)
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
|
||||
def probe_host(host: str, *, cfg: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||
"""Live probe from AiRouter container (same Docker network as services)."""
|
||||
cfg = cfg or {}
|
||||
timeout = float(cfg.get("runtime_probe_timeout_sec") or 8)
|
||||
service, ports = service_for_host(host)
|
||||
https = f"https://{host}/"
|
||||
http = f"http://{host}/"
|
||||
status = _http_status(https, timeout=timeout)
|
||||
if status is None:
|
||||
status = _http_status(http, timeout=timeout)
|
||||
|
||||
port_hits: dict[str, bool] = {}
|
||||
open_ports: list[int] = []
|
||||
for p in ports:
|
||||
ok = _tcp_open(service, p, timeout=min(2.0, timeout))
|
||||
port_hits[f"{service}:{p}"] = ok
|
||||
if ok:
|
||||
open_ports.append(p)
|
||||
|
||||
closed_preferred = not port_hits.get(f"{service}:80", True) and bool(open_ports)
|
||||
hint = ""
|
||||
suggested_url = None
|
||||
if status in (502, 503, 504) and open_ports:
|
||||
# Classic misconfig: Traefik → :80 while app on :4000
|
||||
if 80 not in open_ports and open_ports:
|
||||
suggested_url = f"http://{service}:{open_ports[0]}"
|
||||
hint = (
|
||||
f"HTTP {status}: Traefik likely targets a closed port. "
|
||||
f"{service}:80 open={port_hits.get(f'{service}:80')}; "
|
||||
f"open ports={open_ports}. Prefer {suggested_url} in dynamic_conf.yml."
|
||||
)
|
||||
else:
|
||||
hint = (
|
||||
f"HTTP {status} but {service} has open ports {open_ports or 'none'}. "
|
||||
"Check container health / upstream path."
|
||||
)
|
||||
elif status in (502, 503, 504):
|
||||
hint = (
|
||||
f"HTTP {status}: no open TCP on guessed service `{service}` "
|
||||
f"ports {list(ports)}. Service down or wrong DNS name."
|
||||
)
|
||||
elif status and status < 400:
|
||||
hint = f"HTTP {status}: host responds; issue may be app-level."
|
||||
else:
|
||||
hint = f"HTTP status={status}; ports={port_hits}"
|
||||
|
||||
facts: dict[str, Any] = {
|
||||
"host": host,
|
||||
"http_status": status,
|
||||
"service": service,
|
||||
"ports": port_hits,
|
||||
"open_ports": open_ports,
|
||||
"suggested_backend_url": suggested_url,
|
||||
"hint": hint,
|
||||
"source": "gateway_tcp",
|
||||
"closed_port_80_but_alt_open": closed_preferred,
|
||||
}
|
||||
log.info(
|
||||
"runtime_probe host=%s status=%s service=%s open=%s suggest=%s",
|
||||
host,
|
||||
status,
|
||||
service,
|
||||
open_ports,
|
||||
suggested_url,
|
||||
)
|
||||
return facts
|
||||
|
||||
|
||||
def attach_runtime_probe(
|
||||
plan: dict[str, Any] | None,
|
||||
*,
|
||||
user_text: str = "",
|
||||
cfg: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any] | None:
|
||||
"""If DevOps task — probe hosts and attach runtime_facts to plan."""
|
||||
if not isinstance(plan, dict):
|
||||
return plan
|
||||
cfg = cfg or {}
|
||||
if not cfg.get("runtime_probe_enabled", True):
|
||||
return plan
|
||||
if plan.get("runtime_facts"):
|
||||
return plan
|
||||
if not is_devops_request(plan, user_text):
|
||||
return plan
|
||||
|
||||
out = dict(plan)
|
||||
hosts = extract_hosts(
|
||||
user_text,
|
||||
json.dumps(plan, ensure_ascii=False),
|
||||
str(plan.get("user_goal") or ""),
|
||||
)
|
||||
if not hosts:
|
||||
# DevOps without host — still mark needed for terminal phase
|
||||
out["runtime_probe"] = "needed"
|
||||
out["runtime_facts"] = {
|
||||
"hint": "DevOps task without clear host — curl/docker inspect before edit.",
|
||||
"source": "none",
|
||||
}
|
||||
return out
|
||||
|
||||
facts_list = [probe_host(h, cfg=cfg) for h in hosts[:3]]
|
||||
primary = facts_list[0]
|
||||
out["runtime_facts"] = primary
|
||||
out["runtime_facts_all"] = facts_list
|
||||
out["runtime_probe"] = "gateway"
|
||||
# Strengthen paths toward Traefik dynamic conf for ift hosts
|
||||
if "ift." in primary.get("host", "") or primary.get("suggested_backend_url"):
|
||||
root = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps"
|
||||
yml = root + r"\ift\traefik\dynamic_conf.yml"
|
||||
tasks = list(out.get("subtasks") or [])
|
||||
if tasks and isinstance(tasks[0], dict):
|
||||
t0 = dict(tasks[0])
|
||||
paths = list(t0.get("paths") or [])
|
||||
if yml not in paths:
|
||||
paths = [yml] + paths
|
||||
t0["paths"] = paths[:4]
|
||||
if primary.get("suggested_backend_url"):
|
||||
t0["edit_goal"] = (
|
||||
f"Set Traefik service backend to {primary['suggested_backend_url']} "
|
||||
f"(HTTP {primary.get('http_status')}: wrong upstream port)."
|
||||
)
|
||||
tasks[0] = t0
|
||||
out["subtasks"] = tasks
|
||||
return out
|
||||
|
||||
|
||||
def runtime_facts_line(plan: dict[str, Any] | None) -> str:
|
||||
facts = (plan or {}).get("runtime_facts") if isinstance(plan, dict) else None
|
||||
if not isinstance(facts, dict) or not facts:
|
||||
return ""
|
||||
parts = [
|
||||
f"host={facts.get('host')}",
|
||||
f"http={facts.get('http_status')}",
|
||||
f"service={facts.get('service')}",
|
||||
]
|
||||
if facts.get("open_ports"):
|
||||
parts.append(f"open={facts.get('open_ports')}")
|
||||
if facts.get("suggested_backend_url"):
|
||||
parts.append(f"fix→{facts.get('suggested_backend_url')}")
|
||||
hint = str(facts.get("hint") or "")[:180]
|
||||
return "RUNTIME: " + "; ".join(str(p) for p in parts) + (f" | {hint}" if hint else "")
|
||||
|
||||
|
||||
def devops_blocks_blind_force_edit(plan: dict[str, Any] | None) -> bool:
|
||||
"""Do not blind-edit DevOps YAML until runtime facts exist."""
|
||||
if not isinstance(plan, dict):
|
||||
return False
|
||||
if not is_devops_request(plan):
|
||||
return False
|
||||
facts = plan.get("runtime_facts")
|
||||
if not facts:
|
||||
return True
|
||||
# Facts present — allow targeted edit (including force with goal from facts)
|
||||
return False
|
||||
|
||||
|
||||
def suggested_traefik_edit(
|
||||
plan: dict[str, Any] | None,
|
||||
) -> dict[str, str] | None:
|
||||
"""If probe found closed:80 + open:alt — return old/new URL snippets."""
|
||||
if not isinstance(plan, dict):
|
||||
return None
|
||||
facts = plan.get("runtime_facts")
|
||||
if not isinstance(facts, dict):
|
||||
return None
|
||||
url = facts.get("suggested_backend_url")
|
||||
service = facts.get("service")
|
||||
if not url or not service:
|
||||
return None
|
||||
# Common misconfig in our dynamic_conf
|
||||
old = f'url: "http://{service}:80"'
|
||||
new = f'url: "{url}"'
|
||||
if old == new:
|
||||
return None
|
||||
return {
|
||||
"path": r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml",
|
||||
"old_text": old,
|
||||
"new_text": new,
|
||||
"reason": str(facts.get("hint") or "port mismatch"),
|
||||
}
|
||||
|
||||
|
||||
def force_runtime_fix_edit(
|
||||
*,
|
||||
plan: dict[str, Any] | None,
|
||||
tools: list[Any],
|
||||
model: str,
|
||||
) -> dict[str, Any] | None:
|
||||
"""Synthetic edit_file for clear Traefik port mismatch — skip blind marker edits."""
|
||||
fix = suggested_traefik_edit(plan)
|
||||
if not fix:
|
||||
return None
|
||||
# late import to avoid cycles
|
||||
from agent_hier import _pick_edit_tool, _synthetic_tool_completion
|
||||
|
||||
edit_name = _pick_edit_tool(tools)
|
||||
if not edit_name:
|
||||
return None
|
||||
log.info(
|
||||
"runtime_probe → targeted edit %s => %s",
|
||||
fix["old_text"],
|
||||
fix["new_text"],
|
||||
)
|
||||
return _synthetic_tool_completion(
|
||||
model=model,
|
||||
tool_name=edit_name,
|
||||
arguments={
|
||||
"path": fix["path"],
|
||||
"edits": [{"old_text": fix["old_text"], "new_text": fix["new_text"]}],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def ssh_probe_fallback(host: str, cfg: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Optional: ssh to host and docker inspect (if gateway has keys)."""
|
||||
ssh_host = str(cfg.get("runtime_probe_ssh_host") or "").strip()
|
||||
if not ssh_host:
|
||||
return None
|
||||
service, _ports = service_for_host(host)
|
||||
script = (
|
||||
f"echo HTTP=$(curl -sk -o /dev/null -w '%{{http_code}}' --max-time 5 https://{host}/ || echo err); "
|
||||
f"CID=$(docker ps -q --filter name={service} | head -1); "
|
||||
f"echo CID=$CID; "
|
||||
f"if [ -n \"$CID\" ]; then docker inspect \"$CID\" --format 'PORTS={{{{json .NetworkSettings.Ports}}}}'; fi"
|
||||
)
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[
|
||||
"ssh",
|
||||
"-o",
|
||||
"BatchMode=yes",
|
||||
"-o",
|
||||
"ConnectTimeout=5",
|
||||
ssh_host,
|
||||
"bash",
|
||||
"-lc",
|
||||
script,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=20,
|
||||
check=False,
|
||||
)
|
||||
out = (proc.stdout or "") + "\n" + (proc.stderr or "")
|
||||
return {"host": host, "ssh_raw": out[:1500], "source": "ssh", "service": service}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
log.warning("runtime ssh probe failed: %s", exc)
|
||||
return None
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
|
||||
echo "== Tokens/users now =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT count(*) AS tokens FROM \"LiteLLM_VerificationToken\";
|
||||
SELECT count(*) AS users FROM \"LiteLLM_UserTable\";
|
||||
SELECT \"user_id\", \"user_role\" FROM \"LiteLLM_UserTable\" LIMIT 5;
|
||||
"
|
||||
|
||||
echo "== Master key models probe =="
|
||||
code=$(curl -sS -o /tmp/m.json -w "%{http_code}" --max-time 20 \
|
||||
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
|
||||
https://litellm.ift.calentiq.com/v1/models)
|
||||
echo "models HTTP $code"
|
||||
head -c 120 /tmp/m.json; echo
|
||||
|
||||
echo "== Flush redis key cache (best effort) =="
|
||||
RID=$(docker ps -q -f name=ai-router_redis | head -1)
|
||||
if [[ -n "$RID" ]]; then
|
||||
docker exec "$RID" redis-cli KEYS '*token*' 2>/dev/null | head -20 || true
|
||||
docker exec "$RID" redis-cli KEYS '*litellm*' 2>/dev/null | head -20 || true
|
||||
# do not FLUSHALL — may kill router session cache; only clear litellm-ish if safe
|
||||
fi
|
||||
echo DONE
|
||||
REMOTE
|
||||
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
echo "== curl observer =="
|
||||
curl -sS -o /tmp/obs.body -w "http=%{http_code} time=%{time_total}\n" \
|
||||
--max-time 15 -k https://observer.ift.calentiq.com/ || true
|
||||
head -c 400 /tmp/obs.body; echo
|
||||
echo "== swarm observer_web =="
|
||||
docker service ls 2>/dev/null | grep -i observer || true
|
||||
docker service ps $(docker service ls -q --filter name=observer 2>/dev/null | head -1) --no-trunc 2>/dev/null | head -8 || \
|
||||
docker ps -a --filter name=observer --format '{{.Names}} {{.Status}}' | head -10
|
||||
echo "== network aliases =="
|
||||
docker network inspect eventhub-ift-net --format '{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}' 2>/dev/null | grep -i observer || true
|
||||
REMOTE
|
||||
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env bash
|
||||
# Explain + delete spurious LiteLLM failure logs from restart disconnects
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
USER="${POSTGRES_USER:-litellm}"
|
||||
DB="${POSTGRES_DB:-litellm}"
|
||||
|
||||
echo "== Error message sample =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -t -A -c "
|
||||
SELECT left(metadata->'error_information'->>'error_message', 200)
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE status='failure'
|
||||
LIMIT 3;
|
||||
"
|
||||
|
||||
echo "== Delete empty-model restart failures =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "
|
||||
DELETE FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE status = 'failure'
|
||||
AND (model IS NULL OR model = '')
|
||||
AND (call_type IS NULL OR call_type = '');
|
||||
"
|
||||
|
||||
echo "== Remaining =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "
|
||||
SELECT status, count(*) FROM \"LiteLLM_SpendLogs\" GROUP BY 1;
|
||||
"
|
||||
echo CLEANED_SPURIOUS_FAILS_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env bash
|
||||
# Clear LiteLLM Postgres spend/logs on IFT (keep schema + keys if present)
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
if [[ -z "$PG" ]]; then
|
||||
echo "ERROR: postgres container not running" >&2
|
||||
exit 1
|
||||
fi
|
||||
USER="${POSTGRES_USER:-litellm}"
|
||||
DB="${POSTGRES_DB:-litellm}"
|
||||
echo "== Tables before =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "\dt"
|
||||
|
||||
# LiteLLM Prisma tables — truncate data, keep schema
|
||||
# Prefer spend/usage; also clear invite/audit-ish if present
|
||||
SQL=$(cat <<'EOS'
|
||||
DO $$
|
||||
DECLARE
|
||||
r RECORD;
|
||||
n bigint;
|
||||
BEGIN
|
||||
FOR r IN
|
||||
SELECT tablename
|
||||
FROM pg_tables
|
||||
WHERE schemaname = 'public'
|
||||
AND tablename NOT IN ('_prisma_migrations')
|
||||
LOOP
|
||||
EXECUTE format('SELECT count(*) FROM %I', r.tablename) INTO n;
|
||||
RAISE NOTICE 'truncate % (% rows)', r.tablename, n;
|
||||
EXECUTE format('TRUNCATE TABLE %I RESTART IDENTITY CASCADE', r.tablename);
|
||||
END LOOP;
|
||||
END $$;
|
||||
EOS
|
||||
)
|
||||
|
||||
echo "== Truncate all public tables (except _prisma_migrations) =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" -i "$PG" \
|
||||
psql -U "$USER" -d "$DB" -v ON_ERROR_STOP=1 <<< "$SQL"
|
||||
|
||||
echo "== Counts after =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "
|
||||
SELECT relname AS table, n_live_tup AS approx_rows
|
||||
FROM pg_stat_user_tables
|
||||
ORDER BY relname;
|
||||
"
|
||||
|
||||
echo "== Restart litellm (refresh Admin UI caches) =="
|
||||
docker service update --force ai-router_litellm >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness && echo
|
||||
echo LITELLM_DB_CLEARED_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,103 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deploy Cursor-parity AiRouter (Max plan / Coder routine / DeepSeek hard) to IFT
|
||||
set -euo pipefail
|
||||
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
|
||||
cd "$AIR"
|
||||
|
||||
echo "== Local unit + compile =="
|
||||
python3 -m py_compile \
|
||||
router/agent_hier.py router/router.py router/hierarchical.py router/orchestrator.py
|
||||
python3 -m unittest discover -s test/unit -q
|
||||
PRIMARY_PROVIDER=hybrid python3 scripts/gen-litellm-config.py
|
||||
|
||||
echo "== Sync to eventhub-ift =="
|
||||
scp \
|
||||
router/agent_hier.py \
|
||||
router/router.py \
|
||||
router/hierarchical.py \
|
||||
router/orchestrator.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp \
|
||||
config/orchestration.yaml \
|
||||
config/providers.yaml \
|
||||
config/routing_rules.yaml \
|
||||
eventhub-ift:/opt/ai-router-stack/config/
|
||||
scp litellm_config.yaml eventhub-ift:/opt/ai-router-stack/litellm_config.yaml
|
||||
scp README.md AGENTS.md eventhub-ift:/opt/ai-router-stack/ 2>/dev/null || true
|
||||
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
export PRIMARY_PROVIDER="${PRIMARY_PROVIDER:-hybrid}"
|
||||
|
||||
sed -i 's/\r$//' \
|
||||
router/agent_hier.py router/router.py router/hierarchical.py router/orchestrator.py \
|
||||
config/orchestration.yaml config/providers.yaml config/routing_rules.yaml \
|
||||
litellm_config.yaml || true
|
||||
|
||||
echo "== Build gateway image =="
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
LITC="litellm_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
docker config create "$LITC" ./litellm_config.yaml
|
||||
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
OLD_LIT=$(docker service inspect ai-router_litellm --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep litellm_config | grep -v entrypoint | tail -1 || true)
|
||||
|
||||
echo "== Update ai-router_router =="
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
if [ -n "${OLD_ORCH:-}" ]; then
|
||||
RARGS+=(--config-rm "$OLD_ORCH")
|
||||
fi
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
|
||||
echo "== Update ai-router_litellm =="
|
||||
LARGS=(--force)
|
||||
LARGS+=(--config-add "source=${LITC},target=/app/config.yaml")
|
||||
if [ -n "${OLD_LIT:-}" ]; then
|
||||
LARGS+=(--config-rm "$OLD_LIT")
|
||||
fi
|
||||
docker service update "${LARGS[@]}" ai-router_litellm >/dev/null
|
||||
|
||||
echo "== Wait health =="
|
||||
ok=0
|
||||
for i in $(seq 1 60); do
|
||||
if curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null \
|
||||
&& curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null; then
|
||||
ok=1
|
||||
break
|
||||
fi
|
||||
sleep 3
|
||||
done
|
||||
if [ "$ok" != 1 ]; then
|
||||
echo "ERROR: health not ready" >&2
|
||||
docker service ps ai-router_router --no-trunc | head -8
|
||||
docker service ps ai-router_litellm --no-trunc | head -8
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RID=$(docker ps -q -f name=ai-router_router | head -1)
|
||||
LID=$(docker ps -q -f name=ai-router_litellm | head -1)
|
||||
echo "== Verify orchestration =="
|
||||
docker exec "$RID" grep -E 'planner_model:|verifier_model:|agent_executor_model:|executor_synthetic_mode:|worker_map:|hard:' /app/config/orchestration.yaml
|
||||
echo "== Verify litellm novita-planner =="
|
||||
docker exec "$LID" sh -c 'grep -A6 "model_name: novita-planner" /app/config.yaml | head -8'
|
||||
|
||||
echo "== Smoke classify + chat =="
|
||||
curl -sf --max-time 30 -H "Authorization: Bearer ${ROUTER_API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"text":"поправь timeout в router.py"}' \
|
||||
https://ai-router.ift.calentiq.com/classify | head -c 400
|
||||
echo
|
||||
curl -sf --max-time 90 -H "Authorization: Bearer ${ROUTER_API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"smart-router","messages":[{"role":"user","content":"ping"}],"max_tokens":8}' \
|
||||
https://ai-router.ift.calentiq.com/v1/chat/completions | head -c 500
|
||||
echo
|
||||
echo CURSOR_PARITY_DEPLOY_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
PYTHONPATH=router python3 -m py_compile router/agent_hier.py router/router.py
|
||||
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
|
||||
scp router/agent_hier.py router/router.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/agent_hier.py router/router.py config/orchestration.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
|
||||
grep -E 'executor_tool_loop_timeout_sec|executor_escalate_after_read|executor_rewrite_reread|executor_newest_tool' \
|
||||
/app/config/orchestration.yaml
|
||||
echo FAST_AFTER_READ_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
|
||||
cd "$AIR"
|
||||
python3 -m py_compile router/agent_hier.py router/agent_stream.py router/router.py
|
||||
python3 -m unittest discover -s test/unit -q
|
||||
scp router/agent_hier.py router/agent_stream.py router/router.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/agent_hier.py router/agent_stream.py router/router.py config/orchestration.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
|
||||
grep -E 'executor_synthetic_mode|executor_no_tools_escalate' /app/config/orchestration.yaml
|
||||
echo KICKSTART_NO_TOOLS_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
PYTHONPATH=router python3 -m py_compile \
|
||||
router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py
|
||||
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
|
||||
scp router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
|
||||
grep -E 'path_resolve_' /app/config/orchestration.yaml
|
||||
test -f /opt/ai-router-stack/router/path_resolve.py
|
||||
echo PATH_RESOLVE_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
PYTHONPATH=router python3 -m py_compile \
|
||||
router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py router/progress_ui.py
|
||||
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
|
||||
scp router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
|
||||
grep -E 'executor_force_edit_after_read|path_resolve_enabled' /app/config/orchestration.yaml
|
||||
test -f /opt/ai-router-stack/router/progress_ui.py
|
||||
echo PROGRESS_UI_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bash
|
||||
# Hotfix: quiet progress + executor timeout/escalate
|
||||
set -euo pipefail
|
||||
AIR=/mnt/c/Users/alexc/IdeaProjects/eventHub/EventHubAiRouter
|
||||
cd "$AIR"
|
||||
python3 -m py_compile router/hierarchical.py router/agent_stream.py router/router.py router/agent_hier.py
|
||||
python3 -m unittest discover -s test/unit -q
|
||||
PRIMARY_PROVIDER=hybrid LITELLM_MODEL_TIMEOUT=180 python3 scripts/gen-litellm-config.py
|
||||
|
||||
scp router/hierarchical.py router/agent_stream.py router/router.py router/agent_hier.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
scp litellm_config.yaml eventhub-ift:/opt/ai-router-stack/litellm_config.yaml
|
||||
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/*.py config/orchestration.yaml litellm_config.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
LITC="litellm_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
docker config create "$LITC" ./litellm_config.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
OLD_LIT=$(docker service inspect ai-router_litellm --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep litellm_config | grep -v entrypoint | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
LARGS=(--force --config-add "source=${LITC},target=/app/config.yaml")
|
||||
[[ -n "${OLD_LIT:-}" ]] && LARGS+=(--config-rm "$OLD_LIT")
|
||||
docker service update "${LARGS[@]}" ai-router_litellm >/dev/null
|
||||
for i in $(seq 1 50); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && \
|
||||
curl -sf --max-time 5 https://litellm.ift.calentiq.com/health/liveliness >/dev/null && break
|
||||
sleep 3
|
||||
done
|
||||
RID=$(docker ps -q -f name=ai-router_router | head -1)
|
||||
LID=$(docker ps -q -f name=ai-router_litellm | head -1)
|
||||
docker exec "$RID" grep -E 'progress_verbose|executor_timeout|executor_midloop_escalate|show_context_fill' /app/config/orchestration.yaml
|
||||
docker exec "$LID" sh -c 'grep -A5 "model_name: a-medium-code" /app/config.yaml | head -8'
|
||||
echo QUIET_PROGRESS_TIMEOUT_OK
|
||||
REMOTE
|
||||
@@ -0,0 +1,37 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
PYTHONPATH=router python3 -m py_compile \
|
||||
router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
|
||||
router/runtime_probe.py
|
||||
PYTHONPATH=router python3 -m unittest discover -s test/unit -q
|
||||
scp router/agent_hier.py router/router.py router/agent_stream.py \
|
||||
router/path_resolve.py router/hierarchical.py router/progress_ui.py \
|
||||
router/runtime_probe.py \
|
||||
eventhub-ift:/opt/ai-router-stack/router/
|
||||
scp config/orchestration.yaml eventhub-ift:/opt/ai-router-stack/config/
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
sed -i 's/\r$//' router/*.py config/orchestration.yaml || true
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
TS=$(date +%Y%m%d%H%M%S)
|
||||
ORCH="orchestration_config_${TS}"
|
||||
docker config create "$ORCH" ./config/orchestration.yaml
|
||||
OLD_ORCH=$(docker service inspect ai-router_router --format '{{range .Spec.TaskTemplate.ContainerSpec.Configs}}{{println .ConfigName}}{{end}}' | grep orchestration | tail -1 || true)
|
||||
RARGS=(--image "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" --force)
|
||||
RARGS+=(--config-add "source=${ORCH},target=/app/config/orchestration.yaml")
|
||||
[[ -n "${OLD_ORCH:-}" ]] && RARGS+=(--config-rm "$OLD_ORCH")
|
||||
docker service update "${RARGS[@]}" ai-router_router >/dev/null
|
||||
for i in $(seq 1 40); do
|
||||
curl -sf --max-time 5 https://ai-router.ift.calentiq.com/health >/dev/null && break
|
||||
sleep 2
|
||||
done
|
||||
docker exec "$(docker ps -q -f name=ai-router_router | head -1)" \
|
||||
grep -E 'runtime_probe_enabled|path_resolve_enabled' /app/config/orchestration.yaml
|
||||
test -f /opt/ai-router-stack/router/runtime_probe.py
|
||||
echo RUNTIME_PROBE_OK
|
||||
REMOTE
|
||||
+36
-3
@@ -23,21 +23,48 @@ ensure_secret() {
|
||||
if docker secret inspect "$name" >/dev/null 2>&1; then
|
||||
echo "secret exists: $name"
|
||||
else
|
||||
if [[ -z "$value" ]]; then
|
||||
value="_"
|
||||
fi
|
||||
echo -n "$value" | docker secret create "$name" -
|
||||
echo "created secret: $name"
|
||||
fi
|
||||
}
|
||||
|
||||
echo "== Ensure Swarm secrets =="
|
||||
ensure_secret novita_api_key "${NOVITA_API_KEY:?NOVITA_API_KEY required}"
|
||||
PRIMARY="${PRIMARY_PROVIDER:-hybrid}"
|
||||
if [[ "$PRIMARY" == "hybrid" || "$PRIMARY" == "novita" ]]; then
|
||||
ensure_secret novita_api_key "${NOVITA_API_KEY:?NOVITA_API_KEY required for PRIMARY_PROVIDER=${PRIMARY}}"
|
||||
else
|
||||
ensure_secret novita_api_key "${NOVITA_API_KEY:-}"
|
||||
fi
|
||||
ensure_secret anthropic_api_key "${ANTHROPIC_API_KEY:-}"
|
||||
ensure_secret litellm_master_key "${LITELLM_MASTER_KEY:?LITELLM_MASTER_KEY required}"
|
||||
ensure_secret litellm_salt_key "${LITELLM_SALT_KEY:?LITELLM_SALT_KEY required}"
|
||||
ensure_secret router_api_key "${ROUTER_API_KEY:?ROUTER_API_KEY required}"
|
||||
ensure_secret postgres_password "${POSTGRES_PASSWORD:?POSTGRES_PASSWORD required}"
|
||||
ensure_secret groq_api_key "${GROQ_API_KEY:-}"
|
||||
ensure_secret xai_api_key "${XAI_API_KEY:-}"
|
||||
ensure_secret gemini_api_key "${GEMINI_API_KEY:-}"
|
||||
ensure_secret together_api_key "${TOGETHER_API_KEY:-}"
|
||||
ensure_secret openrouter_api_key "${OPENROUTER_API_KEY:-}"
|
||||
ensure_secret gigachat_credentials "${GIGACHAT_CREDENTIALS:-}"
|
||||
|
||||
vless_conf_is_stub() {
|
||||
local f="${1:-vless/vless.conf}"
|
||||
[[ ! -f "$f" ]] && return 0
|
||||
grep -qE "0\.0\.0\.0|example\.com|UUID@host|^#" "$f" && return 0
|
||||
grep -qE "^vless://" "$f" || return 0
|
||||
return 1
|
||||
}
|
||||
|
||||
if [[ "${VPN_ENABLED:-false}" == "true" ]] && [[ -n "${VLESS_SUB_URL:-}" ]]; then
|
||||
if vless_conf_is_stub "vless/vless.conf"; then
|
||||
echo "== Fetch vless.conf from subscription =="
|
||||
bash scripts/fetch-vless-subscription.sh vless/vless.conf
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f vless/vless.conf ]]; then
|
||||
echo "WARN: vless/vless.conf missing — stub for secret (VPN off until configured)"
|
||||
cp vless/vless.conf.example vless/vless.conf 2>/dev/null || echo "# stub" > vless/vless.conf
|
||||
@@ -46,11 +73,17 @@ ensure_secret vless_conf "$(cat vless/vless.conf)"
|
||||
|
||||
echo "== Build images =="
|
||||
docker build -f router/Dockerfile -t "${ROUTER_IMAGE:-git.sabilin.com/eventhub/ai-router-gateway:ift}" .
|
||||
if [[ "${VPN_ENABLED:-false}" == "true" ]]; then
|
||||
docker build -t ai-router/vless-proxy:local ./vless
|
||||
docker build -t ai-router/vpn-watchdog:local ./watchdog
|
||||
else
|
||||
echo "VPN off — stub images (replicas=0)"
|
||||
docker build -f vless/Dockerfile.stub -t ai-router/vless-proxy:local ./vless
|
||||
docker build -t ai-router/vpn-watchdog:local ./watchdog
|
||||
fi
|
||||
|
||||
echo "== Sync routing config (optional regen) =="
|
||||
bash scripts/sync-routing-config.sh || true
|
||||
echo "== Sync routing config (PRIMARY_PROVIDER=${PRIMARY_PROVIDER:-hybrid}) =="
|
||||
python3 scripts/gen-litellm-config.py
|
||||
|
||||
echo "== Deploy stack: ${STACK_NAME} =="
|
||||
docker stack deploy -c docker-stack.yml --with-registry-auth "${STACK_NAME}"
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== litellm 400/408 detail =="
|
||||
docker service logs --since 40m --raw ai-router_litellm 2>&1 \
|
||||
| grep -iE 'BadRequest|invalid request|400|deepseek|tool|error|trace_id' \
|
||||
| tail -80
|
||||
echo
|
||||
echo "== router 400 bodies =="
|
||||
docker service logs --since 40m --raw ai-router_router 2>&1 \
|
||||
| grep -iE '400|BadRequest|invalid|executor failed|mid-loop' \
|
||||
| tail -40
|
||||
REMOTE
|
||||
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend 15m =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
|
||||
coalesce(\"request_duration_ms\",0) AS ms,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 100) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE \"startTime\" >= now() - interval '20 minutes'
|
||||
ORDER BY \"startTime\" DESC LIMIT 25;
|
||||
"
|
||||
echo "== Router =="
|
||||
docker service logs --since 20m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'path_index|path_resolve|hierarchical_agent phase|executor result|408|escalate|mid-loop|rewrite|kickstart|loadtest|dynamic_conf' \
|
||||
| tail -60
|
||||
REMOTE
|
||||
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
docker service logs --since 20m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'docker-compose|kickstart|tool_calls|executor result|rewrite|hierarchical_agent phase|not found' \
|
||||
| tail -50
|
||||
REMOTE
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend 20m =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT to_char(\"startTime\", 'HH24:MI:SS') AS t, status,
|
||||
CASE WHEN model LIKE '%qwen3.8-max%' THEN 'Max'
|
||||
WHEN model LIKE '%qwen3-coder%' THEN 'Coder'
|
||||
WHEN model LIKE '%deepseek%' THEN 'DeepSeek'
|
||||
ELSE left(model,28) END AS role,
|
||||
total_tokens AS tok, completion_tokens AS out,
|
||||
coalesce(\"request_duration_ms\",0) AS ms
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE \"startTime\" >= now() - interval '20 minutes'
|
||||
ORDER BY \"startTime\" ASC;
|
||||
"
|
||||
echo
|
||||
echo "== Router (edit/force/loop) =="
|
||||
docker service logs --since 20m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'force edit|edit_file|rewrite|path_index|hierarchical_agent phase|executor result|synth|sticky|Стоп|fallback|tool_calls|calentiq-ift' \
|
||||
| grep -vE 'metrics|health|Waiting|Uvicorn|Started|Finished|Shutting' \
|
||||
| tail -80
|
||||
REMOTE
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
|
||||
echo "== Spend last 30m =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, call_type, total_tokens,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 180) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC
|
||||
LIMIT 25;
|
||||
"
|
||||
|
||||
echo "== Router logs (executor/plan) =="
|
||||
docker service logs --since 25m ai-router_router 2>&1 \
|
||||
| grep -iE 'hierarchical|executor|novita-planner|a-medium-code|timeout|408|error|fail|tool' \
|
||||
| tail -50
|
||||
|
||||
echo "== LiteLLM logs errors =="
|
||||
docker service logs --since 25m ai-router_litellm 2>&1 \
|
||||
| grep -iE 'ERROR|Exception|timeout|408|502|qwen3-coder|novita-planner|a-medium-code|Give Feedback' \
|
||||
| tail -40
|
||||
REMOTE
|
||||
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
docker service logs --since 25m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'traefik\.yml|rewrite|bogus|kickstart|executor result|hierarchical_agent phase|paths:|Plan approved|tool_calls' \
|
||||
| tail -60
|
||||
REMOTE
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
echo "== live router logs (last 5m, full) =="
|
||||
docker service logs --since 8m --raw ai-router_router 2>&1 | tail -80
|
||||
echo
|
||||
echo "== litellm last =="
|
||||
docker service logs --since 8m --raw ai-router_litellm 2>&1 | grep -iE 'qwen|deepseek|timeout|408|error|POST' | tail -40
|
||||
echo
|
||||
echo "== in-flight? =="
|
||||
docker service ps ai-router_router --no-trunc | head -5
|
||||
REMOTE
|
||||
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend after 19:06 =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
|
||||
coalesce(\"request_duration_ms\",0) AS ms,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 120) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE \"startTime\" >= '2026-08-12 19:06:00'
|
||||
ORDER BY \"startTime\" DESC;
|
||||
"
|
||||
echo "== router since 19:06 full agent lines =="
|
||||
docker service logs --since 15m --raw ai-router_router 2>&1 \
|
||||
| grep -iE '19:0[6-9]|19:1|executor|tool_loop|408|HTTP Request|kickstart|escalate|fallback|synth|no tool|POST /v1' \
|
||||
| tail -60
|
||||
REMOTE
|
||||
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
|
||||
echo "== Services =="
|
||||
docker stack services ai-router --format '{{.Name}} {{.Replicas}} {{.Image}}' | grep -E 'litellm|postgres|router' || true
|
||||
|
||||
echo "== Health =="
|
||||
curl -sS -o /tmp/llh.txt -w "liveliness:%{http_code}\n" --max-time 10 https://litellm.ift.calentiq.com/health/liveliness || true
|
||||
curl -sS -o /tmp/llr.txt -w "readiness:%{http_code}\n" --max-time 10 https://litellm.ift.calentiq.com/health/readiness || true
|
||||
head -c 300 /tmp/llh.txt; echo; head -c 500 /tmp/llr.txt; echo
|
||||
|
||||
echo "== Recent litellm logs (errors) =="
|
||||
docker service logs --tail 120 ai-router_litellm 2>&1 \
|
||||
| grep -iE 'error|exception|fail|traceback|500|401|prisma|database|budget|key|warning' \
|
||||
| tail -60 || true
|
||||
|
||||
echo "== Last 40 lines raw =="
|
||||
docker service logs --tail 40 ai-router_litellm 2>&1 | tail -40
|
||||
|
||||
echo "== Postgres quick =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c \
|
||||
"SELECT count(*) AS spend FROM \"LiteLLM_SpendLogs\"; SELECT count(*) AS users FROM \"LiteLLM_UserTable\"; SELECT count(*) AS tokens FROM \"LiteLLM_VerificationToken\";"
|
||||
|
||||
echo "== UI/home probe =="
|
||||
curl -sS -o /tmp/ui.txt -w "ui:%{http_code}\n" --max-time 15 https://litellm.ift.calentiq.com/ui/ || true
|
||||
curl -sS -o /tmp/models.txt -w "models:%{http_code}\n" --max-time 20 \
|
||||
-H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
|
||||
https://litellm.ift.calentiq.com/v1/models || true
|
||||
head -c 200 /tmp/models.txt; echo
|
||||
REMOTE
|
||||
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
|
||||
echo "== SpendLogs status breakdown =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT status, count(*) FROM \"LiteLLM_SpendLogs\" GROUP BY 1 ORDER BY 2 DESC;
|
||||
"
|
||||
|
||||
echo "== Recent spend rows =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, \"api_key\",
|
||||
left(coalesce(\"error_information\"::text, ''), 120) AS err,
|
||||
left(coalesce(request_tags::text, ''), 60) AS tags
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC
|
||||
LIMIT 20;
|
||||
"
|
||||
|
||||
echo "== ErrorLogs =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c \
|
||||
"SELECT count(*) FROM \"LiteLLM_ErrorLogs\";"
|
||||
|
||||
echo "== Body-read errors around restart =="
|
||||
docker service logs --since 15m ai-router_litellm 2>&1 \
|
||||
| grep -iE 'Unexpected error reading request body|ClientDisconnect|ConnectionReset|502|504|failed' \
|
||||
| tail -30 || true
|
||||
|
||||
echo "== Uptime-kuma / probes hitting litellm =="
|
||||
docker service logs --tail 30 ai-router_uptime-kuma 2>&1 | tail -20 || true
|
||||
REMOTE
|
||||
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
USER="${POSTGRES_USER:-litellm}"
|
||||
DB="${POSTGRES_DB:-litellm}"
|
||||
|
||||
echo "== SpendLogs columns =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "\d+ \"LiteLLM_SpendLogs\"" | head -80
|
||||
|
||||
echo "== Recent failures =="
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "$USER" -d "$DB" -c "
|
||||
SELECT \"startTime\", status, model,
|
||||
left(coalesce(messages::text,''), 80) AS messages,
|
||||
left(coalesce(metadata::text,''), 200) AS metadata,
|
||||
left(coalesce(\"proxyServerRequest\"::text,''), 120) AS req,
|
||||
left(coalesce(response::text,''), 200) AS response
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC
|
||||
LIMIT 15;
|
||||
"
|
||||
REMOTE
|
||||
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, call_type,
|
||||
left(coalesce(metadata::text,''), 400) AS metadata,
|
||||
left(coalesce(response::text,''), 300) AS response
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC
|
||||
LIMIT 15;
|
||||
"
|
||||
REMOTE
|
||||
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend last 15m =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, call_type, total_tokens, completion_tokens,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 120) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC LIMIT 15;
|
||||
"
|
||||
echo "== Router =="
|
||||
docker service logs --since 15m ai-router_router 2>&1 \
|
||||
| grep -iE 'executor|tool_calls|finish|no_tool|408|escalate|hierarchical_agent|a-medium-code|b-complex' \
|
||||
| tail -40
|
||||
REMOTE
|
||||
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
echo "== curl inside from traefik net =="
|
||||
# find traefik containe
|
||||
TR=$(docker ps --filter name=traefik -q | head -1)
|
||||
OW=$(docker ps --filter name=observer_web -q | head -1)
|
||||
echo "traefik=$TR observer=$OW"
|
||||
if [[ -n "$OW" ]]; then
|
||||
echo "-- observer logs --"
|
||||
docker logs --tail 30 "$OW" 2>&1 | tail -30
|
||||
echo "-- wget localhost from observer --"
|
||||
docker exec "$OW" sh -c 'wget -qO- --timeout=3 http://127.0.0.1/ 2>&1 | head -c 200; echo; wget -qO- --timeout=3 http://127.0.0.1:80/ 2>&1 | head -c 200; echo' || true
|
||||
echo "-- inspect ports --"
|
||||
docker inspect "$OW" --format 'IP={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} Ports={{json .NetworkSettings.Ports}}'
|
||||
fi
|
||||
if [[ -n "$TR" && -n "$OW" ]]; then
|
||||
echo "-- traefik -> observer_web:80 --"
|
||||
docker exec "$TR" wget -qO- --timeout=5 http://observer_web:80/ 2>&1 | head -c 300 || \
|
||||
docker exec "$TR" wget -qO- --timeout=5 http://eventhub-ift-core_observer_web:80/ 2>&1 | head -c 300 || true
|
||||
echo
|
||||
# DNS from traefik
|
||||
docker exec "$TR" getent hosts observer_web 2>&1 || docker exec "$TR" nslookup observer_web 2>&1 | head -10 || true
|
||||
fi
|
||||
echo "== dynamic_conf observer snippet =="
|
||||
grep -n -A6 'observer' /opt/ai-router-stack/../EventHubDevOps/ift/traefik/dynamic_conf.yml 2>/dev/null | head -5 || \
|
||||
grep -n -A5 'observer:' /home/*/IdeaProjects/eventHub/EventHubDevOps/ift/traefik/dynamic_conf.yml 2>/dev/null | head || true
|
||||
# try find compose on host
|
||||
find /opt -name 'dynamic_conf.yml' 2>/dev/null | head -5
|
||||
REMOTE
|
||||
@@ -0,0 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend last 20m =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT \"startTime\", status, model, total_tokens, completion_tokens,
|
||||
coalesce(\"request_duration_ms\",0) AS ms,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 100) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
ORDER BY \"startTime\" DESC LIMIT 20;
|
||||
"
|
||||
echo "== Router recent =="
|
||||
docker service logs --since 20m ai-router_router 2>&1 \
|
||||
| grep -iE 'executor|tool_loop|kickstart|escalate|408|no tool|hierarchical_agent|phase=|waiting|timeout' \
|
||||
| tail -50
|
||||
REMOTE
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
echo "== Spend last 25m =="
|
||||
PG=$(docker ps -q -f name=ai-router_postgres | head -1)
|
||||
docker exec -e PGPASSWORD="$POSTGRES_PASSWORD" "$PG" \
|
||||
psql -U "${POSTGRES_USER:-litellm}" -d "${POSTGRES_DB:-litellm}" -c "
|
||||
SELECT to_char(\"startTime\", 'HH24:MI:SS') AS t, status,
|
||||
CASE
|
||||
WHEN model LIKE '%qwen3.8-max%' THEN 'planner-Max'
|
||||
WHEN model LIKE '%qwen3-coder%' THEN 'coder-30b'
|
||||
WHEN model LIKE '%deepseek%' THEN 'DeepSeek'
|
||||
WHEN model LIKE '%llama%' THEN 'Llama'
|
||||
ELSE left(model, 40)
|
||||
END AS role,
|
||||
total_tokens AS tok, completion_tokens AS out,
|
||||
coalesce(\"request_duration_ms\",0) AS ms,
|
||||
left(coalesce(metadata->'error_information'->>'error_message',''), 80) AS er
|
||||
FROM \"LiteLLM_SpendLogs\"
|
||||
WHERE \"startTime\" >= now() - interval '25 minutes'
|
||||
ORDER BY \"startTime\" ASC;
|
||||
"
|
||||
echo
|
||||
echo "== Router agent timeline =="
|
||||
docker service logs --since 25m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'path_index|path_resolve|hierarchical_agent phase|executor forward|executor result|escalate|408|400|rewrite|kickstart|no tool|mid-loop|synth|Выполн|plan' \
|
||||
| grep -vE 'metrics|health|Waiting for|Application|Uvicorn|Finished server|Started server|Shutting' \
|
||||
| tail -80
|
||||
REMOTE
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env bash
|
||||
# Fetch vless.conf from Happ/mireon subscription (JSON or plain vless:// lines).
|
||||
# Env: VLESS_SUB_URL, VLESS_UA, VLESS_HWID (Happ → Settings / app id)
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
if [[ -f "${ROOT}/.env" ]]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
source "${ROOT}/.env"
|
||||
set +a
|
||||
fi
|
||||
SUB_URL="${VLESS_SUB_URL:?set VLESS_SUB_URL}"
|
||||
OUT="${1:-${ROOT}/vless/vless.conf}"
|
||||
UA="${VLESS_UA:-Happ/3.3.6/Windows/2607171516600}"
|
||||
HWID="${VLESS_HWID:-}"
|
||||
|
||||
TMP="$(mktemp)"
|
||||
trap 'rm -f "${TMP}"' EXIT
|
||||
|
||||
CURL=(curl -fsSL -A "$UA")
|
||||
[[ -n "$HWID" ]] && CURL+=(-H "x-hwid: ${HWID}")
|
||||
|
||||
"${CURL[@]}" "$SUB_URL" | tr -d '\n\r' | base64 -d >"${TMP}" 2>/dev/null || "${CURL[@]}" "$SUB_URL" >"${TMP}"
|
||||
|
||||
pick="$(grep -E '^vless://' "${TMP}" | grep -v '@0\.0\.0\.0:1' | head -1 || true)"
|
||||
if [[ -z "$pick" ]]; then
|
||||
pick="$(python3 "${SCRIPT_DIR}/vless-from-subscription-json.py" "${TMP}")" || true
|
||||
fi
|
||||
|
||||
if [[ -z "$pick" ]]; then
|
||||
echo "fetch-vless-subscription: no valid vless URI" >&2
|
||||
head -c 400 "${TMP}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$(dirname "$OUT")"
|
||||
printf '%s\n' "$pick" >"$OUT"
|
||||
chmod 600 "$OUT"
|
||||
echo "Wrote $(wc -c <"$OUT") bytes -> $OUT"
|
||||
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate litellm_config.yaml from PRIMARY_PROVIDER + config/*.yaml."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
PROFILE = os.environ.get("PRIMARY_PROVIDER", "hybrid").strip().lower()
|
||||
|
||||
|
||||
def load(path: Path) -> dict:
|
||||
with path.open(encoding="utf-8") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
|
||||
|
||||
def api_key_ref(name: str) -> str:
|
||||
return f"os.environ/{name}"
|
||||
|
||||
|
||||
def _apply_thinking(params: dict, spec: dict) -> None:
|
||||
"""Qwen3 defaults to thinking; empty content + hang unless disabled."""
|
||||
model = str(params.get("model") or spec.get("model") or "").lower()
|
||||
force = bool(spec.get("disable_thinking"))
|
||||
auto = ("qwen3" in model) and not spec.get("enable_thinking")
|
||||
if force or auto:
|
||||
params["extra_body"] = {
|
||||
"chat_template_kwargs": {"enable_thinking": False},
|
||||
"enable_thinking": False,
|
||||
}
|
||||
|
||||
|
||||
def lane_entry(name: str, spec: dict, rpm: int | None) -> dict:
|
||||
params: dict = {
|
||||
"model": spec["model"],
|
||||
"api_key": api_key_ref(spec["api_key"]),
|
||||
# Per-deployment timeout: litellm_settings.request_timeout often shows as
|
||||
# Deployment Info timeout: None on stream/tool hangs.
|
||||
"timeout": int(os.environ.get("LITELLM_MODEL_TIMEOUT", "180")),
|
||||
}
|
||||
if rpm:
|
||||
params["rpm"] = rpm
|
||||
_apply_thinking(params, spec)
|
||||
return {"model_name": name, "litellm_params": params}
|
||||
|
||||
|
||||
def fixed_entry(name: str, spec: dict) -> dict:
|
||||
params: dict = {
|
||||
"model": spec["model"],
|
||||
"api_key": api_key_ref(spec["api_key"]),
|
||||
"timeout": int(os.environ.get("LITELLM_MODEL_TIMEOUT", "180")),
|
||||
}
|
||||
for key in ("ssl_verify", "max_tokens", "temperature"):
|
||||
if key in spec:
|
||||
params[key] = spec[key]
|
||||
_apply_thinking(params, spec)
|
||||
return {"model_name": name, "litellm_params": params}
|
||||
|
||||
|
||||
def smart_router_internal(rules: dict) -> dict:
|
||||
litellm_rules = rules.get("litellm", {})
|
||||
return {
|
||||
"model_name": "smart-router-internal",
|
||||
"litellm_params": {
|
||||
"model": "auto_router/complexity_router",
|
||||
"drop_params": True,
|
||||
"complexity_router_default_model": "a-medium-ops",
|
||||
"complexity_router_config": {
|
||||
"tiers": {
|
||||
"SIMPLE": "a-simple",
|
||||
"MEDIUM": "a-medium-ops",
|
||||
"MEDIUM_CODE": "a-medium-code",
|
||||
"COMPLEX": "a-complex",
|
||||
"REASONING": "a-reasoning",
|
||||
},
|
||||
"classifier_fallback": "heuristic",
|
||||
"keyword_tier_rules": litellm_rules.get("keyword_tier_rules", []),
|
||||
"custom_technical_keywords": litellm_rules.get(
|
||||
"custom_technical_keywords", []
|
||||
),
|
||||
"token_thresholds": {"simple": 20, "complex": 500},
|
||||
"tier_boundaries": {
|
||||
"simple_medium": 0.18,
|
||||
"medium_complex": 0.38,
|
||||
"complex_reasoning": 0.62,
|
||||
},
|
||||
"session_affinity": True,
|
||||
"session_affinity_ttl_seconds": 1800,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
providers = load(ROOT / "config/providers.yaml")
|
||||
profiles = providers.get("profiles", {})
|
||||
if PROFILE not in profiles:
|
||||
print(f"ERROR: unknown PRIMARY_PROVIDER={PROFILE!r}", file=sys.stderr)
|
||||
print(f"Available: {', '.join(sorted(profiles))}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
profile = profiles[PROFILE]
|
||||
base = load(ROOT / "config/litellm.base.yaml")
|
||||
matrix = load(ROOT / "config/model_matrix.yaml")
|
||||
rules = load(ROOT / "config/routing_rules.yaml")
|
||||
meta = matrix.get("models", {})
|
||||
|
||||
model_list: list[dict] = []
|
||||
|
||||
# Classifier first
|
||||
fixed = providers.get("fixed_models", {})
|
||||
if "gigachat-classifier" in fixed:
|
||||
model_list.append(fixed_entry("gigachat-classifier", fixed["gigachat-classifier"]))
|
||||
|
||||
# Lane models from active profile
|
||||
lanes = profile.get("lanes", {})
|
||||
for name, spec in lanes.items():
|
||||
rpm = meta.get(name, {}).get("rpm")
|
||||
model_list.append(lane_entry(name, spec, rpm))
|
||||
|
||||
# Fixed models (Claude / OR fallbacks / GigaChat / optional free tiers)
|
||||
for name in (
|
||||
"novita-planner",
|
||||
"novita-verifier",
|
||||
"claude-haiku-planner",
|
||||
"claude-sonnet-verifier",
|
||||
"groq-llama-8b",
|
||||
"groq-qwen-coder",
|
||||
"gemini-flash",
|
||||
"grok-3",
|
||||
"gigachat-pro",
|
||||
):
|
||||
if name in fixed:
|
||||
model_list.append(fixed_entry(name, fixed[name]))
|
||||
|
||||
model_list.append(smart_router_internal(rules))
|
||||
|
||||
# smart-router alias (Zed default)
|
||||
sr = profile.get("smart_router", {})
|
||||
if sr:
|
||||
sr_params: dict = {
|
||||
"model": sr["model"],
|
||||
"api_key": api_key_ref(sr["api_key"]),
|
||||
}
|
||||
_apply_thinking(sr_params, sr)
|
||||
model_list.append({"model_name": "smart-router", "litellm_params": sr_params})
|
||||
|
||||
router = base.setdefault("router_settings", {})
|
||||
router["fallbacks"] = [
|
||||
{k: v for k, v in row.items()}
|
||||
for row in _fallback_list(profile.get("fallbacks", {}))
|
||||
]
|
||||
router["default_fallbacks"] = profile.get(
|
||||
"default_fallbacks", ["a-medium-code", "groq-qwen-coder"]
|
||||
)
|
||||
|
||||
out_cfg = {**base, "model_list": model_list}
|
||||
out_path = ROOT / "litellm_config.yaml"
|
||||
header = (
|
||||
f"# LiteLLM — generated for PRIMARY_PROVIDER={PROFILE}\n"
|
||||
f"# Profile: {profile.get('label', PROFILE)}\n"
|
||||
f"# Regenerate: PRIMARY_PROVIDER={PROFILE} bash scripts/gen-litellm-config.py\n\n"
|
||||
)
|
||||
body = yaml.dump(out_cfg, allow_unicode=True, sort_keys=False)
|
||||
out_path.write_text(header + body, encoding="utf-8")
|
||||
print(f"Wrote {out_path} (profile={PROFILE}, {len(model_list)} models)")
|
||||
return 0
|
||||
|
||||
|
||||
def _fallback_list(fallbacks: dict) -> list[dict]:
|
||||
return [{model: targets} for model, targets in fallbacks.items()]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -20,9 +20,21 @@ fi
|
||||
if [ -f /run/secrets/groq_api_key ]; then
|
||||
export GROQ_API_KEY="$(read_secret /run/secrets/groq_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/xai_api_key ]; then
|
||||
export XAI_API_KEY="$(read_secret /run/secrets/xai_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/gemini_api_key ]; then
|
||||
export GEMINI_API_KEY="$(read_secret /run/secrets/gemini_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/together_api_key ]; then
|
||||
export TOGETHER_API_KEY="$(read_secret /run/secrets/together_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/openrouter_api_key ]; then
|
||||
export OPENROUTER_API_KEY="$(read_secret /run/secrets/openrouter_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/anthropic_api_key ]; then
|
||||
export ANTHROPIC_API_KEY="$(read_secret /run/secrets/anthropic_api_key)"
|
||||
fi
|
||||
if [ -f /run/secrets/gigachat_credentials ]; then
|
||||
export GIGACHAT_CREDENTIALS="$(read_secret /run/secrets/gigachat_credentials)"
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical NO_PROXY when litellm uses HTTP_PROXY (VLESS VPN).
|
||||
#
|
||||
# Direct (bypass VPN): Novita, Groq, Gemini, GigaChat, internal/Calentiq.
|
||||
# VPN-only (NOT listed): Anthropic → api.anthropic.com; OpenRouter → openrouter.ai;
|
||||
# xAI Grok → api.x.ai; Together → api.together.xyz
|
||||
#
|
||||
# shellcheck disable=SC2034
|
||||
NO_PROXY_DEFAULT='localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,api.novita.ai,novita.ai,api.groq.com,groq.com,generativelanguage.googleapis.com,ngw.devices.sberbank.ru,gigachat.devices.sberbank.ru,git.sabilin.com,*.ift.calentiq.com,ift.calentiq.com,*.ift.eventhub.local,*.eventhub.local'
|
||||
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bash
|
||||
# Refresh vless.conf from subscription and reload vless-proxy on Swarm (IFT).
|
||||
set -euo pipefail
|
||||
|
||||
STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
STACK_NAME="${STACK_NAME:-ai-router}"
|
||||
cd "$STACK_DIR"
|
||||
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
source .env
|
||||
set +a
|
||||
|
||||
bash "${STACK_DIR}/scripts/fetch-vless-subscription.sh" "${STACK_DIR}/vless/vless.conf"
|
||||
|
||||
CONF="$(cat "${STACK_DIR}/vless/vless.conf")"
|
||||
SVC="${STACK_NAME}_vless-proxy"
|
||||
|
||||
rotate_vless_secret() {
|
||||
if docker secret inspect vless_conf >/dev/null 2>&1; then
|
||||
echo "== Detach old vless_conf secret =="
|
||||
docker service scale "${SVC}=0" >/dev/null || true
|
||||
sleep 3
|
||||
docker service update --secret-rm vless_conf "${SVC}" >/dev/null 2>&1 || true
|
||||
sleep 2
|
||||
docker secret rm vless_conf >/dev/null 2>&1 || true
|
||||
fi
|
||||
echo -n "$CONF" | docker secret create vless_conf -
|
||||
}
|
||||
|
||||
if docker service inspect "${SVC}" >/dev/null 2>&1; then
|
||||
rotate_vless_secret
|
||||
docker service update \
|
||||
--secret-add "source=vless_conf,target=/app/vless.conf,mode=0444" \
|
||||
"${SVC}" >/dev/null
|
||||
docker service update --force "${SVC}" >/dev/null
|
||||
TARGET="${VPN_ENABLED:-false}"
|
||||
if [[ "$TARGET" == "true" ]]; then
|
||||
docker service scale "${SVC}=1" >/dev/null
|
||||
fi
|
||||
echo "refresh-vless: vless-proxy reloaded"
|
||||
else
|
||||
rotate_vless_secret
|
||||
echo "refresh-vless: secret vless_conf updated (service ${SVC} missing — run deploy.sh)"
|
||||
fi
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# Cron: refresh VPN subscription every 6h (config может меняться на стороне провайдера).
|
||||
set -euo pipefail
|
||||
|
||||
STACK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
CRON_FILE="/etc/cron.d/ai-router-vless-refresh"
|
||||
LOG="/var/log/ai-router-vless-refresh.log"
|
||||
|
||||
sudo tee "$CRON_FILE" >/dev/null <<EOF
|
||||
# AI Router: refresh Happ/mireon vless subscription
|
||||
0 */6 * * * deploy cd ${STACK_DIR} && set -a && . ./.env && set +a && bash scripts/refresh-vless-subscription.sh >>${LOG} 2>&1
|
||||
EOF
|
||||
sudo chmod 644 "$CRON_FILE"
|
||||
echo "Installed ${CRON_FILE} (every 6h)"
|
||||
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
|
||||
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
|
||||
PLAN=$(python3 - <<'PY'
|
||||
import json
|
||||
d=json.load(open("/tmp/agent-plan.json"))
|
||||
msg=(d.get("choices") or [{}])[0].get("message") or {}
|
||||
print(msg.get("content") or "")
|
||||
PY
|
||||
)
|
||||
# Approve with tools like Zed agent
|
||||
python3 - <<'PY' > /tmp/agent-approve-req.json
|
||||
import json
|
||||
plan=open("/tmp/agent-plan.json").read()
|
||||
d=json.loads(plan)
|
||||
content=((d.get("choices") or [{}])[0].get("message") or {}).get("content") or ""
|
||||
tools=[{
|
||||
"type":"function",
|
||||
"function":{
|
||||
"name":"read_file",
|
||||
"description":"Read a file",
|
||||
"parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}
|
||||
}
|
||||
},{
|
||||
"type":"function",
|
||||
"function":{
|
||||
"name":"edit_file",
|
||||
"description":"Edit a file",
|
||||
"parameters":{"type":"object","properties":{
|
||||
"path":{"type":"string"},
|
||||
"edits":{"type":"array","items":{"type":"object","properties":{
|
||||
"old_text":{"type":"string"},"new_text":{"type":"string"}
|
||||
}}}
|
||||
},"required":["path","edits"]}
|
||||
}
|
||||
}]
|
||||
body={
|
||||
"model":"smart-router",
|
||||
"stream": False,
|
||||
"tools": tools,
|
||||
"tool_choice":"auto",
|
||||
"messages":[
|
||||
{"role":"user","content":"observer.ift.calentiq.com не работает - Bad Gateway почини"},
|
||||
{"role":"assistant","content":content},
|
||||
{"role":"user","content":"ok"}
|
||||
]
|
||||
}
|
||||
json.dump(body, open("/tmp/agent-approve-req.json","w"), ensure_ascii=False)
|
||||
print("req bytes", len(json.dumps(body)))
|
||||
PY
|
||||
echo "== approve+tools =="
|
||||
curl -sS --max-time 180 "$BASE/v1/chat/completions" \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-AI-Orchestrate: force" \
|
||||
-H "X-AI-Quality: balanced" \
|
||||
-d @/tmp/agent-approve-req.json | tee /tmp/agent-exec1.json | python3 -c '
|
||||
import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
ch=(d.get("choices") or [{}])[0]
|
||||
msg=ch.get("message") or {}
|
||||
print("finish", ch.get("finish_reason"))
|
||||
print("content", (msg.get("content") or "")[:1200])
|
||||
tcs=msg.get("tool_calls") or []
|
||||
print("tool_calls", len(tcs))
|
||||
for tc in tcs:
|
||||
fn=tc.get("function") or {}
|
||||
print(" TOOL", fn.get("name"), (fn.get("arguments") or "")[:300])
|
||||
meta=d.get("x_router_meta") or {}
|
||||
print("meta phase", meta.get("agent_phase"), "force_edit", meta.get("executor_force_edit_after_read"), "model", meta.get("executor_model") or meta.get("selected_model"))
|
||||
'
|
||||
echo "== router log since approve =="
|
||||
docker service logs --since 3m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'path_index|force edit|stop after|hierarchical_agent|executor result|kickstart|phase=' \
|
||||
| tail -30
|
||||
REMOTE
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift 'bash -s' <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
|
||||
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
|
||||
test -f /tmp/agent-plan.json && echo "plan_ok $(wc -c </tmp/agent-plan.json)" || echo "no plan"
|
||||
python3 <<'PY'
|
||||
import json
|
||||
d=json.load(open("/tmp/agent-plan.json"))
|
||||
content=((d.get("choices") or [{}])[0].get("message") or {}).get("content") or ""
|
||||
tools=[{
|
||||
"type":"function",
|
||||
"function":{
|
||||
"name":"read_file",
|
||||
"description":"Read a file",
|
||||
"parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}
|
||||
}
|
||||
},{
|
||||
"type":"function",
|
||||
"function":{
|
||||
"name":"edit_file",
|
||||
"description":"Edit a file",
|
||||
"parameters":{"type":"object","properties":{
|
||||
"path":{"type":"string"},
|
||||
"edits":{"type":"array","items":{"type":"object","properties":{
|
||||
"old_text":{"type":"string"},"new_text":{"type":"string"}
|
||||
}}}
|
||||
},"required":["path","edits"]}
|
||||
}
|
||||
}]
|
||||
body={
|
||||
"model":"smart-router",
|
||||
"stream": False,
|
||||
"tools": tools,
|
||||
"messages":[
|
||||
{"role":"user","content":"observer.ift.calentiq.com не работает - Bad Gateway почини"},
|
||||
{"role":"assistant","content":content},
|
||||
{"role":"user","content":"ok"}
|
||||
]
|
||||
}
|
||||
json.dump(body, open("/tmp/agent-approve-req.json","w"), ensure_ascii=False)
|
||||
print("req_ok", len(content), "tools", len(tools))
|
||||
PY
|
||||
code=$(curl -sS --max-time 180 -o /tmp/agent-exec1.json -w "%{http_code}" \
|
||||
"$BASE/v1/chat/completions" \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-AI-Orchestrate: force" \
|
||||
-H "X-AI-Quality: balanced" \
|
||||
-d @/tmp/agent-approve-req.json || true)
|
||||
echo "http=$code bytes=$(wc -c </tmp/agent-exec1.json)"
|
||||
head -c 500 /tmp/agent-exec1.json; echo
|
||||
python3 <<'PY'
|
||||
import json
|
||||
raw=open("/tmp/agent-exec1.json").read().strip()
|
||||
if not raw:
|
||||
print("EMPTY"); raise SystemExit
|
||||
d=json.loads(raw)
|
||||
if d.get("error"):
|
||||
print("ERROR", d["error"]); raise SystemExit
|
||||
ch=(d.get("choices") or [{}])[0]
|
||||
msg=ch.get("message") or {}
|
||||
print("finish", ch.get("finish_reason"))
|
||||
print("content_head:")
|
||||
print((msg.get("content") or "")[:1500])
|
||||
tcs=msg.get("tool_calls") or []
|
||||
print("n_tools", len(tcs))
|
||||
for tc in tcs:
|
||||
fn=tc.get("function") or {}
|
||||
print("TOOL", fn.get("name"))
|
||||
print("ARGS", (fn.get("arguments") or "")[:400])
|
||||
meta=d.get("x_router_meta") or {}
|
||||
print("phase", meta.get("agent_phase"), "exec", meta.get("executor_model") or meta.get("selected_model"),
|
||||
"force", meta.get("executor_force_edit_after_read"), "stop", meta.get("executor_stop_after_edit"))
|
||||
PY
|
||||
docker service logs --since 4m --raw ai-router_router 2>&1 \
|
||||
| grep -iE 'path_index|force edit|stop after|phase=|executor result|kickstart|Выполн' \
|
||||
| tail -25
|
||||
REMOTE
|
||||
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
ssh -o BatchMode=yes eventhub-ift bash -s <<'REMOTE'
|
||||
set -euo pipefail
|
||||
cd /opt/ai-router-stack
|
||||
set -a; source .env; set +a
|
||||
BASE="${PUBLIC_URL:-https://ai-router.ift.calentiq.com}"
|
||||
KEY="${ROUTER_API_KEY:-$LITELLM_MASTER_KEY}"
|
||||
echo "BASE=$BASE"
|
||||
# 1) Plan request (no tools) — hierarchical plan confirm
|
||||
curl -sS --max-time 120 "$BASE/v1/chat/completions" \
|
||||
-H "Authorization: Bearer $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-AI-Orchestrate: force" \
|
||||
-H "X-AI-Quality: balanced" \
|
||||
-d '{
|
||||
"model": "smart-router",
|
||||
"stream": false,
|
||||
"messages": [
|
||||
{"role": "user", "content": "observer.ift.calentiq.com не работает - Bad Gateway почини"}
|
||||
]
|
||||
}' | tee /tmp/agent-plan.json | python3 -c '
|
||||
import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
msg=(d.get("choices") or [{}])[0].get("message") or {}
|
||||
print("=== PLAN CONTENT ===")
|
||||
print((msg.get("content") or "")[:2500])
|
||||
meta=d.get("x_router_meta") or {}
|
||||
print("=== META keys ===", sorted(meta.keys())[:30])
|
||||
print("mode", meta.get("mode"), "awaiting", meta.get("awaiting_plan_confirm"), "planner", meta.get("planner_model"))
|
||||
'
|
||||
REMOTE
|
||||
+34
-4
@@ -14,10 +14,29 @@ BASE="${TEST_BASE_URL:-http://127.0.0.1:8000}"
|
||||
LITELLM_BASE="${TEST_LITELLM_URL:-http://127.0.0.1:4000}"
|
||||
ROUTER_KEY="${ROUTER_API_KEY:?ROUTER_API_KEY required}"
|
||||
|
||||
curl_common() {
|
||||
local url="$1"
|
||||
CURL=(curl -sf)
|
||||
if [[ "$BASE" == https:* ]]; then
|
||||
if [[ "$url" == https:* ]]; then
|
||||
CURL+=( -k )
|
||||
fi
|
||||
if [[ -n "${TEST_RESOLVE_IP:-}" ]]; then
|
||||
local host="${url#*://}"
|
||||
host="${host%%/*}"
|
||||
host="${host%%:*}"
|
||||
CURL+=( --resolve "${host}:443:${TEST_RESOLVE_IP}" --resolve "${host}:80:${TEST_RESOLVE_IP}" )
|
||||
fi
|
||||
}
|
||||
|
||||
curl_common "${BASE}"
|
||||
if [[ "$LITELLM_BASE" == https:* && -n "${TEST_RESOLVE_IP:-}" ]]; then
|
||||
LITELLM_CURL=(curl -sf -k)
|
||||
lh="${LITELLM_BASE#*://}"; lh="${lh%%/*}"; lh="${lh%%:*}"
|
||||
LITELLM_CURL+=( --resolve "${lh}:443:${TEST_RESOLVE_IP}" --resolve "${lh}:80:${TEST_RESOLVE_IP}" )
|
||||
else
|
||||
LITELLM_CURL=(curl -sf)
|
||||
[[ "$LITELLM_BASE" == https:* ]] && LITELLM_CURL+=( -k )
|
||||
fi
|
||||
|
||||
echo "== Router health =="
|
||||
"${CURL[@]}" "${BASE}/health" | jq .
|
||||
@@ -34,20 +53,31 @@ echo "== Classify COMPLEX =="
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"messages":[{"role":"user","content":"Спроектируй архитектуру microservices"}]}' | jq .
|
||||
|
||||
echo "== Chat smart-router max_tokens=16 =="
|
||||
echo "== Chat smart-router max_tokens=16 (single-shot / no force hierarchical) =="
|
||||
"${CURL[@]}" "${BASE}/v1/chat/completions" \
|
||||
-H "Authorization: Bearer ${ROUTER_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-AI-Orchestrate: off" \
|
||||
-d '{"model":"smart-router","max_tokens":16,"messages":[{"role":"user","content":"bash docker service ls"}]}' \
|
||||
| jq '.choices[0].message.content, .x_router_meta // empty'
|
||||
|
||||
if [[ "${SMOKE_HIERARCHICAL:-}" == "1" ]]; then
|
||||
echo "== Hierarchical force (Novita plan/verify + workers; VPN not required) =="
|
||||
"${CURL[@]}" "${BASE}/v1/chat/completions" \
|
||||
-H "Authorization: Bearer ${ROUTER_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-AI-Orchestrate: force" \
|
||||
-d '{"model":"smart-router","max_tokens":256,"messages":[{"role":"user","content":"Разбей на 2 шага: 1) что такое Docker 2) одна команда docker ps. Кратко."}]}' \
|
||||
| jq '.x_router_meta.mode, .x_router_meta.planner_model, .x_router_meta.worker_calls, .x_router_meta.verify_skipped, (.choices[0].message.content|.[0:200])'
|
||||
fi
|
||||
|
||||
if [[ -n "${SKIP_LITELLM_SMOKE:-}" ]]; then
|
||||
echo "SKIP_LITELLM_SMOKE set — skipping LiteLLM checks"
|
||||
else
|
||||
echo "== LiteLLM liveliness =="
|
||||
"${CURL[@]}" "${LITELLM_BASE}/health/liveliness" && echo
|
||||
"${LITELLM_CURL[@]}" "${LITELLM_BASE}/health/liveliness" && echo
|
||||
echo "== LiteLLM UI =="
|
||||
curl -sfI -k "${LITELLM_BASE}/ui" | head -3 || true
|
||||
"${LITELLM_CURL[@]}" -I "${LITELLM_BASE}/ui" | head -3 || true
|
||||
fi
|
||||
|
||||
echo "All smoke checks passed."
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env bash
|
||||
# Switch PRIMARY_PROVIDER and regenerate litellm_config.yaml
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
PROFILE="${1:-}"
|
||||
if [[ -z "$PROFILE" ]]; then
|
||||
echo "Usage: bash scripts/switch-provider.sh <profile>" >&2
|
||||
echo "Profiles:" >&2
|
||||
python3 -c "
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
p = yaml.safe_load(Path('config/providers.yaml').read_text(encoding='utf-8'))
|
||||
for k, v in sorted(p.get('profiles', {}).items()):
|
||||
wc = ' [welcome credit]' if v.get('welcome_credit') else ''
|
||||
print(f' {k:12} {v.get(\"label\", \"\")}{wc}')
|
||||
"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! -f .env ]]; then
|
||||
echo "ERROR: .env missing — copy from .env.example" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if grep -q '^PRIMARY_PROVIDER=' .env; then
|
||||
sed -i "s/^PRIMARY_PROVIDER=.*/PRIMARY_PROVIDER=${PROFILE}/" .env
|
||||
else
|
||||
echo "PRIMARY_PROVIDER=${PROFILE}" >> .env
|
||||
fi
|
||||
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
source .env
|
||||
set +a
|
||||
|
||||
python3 scripts/gen-litellm-config.py
|
||||
echo ""
|
||||
echo "Switched to PRIMARY_PROVIDER=${PROFILE}"
|
||||
echo "Redeploy on IFT: bash scripts/deploy.sh (Swarm configs are immutable — stack rm if deploy fails)"
|
||||
@@ -1,78 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# Sync model_list fragment from config/*.yaml
|
||||
# Regenerate litellm_config.yaml from PRIMARY_PROVIDER + config/providers.yaml
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
export ROOT
|
||||
|
||||
python3 <<'PY'
|
||||
import os
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
|
||||
root = Path(os.environ["ROOT"])
|
||||
rules = yaml.safe_load((root / "config/routing_rules.yaml").read_text(encoding="utf-8"))
|
||||
matrix = yaml.safe_load((root / "config/model_matrix.yaml").read_text(encoding="utf-8"))
|
||||
|
||||
entries = []
|
||||
for name, cfg in matrix.get("models", {}).items():
|
||||
novita = cfg.get("novita")
|
||||
if not novita:
|
||||
continue
|
||||
entry = {
|
||||
"model_name": name,
|
||||
"litellm_params": {
|
||||
"model": novita,
|
||||
"api_key": "os.environ/NOVITA_API_KEY",
|
||||
},
|
||||
}
|
||||
if cfg.get("rpm"):
|
||||
entry["litellm_params"]["rpm"] = cfg["rpm"]
|
||||
entries.append(entry)
|
||||
|
||||
for name, cfg in matrix.get("optional_providers", {}).items():
|
||||
entries.append({
|
||||
"model_name": name,
|
||||
"litellm_params": {
|
||||
"model": cfg["model"],
|
||||
"api_key": cfg.get("api_key", "os.environ/GROQ_API_KEY"),
|
||||
},
|
||||
})
|
||||
|
||||
litellm_rules = rules.get("litellm", {})
|
||||
entries.append({
|
||||
"model_name": "smart-router-internal",
|
||||
"litellm_params": {
|
||||
"model": "auto_router/complexity_router",
|
||||
"drop_params": True,
|
||||
"complexity_router_default_model": "a-medium-ops",
|
||||
"complexity_router_config": {
|
||||
"tiers": {
|
||||
"SIMPLE": "a-simple",
|
||||
"MEDIUM": "a-medium-ops",
|
||||
"MEDIUM_CODE": "a-medium-code",
|
||||
"COMPLEX": "a-complex",
|
||||
"REASONING": "a-reasoning",
|
||||
},
|
||||
"classifier_fallback": "heuristic",
|
||||
"keyword_tier_rules": litellm_rules.get("keyword_tier_rules", []),
|
||||
"custom_technical_keywords": litellm_rules.get("custom_technical_keywords", []),
|
||||
"token_thresholds": {"simple": 20, "complex": 500},
|
||||
"tier_boundaries": {
|
||||
"simple_medium": 0.18,
|
||||
"medium_complex": 0.38,
|
||||
"complex_reasoning": 0.62,
|
||||
},
|
||||
"session_affinity": True,
|
||||
"session_affinity_ttl_seconds": 1800,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
out = root / "litellm_config.generated.yaml"
|
||||
out.write_text(
|
||||
yaml.dump({"model_list": entries}, allow_unicode=True, sort_keys=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(f"Wrote {out} ({len(entries)} models)")
|
||||
PY
|
||||
cd "$ROOT"
|
||||
if [[ -f .env ]]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
source .env
|
||||
set +a
|
||||
fi
|
||||
python3 scripts/gen-litellm-config.py
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract first working vless:// URI from Happ/mireon subscription JSON."""
|
||||
import json
|
||||
import sys
|
||||
import urllib.parse
|
||||
|
||||
path = sys.argv[1]
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
|
||||
configs = data if isinstance(data, list) else [data]
|
||||
for j in configs:
|
||||
if not isinstance(j, dict):
|
||||
continue
|
||||
for ob in j.get("outbounds", []):
|
||||
if ob.get("protocol") != "vless":
|
||||
continue
|
||||
tag = ob.get("tag") or "proxy"
|
||||
st = ob.get("settings") or {}
|
||||
vnext = (st.get("vnext") or [{}])[0]
|
||||
addr, port = vnext.get("address"), vnext.get("port")
|
||||
users = (vnext.get("users") or [{}])[0]
|
||||
uid = users.get("id")
|
||||
if not all([addr, port, uid]) or addr in ("0.0.0.0", "127.0.0.1"):
|
||||
continue
|
||||
flow = users.get("flow") or ""
|
||||
stream = ob.get("streamSettings") or {}
|
||||
net = stream.get("network") or "tcp"
|
||||
sec = stream.get("security") or "none"
|
||||
rs = stream.get("realitySettings") or {}
|
||||
ts = stream.get("tlsSettings") or {}
|
||||
sni = rs.get("serverName") or ts.get("serverName") or ""
|
||||
params = {"encryption": "none", "security": sec, "type": net}
|
||||
if sni:
|
||||
params["sni"] = sni
|
||||
if rs.get("publicKey"):
|
||||
params["pbk"] = rs["publicKey"]
|
||||
if rs.get("shortId"):
|
||||
params["sid"] = rs["shortId"]
|
||||
if rs.get("fingerprint"):
|
||||
params["fp"] = rs["fingerprint"]
|
||||
if flow:
|
||||
params["flow"] = flow
|
||||
q = urllib.parse.urlencode(params)
|
||||
print(f"vless://{uid}@{addr}:{port}?{q}#{tag}")
|
||||
sys.exit(0)
|
||||
sys.exit(1)
|
||||
@@ -10,7 +10,9 @@ set -a
|
||||
source .env
|
||||
set +a
|
||||
|
||||
NO_PROXY_VAL="${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}"
|
||||
# shellcheck disable=SC1091
|
||||
source "${STACK_DIR}/scripts/no-proxy-default.sh"
|
||||
NO_PROXY_VAL="${NO_PROXY:-${NO_PROXY_DEFAULT}}"
|
||||
|
||||
echo "== Scale vless-proxy and vpn-watchdog to 0 =="
|
||||
docker service scale "${STACK_NAME}_vless-proxy=0" "${STACK_NAME}_vpn-watchdog=0" || true
|
||||
|
||||
@@ -11,7 +11,9 @@ source .env
|
||||
set +a
|
||||
|
||||
VLESS_URL="${VLESS_PROXY_URL:-http://vless-proxy:8080}"
|
||||
NO_PROXY_VAL="${NO_PROXY:-localhost,127.0.0.1,api.novita.ai,novita.ai}"
|
||||
# shellcheck disable=SC1091
|
||||
source "${STACK_DIR}/scripts/no-proxy-default.sh"
|
||||
NO_PROXY_VAL="${NO_PROXY:-${NO_PROXY_DEFAULT}}"
|
||||
|
||||
echo "== Scale vless-proxy and vpn-watchdog to 1 =="
|
||||
docker service scale "${STACK_NAME}_vless-proxy=1" "${STACK_NAME}_vpn-watchdog=1"
|
||||
@@ -42,4 +44,4 @@ docker service update \
|
||||
--env-add "NO_PROXY=${NO_PROXY_VAL}" \
|
||||
"${STACK_NAME}_litellm"
|
||||
|
||||
echo "VPN enabled. Novita bypasses via NO_PROXY."
|
||||
echo "VPN enabled. Direct: Novita/Groq/Gemini/GigaChat. VPN: Anthropic/OpenRouter/Grok/Together."
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,154 @@
|
||||
"""Unit tests for deterministic path index + resolve gating."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import unittest
|
||||
|
||||
from path_resolve import (
|
||||
apply_deterministic_path_index,
|
||||
enrich_plan_from_discovery_tools,
|
||||
match_index_paths,
|
||||
path_resolve_needed,
|
||||
)
|
||||
|
||||
|
||||
class TestPathResolve(unittest.TestCase):
|
||||
def test_index_hits_traefik_not_fake_yml(self) -> None:
|
||||
plan = {
|
||||
"user_goal": "fix observer.ift.calentiq.com traefik route",
|
||||
"subtasks": [
|
||||
{
|
||||
"id": "1",
|
||||
"prompt": "edit traefik for observer",
|
||||
"paths": [
|
||||
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\traefik.yml"
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
# sanitize would drop fake; index should still hit via keywords
|
||||
indexed = match_index_paths(plan)
|
||||
self.assertTrue(indexed)
|
||||
self.assertTrue(
|
||||
indexed[0].lower().replace("/", "\\").endswith(
|
||||
r"ift\traefik\dynamic_conf.yml"
|
||||
)
|
||||
)
|
||||
fixed, unresolved = apply_deterministic_path_index(plan)
|
||||
self.assertEqual(unresolved, [])
|
||||
self.assertFalse(path_resolve_needed(fixed))
|
||||
self.assertTrue(
|
||||
fixed["subtasks"][0]["paths"][0]
|
||||
.lower()
|
||||
.replace("/", "\\")
|
||||
.endswith(r"ift\traefik\dynamic_conf.yml")
|
||||
)
|
||||
|
||||
def test_index_compose_core(self) -> None:
|
||||
plan = {
|
||||
"user_goal": "bump timeout in docker compose swarm",
|
||||
"subtasks": [{"id": "1", "prompt": "edit compose", "paths": []}],
|
||||
}
|
||||
fixed, unresolved = apply_deterministic_path_index(plan)
|
||||
self.assertEqual(unresolved, [])
|
||||
self.assertIn("docker-compose.core.yml", fixed["subtasks"][0]["paths"][0])
|
||||
|
||||
def test_miss_triggers_queries(self) -> None:
|
||||
plan = {
|
||||
"user_goal": "EventHubDevOps mystery widget xyz",
|
||||
"subtasks": [{"id": "1", "prompt": "widget", "paths": []}],
|
||||
}
|
||||
fixed, unresolved = apply_deterministic_path_index(plan)
|
||||
self.assertTrue(unresolved)
|
||||
self.assertTrue(path_resolve_needed(fixed))
|
||||
|
||||
def test_enrich_from_find_tool(self) -> None:
|
||||
plan = {
|
||||
"path_resolve_queries": ["dynamic_conf.yml"],
|
||||
"path_resolve": "needed",
|
||||
"subtasks": [{"id": "1", "prompt": "x", "paths": []}],
|
||||
}
|
||||
yml = r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "f1",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "find_path",
|
||||
"arguments": json.dumps({"query": "dynamic_conf"}),
|
||||
},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "f1", "content": yml + "\n"},
|
||||
]
|
||||
enriched = enrich_plan_from_discovery_tools(plan, messages)
|
||||
self.assertEqual(enriched["path_resolve"], "find")
|
||||
self.assertFalse(path_resolve_needed(enriched))
|
||||
self.assertEqual(enriched["subtasks"][0]["paths"][0], yml)
|
||||
|
||||
|
||||
def test_enrich_ignores_yaml_etc_artifacts(self) -> None:
|
||||
plan = {
|
||||
"path_resolve": "index",
|
||||
"subtasks": [
|
||||
{
|
||||
"id": "1",
|
||||
"prompt": "traefik",
|
||||
"paths": [
|
||||
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
|
||||
],
|
||||
"path_source": "index",
|
||||
}
|
||||
],
|
||||
}
|
||||
# read_file body mentions container paths — must NOT poison plan
|
||||
yml_body = (
|
||||
"http:\n routers:\n"
|
||||
" # volume: /etc/traefik/dynamic_conf.yml\n"
|
||||
" # also null:/etc/nginx/conf.d/default.conf\n"
|
||||
)
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "r1",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "read_file",
|
||||
"arguments": json.dumps(
|
||||
{
|
||||
"path": r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
|
||||
}
|
||||
),
|
||||
},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "r1", "content": yml_body},
|
||||
]
|
||||
enriched = enrich_plan_from_discovery_tools(plan, messages)
|
||||
self.assertEqual(enriched["path_resolve"], "index")
|
||||
self.assertIn(
|
||||
"dynamic_conf.yml",
|
||||
enriched["subtasks"][0]["paths"][0],
|
||||
)
|
||||
self.assertNotIn("nginx", json.dumps(enriched).lower())
|
||||
# Without index, still ignore read bodies (no find_* call ids)
|
||||
plan2 = {
|
||||
"path_resolve_queries": ["x"],
|
||||
"path_resolve": "needed",
|
||||
"subtasks": [{"id": "1", "prompt": "x", "paths": []}],
|
||||
}
|
||||
enriched2 = enrich_plan_from_discovery_tools(plan2, messages)
|
||||
self.assertEqual(enriched2.get("path_resolve"), "needed")
|
||||
self.assertEqual(enriched2["subtasks"][0]["paths"], [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Tests for Zed-facing progress UI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from progress_ui import (
|
||||
execution_banner,
|
||||
format_progress_block,
|
||||
humanize_line,
|
||||
short_path,
|
||||
tool_status,
|
||||
)
|
||||
|
||||
|
||||
class TestProgressUi(unittest.TestCase):
|
||||
def test_humanize_drops_noise(self) -> None:
|
||||
self.assertIsNone(humanize_line("ждём LiteLLM `b-complex` (до 45с)…"))
|
||||
self.assertIsNone(humanize_line("ctx 7358/12000"))
|
||||
self.assertEqual(humanize_line("план…"), "Планирую задачу")
|
||||
self.assertIn(
|
||||
"утверждён",
|
||||
humanize_line("план: утверждён пользователем (3 подзадач)") or "",
|
||||
)
|
||||
|
||||
def test_format_block_numbered(self) -> None:
|
||||
block = format_progress_block(
|
||||
[
|
||||
"план…",
|
||||
"ждём LiteLLM x",
|
||||
"план: утверждён пользователем (3 подзадач)",
|
||||
"agent: план готов → executor с tools (Zed)",
|
||||
]
|
||||
)
|
||||
self.assertIn("**Ход**", block)
|
||||
self.assertIn("1. Планирую задачу", block)
|
||||
self.assertIn("2. План утверждён", block)
|
||||
self.assertNotIn("LiteLLM", block)
|
||||
|
||||
def test_execution_banner_shows_short_path(self) -> None:
|
||||
plan = {
|
||||
"subtasks": [
|
||||
{
|
||||
"paths": [
|
||||
r"C:\Users\alexc\IdeaProjects\eventHub\EventHubDevOps\ift\traefik\dynamic_conf.yml"
|
||||
],
|
||||
"path_source": "index",
|
||||
}
|
||||
]
|
||||
}
|
||||
text = execution_banner(path_mode=False, plan=plan, model="a-medium-code")
|
||||
self.assertIn("**Выполнение**", text)
|
||||
self.assertIn("dynamic_conf.yml", text)
|
||||
self.assertIn("индекс", text)
|
||||
self.assertNotIn("a-medium-code", text)
|
||||
|
||||
def test_short_path_and_tool_status(self) -> None:
|
||||
self.assertEqual(
|
||||
short_path(r"C:\x\EventHubDevOps\ift\traefik\dynamic_conf.yml"),
|
||||
r"ift\traefik\dynamic_conf.yml",
|
||||
)
|
||||
self.assertIn("Читаю", tool_status(["read_file"], path=r"a\b\c.yml"))
|
||||
self.assertIn("Правлю", tool_status(["edit_file"]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Unit tests for DevOps runtime_probe."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from runtime_probe import (
|
||||
attach_runtime_probe,
|
||||
devops_blocks_blind_force_edit,
|
||||
extract_hosts,
|
||||
is_devops_request,
|
||||
suggested_traefik_edit,
|
||||
)
|
||||
|
||||
|
||||
class TestRuntimeProbe(unittest.TestCase):
|
||||
def test_extract_hosts(self) -> None:
|
||||
hosts = extract_hosts(
|
||||
"observer.ift.calentiq.com не работает — Bad Gateway"
|
||||
)
|
||||
self.assertEqual(hosts, ["observer.ift.calentiq.com"])
|
||||
|
||||
def test_is_devops_request(self) -> None:
|
||||
self.assertTrue(
|
||||
is_devops_request(
|
||||
{"user_goal": "fix traefik"},
|
||||
"observer.ift.calentiq.com Bad Gateway",
|
||||
)
|
||||
)
|
||||
self.assertFalse(is_devops_request({"user_goal": "rename button"}))
|
||||
|
||||
def test_attach_probe_sets_facts(self) -> None:
|
||||
plan = {
|
||||
"user_goal": "observer.ift.calentiq.com Bad Gateway",
|
||||
"subtasks": [
|
||||
{
|
||||
"id": "1",
|
||||
"prompt": "fix traefik",
|
||||
"paths": [],
|
||||
}
|
||||
],
|
||||
}
|
||||
fake = {
|
||||
"host": "observer.ift.calentiq.com",
|
||||
"http_status": 502,
|
||||
"service": "observer_web",
|
||||
"ports": {"observer_web:80": False, "observer_web:4000": True},
|
||||
"open_ports": [4000],
|
||||
"suggested_backend_url": "http://observer_web:4000",
|
||||
"hint": "port mismatch",
|
||||
"source": "gateway_tcp",
|
||||
"closed_port_80_but_alt_open": True,
|
||||
}
|
||||
with patch("runtime_probe.probe_host", return_value=fake):
|
||||
out = attach_runtime_probe(
|
||||
plan,
|
||||
user_text="observer.ift.calentiq.com не работает",
|
||||
cfg={"runtime_probe_enabled": True},
|
||||
)
|
||||
self.assertEqual(out["runtime_probe"], "gateway")
|
||||
self.assertEqual(
|
||||
out["runtime_facts"]["suggested_backend_url"],
|
||||
"http://observer_web:4000",
|
||||
)
|
||||
self.assertIn("4000", out["subtasks"][0]["edit_goal"])
|
||||
path0 = out["subtasks"][0]["paths"][0]
|
||||
self.assertTrue(path0.endswith("dynamic_conf.yml"))
|
||||
|
||||
def test_suggested_traefik_edit(self) -> None:
|
||||
plan = {
|
||||
"runtime_facts": {
|
||||
"service": "observer_web",
|
||||
"suggested_backend_url": "http://observer_web:4000",
|
||||
"hint": "x",
|
||||
}
|
||||
}
|
||||
fix = suggested_traefik_edit(plan)
|
||||
assert fix is not None
|
||||
self.assertEqual(fix["old_text"], 'url: "http://observer_web:80"')
|
||||
self.assertEqual(fix["new_text"], 'url: "http://observer_web:4000"')
|
||||
|
||||
def test_blocks_blind_without_facts(self) -> None:
|
||||
plan = {"user_goal": "Bad Gateway traefik observer.ift.calentiq.com"}
|
||||
self.assertTrue(devops_blocks_blind_force_edit(plan))
|
||||
plan["runtime_facts"] = {"host": "x", "http_status": 502}
|
||||
self.assertFalse(devops_blocks_blind_force_edit(plan))
|
||||
|
||||
def test_disabled(self) -> None:
|
||||
plan = {"user_goal": "observer.ift.calentiq.com"}
|
||||
out = attach_runtime_probe(
|
||||
plan, user_text="x", cfg={"runtime_probe_enabled": False}
|
||||
)
|
||||
self.assertIsNone(out.get("runtime_facts"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+8
-2
@@ -1,3 +1,9 @@
|
||||
FROM thejohnd0e/vless-to-http:latest
|
||||
FROM alpine:3.20
|
||||
|
||||
# vless.conf mounted via Swarm secret at /app/vless.conf
|
||||
WORKDIR /app
|
||||
RUN apk add --no-cache curl unzip ca-certificates wget procps && curl -L -o /tmp/xray.zip https://github.com/XTLS/Xray-core/releases/latest/download/Xray-linux-64.zip && unzip /tmp/xray.zip -d /tmp/xray && install -m755 /tmp/xray/xray /usr/local/bin/Xray && mkdir -p /usr/local/share/xray && mv /tmp/xray/geoip.dat /tmp/xray/geosite.dat /usr/local/share/xray/ && rm -rf /tmp/xray /tmp/xray.zip
|
||||
|
||||
COPY entrypoint.sh /app/entrypoint.sh
|
||||
RUN chmod +x /app/entrypoint.sh
|
||||
EXPOSE 8080
|
||||
CMD ["/app/entrypoint.sh"]
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
# Stub when VPN_ENABLED=false (vless replicas=0). Replace with Dockerfile when enabling VPN.
|
||||
FROM alpine:3.20
|
||||
RUN apk add --no-cache wget
|
||||
EXPOSE 8080
|
||||
CMD ["sleep", "infinity"]
|
||||
@@ -0,0 +1,184 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
CFG=/app/config.json
|
||||
|
||||
# ---- read vless.conf (compose mount or swarm secret rotate) ----
|
||||
CONF=""
|
||||
for p in /app/vless.conf /run/secrets/vless_conf; do
|
||||
if [ -s "$p" ]; then CONF="$p"; break; fi
|
||||
done
|
||||
if [ -z "$CONF" ]; then
|
||||
echo "vless.conf not found or empty" >&2
|
||||
exit 23
|
||||
fi
|
||||
VLESS_URL="$(tr -d '\r\n' < "$CONF")"
|
||||
VLESS_URL="${VLESS_URL%%#*}"
|
||||
|
||||
# ---- extract fields from URL ----
|
||||
USER_ID="$(printf '%s' "$VLESS_URL" | sed -n 's#^vless://\([^@/]*\).*#\1#p')"
|
||||
SERVER="$( printf '%s' "$VLESS_URL" | sed -n 's#.*@\([^:/?]*\).*#\1#p')"
|
||||
PORT_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*:\([0-9][0-9]*\).*#\1#p')"
|
||||
PORT="$(printf '%s' "$PORT_RAW" | tr -cd '0-9')"
|
||||
|
||||
PUBKEY="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]pbk=\([^&]*\).*#\1#p')"
|
||||
SNI="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]sni=\([^&]*\).*#\1#p')"
|
||||
FP="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]fp=\([^&]*\).*#\1#p')"
|
||||
SID="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]sid=\([^&]*\).*#\1#p')"
|
||||
SPX="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]spx=\([^&]*\).*#\1#p' | sed 's/%2F/\//g')"
|
||||
FLOW="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]flow=\([^&]*\).*#\1#p')"
|
||||
TYPE="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]type=\([^&]*\).*#\1#p')"
|
||||
PATH_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]path=\([^&]*\).*#\1#p')"
|
||||
HOST_RAW="$(printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]host=\([^&]*\).*#\1#p')"
|
||||
MODE="$( printf '%s' "$VLESS_URL" | sed -n 's#.*[?&]mode=\([^&]*\).*#\1#p')"
|
||||
|
||||
[ -z "${FP:-}" ] && FP="firefox"
|
||||
[ -z "${SPX:-}" ] && SPX="/"
|
||||
[ -z "${TYPE:-}" ] && TYPE="tcp"
|
||||
XHTTP_PATH="$(printf '%s' "${PATH_RAW:-}" | sed 's/%2[Ff]/\//g')"
|
||||
HOST="$(printf '%s' "${HOST_RAW:-}" | sed 's/%2[Cc]/,/g')"
|
||||
|
||||
# ---- minimal validation ----
|
||||
[ -n "${USER_ID:-}" ] || { echo "ERR: empty USER_ID"; exit 23; }
|
||||
[ -n "${SERVER:-}" ] || { echo "ERR: empty SERVER"; exit 23; }
|
||||
[ -n "${PORT:-}" ] || { echo "ERR: empty PORT"; exit 23; }
|
||||
[ -n "${PUBKEY:-}" ] || { echo "ERR: empty PUBKEY"; exit 23; }
|
||||
[ -n "${SNI:-}" ] || { echo "ERR: empty SNI"; exit 23; }
|
||||
[ -n "${SID:-}" ] || { echo "ERR: empty SID"; exit 23; }
|
||||
|
||||
case "$TYPE" in
|
||||
tcp|xhttp) ;;
|
||||
*)
|
||||
echo "ERR: unsupported transport type '$TYPE' (supported: tcp, xhttp)" >&2
|
||||
exit 23
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ "$TYPE" = "xhttp" ] && [ -z "${XHTTP_PATH:-}" ]; then
|
||||
echo "ERR: empty PATH for xhttp transport" >&2
|
||||
exit 23
|
||||
fi
|
||||
|
||||
# ---- build user block safely (with/without flow) ----
|
||||
if [ -n "${FLOW:-}" ]; then
|
||||
USER_BLOCK=$(cat <<JSON
|
||||
{
|
||||
"id": "${USER_ID}",
|
||||
"encryption": "none",
|
||||
"level": 0,
|
||||
"flow": "${FLOW}"
|
||||
}
|
||||
JSON
|
||||
)
|
||||
else
|
||||
USER_BLOCK=$(cat <<JSON
|
||||
{
|
||||
"id": "${USER_ID}",
|
||||
"encryption": "none",
|
||||
"level": 0
|
||||
}
|
||||
JSON
|
||||
)
|
||||
fi
|
||||
|
||||
if [ "$TYPE" = "xhttp" ]; then
|
||||
XHTTP_HOST_LINE=""
|
||||
XHTTP_MODE_LINE=""
|
||||
if [ -n "${HOST:-}" ]; then
|
||||
XHTTP_HOST_LINE=$(cat <<JSON
|
||||
,
|
||||
"host": "${HOST}"
|
||||
JSON
|
||||
)
|
||||
fi
|
||||
if [ -n "${MODE:-}" ]; then
|
||||
XHTTP_MODE_LINE=$(cat <<JSON
|
||||
,
|
||||
"mode": "${MODE}"
|
||||
JSON
|
||||
)
|
||||
fi
|
||||
STREAM_SETTINGS_BLOCK=$(cat <<JSON
|
||||
"streamSettings": {
|
||||
"network": "xhttp",
|
||||
"security": "reality",
|
||||
"realitySettings": {
|
||||
"show": false,
|
||||
"publicKey": "${PUBKEY}",
|
||||
"shortId": "${SID}",
|
||||
"spiderX": "${SPX}",
|
||||
"fingerprint": "${FP}",
|
||||
"serverName": "${SNI}"
|
||||
},
|
||||
"xhttpSettings": {
|
||||
"path": "${XHTTP_PATH}"${XHTTP_HOST_LINE}${XHTTP_MODE_LINE}
|
||||
}
|
||||
},
|
||||
JSON
|
||||
)
|
||||
else
|
||||
STREAM_SETTINGS_BLOCK=$(cat <<JSON
|
||||
"streamSettings": {
|
||||
"network": "tcp",
|
||||
"security": "reality",
|
||||
"realitySettings": {
|
||||
"show": false,
|
||||
"publicKey": "${PUBKEY}",
|
||||
"shortId": "${SID}",
|
||||
"spiderX": "${SPX}",
|
||||
"fingerprint": "${FP}",
|
||||
"serverName": "${SNI}"
|
||||
}
|
||||
},
|
||||
JSON
|
||||
)
|
||||
fi
|
||||
|
||||
# ---- generate config.json ----
|
||||
cat > "$CFG" <<EOF
|
||||
{
|
||||
"log": { "loglevel": "debug" },
|
||||
"inbounds": [
|
||||
{
|
||||
"port": 8080,
|
||||
"protocol": "http",
|
||||
"listen": "0.0.0.0",
|
||||
"settings": { "allowTransparent": true, "timeout": 300 },
|
||||
"sniffing": { "enabled": true, "destOverride": ["http","tls"] }
|
||||
},
|
||||
{
|
||||
"port": 1080,
|
||||
"protocol": "socks",
|
||||
"listen": "0.0.0.0",
|
||||
"settings": { "auth": "noauth", "udp": true }
|
||||
}
|
||||
],
|
||||
"outbounds": [
|
||||
{
|
||||
"protocol": "vless",
|
||||
"settings": {
|
||||
"vnext": [
|
||||
{
|
||||
"address": "${SERVER}",
|
||||
"port": ${PORT},
|
||||
"users": [
|
||||
${USER_BLOCK}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
${STREAM_SETTINGS_BLOCK}
|
||||
"tag": "proxy"
|
||||
},
|
||||
{ "protocol": "freedom", "settings": {}, "tag": "direct" }
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
# ---- print generated config for debugging ----
|
||||
echo "===== GENERATED CONFIG ====="
|
||||
cat "$CFG"
|
||||
echo "============================"
|
||||
|
||||
# ---- start Xray ----
|
||||
exec /usr/local/bin/Xray run -config "$CFG"
|
||||
Reference in New Issue
Block a user