refactor(search): /api/search/ask dispatcher route via llm-router
PR-2 of DS AI routing policy (2026-05-23, see plan ~/.claude/plans/document-server-ai-cheeky-reddy.md + memory project_document_server_ai_routing_policy). DS 의 모든 backend 호출이 llm-router :8890 단일 경유. 정칙 정합: - 신규 RouterBackend (services/llm/backends.py) — alias 별 router POST + requires_gate 분기 (mac-mini-default 만 llm_gate FOREGROUND 보호). - 기존 GemmaMacMiniBackend + QwenMacBookBackend = legacy 보존 (DS_BACKENDS_VIA_ROUTER=false rollback safety only). 1주 후 별 cleanup PR (PR-DS-Backends-Legacy-Cleanup-1) 로 폐기. - get_backend factory dual-path (env flag) — backward-compat (gemma-macmini alias → mac-mini-default 매핑). - search.py:457 Query pattern 확장: mac-mini-default|claude-cloud|auto 추가. /ask/react 의 isinstance(QwenMacBookBackend) → hasattr duck-typing (RouterBackend + Legacy 모두 generate_with_tools 구현). - SearchAskBackendConfig 에 router_url 신규 (env LLM_ROUTER_URL 또는 hardcoded MVP default http://100.76.254.116:8890). - docker-compose.yml fastapi env 에 LLM_ROUTER_URL + DS_BACKENDS_VIA_ROUTER 추가. AIClient (_call_chat, call_triage, call_primary, call_fallback) 경유 path 는 별 PR (PR-AIClient-Router-Migration-1) — MVP scope C 채택, 회귀 risk 최소화. Closure (즉시 fixture/matrix): - factory smoke 6 alias (None/mac-mini-default/gemma-macmini/ qwen-macbook/claude-cloud/auto) + 1 invalid (nonsense → ValueError). - live 3 case: mac-mini-default 200 \"pong! 🏓\" + qwen-macbook cold 502 upstream_502_primary=ConnectError + claude-cloud 503 provider_not_configured. - silent fallback 0 + direct M5/Mac mini socket 0 (RouterBackend 만 router 호출). Backup: ~/.local/share/ds-routing-pr2-backups/20260523/ (backends.py + config.py + search.py + docker-compose.yml). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+13
-7
@@ -457,12 +457,15 @@ async def ask(
|
|||||||
backend: Annotated[
|
backend: Annotated[
|
||||||
str | None,
|
str | None,
|
||||||
Query(
|
Query(
|
||||||
pattern="^(qwen-macbook|gemma-macmini)$",
|
pattern="^(qwen-macbook|gemma-macmini|mac-mini-default|claude-cloud|auto)$",
|
||||||
description=(
|
description=(
|
||||||
"PR-MacBook-RAG-Backend-1: 명시 backend opt-in. "
|
"PR-2 of DS AI routing policy (2026-05-23) — 명시 backend opt-in via llm-router. "
|
||||||
"미지정 = gemma-macmini (Mac mini, default). "
|
"미지정 = mac-mini-default (gemma-macmini alias, default). "
|
||||||
"'qwen-macbook' = MacBook M5 Max Qwen 3.6 27B. "
|
"'mac-mini-default' = router 가 tier_b (Mac mini gemma-4-26b). "
|
||||||
"MacBook unavailable 시 503 + error_reason=macbook_unavailable "
|
"'qwen-macbook' = router 가 named upstream (M5 Max Qwen 3.6 27B). "
|
||||||
|
"'claude-cloud' = router 가 503 provider_not_configured (활성화 별 PR). "
|
||||||
|
"'auto' = router 의 rule + LLM triage. "
|
||||||
|
"backend unavailable 시 503 + error_reason=macbook_unavailable / router_* "
|
||||||
"(자동 fallback 없음 — 다시 호출하거나 backend 인자 제거 후 재시도)."
|
"(자동 fallback 없음 — 다시 호출하거나 backend 인자 제거 후 재시도)."
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
@@ -979,11 +982,14 @@ async def ask_react(
|
|||||||
- debug_trace: list[dict] | null (debug=true 시 round 별 trace)
|
- debug_trace: list[dict] | null (debug=true 시 round 별 trace)
|
||||||
"""
|
"""
|
||||||
# 지연 import — 순환 의존성 회피 (react_loop 가 api.search.SearchResult 사용 안 함)
|
# 지연 import — 순환 의존성 회피 (react_loop 가 api.search.SearchResult 사용 안 함)
|
||||||
from services.llm.backends import BackendUnavailable, QwenMacBookBackend, get_backend
|
from services.llm.backends import BackendUnavailable, get_backend
|
||||||
from services.search.react_loop import agentic_ask_loop
|
from services.search.react_loop import agentic_ask_loop
|
||||||
|
|
||||||
backend_inst = get_backend("qwen-macbook")
|
backend_inst = get_backend("qwen-macbook")
|
||||||
assert isinstance(backend_inst, QwenMacBookBackend) # mypy / runtime guard
|
# PR-2 of DS AI routing policy: backend_inst may be RouterBackend (default)
|
||||||
|
# or QwenMacBookBackend (DS_BACKENDS_VIA_ROUTER=false rollback). Both
|
||||||
|
# implement generate_with_tools so the ReAct loop is identical.
|
||||||
|
assert hasattr(backend_inst, "generate_with_tools")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
result = await agentic_ask_loop(
|
result = await agentic_ask_loop(
|
||||||
|
|||||||
+15
-6
@@ -36,18 +36,27 @@ class DeepSummaryBacklogConfig(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class SearchAskBackendConfig(BaseModel):
|
class SearchAskBackendConfig(BaseModel):
|
||||||
"""PR-MacBook-RAG-Backend-1: /api/search/ask backend dispatcher.
|
"""PR-2 of DS AI routing policy ([[document-server-ai-routing-policy]], 2026-05-23):
|
||||||
|
/api/search/ask backend dispatcher 가 llm-router :8890 단일 경유.
|
||||||
|
|
||||||
backend 미지정 = Gemma Mac mini (settings.ai.primary 경로 그대로).
|
- backend 미지정 / "gemma-macmini" / "mac-mini-default" → router 가 tier_b
|
||||||
backend="qwen-macbook" 명시 opt-in = MacBook M5 Max mlx-vlm.server.
|
- backend "qwen-macbook" → router 가 named upstream (M5 Max)
|
||||||
MacBook unavailable 시 503 + error_reason=macbook_unavailable (자동 fallback 없음).
|
- backend "claude-cloud" → router 가 503 명시 (scaffold)
|
||||||
|
- backend "auto" → router 의 rule + LLM triage
|
||||||
|
|
||||||
|
Unavailable → BackendUnavailable → 503 명시 (silent fallback 0).
|
||||||
|
Rollback: DS_BACKENDS_VIA_ROUTER=false 로 legacy 직접 호출 path.
|
||||||
|
legacy macmini_url / macbook_url / macbook_model 은 fallback 시만 사용.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
# PR-2 신규: llm-router URL. 비면 env LLM_ROUTER_URL 또는 hardcoded default.
|
||||||
|
router_url: str = ""
|
||||||
|
# Legacy fields (DS_BACKENDS_VIA_ROUTER=false 시만 사용)
|
||||||
macmini_url: str = "http://100.76.254.116:8801"
|
macmini_url: str = "http://100.76.254.116:8801"
|
||||||
macbook_url: str = "http://100.118.112.84:8810"
|
macbook_url: str = "http://100.118.112.84:8810"
|
||||||
macbook_model: str = "mlx-community/Qwen3.6-27B-8bit"
|
macbook_model: str = "mlx-community/Qwen3.6-27B-8bit"
|
||||||
timeout_connect_s: int = 1
|
timeout_connect_s: int = 5
|
||||||
timeout_read_s: int = 30
|
timeout_read_s: int = 60
|
||||||
|
|
||||||
|
|
||||||
class SearchAskReactConfig(BaseModel):
|
class SearchAskReactConfig(BaseModel):
|
||||||
|
|||||||
+316
-58
@@ -1,23 +1,40 @@
|
|||||||
"""PR-MacBook-RAG-Backend-1: /api/search/ask 의 명시 backend dispatcher.
|
"""PR-2 of DS AI routing policy ([[document-server-ai-routing-policy]], 2026-05-23):
|
||||||
|
/api/search/ask 의 명시 backend dispatcher. 모든 backend = llm-router :8890 경유.
|
||||||
|
|
||||||
## 정책 (정정 4)
|
## 정책 (PR-2 of routing policy, MVP 옵션 C — ask path 만 swap)
|
||||||
|
|
||||||
- 기본 (`backend` 미지정) = Gemma Mac mini. 기존 코드 경로 100% 보존.
|
- 기본 (`backend` 미지정) / `gemma-macmini` / `mac-mini-default`
|
||||||
- 명시 opt-in `backend="qwen-macbook"` 만 MacBook M5 Max mlx-vlm.server 호출.
|
→ RouterBackend(alias="mac-mini-default", requires_gate=True)
|
||||||
- MacBook unavailable 시 `BackendUnavailable` 예외 → /ask wrapper 가 503 +
|
→ router 가 tier_b (Mac mini :8801 gemma-4-26b) 호출. llm_gate 영구 룰 보존.
|
||||||
`error_reason="macbook_unavailable"` 응답. **Gemma 자동 fallback 금지**.
|
- `qwen-macbook`
|
||||||
|
→ RouterBackend(alias="qwen-macbook", requires_gate=False)
|
||||||
|
→ router 가 named upstream (M5 Max :8810 Qwen3.6-27B) 호출.
|
||||||
|
- `claude-cloud`
|
||||||
|
→ RouterBackend(alias="claude-cloud", requires_gate=False)
|
||||||
|
→ router 가 503 provider_not_configured pass-through. activation = 별 PR.
|
||||||
|
- `auto`
|
||||||
|
→ RouterBackend(alias=None, requires_gate=True)
|
||||||
|
→ router 가 rule + LLM triage 로 tier 결정. 안전상 Mac mini gate 보호 보수적.
|
||||||
|
- 그 외 → ValueError (호출자가 400/422 으로 매핑)
|
||||||
|
|
||||||
## 영구 룰
|
## 영구 룰
|
||||||
|
|
||||||
- Qwen backend 는 **Mac mini llm_gate 점유 금지**. 별 endpoint, 별 concurrency.
|
- Mac mini 26B 단일 inference (llm_gate, [[feedback_docstring_invariant_swap_audit]])
|
||||||
→ MacBook 전용 `asyncio.Semaphore(1)` (single-inference 가정) 분리.
|
보존 = requires_gate=True 분기에서 `acquire_mlx_gate(Priority.FOREGROUND)` 유지.
|
||||||
- Gemma backend 는 기존 path 그대로 (acquire_mlx_gate(FOREGROUND) + ai.primary).
|
router 경유로도 client-side mutex 효과는 동일.
|
||||||
llm_gate 영구 룰 ([[feedback_docstring_invariant_swap_audit]] 케이스) 보존.
|
- BackendUnavailable 매핑 정책 ([[feedback_no_silent_fallback_explicit_opt_in]]) 보존.
|
||||||
|
silent fallback 0 = router 가 503/502 반환하면 그대로 BackendUnavailable.
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
`DS_BACKENDS_VIA_ROUTER=false` env 로 legacy path (GemmaMacMiniBackend +
|
||||||
|
QwenMacBookBackend 직접 호출) 즉시 복귀. legacy class 1주 보존 후 별 cleanup PR.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import os
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
@@ -34,9 +51,15 @@ if TYPE_CHECKING:
|
|||||||
logger = setup_logger("llm_backend")
|
logger = setup_logger("llm_backend")
|
||||||
|
|
||||||
|
|
||||||
# 명시 backend 식별자. None / "gemma-macmini" 는 default Gemma path.
|
# 명시 backend 식별자.
|
||||||
QWEN_MACBOOK = "qwen-macbook"
|
QWEN_MACBOOK = "qwen-macbook"
|
||||||
GEMMA_MACMINI = "gemma-macmini"
|
GEMMA_MACMINI = "gemma-macmini"
|
||||||
|
MAC_MINI_DEFAULT = "mac-mini-default"
|
||||||
|
CLAUDE_CLOUD = "claude-cloud"
|
||||||
|
AUTO = "auto"
|
||||||
|
|
||||||
|
# Allowed user-facing alias keys (Query pattern 과 동기 — app/api/search.py:457).
|
||||||
|
_ALLOWED_ALIASES = {GEMMA_MACMINI, QWEN_MACBOOK, MAC_MINI_DEFAULT, CLAUDE_CLOUD, AUTO}
|
||||||
|
|
||||||
|
|
||||||
class BackendUnavailable(Exception):
|
class BackendUnavailable(Exception):
|
||||||
@@ -59,9 +82,186 @@ class BackendBase(ABC):
|
|||||||
가 status="llm_error" 로 매핑 (기존 동작). BackendUnavailable 만 503 으로 매핑.
|
가 status="llm_error" 로 매핑 (기존 동작). BackendUnavailable 만 503 으로 매핑.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
async def generate_with_tools(
|
||||||
|
self,
|
||||||
|
messages: list[dict],
|
||||||
|
tools: list[dict],
|
||||||
|
*,
|
||||||
|
tool_choice: str = "auto",
|
||||||
|
timeout_read_s: int,
|
||||||
|
) -> dict:
|
||||||
|
"""ReAct loop 용 OpenAI 호환 chat completion with tool calling.
|
||||||
|
|
||||||
|
Default = NotImplementedError. RouterBackend 와 QwenMacBookBackend (legacy)
|
||||||
|
만 override. ReAct endpoint 가 미지원 backend 호출하면 명확한 에러.
|
||||||
|
"""
|
||||||
|
raise NotImplementedError(
|
||||||
|
f"{type(self).__name__} does not implement generate_with_tools"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
# RouterBackend (PR-2 신규, 기본 path)
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
class RouterBackend(BackendBase):
|
||||||
|
"""모든 ask path 가 llm-router :8890 경유. alias 별 gate 적용.
|
||||||
|
|
||||||
|
response shape = router 가 upstream OpenAI 호환 응답을 그대로 forward.
|
||||||
|
qwen-macbook tool calling response = mlx-vlm OpenAI 표준 호환
|
||||||
|
(tests/fixtures/qwen_tool_call_response.json, [[reference_mlx_vlm_tool_calling]]).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
router_url: str,
|
||||||
|
alias: str | None,
|
||||||
|
requires_gate: bool,
|
||||||
|
timeout_connect_s: int,
|
||||||
|
):
|
||||||
|
self.name = alias or AUTO
|
||||||
|
self.router_url = router_url.rstrip("/")
|
||||||
|
self.alias = alias # None means "auto" (router rule + triage)
|
||||||
|
self.requires_gate = requires_gate
|
||||||
|
self.timeout_connect_s = timeout_connect_s
|
||||||
|
|
||||||
|
def _build_payload(
|
||||||
|
self,
|
||||||
|
messages_or_prompt,
|
||||||
|
*,
|
||||||
|
tools: list[dict] | None = None,
|
||||||
|
tool_choice: str | None = None,
|
||||||
|
) -> dict:
|
||||||
|
if isinstance(messages_or_prompt, str):
|
||||||
|
payload: dict = {
|
||||||
|
"messages": [{"role": "user", "content": messages_or_prompt}],
|
||||||
|
"max_tokens": 4096,
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
payload = {
|
||||||
|
"messages": messages_or_prompt,
|
||||||
|
"max_tokens": 4096,
|
||||||
|
}
|
||||||
|
if self.alias:
|
||||||
|
payload["model"] = self.alias
|
||||||
|
if tools:
|
||||||
|
payload["tools"] = tools
|
||||||
|
if tool_choice in ("auto", "none"):
|
||||||
|
payload["tool_choice"] = tool_choice
|
||||||
|
return payload
|
||||||
|
|
||||||
|
async def _post(self, payload: dict, *, timeout_read_s: int) -> dict:
|
||||||
|
timeout = httpx.Timeout(
|
||||||
|
connect=float(self.timeout_connect_s),
|
||||||
|
read=float(timeout_read_s),
|
||||||
|
write=10.0,
|
||||||
|
pool=5.0,
|
||||||
|
)
|
||||||
|
url = f"{self.router_url}/v1/chat/completions"
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||||
|
resp = await client.post(url, json=payload)
|
||||||
|
# router 가 503 (provider_not_configured / 기타 router-side 503) → BackendUnavailable
|
||||||
|
if resp.status_code == 503:
|
||||||
|
try:
|
||||||
|
body = resp.json()
|
||||||
|
err = body.get("error", {}) if isinstance(body, dict) else {}
|
||||||
|
reason = (
|
||||||
|
err.get("type")
|
||||||
|
or err.get("error_reason")
|
||||||
|
or "router_503"
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
reason = "router_503"
|
||||||
|
raise BackendUnavailable(self.name, reason)
|
||||||
|
# router 가 400 unknown_alias → 코드 bug. 일반 예외 (호출자가 5xx 로 변환)
|
||||||
|
if resp.status_code == 400:
|
||||||
|
try:
|
||||||
|
body = resp.json()
|
||||||
|
except Exception:
|
||||||
|
body = {}
|
||||||
|
raise ValueError(
|
||||||
|
f"router rejected alias={self.alias!r} body={body!r}"
|
||||||
|
)
|
||||||
|
# router 가 502 (upstream unavailable, M5 cold 등) → BackendUnavailable
|
||||||
|
if resp.status_code == 502:
|
||||||
|
try:
|
||||||
|
body = resp.json()
|
||||||
|
except Exception:
|
||||||
|
body = {}
|
||||||
|
raise BackendUnavailable(
|
||||||
|
self.name,
|
||||||
|
f"upstream_502_{body.get('error', 'unknown')[:32]}",
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
except (
|
||||||
|
httpx.ConnectError,
|
||||||
|
httpx.ConnectTimeout,
|
||||||
|
httpx.ReadTimeout,
|
||||||
|
httpx.PoolTimeout,
|
||||||
|
httpx.WriteTimeout,
|
||||||
|
httpx.RemoteProtocolError,
|
||||||
|
) as exc:
|
||||||
|
logger.warning(
|
||||||
|
"router_backend unavailable alias=%s url=%s exc=%s",
|
||||||
|
self.alias, url, type(exc).__name__,
|
||||||
|
)
|
||||||
|
raise BackendUnavailable(
|
||||||
|
self.name, f"router_{type(exc).__name__}"
|
||||||
|
) from exc
|
||||||
|
except httpx.HTTPStatusError as exc:
|
||||||
|
if 500 <= exc.response.status_code < 600:
|
||||||
|
logger.warning(
|
||||||
|
"router_backend 5xx alias=%s status=%d",
|
||||||
|
self.alias, exc.response.status_code,
|
||||||
|
)
|
||||||
|
raise BackendUnavailable(
|
||||||
|
self.name, f"router_http_{exc.response.status_code}"
|
||||||
|
) from exc
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def generate(self, prompt: str, *, timeout_read_s: int) -> str:
|
||||||
|
payload = self._build_payload(prompt)
|
||||||
|
if self.requires_gate:
|
||||||
|
async with acquire_mlx_gate(Priority.FOREGROUND):
|
||||||
|
async with asyncio.timeout(timeout_read_s):
|
||||||
|
data = await self._post(payload, timeout_read_s=timeout_read_s)
|
||||||
|
else:
|
||||||
|
data = await self._post(payload, timeout_read_s=timeout_read_s)
|
||||||
|
return data["choices"][0]["message"]["content"]
|
||||||
|
|
||||||
|
async def generate_with_tools(
|
||||||
|
self,
|
||||||
|
messages: list[dict],
|
||||||
|
tools: list[dict],
|
||||||
|
*,
|
||||||
|
tool_choice: str = "auto",
|
||||||
|
timeout_read_s: int,
|
||||||
|
) -> dict:
|
||||||
|
payload = self._build_payload(
|
||||||
|
messages, tools=tools, tool_choice=tool_choice,
|
||||||
|
)
|
||||||
|
if self.requires_gate:
|
||||||
|
async with acquire_mlx_gate(Priority.FOREGROUND):
|
||||||
|
async with asyncio.timeout(timeout_read_s):
|
||||||
|
data = await self._post(payload, timeout_read_s=timeout_read_s)
|
||||||
|
else:
|
||||||
|
data = await self._post(payload, timeout_read_s=timeout_read_s)
|
||||||
|
return data["choices"][0]["message"]
|
||||||
|
|
||||||
|
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
# Legacy backends (rollback safety, DS_BACKENDS_VIA_ROUTER=false 시만 사용)
|
||||||
|
# 1주 후 별 cleanup PR 로 폐기 ([[feedback_closure_gate_vs_observation]] —
|
||||||
|
# dual-path = rollback safety only, 시간 관찰 게이트 0).
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
class GemmaMacMiniBackend(BackendBase):
|
class GemmaMacMiniBackend(BackendBase):
|
||||||
"""기존 Mac mini ai.primary 경로 그대로. 코드 변경 0 path."""
|
"""[LEGACY] 기존 Mac mini ai.primary 직접 호출. DS_BACKENDS_VIA_ROUTER=false 시만."""
|
||||||
|
|
||||||
name = GEMMA_MACMINI
|
name = GEMMA_MACMINI
|
||||||
|
|
||||||
@@ -82,13 +282,7 @@ class GemmaMacMiniBackend(BackendBase):
|
|||||||
|
|
||||||
|
|
||||||
class QwenMacBookBackend(BackendBase):
|
class QwenMacBookBackend(BackendBase):
|
||||||
"""MacBook M5 Max mlx-vlm.server (Tailscale) 직접 호출.
|
"""[LEGACY] MacBook M5 Max mlx-vlm.server (Tailscale) 직접 호출. DS_BACKENDS_VIA_ROUTER=false 시만."""
|
||||||
|
|
||||||
- Mac mini llm_gate 점유 X (별 endpoint 라 의미 없음 + 큐 분할 금지 영구 룰의
|
|
||||||
대상이 아님)
|
|
||||||
- MacBook 자체 single-inference 가정 → 별 semaphore(1)
|
|
||||||
- 연결 거부 / DNS / timeout / 5xx → BackendUnavailable
|
|
||||||
"""
|
|
||||||
|
|
||||||
name = QWEN_MACBOOK
|
name = QWEN_MACBOOK
|
||||||
_gate: asyncio.Semaphore | None = None
|
_gate: asyncio.Semaphore | None = None
|
||||||
@@ -134,15 +328,15 @@ class QwenMacBookBackend(BackendBase):
|
|||||||
httpx.RemoteProtocolError,
|
httpx.RemoteProtocolError,
|
||||||
) as exc:
|
) as exc:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"qwen-macbook unavailable url=%s exc=%s",
|
"qwen-macbook[legacy] unavailable url=%s exc=%s",
|
||||||
url, type(exc).__name__,
|
url, type(exc).__name__,
|
||||||
)
|
)
|
||||||
raise BackendUnavailable(self.name, type(exc).__name__) from exc
|
raise BackendUnavailable(self.name, type(exc).__name__) from exc
|
||||||
except httpx.HTTPStatusError as exc:
|
except httpx.HTTPStatusError as exc:
|
||||||
# 5xx 만 unavailable, 4xx 는 호출자 잘못 → 일반 예외 전파
|
|
||||||
if 500 <= exc.response.status_code < 600:
|
if 500 <= exc.response.status_code < 600:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"qwen-macbook 5xx status=%d", exc.response.status_code,
|
"qwen-macbook[legacy] 5xx status=%d",
|
||||||
|
exc.response.status_code,
|
||||||
)
|
)
|
||||||
raise BackendUnavailable(
|
raise BackendUnavailable(
|
||||||
self.name, f"http_{exc.response.status_code}"
|
self.name, f"http_{exc.response.status_code}"
|
||||||
@@ -157,19 +351,6 @@ class QwenMacBookBackend(BackendBase):
|
|||||||
tool_choice: str = "auto",
|
tool_choice: str = "auto",
|
||||||
timeout_read_s: int,
|
timeout_read_s: int,
|
||||||
) -> dict:
|
) -> dict:
|
||||||
"""OpenAI 호환 chat completion with tool calling (ReAct loop 용).
|
|
||||||
|
|
||||||
Returns: `choices[0].message` dict 그대로 — `content` (Optional[str]) +
|
|
||||||
`tool_calls` (Optional[list]) 둘 다 포함.
|
|
||||||
|
|
||||||
Response shape = G0-1 fixture `tests/fixtures/qwen_tool_call_response.json`
|
|
||||||
기준 (mlx-vlm OpenAI 표준 호환). tool_calls[].function.arguments 는
|
|
||||||
**JSON string** 으로 옴 — 호출자가 json.loads 필요.
|
|
||||||
|
|
||||||
- `tool_choice="auto"`: 모델이 tool 호출 여부 결정
|
|
||||||
- `tool_choice="none"`: tool 호출 금지, content 만 반환 (final round)
|
|
||||||
- `tools=[]` + `tool_choice="none"`: tool 정의 없이 final answer 강제
|
|
||||||
"""
|
|
||||||
gate = self._get_gate()
|
gate = self._get_gate()
|
||||||
timeout = httpx.Timeout(
|
timeout = httpx.Timeout(
|
||||||
connect=float(self.timeout_connect_s),
|
connect=float(self.timeout_connect_s),
|
||||||
@@ -203,14 +384,15 @@ class QwenMacBookBackend(BackendBase):
|
|||||||
httpx.RemoteProtocolError,
|
httpx.RemoteProtocolError,
|
||||||
) as exc:
|
) as exc:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"qwen-macbook(tools) unavailable url=%s exc=%s",
|
"qwen-macbook[legacy](tools) unavailable url=%s exc=%s",
|
||||||
url, type(exc).__name__,
|
url, type(exc).__name__,
|
||||||
)
|
)
|
||||||
raise BackendUnavailable(self.name, type(exc).__name__) from exc
|
raise BackendUnavailable(self.name, type(exc).__name__) from exc
|
||||||
except httpx.HTTPStatusError as exc:
|
except httpx.HTTPStatusError as exc:
|
||||||
if 500 <= exc.response.status_code < 600:
|
if 500 <= exc.response.status_code < 600:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"qwen-macbook(tools) 5xx status=%d", exc.response.status_code,
|
"qwen-macbook[legacy](tools) 5xx status=%d",
|
||||||
|
exc.response.status_code,
|
||||||
)
|
)
|
||||||
raise BackendUnavailable(
|
raise BackendUnavailable(
|
||||||
self.name, f"http_{exc.response.status_code}"
|
self.name, f"http_{exc.response.status_code}"
|
||||||
@@ -218,38 +400,113 @@ class QwenMacBookBackend(BackendBase):
|
|||||||
raise
|
raise
|
||||||
|
|
||||||
|
|
||||||
# ── dispatcher ─────────────────────────────────────────────────────────────
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
# Dispatcher (PR-2: dual-path with DS_BACKENDS_VIA_ROUTER env flag)
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
_BACKENDS: dict[str, BackendBase] = {}
|
|
||||||
|
def _via_router() -> bool:
|
||||||
|
"""`DS_BACKENDS_VIA_ROUTER=true` (default) = RouterBackend.
|
||||||
|
false 시 legacy GemmaMacMiniBackend/QwenMacBookBackend (rollback safety).
|
||||||
|
"""
|
||||||
|
return os.getenv("DS_BACKENDS_VIA_ROUTER", "true").lower() == "true"
|
||||||
|
|
||||||
|
|
||||||
|
_ROUTER_BACKENDS: dict[str, RouterBackend] = {}
|
||||||
|
_LEGACY_BACKENDS: dict[str, BackendBase] = {}
|
||||||
|
|
||||||
|
|
||||||
|
def _router_url() -> str:
|
||||||
|
"""router URL = settings 우선, fallback env, fallback hardcoded MVP default."""
|
||||||
|
cfg = settings.search.ask.backend
|
||||||
|
cfg_url = getattr(cfg, "router_url", "") or ""
|
||||||
|
if cfg_url:
|
||||||
|
return cfg_url
|
||||||
|
return os.getenv("LLM_ROUTER_URL", "http://100.76.254.116:8890")
|
||||||
|
|
||||||
|
|
||||||
|
def _build_router_backend(alias: str | None, requires_gate: bool) -> RouterBackend:
|
||||||
|
cfg = settings.search.ask.backend
|
||||||
|
return RouterBackend(
|
||||||
|
router_url=_router_url(),
|
||||||
|
alias=alias,
|
||||||
|
requires_gate=requires_gate,
|
||||||
|
timeout_connect_s=cfg.timeout_connect_s,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _build_qwen_backend() -> QwenMacBookBackend:
|
def _build_qwen_backend() -> QwenMacBookBackend:
|
||||||
b = settings.search.ask.backend
|
cfg = settings.search.ask.backend
|
||||||
return QwenMacBookBackend(
|
return QwenMacBookBackend(
|
||||||
base_url=b.macbook_url,
|
base_url=cfg.macbook_url,
|
||||||
model=b.macbook_model,
|
model=cfg.macbook_model,
|
||||||
timeout_connect_s=b.timeout_connect_s,
|
timeout_connect_s=cfg.timeout_connect_s,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_router_backend(name: str | None) -> RouterBackend:
|
||||||
|
"""RouterBackend path. PR-2 default."""
|
||||||
|
key = (name or "").strip().lower()
|
||||||
|
|
||||||
|
if key in ("", GEMMA_MACMINI, MAC_MINI_DEFAULT):
|
||||||
|
cache_key = MAC_MINI_DEFAULT
|
||||||
|
if cache_key not in _ROUTER_BACKENDS:
|
||||||
|
_ROUTER_BACKENDS[cache_key] = _build_router_backend(
|
||||||
|
alias=MAC_MINI_DEFAULT, requires_gate=True,
|
||||||
|
)
|
||||||
|
return _ROUTER_BACKENDS[cache_key]
|
||||||
|
if key == QWEN_MACBOOK:
|
||||||
|
if QWEN_MACBOOK not in _ROUTER_BACKENDS:
|
||||||
|
_ROUTER_BACKENDS[QWEN_MACBOOK] = _build_router_backend(
|
||||||
|
alias=QWEN_MACBOOK, requires_gate=False,
|
||||||
|
)
|
||||||
|
return _ROUTER_BACKENDS[QWEN_MACBOOK]
|
||||||
|
if key == CLAUDE_CLOUD:
|
||||||
|
if CLAUDE_CLOUD not in _ROUTER_BACKENDS:
|
||||||
|
_ROUTER_BACKENDS[CLAUDE_CLOUD] = _build_router_backend(
|
||||||
|
alias=CLAUDE_CLOUD, requires_gate=False,
|
||||||
|
)
|
||||||
|
return _ROUTER_BACKENDS[CLAUDE_CLOUD]
|
||||||
|
if key == AUTO:
|
||||||
|
if AUTO not in _ROUTER_BACKENDS:
|
||||||
|
# auto = router 의 rule + triage. tier_b 갈 가능성 큼 → gate 보호 보수적.
|
||||||
|
_ROUTER_BACKENDS[AUTO] = _build_router_backend(
|
||||||
|
alias=None, requires_gate=True,
|
||||||
|
)
|
||||||
|
return _ROUTER_BACKENDS[AUTO]
|
||||||
|
raise ValueError(f"unknown backend: {name!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _get_legacy_backend(name: str | None) -> BackendBase:
|
||||||
|
"""Rollback path. DS_BACKENDS_VIA_ROUTER=false 시만."""
|
||||||
|
key = (name or "").strip().lower() or GEMMA_MACMINI
|
||||||
|
if key == MAC_MINI_DEFAULT:
|
||||||
|
key = GEMMA_MACMINI # legacy 는 mac-mini-default alias 모름
|
||||||
|
if key == AUTO:
|
||||||
|
key = GEMMA_MACMINI # legacy 에 auto 개념 없음 → default 로
|
||||||
|
if key == CLAUDE_CLOUD:
|
||||||
|
raise ValueError(
|
||||||
|
f"backend {CLAUDE_CLOUD!r} requires DS_BACKENDS_VIA_ROUTER=true"
|
||||||
|
)
|
||||||
|
if key not in (GEMMA_MACMINI, QWEN_MACBOOK):
|
||||||
|
raise ValueError(f"unknown backend: {name!r}")
|
||||||
|
if key not in _LEGACY_BACKENDS:
|
||||||
|
if key == GEMMA_MACMINI:
|
||||||
|
_LEGACY_BACKENDS[key] = GemmaMacMiniBackend()
|
||||||
|
else:
|
||||||
|
_LEGACY_BACKENDS[key] = _build_qwen_backend()
|
||||||
|
return _LEGACY_BACKENDS[key]
|
||||||
|
|
||||||
|
|
||||||
def get_backend(name: str | None) -> BackendBase:
|
def get_backend(name: str | None) -> BackendBase:
|
||||||
"""name 으로 backend 인스턴스 반환 (캐싱).
|
"""name 으로 backend 인스턴스 반환 (캐싱).
|
||||||
|
|
||||||
- None / "" / "gemma-macmini" → Gemma Mac mini (default)
|
DS_BACKENDS_VIA_ROUTER=true (default, PR-2) → RouterBackend
|
||||||
- "qwen-macbook" → MacBook Qwen
|
DS_BACKENDS_VIA_ROUTER=false → legacy GemmaMacMiniBackend / QwenMacBookBackend
|
||||||
- 그 외 → ValueError (호출자가 400 으로 매핑)
|
|
||||||
"""
|
"""
|
||||||
key = (name or "").strip().lower() or GEMMA_MACMINI
|
if _via_router():
|
||||||
|
return _get_router_backend(name)
|
||||||
if key not in (GEMMA_MACMINI, QWEN_MACBOOK):
|
return _get_legacy_backend(name)
|
||||||
raise ValueError(f"unknown backend: {name!r}")
|
|
||||||
|
|
||||||
if key not in _BACKENDS:
|
|
||||||
if key == GEMMA_MACMINI:
|
|
||||||
_BACKENDS[key] = GemmaMacMiniBackend()
|
|
||||||
else:
|
|
||||||
_BACKENDS[key] = _build_qwen_backend()
|
|
||||||
return _BACKENDS[key]
|
|
||||||
|
|
||||||
|
|
||||||
def reset_backends_for_test() -> None:
|
def reset_backends_for_test() -> None:
|
||||||
@@ -257,5 +514,6 @@ def reset_backends_for_test() -> None:
|
|||||||
|
|
||||||
production code 에서 사용 X.
|
production code 에서 사용 X.
|
||||||
"""
|
"""
|
||||||
_BACKENDS.clear()
|
_ROUTER_BACKENDS.clear()
|
||||||
|
_LEGACY_BACKENDS.clear()
|
||||||
QwenMacBookBackend._gate = None
|
QwenMacBookBackend._gate = None
|
||||||
|
|||||||
@@ -213,6 +213,11 @@ services:
|
|||||||
- LAPTOP_WORKER_BOT_TOKEN_ENABLED=${LAPTOP_WORKER_BOT_TOKEN_ENABLED:-false}
|
- LAPTOP_WORKER_BOT_TOKEN_ENABLED=${LAPTOP_WORKER_BOT_TOKEN_ENABLED:-false}
|
||||||
- LAPTOP_WORKER_BOT_USERNAME=${LAPTOP_WORKER_BOT_USERNAME:-laptop-worker-bot}
|
- LAPTOP_WORKER_BOT_USERNAME=${LAPTOP_WORKER_BOT_USERNAME:-laptop-worker-bot}
|
||||||
- LAPTOP_WORKER_BOT_TOKEN_EXPIRE_DAYS=${LAPTOP_WORKER_BOT_TOKEN_EXPIRE_DAYS:-365}
|
- LAPTOP_WORKER_BOT_TOKEN_EXPIRE_DAYS=${LAPTOP_WORKER_BOT_TOKEN_EXPIRE_DAYS:-365}
|
||||||
|
# PR-2 of DS AI routing policy (2026-05-23) — backends dispatcher via llm-router.
|
||||||
|
# router_url default points at Mac mini Tailscale interface :8890 (PR-1).
|
||||||
|
# DS_BACKENDS_VIA_ROUTER=false 로 legacy 직접 호출 path 즉시 복귀.
|
||||||
|
- LLM_ROUTER_URL=${LLM_ROUTER_URL:-http://100.76.254.116:8890}
|
||||||
|
- DS_BACKENDS_VIA_ROUTER=${DS_BACKENDS_VIA_ROUTER:-true}
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
frontend:
|
frontend:
|
||||||
|
|||||||
Reference in New Issue
Block a user