feat(eval): Phase 2B Reranker Diagnose — dispatcher + gte 측정 + decision (H3 bge-reranker-v2-m3 유지)

round-2-review-mighty-starfish.md v2.1 (Phase 2B Reranker Diagnose) plan 실행.
Phase 2A 의 CANDIDATE_BACKEND_MAP 패턴 재사용 + RERANKER_BACKEND_MAP 신규.

코드 변경 (4 파일):
- app/services/search/rerank_service.py:
  - RERANKER_BACKEND_MAP allowlist (baseline / cand_gte_ml_base, slug-based resolve)
  - _resolve_reranker(slug) → endpoint URL or None
  - _rerank_via_candidate_endpoint() — 후보 TEI POST /rerank
  - rerank_chunks() 시그니처에 reranker_backend + snapshot_*_id_max 추가 + dispatch log
- app/services/search/search_pipeline.py: run_search() threading
- app/api/search.py: reranker_backend Query parameter + 400 unknown_reranker_backend 에러 매핑
- tests/search_eval/run_eval.py: --reranker-backend flag + call_search/evaluate threading

infra:
- docker-compose.override.rerank-cand.yml: 3 후보 service (gte_ml_base / mxbai_large / bge_v2_gemma_2b),
  profile 'rerank-cand' 격리, restart=unless-stopped

측정 산출물 (51 case, scored=46, failure=5):
- reports/v0_2_phase2b_baseline_snapshot_2026-05-23.csv (NDCG 0.659, Phase 2A 와 일치 = 재현성 PASS)
- reports/v0_2_phase2b_gte_ml_base_2026-05-23.csv
- tests/search_eval/baselines/v0_2_phase2b_{baseline_snapshot,gte_ml_base}_2026-05-23.json
- reports/phase_2b_reranker_decision_2026-05-23.md
- tests/fixtures/tei_rerank_response.json (G0-1 한국어+영어 mixed sample sanity PASS)

후보 TEI 1.7 호환성 (Phase 1 smoke gate):
- cand_gte_ml_base       : ✅ PASS (xlm-roberta-based, TEI 호환)
- cand_mxbai_large       : ❌ deberta-v2 미지원 → Phase 2B-Extended (sentence-transformers wrapper)
- cand_bge_v2_gemma_2b   : ❌ LLM-based reranker, 1_Pooling/config.json 부재 → Phase 2B-Extended (FlagEmbedding wrapper)

결과 (1 후보 측정 + baseline rebaseline):
| Candidate                          | NDCG  | Δ baseline | mixed | korean | exam  | p50 ms |
|------------------------------------|------:|-----------:|------:|-------:|------:|-------:|
| bge-reranker-v2-m3 (baseline)      | 0.659 | —          | 0.39  | 0.51   | 0.74  | 454    |
| cand_gte_ml_base                   | 0.604 | -0.055     | 0.38  | 0.41   | 0.62  | 345    |

Decision (H3): bge-reranker-v2-m3 유지. gte 의 reranker quality 가 production 보다 약함 (korean_only -0.10, exam -0.12, overall -0.055).

후속 PR 백로그 (6건):
- PR-Search-Query-Rewrite-1 (Phase 2Q, korean_only/mixed 보완 권고)
- PR-2B-Extended-Mxbai-Large (sentence-transformers wrapper)
- PR-2B-Extended-Bge-V2-Gemma (FlagEmbedding LayerwiseReranker wrapper)
- PR-2B-Extended-Jina-V2-ML (license 결정 후, 개인 비영리 가정)
- PR-2B-Cloud-Reranker-Scaffold-1 (Cohere scaffold-only, 선택)
- PR-2B-Rerank-Cand-Cleanup-1 (1주 후 cand 컨테이너 정리)

production 영향:
- production reranker (bge-reranker-v2-m3) 변경 0
- config.yaml ai.models.rerank.endpoint 변경 0
- embedding (bge-m3 ollama) 변경 0 (Phase 2A 결정 보존)
- documents / document_chunks 변경 0 (21365 docs / 30605 chunks 그대로)
- 4 smoke PASS (baseline / baseline+snapshot / cand_gte_ml_base / cand_invalid → 400)
- dispatch log 박제 verify (endpoint + snapshot id)

closure gate: 16 항목 PASS (flex closure 조항 적용 — 1 후보 측정, 2 후보 TEI 호환 탈락 사유 명시).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

tests/fixtures/tei_rerank_response.json

@@ -0,0 +1,46 @@
+{
+  "fixture_purpose": "Phase 2B G0-1 — TEI rerank endpoint 응답 spec 박제. mixed (한국어+영어) sanity check.",
+  "request": {
+    "endpoint_examples": [
+      "http://reranker:80/rerank (production baseline bge-reranker-v2-m3)",
+      "http://rerank-cand-gte-ml-base:80/rerank",
+      "http://rerank-cand-mxbai-large:80/rerank",
+      "http://rerank-cand-bge-v2-gemma-2b:80/rerank"
+    ],
+    "method": "POST",
+    "headers": {"Content-Type": "application/json"},
+    "body": {
+      "query": "압력용기 설계 기준",
+      "texts": [
+        "ASME Section VIII Division 1 pressure vessel design rules and material selection criteria for high-pressure applications.",
+        "고압가스 안전관리법에 따른 압력용기 검사 기준 — 정기 검사 주기 및 안전 밸브 설정.",
+        "Today weather forecast for Seoul: partly cloudy with chance of rain in the afternoon."
+      ]
+    }
+  },
+  "response_shape": "[{index: int, score: float}, ...] sorted score desc",
+  "captured_responses": {
+    "baseline_bge_v2_m3": {
+      "endpoint": "http://reranker:80/rerank",
+      "model": "BAAI/bge-reranker-v2-m3 (production)",
+      "raw": [
+        {"index": 0, "score": 0.9091032},
+        {"index": 1, "score": 0.7514658},
+        {"index": 2, "score": 0.0000165714}
+      ],
+      "interpretation": "ASME(en)+고압가스(ko) 둘 다 무관(weather) 보다 명확 높음. 한국어/영어 score gap 작음 (0.91 vs 0.75 = 0.16) — 한국어 능력 강함."
+    },
+    "cand_gte_ml_base": {
+      "endpoint": "http://rerank-cand-gte-ml-base:80/rerank",
+      "model": "Alibaba-NLP/gte-multilingual-reranker-base",
+      "raw": [
+        {"index": 0, "score": 0.6365791},
+        {"index": 1, "score": 0.4685475},
+        {"index": 2, "score": 0.034488525}
+      ],
+      "interpretation": "ASME(en)+고압가스(ko) 둘 다 weather 보다 명확 높음. 한국어/영어 score gap 0.17 — baseline 과 비슷. score 절대값 baseline 보다 낮음 (model 별 calibration 차이, rank 순서는 동일)."
+    }
+  },
+  "sanity_check": "ASME(en) > 고압가스(ko) > weather(noise) 순서 — 두 모델 모두 통과. 후보가 한국어 무관하지 않은지 검증.",
+  "captured_at": "2026-05-23"
+}

tests/search_eval/baselines/v0_2_phase2b_baseline_snapshot_2026-05-23.json

@@ -0,0 +1,48 @@
+{
+  "version": "v0.2-phase2b",
+  "label": "baseline_snapshot",
+  "date": "2026-05-23",
+  "snapshot": {
+    "doc_id_max": 25180,
+    "chunk_id_max": 56526,
+    "documents_n": 21365,
+    "chunks_n": 30605
+  },
+  "eval_set": {
+    "total_cases": 51,
+    "scored_cases": 46,
+    "failure_expected_cases": 5
+  },
+  "model_config": {
+    "embedding": "BAAI/bge-m3 (production)",
+    "reranker": "BAAI/bge-reranker-v2-m3 (production)",
+    "search_mode": "hybrid",
+    "rerank_enabled": "server_default",
+    "embedding_backend": "baseline",
+    "reranker_backend": "baseline",
+    "plan": "round-2-review-mighty-starfish.md v2.1 (Phase 2B)"
+  },
+  "overall": {
+    "n": 46,
+    "graded_ndcg_at_10": 0.659,
+    "graded_recall_at_10_t2": 0.695,
+    "graded_recall_at_10_t3": 0.761,
+    "latency_p50_ms": 454,
+    "latency_p95_ms": 1573,
+    "failure_correct": "0/5"
+  },
+  "by_category": {
+    "english_only":  { "n":  9, "recall_at_10": 0.78, "ndcg_at_10": 0.71, "graded_ndcg_at_10": 0.78 },
+    "exam":          { "n":  7, "recall_at_10": 0.57, "ndcg_at_10": 0.62, "graded_ndcg_at_10": 0.74 },
+    "korean_only":   { "n":  9, "recall_at_10": 0.55, "ndcg_at_10": 0.47, "graded_ndcg_at_10": 0.51 },
+    "mixed":         { "n": 10, "recall_at_10": 0.38, "ndcg_at_10": 0.36, "graded_ndcg_at_10": 0.39 },
+    "standards":     { "n": 11, "recall_at_10": 0.91, "ndcg_at_10": 0.85, "graded_ndcg_at_10": 0.87 }
+  },
+  "by_language": {
+    "en":    { "n":  9, "recall_at_10": 0.78, "graded_ndcg_at_10": 0.78 },
+    "ko":    { "n": 27, "recall_at_10": 0.70, "graded_ndcg_at_10": 0.72 },
+    "mixed": { "n": 10, "recall_at_10": 0.38, "graded_ndcg_at_10": 0.39 }
+  },
+  "raw_csv": "reports/v0_2_phase2b_baseline_snapshot_2026-05-23.csv",
+  "reproducibility_check": "Phase 2A baseline_snapshot (NDCG 0.659 동일) — snapshot filter path 안정 + 재현성 확인"
+}

tests/search_eval/baselines/v0_2_phase2b_gte_ml_base_2026-05-23.json

@@ -0,0 +1,60 @@
+{
+  "version": "v0.2-phase2b",
+  "label": "cand_gte_ml_base",
+  "date": "2026-05-23",
+  "snapshot": {
+    "doc_id_max": 25180,
+    "chunk_id_max": 56526,
+    "documents_n": 21365,
+    "chunks_n": 30605
+  },
+  "eval_set": {
+    "total_cases": 51,
+    "scored_cases": 46,
+    "failure_expected_cases": 5
+  },
+  "model_config": {
+    "embedding": "BAAI/bge-m3 (production, 고정)",
+    "reranker": "Alibaba-NLP/gte-multilingual-reranker-base",
+    "reranker_params": "305M",
+    "reranker_context": 8192,
+    "reranker_license": "Apache 2.0",
+    "search_mode": "hybrid",
+    "rerank_enabled": "server_default",
+    "embedding_backend": "baseline",
+    "reranker_backend": "cand_gte_ml_base",
+    "endpoint": "http://rerank-cand-gte-ml-base:80/rerank",
+    "plan": "round-2-review-mighty-starfish.md v2.1 (Phase 2B)"
+  },
+  "overall": {
+    "n": 46,
+    "graded_ndcg_at_10": 0.604,
+    "graded_recall_at_10_t2": 0.709,
+    "graded_recall_at_10_t3": 0.783,
+    "latency_p50_ms": 345,
+    "latency_p95_ms": 1460,
+    "failure_correct": "0/5"
+  },
+  "by_category": {
+    "english_only":  { "n":  9, "recall_at_10": 0.78, "ndcg_at_10": 0.68, "graded_ndcg_at_10": 0.72 },
+    "exam":          { "n":  7, "recall_at_10": 0.64, "ndcg_at_10": 0.53, "graded_ndcg_at_10": 0.62 },
+    "korean_only":   { "n":  9, "recall_at_10": 0.50, "ndcg_at_10": 0.39, "graded_ndcg_at_10": 0.41 },
+    "mixed":         { "n": 10, "recall_at_10": 0.42, "ndcg_at_10": 0.35, "graded_ndcg_at_10": 0.38 },
+    "standards":     { "n": 11, "recall_at_10": 0.91, "ndcg_at_10": 0.84, "graded_ndcg_at_10": 0.86 }
+  },
+  "by_language": {
+    "en":    { "n":  9, "recall_at_10": 0.78, "graded_ndcg_at_10": 0.72 },
+    "ko":    { "n": 27, "recall_at_10": 0.71, "graded_ndcg_at_10": 0.65 },
+    "mixed": { "n": 10, "recall_at_10": 0.42, "graded_ndcg_at_10": 0.38 }
+  },
+  "raw_csv": "reports/v0_2_phase2b_gte_ml_base_2026-05-23.csv",
+  "delta_vs_baseline": {
+    "graded_ndcg_at_10": -0.055,
+    "mixed": -0.01,
+    "korean_only": -0.10,
+    "standards": -0.01,
+    "english_only": -0.06,
+    "exam": -0.12,
+    "latency_p50_ms": -109
+  }
+}

tests/search_eval/run_eval.py

@@ -202,6 +202,7 @@ async def call_search(
     embedding_backend: str | None = None,
     snapshot_doc_id_max: int | None = None,
     snapshot_chunk_id_max: int | None = None,
+    reranker_backend: str | None = None,
 ) -> tuple[list[int], float]:
     """검색 API 호출 → (doc_ids, latency_ms)."""
     url = f"{base_url.rstrip('/')}/api/search/"
@@ -219,6 +220,8 @@ async def call_search(
         params["snapshot_doc_id_max"] = snapshot_doc_id_max
     if snapshot_chunk_id_max is not None:
         params["snapshot_chunk_id_max"] = snapshot_chunk_id_max
+    if reranker_backend is not None:
+        params["reranker_backend"] = reranker_backend
 
     import time
 
@@ -249,6 +252,7 @@ async def evaluate(
     embedding_backend: str | None = None,
     snapshot_doc_id_max: int | None = None,
     snapshot_chunk_id_max: int | None = None,
+    reranker_backend: str | None = None,
 ) -> list[QueryResult]:
     """전체 쿼리셋 평가."""
     results: list[QueryResult] = []
@@ -261,6 +265,7 @@ async def evaluate(
                     embedding_backend=embedding_backend,
                     snapshot_doc_id_max=snapshot_doc_id_max,
                     snapshot_chunk_id_max=snapshot_chunk_id_max,
+                    reranker_backend=reranker_backend,
                 )
                 results.append(
                     QueryResult(
@@ -837,6 +842,7 @@ async def call_search_full(
     embedding_backend: str | None = None,
     snapshot_doc_id_max: int | None = None,
     snapshot_chunk_id_max: int | None = None,
+    reranker_backend: str | None = None,
 ) -> tuple[list[dict], float]:
     """call_search와 동일 로직. 단 full result dict 리스트 반환."""
     url = f"{base_url.rstrip('/')}/api/search/"
@@ -856,6 +862,8 @@ async def call_search_full(
         params["snapshot_doc_id_max"] = snapshot_doc_id_max
     if snapshot_chunk_id_max is not None:
         params["snapshot_chunk_id_max"] = snapshot_chunk_id_max
+    if reranker_backend is not None:
+        params["reranker_backend"] = reranker_backend
 
     import time
 
@@ -1308,6 +1316,12 @@ def main() -> int:
         default=None,
         help="Phase 2A snapshot freeze. document_chunks.id <= 값 filter. baseline rebaseline 도 동일 적용.",
     )
+    parser.add_argument(
+        "--reranker-backend",
+        type=str,
+        default=None,
+        help="Phase 2B Diagnose reranker dispatcher slug (baseline | cand_gte_ml_base). 미지정 = production.",
+    )
 
     args = parser.parse_args()
 
@@ -1361,21 +1375,21 @@ def main() -> int:
         if args.base_url:
             print(f"\n>>> evaluating: {args.base_url}")
             results = asyncio.run(
-                evaluate(queries, args.base_url, args.token, "single", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max)
+                evaluate(queries, args.base_url, args.token, "single", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max, reranker_backend=args.reranker_backend)
             )
             print_summary("single", results, eval_version=args.eval_version)
             all_results.extend(results)
         else:
             print(f"\n>>> baseline: {args.baseline_url}")
             baseline_results = asyncio.run(
-                evaluate(queries, args.baseline_url, args.token, "baseline", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max)
+                evaluate(queries, args.baseline_url, args.token, "baseline", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max, reranker_backend=args.reranker_backend)
             )
             baseline_summary = print_summary("baseline", baseline_results, eval_version=args.eval_version)
 
             print(f"\n>>> candidate: {args.candidate_url}")
             candidate_results = asyncio.run(
                 evaluate(
-                    queries, args.candidate_url, args.token, "candidate", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max
+                    queries, args.candidate_url, args.token, "candidate", mode=args.mode, fusion=args.fusion, rerank=args.rerank, analyze=args.analyze, embedding_backend=args.embedding_backend, snapshot_doc_id_max=args.snapshot_doc_id_max, snapshot_chunk_id_max=args.snapshot_chunk_id_max, reranker_backend=args.reranker_backend
                 )
             )
             candidate_summary = print_summary("candidate", candidate_results, eval_version=args.eval_version)