hyungi/hyungi_document_server
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(search): confidence 임계값 완화 + hybrid +vector boost 가산

Browse Source 
baseline 평가셋 실행 시 'summary+vector' top_score 2.39가 임계값 2.5에
미달해 정답 쿼리(산업안전보건법 제6장)가 low_confidence로 잘못 잡힘.

- 텍스트 매치 임계값 0.5씩 완화 (실측 분포 반영)
- '+vector' 접미사가 있으면 hybrid 합성 매치이므로 confidence +0.10 가산
- 정답률 5/5 → 4/5 false-positive 1건 제거 기대

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hyungi Ahn
2026-04-07 08:37:13 +09:00
parent 50e6b5ad90
commit 70b27d4a51
1 changed files with 15 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
app/services/search_telemetry.py
View File

@@ -103,17 +103,22 @@ def compute_confidence(results: list[Any], mode: str) -> float:
# 코사인 유사도 그대로 # 코사인 유사도 그대로
return _cosine_to_confidence(top_score) return _cosine_to_confidence(top_score)
# text / hybrid: 강한 텍스트 매치 우선 판정 # hybrid에서 텍스트+벡터 합성 매치는 reason에 "+vector" 접미. 신뢰 가산.
if "title" in reason and top_score >= 4.0: has_vector_boost = "+vector" in reason
return 0.95 boost = 0.10 if has_vector_boost else 0.0
if any(k in reason for k in ("tags", "note")) and top_score >= 3.0:
return 0.85 # text / hybrid: 강한 텍스트 매치 우선 판정.
if "summary" in reason and top_score >= 2.5: # 임계값은 search.py의 가중치 합산 분포(텍스트base + FTS bonus + 0.5*cosine)를 반영.
return 0.75 if "title" in reason and top_score >= 3.5:
if "content" in reason and top_score >= 2.0: return min(1.0, 0.95 + boost)
return 0.65 if any(k in reason for k in ("tags", "note")) and top_score >= 2.5:
return min(1.0, 0.85 + boost)
if "summary" in reason and top_score >= 2.0:
return min(1.0, 0.75 + boost)
if "content" in reason and top_score >= 1.5:
return min(1.0, 0.65 + boost)
if "fts" in reason and top_score >= 1.0: if "fts" in reason and top_score >= 1.0:
return 0.55 return min(1.0, 0.55 + boost)
# vector-only hit (텍스트 0건 → 코사인 raw, amplify 금지) # vector-only hit (텍스트 0건 → 코사인 raw, amplify 금지)
if reason == "vector": if reason == "vector":