hyungi/hyungi_document_server
1
1
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

fix(search): reranker MAX_CLIENT_BATCH_SIZE 64→256

Browse Source 
rerank_service.py 가 후보를 MAX_RERANK_INPUT=200 까지 청크 없이 한 번에 TEI 로 POST → TEI 한도 64 초과(85) 시 HTTPError → RRF silent fallback(리랭크 누락=검색 품질 저하, 48h 4회). MAX_BATCH_TOKENS=16384 가 VRAM 상한이라 client batch entries 한도만 256(MAX_RERANK_INPUT 200 커버)으로 상향, reranker 만 재생성. 검증: 85-text rerank HTTP 200, batch 에러 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
hyungi
2026-06-17 23:35:43 +00:00
parent cd694e7386
commit 5cabf728e6
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker-compose.yml
+1 -1
View File
@@ -149,7 +149,7 @@ services:
# → 32 한도 초과 → 413. 64 로 늘림. # → 32 한도 초과 → 413. 64 로 늘림.
# GPU VRAM free 6199MiB 충분. baseline path (MAX_RERANK_INPUT=200) 영향 0. # GPU VRAM free 6199MiB 충분. baseline path (MAX_RERANK_INPUT=200) 영향 0.
- MAX_BATCH_TOKENS=16384 - MAX_BATCH_TOKENS=16384
- MAX_CLIENT_BATCH_SIZE=64 - MAX_CLIENT_BATCH_SIZE=256 # 2026-06-18 fix: 64→256, MAX_RERANK_INPUT=200 커버 (batch>64 ERROR=RRF silent fallback 해소; MAX_BATCH_TOKENS가 VRAM 상한이라 entries 증가는 VRAM 무관)
- MAX_CONCURRENT_REQUESTS=4 - MAX_CONCURRENT_REQUESTS=4
volumes: volumes:
- reranker_cache:/data - reranker_cache:/data