Commit Graph

10 Commits

Author SHA1 Message Date
Hyungi Ahn 490bef1136 feat(ai): B-0 3-tier routing — triage/primary/fallback 슬롯 + AIClient
- config.yaml: ai.models 에 triage (gemma4:e4b-it-q8_0, GPU Ollama,
  context_char_limit=120k, timeout 30s) 신규. primary (MLX gemma-4-26b)
  는 에스컬레이션 전용 역할 명시. fallback 을 gemma4:e4b 로 통일
  (exaone 제거 이미 반영). classifier/verifier 는 optional 유지,
  vision 은 optional 로 완화 (미사용 정리 준비).
- core/config.py: AIConfig 에 triage 필드 추가, vision 은 Optional 로
  전환. AIModelConfig.context_char_limit + DeepSummaryBacklogConfig
  (R2 backlog guard 임계치 ratio 0.3 / pending 5 / window 30min)
  스키마 신설. load_settings 가 models.get("vision") graceful.
- ai/client.py: call_triage / call_primary / call_fallback 3-tier
  진입점 신규. primary 는 caller 가 get_mlx_gate() 블록 안에서 호출
  해야 한다는 계약 docstring. classify/summarize 는 DEPRECATED 주석
  만 추가, 기존 호출부 (eval runner 등) 를 위해 유지.

PR-B B-0 Day 1. 기존 primary 경로 변경 없음 — 회귀 0 기대.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 10:05:24 +09:00
Hyungi Ahn b401085518 feat(ai): EscalationEnvelope contract (4B→26B handoff)
frozen dataclass with from_stage / escalation_reasons / risk_flags /
distilled_context / original_pointers / synthesis_directives / user_intent /
draft_hint. JSON round-trip (to_json/from_json). to_system_injection() 으로
26B system prompt 에 주입할 텍스트 블록 생성 (risk_flags + directives +
distilled_context 순).

from_stage 는 whitelist 검증 (triage/classify/summarize_short/advice_trigger/
night_sweep/ask_pre/unknown). tuple 타입 강제 (mutability 방지).

PR-B 의 escalation_service 가 이 계약을 사용. PR-A 는 계약만 정의.

plan: ~/.claude/plans/wise-gliding-hippo.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 09:34:48 +09:00
Hyungi Ahn f8f72ceae2 fix(ocr): Surya 0.17 API + NFC/NFD path normalize
- services/ocr/server.py: surya 0.17.x predictors 기반으로 재작성
  (구 `from surya.ocr import run_ocr` 제거됨 → import error → 빈 텍스트 반환)
- NFC(DB 경로) vs NFD(NFS 파일시스템) 한글 정규화 mismatch 보정
- surya-ocr 버전 0.17.1 고정 (0.6~1.0 범위는 breaking change 노출)
- AIClient.ocr() NotImplementedError 제거 (호출처 0건, extract_worker 가
  ocr-service HTTP 호출을 직접 사용)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 13:52:19 +09:00
Hyungi Ahn 5070ac45ff fix(extract): LibreOffice 추출 절단 제거 및 요약 입력 확대
- extract_worker: LibreOffice 15000자 절단 제거 (full text 저장 원칙)
- classify_worker/summarize_worker: 요약 입력 15000→50000자 확대
- client.py: 길이 기반 Claude 자동전환 제거 (require_explicit_trigger 정책 준수)
  _call_chat의 primary→fallback(exaone3.5) 체인은 유지

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:00:23 +09:00
Hyungi Ahn 76e723cdb1 feat(search): Phase 1.3 TEI reranker 통합 (코드 골격)
데이터 흐름 원칙: fusion=doc 기준 / reranker=chunk 기준 — 절대 섞지 말 것.

신규/수정:
- ai/client.py: rerank() 메서드 추가 (TEI POST /rerank API)
- services/search/rerank_service.py:
    - rerank_chunks() — asyncio.Semaphore(2) + 5s soft timeout + RRF fallback
    - _make_snippet/_extract_window — title + query 중심 200~400 토큰
      (keyword 매치 없으면 첫 800자 fallback)
    - apply_diversity() — max_per_doc=2, top score>=0.90 unlimited
    - warmup_reranker() — 10회 retry + 3초 간격 (TEI 모델 로딩 대기)
    - MAX_RERANK_INPUT=200, MAX_CHUNKS_PER_DOC=2 hard cap
- services/search_telemetry.py: compute_confidence_reranked() — sigmoid score 임계값
- api/search.py:
    - ?rerank=true|false 파라미터 (기본 true, hybrid 모드만)
    - 흐름: fused_docs(limit*5) → chunks_by_doc 회수 → rerank_chunks → apply_diversity
    - text-only 매치 doc은 doc 자체를 chunk처럼 wrap (fallback)
    - rerank 활성 시 confidence는 reranker score 기반
- tests/search_eval/run_eval.py: --rerank true|false 플래그

GPU 적용 보류:
- TEI 컨테이너 추가 (docker-compose.yml) — 별도 작업
- config.yaml rerank.endpoint 갱신 — GPU 직접 (commit 없음)
- 재인덱싱 완료 후 build + warmup + 평가셋 측정
2026-04-08 12:41:47 +09:00
Hyungi Ahn 63f75de89d fix: Qwen3.5 thinking 모드 비활성화 (enable_thinking: false)
JSON 응답에 Thinking Process 텍스트가 섞이는 문제 해결.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:38:10 +09:00
Hyungi Ahn 47e9981660 fix: Qwen3.5 Thinking Process 텍스트 제거 — JSON 파싱 개선
첫 번째 { 이전의 모든 비-JSON 텍스트를 제거하여
thinking/reasoning preamble이 있어도 JSON 추출 가능.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:44:21 +09:00
Hyungi Ahn d93e50b55c security: fix 5 review findings (2 high, 3 medium)
HIGH:
- Lock setup TOTP/NAS endpoints behind _require_setup() guard
  (prevented unauthenticated admin 2FA takeover after setup)
- Sanitize upload filename with Path().name + resolve() validation
  (prevented path traversal writing outside Inbox)

MEDIUM:
- Add score > 0.01 filter to hybrid search via subquery
  (prevented returning irrelevant documents with zero score)
- Implement Inbox → Knowledge file move after classification
  (classify_worker now moves files based on ai_domain)
- Add Anthropic Messages API support in _request()
  (premium/Claude path now sends correct format and parses
  content[0].text instead of choices[0].message.content)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:33:31 +09:00
Hyungi Ahn 299fac3904 feat: implement Phase 1 data pipeline and migration
- Implement kordoc /parse endpoint (HWP/HWPX/PDF via kordoc lib,
  text files direct read, images flagged for OCR)
- Add queue consumer with APScheduler (1min interval, stage chaining
  extract→classify→embed, stale item recovery, retry logic)
- Add extract worker (kordoc HTTP call + direct text read)
- Add classify worker (Qwen3.5 AI classification with think-tag
  stripping and robust JSON extraction from AI responses)
- Add embed worker (GPU server nomic-embed-text, graceful failure)
- Add DEVONthink migration script with folder mapping for 16 DBs,
  dry-run mode, batch commits, and idempotent file_path UNIQUE
- Enhance ai/client.py with strip_thinking() and parse_json_response()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:35:36 +09:00
Hyungi Ahn 131dbd7b7c feat: scaffold v2 project structure with Docker, FastAPI, and config
동작하는 최소 코드 수준의 v2 스캐폴딩:

- docker-compose.yml: postgres, fastapi, kordoc, frontend, caddy
- app/: FastAPI 백엔드 (main, core, models, ai, prompts)
- services/kordoc/: Node.js 문서 파싱 마이크로서비스
- gpu-server/: AI Gateway + GPU docker-compose
- frontend/: SvelteKit 기본 구조
- migrations/: PostgreSQL 초기 스키마 (documents, tasks, processing_queue)
- tests/: pytest conftest 기본 설정
- config.yaml, Caddyfile, credentials.env.example 갱신

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 10:20:15 +09:00