Commit Graph

7 Commits

Author SHA1 Message Date
hyungi f4eef9e6e0 feat(email): IMAP ingest service for inbox@hyungi.net
신규 워커 app/workers/inbox_ingest.py (337줄):
- 5분 APScheduler cron (mailplus_archive 와 분리 — INBOX root archive vs DocumentServer/Ingest folder)
- UID SEARCH SINCE 14일 (UNSEEN 단독 의존 X, 사용자가 MailPlus UI 에서 먼저 읽어도 누락 회피)
- Message-ID 정규화 또는 imap:{folder}:{uidvalidity}:{uid} fallback → source_external_id always non-null
- ON CONFLICT DO NOTHING (DB unique 진실원장)
- 신규 row 만 BODY parse: snippet + HTML stripping + attachment metadata (binary 저장 X)
- enqueue_stage(doc.id, classify) 로 기존 classify pipeline 진입
- HC.io heartbeat (옵션, INBOX_INGEST_HC_URL)
- parse 실패 분기: row 생성 전 (logger.error + HC fail) / 후 (email_metadata.parse_error 기록)

env (credentials.env.example):
- INBOX_INGEST_ENABLED=false (기본 dormant, 사용자가 alias/folder 셋업 후 true)
- INBOX_INGEST_FOLDER=DocumentServer/Ingest
- INBOX_INGEST_DAYS=14
- INBOX_INGEST_HC_URL=

main.py:
- inbox_ingest_run import + scheduler.add_job interval 5m

email_ingest 정책 (사용자 라운드 2026-05-12):
- 직접 events row 생성 X
- 이메일은 universal inbox item, source_channel=email memo 로 저장
- classify_worker 가 ai_event_kind 채움 (별 PR 의 4B robustness fix 선결)
- 사용자 1-click promote 만이 events row 생성 path

plan: ~/.claude/plans/document-enchanted-candy.md
2026-05-12 06:56:35 +00:00
Hyungi Ahn 5bfbb79641 feat(verifier): Phase 3.5 B2 — numeric_conflict promote (env flag) + Tier 4
VERIFIER_NUMERIC_PROMOTE 환경변수로 numeric_conflict severity 승격 실험.

verifier_service.py:
- _NUMERIC_PROMOTE = os.getenv('VERIFIER_NUMERIC_PROMOTE', '0') == '1'
  (import time 평가 — env 변경 시 process restart 필수)
- _SEVERITY_MAP['numeric_conflict']: env=1 → critical=strong / minor=medium,
  env=0 (기본) → 둘 다 medium (기존 동작 유지)
- direct_negation 은 env 무관 항상 strong (안전장치)

verifier.txt:
- numeric_conflict 정의에 critical/minor 분리 명시 (core quantity vs peripheral)
- "Range values satisfy any answer within range" rule 추가
- severity mapping 갱신: numeric_conflict 분기 명시

search.py re-gate (Tier 1~7 재번호, B2 신규 Tier 4):
- v_strong_numeric = sum(1 for f in v_strong
                         if f.startswith('verifier_numeric_conflict'))
- Tier 4 (신규): g_strong + v_strong_numeric >= 1 + low_conf → refuse
  re_gate value: 'refuse(grounding+verifier_numeric)'
- 원칙 유지: verifier strong 단독 refuse 금지 — g_strong 교차 필수
- 호환성: 기존 re_gate string literals 그대로 유지, 신규 1개만 추가

credentials.env.example: VERIFIER_NUMERIC_PROMOTE=0 (off, B3 통과 후 production 전환)

tests/test_verifier_numeric_promote.py: 4 케이스 (env off / on / explicit 0 /
direct_negation invariant). monkeypatch.setenv + importlib.reload 패턴.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 08:11:06 +09:00
Hyungi Ahn 09883d0358 feat(ask): Phase 3.5 A0 — ask_events source/eval_case_id + eval auth boundary
- migrations 138~142: source TEXT DEFAULT 'document_server' + eval_case_id TEXT
  추가, 인덱스 2개, backfill, 1주 관찰 후 NOT NULL (140 적용 분리)
- app/models/ask_event.py: source / eval_case_id ORM 필드 (138~141 단계 nullable)
- app/services/search_telemetry.py: record_ask_event 시그니처에 source / eval_case_id
- app/core/config.py: settings.eval_runner_token + EVAL_RUNNER_TOKEN env 로드
- app/api/search.py:
  - X-Source / X-Eval-Case-Id / X-Eval-Token 헤더 수신
  - _resolve_eval_identity(): hmac.compare_digest 로 token 검증, 실패 시 source
    'document_server' 강등 + warning log + eval_case_id=None
  - 두 record_ask_event 호출에 검증된 source/eval_case_id 전달
- credentials.env.example: EVAL_RUNNER_TOKEN= (empty default = 모든 eval claim 거부)
- tests/test_ask_eval_auth.py: 9 케이스 — token 없음/틀림/일치, env 미설정,
  case_id only, non-eval source forces case_id None

trust boundary: 일반 client 의 X-Source=eval / X-Eval-Case-Id 시도는 무시되어
calibration telemetry 오염 불가. eval runner 만 EVAL_RUNNER_TOKEN 으로 인증.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 08:11:06 +09:00
Hyungi Ahn 6b189f0d47 fix(digest): multi-word ai_sub_group matching + NYT_API_KEY example
- loader.py: first-token + all-but-last-token 이중 키 매칭 (Le Monde, Der Spiegel 대응)
- chunk_worker.py: startswith 매칭 보강
- credentials.env.example: NYT_API_KEY 항목 추가

핫픽스 — 단계 3에서 news_source_id FK 정규화로 문자열 매칭 제거 예정

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 14:33:04 +09:00
Hyungi Ahn 0ca78640ee infra: migrate application from Mac mini to GPU server
- Integrate ollama + ai-gateway into root docker-compose.yml
  (NVIDIA GPU runtime, single compose for all services)
- Change NAS mount from SMB (NAS_SMB_PATH) to NFS (NAS_NFS_PATH)
  Default: /mnt/nas/Document_Server (fstab registered on GPU server)
- Update config.yaml AI endpoints:
  primary → Mac mini MLX via Tailscale (100.76.254.116:8800)
  fallback/embedding/vision/rerank → ollama (same Docker network)
  gateway → ai-gateway (same Docker network)
- Update credentials.env.example (remove GPU_SERVER_IP, add NFS path)
- Mark gpu-server/docker-compose.yml as deprecated
- Update CLAUDE.md network diagram and AI model config
- Update architecture.md, deploy.md, devlog.md for GPU server as main
- Caddyfile: auto_https off, HTTP only (TLS at upstream proxy)
- Caddy port: 127.0.0.1:8080:80 (localhost only)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 07:47:09 +09:00
Hyungi Ahn 131dbd7b7c feat: scaffold v2 project structure with Docker, FastAPI, and config
동작하는 최소 코드 수준의 v2 스캐폴딩:

- docker-compose.yml: postgres, fastapi, kordoc, frontend, caddy
- app/: FastAPI 백엔드 (main, core, models, ai, prompts)
- services/kordoc/: Node.js 문서 파싱 마이크로서비스
- gpu-server/: AI Gateway + GPU docker-compose
- frontend/: SvelteKit 기본 구조
- migrations/: PostgreSQL 초기 스키마 (documents, tasks, processing_queue)
- tests/: pytest conftest 기본 설정
- config.yaml, Caddyfile, credentials.env.example 갱신

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 10:20:15 +09:00
Hyungi Ahn bec9579a8a chore: 프로젝트 구조 + 설계 문서 초기 커밋
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 11:22:04 +09:00