Compare commits

...

36 Commits

Author SHA1 Message Date
hyungi 6d447f9cba feat(study): 이론↔문제 브리지 (Stage B) — 개념별 정답률·약점 개념 지도
이론공부 B→A→C 의 B. 완성된 문제풀이에 이론 연결(약점 구동).
- 마이그 382 study_concept_links(개념 doc↔기출, FK 없음) + 백필 SQL(임베딩 코사인 top-k=10·threshold 0.62 → 2362링크·284개념·964문항)
- concept_links 서비스(related_questions·weakness_map 롤업) + GET /concepts/{id}/questions·/concepts/weakness-map(라우트 순서=weakness-map 먼저)
- 리더 관련기출 섹션(정답률·문항 stub→문항상세) + 홈 약점개념 위젯
- 적대리뷰 반영: Promise.all 격리(weakness-map 실패→코어 대시보드 블랙아웃 방지)·q.subject null 폴백. 백필=배포 후 트랜잭션 래핑 실행. 문제풀이 무접촉

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 12:05:09 +09:00
hyungi f38ec177d7 feat(study): 개념 학습 리더 (Stage A) — 구조 파싱·떠올리기·백링크
이론공부 개선 B→A→C 의 A. 개념노트를 구조(요약/본문/빈출★/관련개념)로 렌더 + 능동 회상(떠올리기) + 관련개념 백링크 + 이전/다음.
- concept_parser: md 골격 파서(273/273 불변식) + 관련개념 백링크 해소(exact→title⊆phrase substring, 과대매치 가드)
- concept_curriculum.concept_detail + GET /api/study/concepts/{id} (개념문서 태그 스코프)
- /study/read/[docId] 리더(MarkdownDoc KaTeX+docimg 재사용·읽기/떠올리기 모드) + 홈 오늘의개념 링크 연결
- 적대리뷰 5건 반영(이중로드·substring 오결선·엔드포인트 스코프·prev/next 결정성·in-flight 가드). 마이그 없음·문제풀이 무접촉

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:51:40 +09:00
hyungi da4a2e81c3 feat(study): 이론공부 홈 — 오늘의 개념·진도·회독 SR (Stage S)
개념문서(가스기사 289) 소비 표면 개선 1단계. /study 허브를 데일리 랜딩으로.
- 마이그 381 study_concept_progress (개념 SR, sr_schedule 공용, documents FK 없음=락 회피)
- concept_curriculum 서비스 + /api/study (curriculum·today-concepts·concepts/{id}/read)
- read 상태 정본 = document_reads (is_read 컬럼 아님), mark_read=회독+SR 입고
- 문제풀이 표면 무접촉·additive

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:11:30 +09:00
hyungi 966a4315c8 feat(shell): 시안B 슬림 아이콘 레일 — 사이드바 접힘=54px 글로벌 네비(숨김 대신) 2026-06-30 06:29:33 +00:00
hyungi 3c42b7b97a feat(book): 공부도구 배선 — 노트/형광펜/암기카드(clause_study) + 책 리더 패널 2026-06-30 06:26:55 +00:00
hyungi 91ce54c1cd chore(paper): OpenAlex 매치율 측정 스크립트(결론=인용보강 부적합) 2026-06-30 06:20:59 +00:00
hyungi 9ec0a184a0 feat(book): /book 몰입 — 글로벌 분류 사이드바 숨김(더블사이드바 해소) 2026-06-30 06:16:28 +00:00
hyungi a22b2c7647 feat(docs): 관련 문서(유사도 KNN) 엔드포인트+패널 + 법령/지침 splitter 2026-06-30 06:10:11 +00:00
hyungi c44692fddc feat(clause-kb): 코드북 리더 r2 — 세이지 코드북 미감(인덱스/세리프/책내검색/양방향 백링크/페이저)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 05:02:35 +00:00
hyungi 7487739aec fix(clause-kb): 절-문서 이미지를 부모 표준 document_images 로 폴백 해소(docimg 404 수정)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:38:37 +00:00
hyungi a8d3af2b62 fix(clause-kb): backlinks 엔드포인트 parent_id ORM 미매핑 → raw SQL 조회 (500 수정)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:34:18 +00:00
hyungi 51a7c96b56 feat(clause-kb): over-CAP 절 본문 페이지네이션(~11K tok/page) 2026-06-29 23:20:16 +00:00
hyungi eb83d41ba5 feat(clause-kb): 책 API(절 목차/백링크) + /book/[id] 유기적 책 리더 + persist 스크립트 2026-06-29 23:13:34 +00:00
hyungi 62794b3857 feat(search): ASME 절-KB schema 379 + doc_kind retrieval 필터
- migration 379: documents +parent_id/doc_kind/clause_code/clause_part/clause_order + clause_links/document_tags
- _license_sql 에 doc_kind=standard 필터(절-문서 read/nav 전용, 검색 제외; 전 문서 standard=동작보존)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:56:59 +00:00
hyungi 8cdfe6006d feat(search): cloud-egress 게이트를 단건 문서 fetch 로 확장
GET /api/documents/{id} 가 egress=cloud 토큰일 때 search 와 동일한
cloud-eligibility 게이트(egress allowlist 갭2 + license 제한 B-4)를 통과한
문서만 반환. id 직접 fetch 로 비공개/인프라/개인/restricted 문서를 우회
열람하는 경로 차단 — 부적격은 404(존재 비노출). local 토큰=무회귀.

술어는 retrieval_service.cloud_eligible_doc_sql 로 단일화(_axis_sql
cloud_egress + _license_sql 합성) → search retrieval 과 byte-동일 게이트
공유, 경로별 드리프트 방지. MCP fetch_document 툴의 서버사이드 강제.

e2e: cloud 토큰 적격 Eng 200 / 인프라알림·리디북스memo·개인노트 404,
local 토큰 전부 200(무회귀).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:52:41 +00:00
hyungi 3fb613916a feat(search): cloud-egress allowlist gate for cloud consumers (gap2)
클라우드 소비자(Claude/MCP)에 cloud-eligibility allowlist 강제 — DS 접근규격 갭2.
- auth: create_access_token egress claim(기본 local·비파괴) + get_egress_class 의존성
- AxisFilter.cloud_egress + _axis_sql allowlist 술어(토큰 claim 유래·쿼리파라미터 아님=우회불가)
- 규칙: external OR (work ∩ bucket∈{Eng,Safety,Law} ∩ ∉{voice,chat,memo} ∩ ≠memo ∩ user_note없음)
검증(cloud vs local): 인프라알림([Hyungi_NAS] tk-*api)·work/Programming(리디북스) 차단,
work/Engineering(hoop stress·ASME) 통과, external 통과. local=전부(무회귀).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 05:19:23 +00:00
hyungi 0c7211e24b feat(search): domain_bucket scope filter on AxisFilter (include/exclude)
검색 retrieval 에 domain_bucket(377) 포함/제외 필터 추가.
- AxisFilter.domain_buckets(= ANY) / exclude_buckets(<> ALL) + active()
- _axis_sql 2절 — 전 leg documents alias(d / chunk df JOIN) 경유, 미지정시 byte-불변(무회귀)
- search.py: domain_bucket / exclude_bucket Query 파라미터(CSV)
검증: exclude_bucket=News → News 0건(금리 10→0·인공지능 15→0·반도체 11→0),
domain_bucket=Safety → Knowledge/Industrial_Safety 드리프트까지 정규화 포함.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:35:12 +00:00
hyungi 94b172e314 ops(ci): boot_smoke 스키마 어서션 max_migration 361→378 (현재 마이그 헤드)
지난 감사(361) 이후 마이그가 378(이번 publish_outbox attempts/failed 포함)까지 전진 →
boot_smoke 스키마 게이트의 하드코딩 기대값 갱신. purge/cand/uq 기대는 동일.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:30:53 +09:00
hyungi 9357d1592d fix(publish): 마이그 번호 377→378 (멀티세션 prod 377_domain_bucket 충돌 회피)
검수 fix 작업 중 다른 세션이 prod 에 377_domain_bucket(ee3b347)을 선점·배포 →
publish_outbox attempts/failed 마이그를 378 로 리넘버(브랜치를 ee3b347 로 rebase).
모델 주석도 mig378 로 정정. 내 fix 8건은 새 prod 커밋과 파일 무충돌(번호만 조정).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:23:16 +09:00
hyungi 832ea72784 fix(publish): backfill 스크립트 after_id 페이징 루프 (overflow 누락 방지)
backfill_publish_* 가 단일 호출(after_id=0, limit=PAGE)이라 PAGE 초과분이 누락(경고만)됐다.
docstring 은 이미 페이지 반복을 명시했으나 스크립트가 미구현. 함수 반환을 (count, last_id)로
바꾸고 3 스크립트를 last_id 기반 while 루프로 전량 처리. PAGE=5000 bounded tx.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi d26b1150d8 fix(workers): presegment/csb 이벤트루프 blocking I/O to_thread 오프로드
- presegment_worker: fitz open/get_toc(동기 blocking, live 스테이지)를 to_thread 로 — 거대/손상
  PDF 파싱이 같은 루프의 1분 consumer + FastAPI 요청을 수백 ms~초 정지시키던 것 해소.
- csb_collector: 50MB PDF write_bytes + read_bytes(해시)를 to_thread 로 (R5 동형).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi dcfed09530 fix(workers): marker 200-malformed json transient 분류 + classify summary 가시성
- marker_worker: resp.json() JSONDecodeError(200 응답 truncated/malformed body)가 catch-all
  except 로 _fail(non-retryable) 되던 것을 별도 except 로 raise → queue retry. transient
  연결흔들림이 영구 failed 로 박히는 것 차단.
- classify_worker: ai_summary fallback 비-deferrable 실패를 warning→error 로 격상. ai_summary
  NULL 완료는 digest/briefing 에서 조용히 제외되므로 운영 추적성 보강(best-effort 강등은 유지).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi 7d882352b8 fix(mineru): 변환/워밍 self-timeout + OOM·행 시 엔진 재워밍 escalate
aio_do_parse 에 자체 타임아웃이 없어 vLLM 행 시 _engine_lock 을 영구 점유 → markdown
변환 전체 마비(컨테이너 재시작 전까지). 클라이언트(marker_worker)는 300s 로 포기하나
서버측 inflight 는 자동 취소 안 됨.

- _run_mineru 를 asyncio.wait_for(convert 600s / warmup 1200s)로 감싸 lock 점유 상한.
- 타임아웃·OOM/CUDA 류 실패 시 _warmup_done 리셋 → 다음 요청 재워밍. 재워밍도 실패하면
  _warmup_error → /ready 503 → healthcheck 재시작으로 escalate(영구 degradation 차단).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi 7a8aced2a9 fix(workers): file_watcher 파일별 세션 격리 (사이클 전체 롤백 방지)
스캔 전체(Web+PKM)가 단일 세션·단일 commit 이라 한 파일 예외(rglob↔stat 사이 삭제로
FileNotFoundError, flush 오류 등)가 watch_inbox 전체를 raise·롤백 → 그 사이클 등록분을
모두 잃거나, 결정적 poison 파일이 매 사이클 같은 지점에서 중단시켜 그 뒤 파일 영구 미등록.

파일별 독립 세션+commit + try/continue 격리 (news_collector/csb_collector 동형).
file_hash 는 세션 밖에서 계산(커넥션 미점유), 무변경 파일은 쓰기/commit 없음.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi d50be9f2e7 fix(publish): ingest_study 동시경합을 already_ingested 로 흡수 (500 회피)
같은 client_session_uuid 동시 POST 2건이 최초 멱등체크를 둘 다 통과 → 둘째가
(client_session_uuid, study_topic_id) uq(mig376) 위반으로 IntegrityError → 미처리 500.
데이터는 안전(원자 1-tx 롤백, SR 이중 advance 없음)이나 비우아한 500이 문제.

변이 구간(quiz_session insert ~ commit)을 try/except IntegrityError 로 감싸 승자 결과
재조회 후 already_ingested 반환. uuid 경합이 아닌 진짜 무결성 오류는 재조회 비어 re-raise.
멱등 응답 빌더 _already_ingested 헬퍼 추출(최초 체크 + 경합 흡수 공용).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi b9f9d88d99 fix(publish): publish_outbox poison row head-of-line block 차단
배치 단일 트랜잭션이라 한 행의 예외가 배치 전체(앞 행 processed_at 포함)를 롤백 →
poison 행이 매 사이클 최저 id 로 재선택되어 후속 발행이 영구 정지. outbox 모델에
재시도/terminal 컬럼이 전무(processing_queue·study_jobs 의 per-item 격리 패턴 미적용).

- mig377: publish_outbox 에 attempts/failed_at 추가
- 워커: 행별 savepoint(begin_nested) 격리 — 예외 시 attempts++, MAX(5) 초과 시
  failed_at 스탬프(terminal) 후 select 제외. 실패 행은 rev 미소모(드문 gap 은 단일
  라이터·커밋순 부여라 viewer since-rev 증분 동기에 무해).

study_publish_enabled=false 기본이라 현재 inert, 발행 활성화(P0-1b) 전 선결.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi d030a2b7b0 fix(deploy): fresh-DB/DR 부팅 — postgres initdb.d 마운트 제거
빈 볼륨 첫 기동 시 postgres 가 migrations/*.sql 을 psql autocommit 으로 실행해
스키마는 만들되 schema_migrations 스탬프를 안 남김 → fastapi init_db 가 documents
존재로 'fresh' 오판해 baseline 로드를 건너뛰고 001 부터 재replay → CREATE TABLE
users(IF NOT EXISTS 없음) 충돌 → DR/신규환경 부팅 크래시.

fresh-boot 을 init_db 의 baseline + migration runner 단일 경로로 일원화.
기존 prod 볼륨은 비어있지 않아 init scripts 미발동 = 무영향. 관련 docs 정정.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:22:36 +09:00
hyungi ee3b347fa7 feat(search): add domain_bucket rollup column (migration 377)
ai_domain(반자유 AI 분류, 드리프트)을 검색 스코프용 7버킷으로 결정적 롤업하는
STORED generated column. News 86% 기본제외 + 도메인 스코프 검색의 토대.
축: ai_domain(routing) 롤업 — category(UI) 아님.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:16:30 +00:00
hyungi a826872b0d ops(ai): deep 슬롯 제거 — 맥북 night-drain 보류, deep_summary 맥미니 일원화
사용자 결정(2026-06-29 '맥북 야간운행 의미없어'): config deep 슬롯(qwen-macbook) 제거 → deep_summary 가 primary(맥미니) 경로 복귀(config 주석 보증), use_deep/drain 도 맥미니 폴백. drain-keeper(GPU cron) 비활성. 맥북 mlx-vlm-server 는 OpenCode 로컬용 보존. inventory 선행 갱신(Update Rule). 효과: 멈췄던 deep_summary(ai_detail_summary, last id 59773)가 맥미니에서 재개 → 3→2 짧은 ai_summary 의 풀버전 백스톱 복원.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 08:23:23 +09:00
hyungi 4cdd30950c refactor(classify): summarize 콜을 tier triage 에 병합 (3콜→2콜)
B-1 3→2: p3a_short_summary triage 가 ai_summary 도 생산 → 별도 summarize 콜 제거. classify(domain/type)은 분리 유지(shadow probe 결과 결합 시 domain 노이즈 → 안전하게 요약만 병합). 본문 prefill 3회→2회 = Mac mini 부하 절감. >120K long_context·triage 파싱실패 시 summarize fallback 보존. shadow probe(Industrial_Safety 5문서) 검증: triage ai_summary 품질 legacy summarize 동급.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 16:55:12 +09:00
hyungi 495e1c786f refactor(search)!: /ask 고아 service·테스트·프롬프트 정리 (검색 단일화 Phase 2)
/ask 삭제로 0-consumer 된 자산 제거(3-gate 실증): search.py /ask 섹션(Citation/ConfirmedItem/AskDebug/AskResponse 모델 + 헬퍼 + _resolve_eval_identity) + 죽은 import 13개. service 4(classifier/verifier/refusal_gate/grounding_check). AIClient.call_classifier/call_verifier(고아). 프롬프트 2(classifier/verifier.txt). broken test 6. evidence/synthesis 는 공유(documents.py 등)라 유지. 실 pyflakes 클린(이전 세션 pyflakes 미설치로 검증 누락 → 설치 후 실검증).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:39:53 +09:00
hyungi 86a71ec4d1 refactor(search)!: /ask 프론트 UI 제거 (검색 단일화 — AI답변=eid /chat)
/documents 인라인 AI카드(askSearch·AskAnswerCard·Sparkles→/ask) + /ask 페이지·컴포넌트(components/ask, AskAnswer/Evidence/Results) + 고아 util(isQuestion)·type(types/ask) 제거. /documents=순수 문서검색, AI답변은 eid /chat 사이드바로 일원화. dangling ref 0(grep).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 12:48:13 +09:00
hyungi b6717c537f refactor(search)!: /ask + /ask/react 엔드포인트 삭제 (검색 단일화 1단계)
검색 단일화 결정(PKM 현황/계획서 2026-06-27): AI 답변을 eid /chat 으로 일원화. /ask(grounding-heavy 3-panel, 사용자 숨김) + /ask/react(eid /chat deep 과 동일 agentic_ask_loop 중복) 엔드포인트 제거. GET /(plain 검색) 유지. py_compile + pyflakes undefined-name 0. 잔여(AskResponse 모델·_resolve_eval_identity·/ask 전용 service)는 Phase 2 dead-code 정리.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 07:47:36 +09:00
hyungi 842ad14930 refactor(search): /ask 핸들러 오케스트레이션을 _run_ask 로 분리 (라우터=deps 해소만)
api 리뷰: ask() 529줄에 검색→evidence/classifier→refusal→synthesis→grounding/verifier→7-tier 재게이트→telemetry 가 인라인. body 를 _run_ask(plain params)로 분리, ask 는 FastAPI deps 해소 후 return await _run_ask(...). body verbatim(동작 무변경), 12 params 전부 전달, 다중 return/background_tasks 보존. py_compile + pyflakes undefined-name 0 으로 충실이동 검증.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 07:25:58 +09:00
hyungi 2fedaa065b fix(study): subject_note_rag 에 licensed_restricted 필터 누락 — 구매자료 분야노트 RAG 누수
explanation_rag 는 restricted_exclude_orm() 으로 licensed_restricted 문서를 제외하는데(B-4, a안 U-2① 단일술어), 복제된 subject_note_rag._gather_document_evidence 는 이 술어를 빠뜨려 구매 자료 verbatim 이 분야노트 RAG 로 샐 수 있었음(services 리뷰 P1 보안 drift). doc_meta 쿼리에 필터 추가 → valid_doc_ids → 청크 쿼리까지 자동 전파(explanation_rag 동일 구조).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 07:10:24 +09:00
hyungi 274d2009c4 fix(migration): fresh DB/DR 부트스트랩 깨짐 3건 수정 (validator 오탐 + multi-statement)
verification env(ephemeral postgres + init_db) 실측으로 fresh DB 부트스트랩이 359~376 replay 중 깨지는 3건 발견·수정:
1. _validate_sql_content 가 인라인 주석(SQL -- ...) 미제거 → 365 의 '-- commit 시 ...' 설명주석을 트랜잭션 제어문 오탐. 줄별 -- 이후 제거.
2. raw '"schema_migrations" in sql.lower()' 체크도 주석 미제외 → 365 의 '-- ... schema_migrations 건드리지 않음' 오탐. _validate_sql_content 로 통합(주석 제외).
3. 마이그 루프가 exec_driver_sql(prepared)이라 multi-statement(365=테이블+시드+인덱스) 불허 → baseline 적재와 동일한 raw asyncpg simple execute 로 통일.
(에이전트가 P0로 본 320/326 enum-same-txn 은 오탐 — baseline 0358 이 이미 방어.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 07:00:32 +09:00
73 changed files with 2985 additions and 4238 deletions
-17
View File
@@ -260,23 +260,6 @@ class AIClient:
cfg = self.ai.deep or self.ai.primary
return await self._request(cfg, prompt, system=system)
async def call_classifier(self, prompt: str) -> str:
"""answerability classifier (config ai.classifier, Mac mini 26B MLX).
private _request 직접 호출(classifier_service)을 봉인하는 public 진입점. gate 는
caller(classifier_service)가 acquire_mlx_gate 로 관리 — call_primary 와 동일한
caller-managed 계약(여기서 self-gate 하면 caller 와 double-acquire 데드락).
"""
return await self._request(self.ai.classifier, prompt)
async def call_verifier(self, prompt: str) -> str:
"""semantic verifier (config ai.verifier, Mac mini 26B MLX).
private _request 직접 호출(verifier_service)을 봉인. gate 는 caller(verifier_service)
가 관리(caller-managed — self-gate 금지).
"""
return await self._request(self.ai.verifier, prompt)
# ─── Legacy API (classify_worker 교체 시 제거 예정) ───────────────────
async def classify(self, text: str, cfg=None) -> dict:
+337 -2
View File
@@ -28,7 +28,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from starlette.requests import ClientDisconnect
from ai.client import AIClient, _load_prompt, parse_json_response
from core.auth import get_current_user
from core.auth import get_current_user, get_egress_class
from core.config import settings
from core.database import async_session, get_session
from core.utils import file_hash
@@ -742,11 +742,31 @@ async def get_document(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
egress_class: Annotated[str, Depends(get_egress_class)],
):
"""문서 단건 조회. 본문(extracted_text)·canonical markdown 동봉."""
"""문서 단건 조회. 본문(extracted_text)·canonical markdown 동봉.
cloud egress(갭2): egress=cloud 토큰(예: Claude/MCP)은 search 와 동일한 cloud-eligibility
게이트를 통과한 문서만 열람 가능 — id 직접 fetch 로 비공개/인프라/개인/restricted 문서를
우회 열람하는 경로를 차단한다. 부적격은 404(존재 자체 비노출). local 토큰=게이트 미발동(무회귀).
"""
from sqlalchemy import text as sql_text
from services.search.retrieval_service import cloud_eligible_doc_sql
doc = await session.get(Document, doc_id)
if not doc or doc.deleted_at is not None:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
if egress_class == "cloud":
eligible = (
await session.execute(
sql_text(
"SELECT 1 FROM documents WHERE id = :doc_id AND deleted_at IS NULL"
+ cloud_eligible_doc_sql("")
).bindparams(doc_id=doc_id)
)
).first()
if eligible is None:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
return DocumentDetailResponse.model_validate(doc)
@@ -1028,6 +1048,19 @@ async def get_document_image_raw(
DocumentImage.image_key == image_key,
)
)
if img is None:
# clause-KB: 절-문서는 부모 표준 이미지를 공유(md_content=부모 슬라이스) → parent_id 폴백.
from sqlalchemy import text as sql_text
_par = (await session.execute(
sql_text("SELECT parent_id FROM documents WHERE id = :id").bindparams(id=doc_id)
)).scalar()
if _par is not None:
img = await session.scalar(
select(DocumentImage).where(
DocumentImage.document_id == _par,
DocumentImage.image_key == image_key,
)
)
if img is None:
raise HTTPException(status_code=404, detail="이미지를 찾을 수 없습니다")
@@ -1801,3 +1834,305 @@ async def analyze_document(
error_code=error_code,
source=source,
)
# ─── ASME 절-지식베이스: 유기적 책 네비 (clause-KB, doc_kind='clause' 자식 문서 기반) ───
class ClauseTocItem(BaseModel):
id: int
clause_code: str | None = None
clause_part: str | None = None
clause_order: int | None = None
title: str | None = None
class ClauseBookResponse(BaseModel):
parent_id: int
parent_title: str | None = None
clauses: list[ClauseTocItem]
@router.get("/{doc_id}/clauses", response_model=ClauseBookResponse)
async def get_document_clauses(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""부모 표준 doc 의 절-문서 목차(유기적 책 TOC). doc_kind='clause' 자식을 clause_order 순 반환.
절-문서는 in_corpus=false + doc_kind='clause'(검색 제외)라 일반 목록/검색엔 안 뜨지만,
이 책-내 네비는 부모 표준에서 자식 절로 진입하는 전용 경로다(ASME 2025판=한 권의 책).
"""
from sqlalchemy import text as sql_text
parent = await session.get(Document, doc_id)
if not parent or parent.deleted_at is not None:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
rows = (
await session.execute(
sql_text(
"""
SELECT id, clause_code, clause_part, clause_order, title
FROM documents
WHERE parent_id = :pid AND doc_kind = 'clause' AND deleted_at IS NULL
ORDER BY clause_order
"""
).bindparams(pid=doc_id)
)
).mappings().all()
return ClauseBookResponse(
parent_id=doc_id,
parent_title=parent.title,
clauses=[ClauseTocItem(**dict(r)) for r in rows],
)
class BacklinkRef(BaseModel):
code: str
doc_id: int | None = None # 해소된 절-문서(같은 부모) — dangling 이면 None
title: str | None = None
anchor: str | None = None
ctx: str | None = None
class BacklinksResponse(BaseModel):
doc_id: int
clause_code: str | None = None
parent_id: int | None = None
prev: ClauseTocItem | None = None
next: ClauseTocItem | None = None
forward: list[BacklinkRef] # 이 절이 참조하는 절들
back: list[BacklinkRef] # 이 절을 참조하는 절들
@router.get("/{doc_id}/backlinks", response_model=BacklinksResponse)
async def get_document_backlinks(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""절-문서의 양방향 백링크 + 같은 부모 내 이전/다음 절(유기적 책 흐름)."""
from sqlalchemy import text as sql_text
doc = await session.get(Document, doc_id)
if not doc or doc.deleted_at is not None:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
_meta = (await session.execute(sql_text(
"SELECT parent_id, clause_code, clause_order FROM documents WHERE id = :id"
).bindparams(id=doc_id))).mappings().first()
_parent_id = _meta["parent_id"] if _meta else None
_clause_code = _meta["clause_code"] if _meta else None
_clause_order = _meta["clause_order"] if _meta else None
forward = (
await session.execute(
sql_text(
"""
SELECT cl.dst_code AS code, cl.dst_doc_id AS doc_id, cl.anchor, cl.ctx, d.title
FROM clause_links cl
LEFT JOIN documents d ON d.id = cl.dst_doc_id
WHERE cl.src_doc_id = :id
ORDER BY cl.char_off NULLS LAST
LIMIT 300
"""
).bindparams(id=doc_id)
)
).mappings().all()
back = (
await session.execute(
sql_text(
"""
SELECT s.clause_code AS code, cl.src_doc_id AS doc_id, s.title, cl.ctx
FROM clause_links cl
JOIN documents s ON s.id = cl.src_doc_id
WHERE cl.dst_doc_id = :id
ORDER BY s.clause_order NULLS LAST
LIMIT 300
"""
).bindparams(id=doc_id)
)
).mappings().all()
prev = nxt = None
if _parent_id is not None and _clause_order is not None:
prow = (
await session.execute(
sql_text(
"""
SELECT id, clause_code, clause_part, clause_order, title FROM documents
WHERE parent_id = :pid AND doc_kind='clause' AND deleted_at IS NULL
AND clause_order < :ord
ORDER BY clause_order DESC LIMIT 1
"""
).bindparams(pid=_parent_id, ord=_clause_order)
)
).mappings().first()
nrow = (
await session.execute(
sql_text(
"""
SELECT id, clause_code, clause_part, clause_order, title FROM documents
WHERE parent_id = :pid AND doc_kind='clause' AND deleted_at IS NULL
AND clause_order > :ord
ORDER BY clause_order ASC LIMIT 1
"""
).bindparams(pid=_parent_id, ord=_clause_order)
)
).mappings().first()
prev = ClauseTocItem(**dict(prow)) if prow else None
nxt = ClauseTocItem(**dict(nrow)) if nrow else None
return BacklinksResponse(
doc_id=doc_id,
clause_code=_clause_code,
parent_id=_parent_id,
prev=prev,
next=nxt,
forward=[BacklinkRef(**dict(r)) for r in forward],
back=[BacklinkRef(**dict(r)) for r in back],
)
# ─── 관련 문서 (유사도, on-demand pgvector KNN — 저부하·무저장) ───
class RelatedItem(BaseModel):
id: int
title: str | None = None
ai_domain: str | None = None
material_type: str | None = None
year: int | None = None
sim: float | None = None
class RelatedResponse(BaseModel):
doc_id: int
related: list[RelatedItem]
@router.get("/{doc_id}/related", response_model=RelatedResponse)
async def get_related_documents(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
limit: int = 8,
same_type: bool = True,
):
"""문서-레벨 임베딩 코사인 최근접 = '관련 문서'. on-demand(저장/배치 없음).
인용그래프가 부적합한 코퍼스(업계 기술기사=인용망 부재)의 대안 연결 레이어.
same_type=true면 같은 material_type 내, false면 전 코퍼스. doc_kind='clause'(절-문서)는 제외.
"""
from sqlalchemy import text as sql_text
lim = max(1, min(limit, 30))
type_clause = "AND d.material_type = src.material_type" if same_type else ""
rows = (
await session.execute(
sql_text(
f"""
WITH src AS (
SELECT embedding, material_type FROM documents WHERE id = :id
)
SELECT d.id, d.title, d.ai_domain, d.material_type, d.facet_year AS year,
round((1 - (d.embedding <=> (SELECT embedding FROM src)))::numeric, 3) AS sim
FROM documents d, src
WHERE d.doc_kind = 'standard' AND d.deleted_at IS NULL
AND d.id <> :id AND d.embedding IS NOT NULL
AND (SELECT embedding FROM src) IS NOT NULL
{type_clause}
ORDER BY d.embedding <=> (SELECT embedding FROM src)
LIMIT :lim
"""
).bindparams(id=doc_id, lim=lim)
)
).mappings().all()
return RelatedResponse(
doc_id=doc_id,
related=[RelatedItem(**{k: r[k] for k in ("id", "title", "ai_domain", "material_type", "year")}, sim=float(r["sim"]) if r["sim"] is not None else None) for r in rows],
)
# ─── 절 공부도구 (노트/형광펜/암기카드) — clause_study ───
class StudyItem(BaseModel):
id: int
kind: str
payload: dict = {}
created_at: datetime | None = None
class StudyListResponse(BaseModel):
doc_id: int
items: list[StudyItem]
class StudyCreate(BaseModel):
kind: str # note | highlight | card
payload: dict = {}
def _parse_payload(p):
import json
if isinstance(p, str):
try:
return json.loads(p)
except Exception:
return {}
return p or {}
@router.get("/{doc_id}/study", response_model=StudyListResponse)
async def list_study(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""절-문서의 공부도구 항목(노트/형광펜/암기카드) 목록."""
from sqlalchemy import text as sql_text
rows = (
await session.execute(
sql_text("SELECT id, kind, payload, created_at FROM clause_study "
"WHERE doc_id = :id ORDER BY created_at DESC").bindparams(id=doc_id)
)
).mappings().all()
return StudyListResponse(
doc_id=doc_id,
items=[StudyItem(id=r["id"], kind=r["kind"], payload=_parse_payload(r["payload"]),
created_at=r["created_at"]) for r in rows],
)
@router.post("/{doc_id}/study", response_model=StudyItem, status_code=201)
async def add_study(
doc_id: int,
body: StudyCreate,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""노트/형광펜/암기카드 1건 추가."""
import json
from sqlalchemy import text as sql_text
if body.kind not in ("note", "highlight", "card"):
raise HTTPException(status_code=400, detail="kind 는 note/highlight/card")
row = (
await session.execute(
sql_text("INSERT INTO clause_study(doc_id, kind, payload) "
"VALUES (:d, :k, cast(:p AS jsonb)) RETURNING id, kind, payload, created_at")
.bindparams(d=doc_id, k=body.kind, p=json.dumps(body.payload, ensure_ascii=False))
)
).mappings().first()
await session.commit()
return StudyItem(id=row["id"], kind=row["kind"], payload=_parse_payload(row["payload"]),
created_at=row["created_at"])
@router.delete("/{doc_id}/study/{study_id}", status_code=204)
async def delete_study(
doc_id: int,
study_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
from sqlalchemy import text as sql_text
await session.execute(
sql_text("DELETE FROM clause_study WHERE id = :s AND doc_id = :d")
.bindparams(s=study_id, d=doc_id)
)
await session.commit()
+100 -75
View File
@@ -23,6 +23,7 @@ from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Header, HTTPException
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from core.config import settings
@@ -66,6 +67,22 @@ class IngestBody(BaseModel):
attempts: list[IngestAttempt]
def _already_ingested(rows) -> dict:
"""이미 적재된 세션들의 캐시 요약(멱등 응답). 최초 멱등체크 + 동시경합 흡수 양쪽에서 사용."""
return {
"status": "already_ingested",
"sessions": [
{
"topic_id": s.study_topic_id,
"correct": s.correct_count,
"wrong": s.wrong_count,
"unsure": s.unsure_count,
}
for s in rows
],
}
def _parse_answered_at(s: str | None, now: datetime) -> datetime:
if not s:
return now
@@ -98,18 +115,7 @@ async def ingest_attempts(
)
).scalars().all()
if existing:
return {
"status": "already_ingested",
"sessions": [
{
"topic_id": s.study_topic_id,
"correct": s.correct_count,
"wrong": s.wrong_count,
"unsure": s.unsure_count,
}
for s in existing
],
}
return _already_ingested(existing)
# pub_id → source_id(내부 질문 id) 해소. deleted tombstone 제외.
pub_ids = list({a.question_pub_id for a in body.attempts})
@@ -156,73 +162,92 @@ async def ingest_attempts(
if not by_topic:
raise HTTPException(status_code=404, detail="해소된 attempt 없음")
summaries = []
for topic_id, items in by_topic.items():
qids = [q.id for (_, q) in items]
qs = StudyQuizSession(
user_id=user_id,
study_topic_id=topic_id,
question_ids=qids,
subject_distribution={},
status="done",
cursor=len(qids),
source="viewer",
client_session_uuid=body.client_session_uuid,
finished_at=now,
created_at=now,
updated_at=now,
)
session.add(qs)
await session.flush() # qs.id
try:
summaries = []
for topic_id, items in by_topic.items():
qids = [q.id for (_, q) in items]
qs = StudyQuizSession(
user_id=user_id,
study_topic_id=topic_id,
question_ids=qids,
subject_distribution={},
status="done",
cursor=len(qids),
source="viewer",
client_session_uuid=body.client_session_uuid,
finished_at=now,
created_at=now,
updated_at=now,
)
session.add(qs)
await session.flush() # qs.id
c = w = u = 0
for a, q in items:
try:
sel, is_corr, outcome = derive_outcome(a.selected_choice, a.is_unsure, q.correct_choice)
except ValueError:
skipped.append(a.question_pub_id) # 선택 없고 unsure 아님 = 무효 → skip
continue
if outcome == "correct":
c += 1
elif outcome == "wrong":
w += 1
elif outcome == "unsure":
u += 1
session.add(
StudyQuestionAttempt(
user_id=user_id,
study_question_id=q.id,
study_topic_id=topic_id,
selected_choice=sel,
correct_choice=q.correct_choice,
is_correct=is_corr,
outcome=outcome,
quiz_session_id=qs.id,
answered_at=_parse_answered_at(a.answered_at, now),
c = w = u = 0
for a, q in items:
try:
sel, is_corr, outcome = derive_outcome(a.selected_choice, a.is_unsure, q.correct_choice)
except ValueError:
skipped.append(a.question_pub_id) # 선택 없고 unsure 아님 = 무효 → skip
continue
if outcome == "correct":
c += 1
elif outcome == "wrong":
w += 1
elif outcome == "unsure":
u += 1
session.add(
StudyQuestionAttempt(
user_id=user_id,
study_question_id=q.id,
study_topic_id=topic_id,
selected_choice=sel,
correct_choice=q.correct_choice,
is_correct=is_corr,
outcome=outcome,
quiz_session_id=qs.id,
answered_at=_parse_answered_at(a.answered_at, now),
)
)
qs.correct_count, qs.wrong_count, qs.unsure_count = c, w, u
await session.flush()
# finalize 무수정 재생(progress/SR/pattern + 4-A/4-B enqueue). 그 후 멱등 마커.
summary = await finalize_session(
session, user_id=user_id, study_topic_id=topic_id, quiz_session_id=qs.id
)
qs.finalized_at = now
summaries.append(
{
"topic_id": topic_id,
"quiz_session_id": qs.id,
"correct": summary.correct,
"wrong": summary.wrong,
"unsure": summary.unsure,
"newly_correct": summary.newly_correct,
"relapsed": summary.relapsed,
"recovered": summary.recovered,
}
)
await session.commit()
except IntegrityError:
# 동시 같은 client_session_uuid 경합 — 상대가 먼저 commit → (client_session_uuid,
# study_topic_id) uq(mig376) 위반. 데이터는 안전(원자 1-tx 전체 롤백 → SR 이중 advance
# 없음). 승자 결과로 graceful 수렴(500 대신 already_ingested). uuid 경합이 아닌 진짜
# 무결성 오류면 재조회가 비어 → re-raise 로 표면화.
await session.rollback()
winner = (
await session.execute(
select(StudyQuizSession).where(
StudyQuizSession.client_session_uuid == body.client_session_uuid
)
)
qs.correct_count, qs.wrong_count, qs.unsure_count = c, w, u
await session.flush()
).scalars().all()
if not winner:
raise
logger.info("study_ingest uuid=%s 동시경합 흡수 → already_ingested", body.client_session_uuid)
return _already_ingested(winner)
# finalize 무수정 재생(progress/SR/pattern + 4-A/4-B enqueue). 그 후 멱등 마커.
summary = await finalize_session(
session, user_id=user_id, study_topic_id=topic_id, quiz_session_id=qs.id
)
qs.finalized_at = now
summaries.append(
{
"topic_id": topic_id,
"quiz_session_id": qs.id,
"correct": summary.correct,
"wrong": summary.wrong,
"unsure": summary.unsure,
"newly_correct": summary.newly_correct,
"relapsed": summary.relapsed,
"recovered": summary.recovered,
}
)
await session.commit()
logger.info(
"study_ingest uuid=%s user=%s sessions=%s skipped=%s",
body.client_session_uuid, user_id, len(summaries), len(skipped),
+11 -852
View File
@@ -3,42 +3,28 @@
실제 검색 파이프라인(retrieval fusion rerank diversity confidence)
`services/search/search_pipeline.py::run_search()` 분리되어 있다.
파일은 다음만 담당:
- Pydantic 스키마 (SearchResult / SearchResponse / SearchDebug / DebugCandidate
/ Citation / AskResponse / AskDebug)
- Pydantic 스키마 (SearchResult / SearchResponse / SearchDebug / DebugCandidate)
- `/search` endpoint wrapper (run_search 호출 + logger + telemetry + 직렬화)
- `/ask` endpoint wrapper (Phase 3.3 에서 추가)
"""
import asyncio
import hmac
import time
from datetime import date
from typing import Annotated, Literal
from typing import Annotated
from fastapi import APIRouter, BackgroundTasks, Depends, Header, Query
from fastapi import APIRouter, BackgroundTasks, Depends, Query
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import get_current_user
from core.config import settings
from core.auth import get_current_user, get_egress_class
from core.database import get_session
from core.utils import setup_logger
from models.user import User
from services.document_telemetry import sanitize_source
from services.search.classifier_service import ClassifierResult, classify
from services.search.evidence_service import EvidenceItem, extract_evidence
from services.search.fusion_service import DEFAULT_FUSION
from services.search.grounding_check import check as grounding_check
from services.search.refusal_gate import RefusalDecision, decide as refusal_decide
from services.search import query_rewriter
from services.search.retrieval_service import AxisFilter
from services.search.result_decorate import compute_facets, decorate_version_status
from services.search.search_pipeline import PipelineResult, run_search
from services.search.synthesis_service import SynthesisResult, synthesize
from services.search.verifier_service import VerifierResult, verify
from services.prompt_versions import ASK_PROMPT_VERSION, resolve_primary_model
from services.search_telemetry import record_ask_event, record_search_event
from services.search_telemetry import record_search_event
# logs/search.log + stdout 동시 출력 (Phase 0.4)
logger = setup_logger("search")
@@ -153,6 +139,7 @@ def _build_search_debug(pr: PipelineResult) -> SearchDebug:
async def search(
q: str,
user: Annotated[User, Depends(get_current_user)],
egress_class: Annotated[str, Depends(get_egress_class)],
session: Annotated[AsyncSession, Depends(get_session)],
background_tasks: BackgroundTasks,
mode: str = Query("hybrid", pattern="^(fts|trgm|vector|hybrid)$"),
@@ -225,6 +212,8 @@ async def search(
None, description="안전 자료실 C-1: 관할 필터 (KR/US/EU/JP/GB/INT)"),
year_from: int | None = Query(None, ge=1900, le=2100, description="published_date 연도 하한 (NULL=created_at fallback)"),
year_to: int | None = Query(None, ge=1900, le=2100, description="published_date 연도 상한"),
domain_bucket: str | None = Query(None, description="377: domain_bucket 스코프 CSV (Safety,Engineering,Law,Philosophy,Programming,General,News). domain_bucket = ANY"),
exclude_bucket: str | None = Query(None, description="377: domain_bucket 제외 CSV (예: News). 지식질의 시 News 기본제외용"),
facets: bool = Query(False, description="안전 자료실 C-1 후속: top-K 결과 분류 축 분포(material_type/jurisdiction/version_status)를 응답 facets 에 집계. 미지정=계산/노출 0"),
):
"""문서 검색 — FTS + ILIKE + 벡터 결합 (Phase 3.1 이후 run_search wrapper)"""
@@ -235,6 +224,9 @@ async def search(
jurisdiction=jurisdiction,
year_from=year_from,
year_to=year_to,
domain_buckets=[b.strip() for b in domain_bucket.split(",") if b.strip()] if domain_bucket else None,
exclude_buckets=[b.strip() for b in exclude_bucket.split(",") if b.strip()] if exclude_bucket else None,
cloud_egress=(egress_class == "cloud"),
)
pr = await run_search(
session,
@@ -354,836 +346,3 @@ async def search(
debug=debug_obj,
facets=facets_obj,
)
# ═══════════════════════════════════════════════════════════
# Phase 3.3: /api/search/ask — Evidence + Grounded Synthesis
# ═══════════════════════════════════════════════════════════
class Citation(BaseModel):
"""answer 본문의 [n] 에 해당하는 근거 단일 행."""
n: int
chunk_id: int | None
doc_id: int
title: str | None
section_title: str | None
span_text: str # evidence LLM 이 추출한 50~300자
full_snippet: str # 원본 800자 (citation 원문 보기 전용)
relevance: float
rerank_score: float
class ConfirmedItem(BaseModel):
"""Partial answer 의 개별 aspect 답변."""
aspect: str
text: str
citations: list[int]
class AskDebug(BaseModel):
"""`/ask?debug=true` 응답 확장."""
timing_ms: dict[str, float]
search_notes: list[str]
query_analysis: dict | None = None
confidence_signal: float
evidence_candidate_count: int
evidence_kept_count: int
evidence_skip_reason: str | None
synthesis_cache_hit: bool
synthesis_prompt_preview: str | None = None
synthesis_raw_preview: str | None = None
hallucination_flags: list[str] = []
# Phase 3.5a: per-layer defense 로깅
defense_layers: dict | None = None
class AskResponse(BaseModel):
"""`/ask` 응답. Phase 3.5a: completeness + aspects 추가."""
results: list[SearchResult]
ai_answer: str | None
citations: list[Citation]
synthesis_status: Literal[
"completed", "timeout", "skipped", "no_evidence", "parse_failed", "llm_error",
# PR-MacBook-RAG-Backend-1: 200 응답에는 등장하지 않음 (해당 status 는 503 분기).
# Literal 호환성 위해 포함.
"backend_unavailable",
]
synthesis_ms: float
confidence: Literal["high", "medium", "low"] | None
refused: bool
no_results_reason: str | None
query: str
total: int
# Phase 3.5a
completeness: Literal["full", "partial", "insufficient"] = "full"
covered_aspects: list[str] | None = None
missing_aspects: list[str] | None = None
confirmed_items: list[ConfirmedItem] | None = None
# PR-MacBook-RAG-Backend-1: backend dispatcher metadata.
# backend 미지정 호출은 둘 다 None 으로 유지 (기존 호출자 호환 — Hermes docsrv_ask /
# voice-memo-bot 응답 형식 변동 0). 명시 opt-in 시만 채워짐.
backend_requested: str | None = None
backend_used: str | None = None
debug: AskDebug | None = None
def _map_no_results_reason(
pr: PipelineResult,
evidence: list[EvidenceItem],
ev_skip: str | None,
sr: SynthesisResult,
) -> str | None:
"""사용자에게 보여줄 한국어 메시지 매핑.
Failure mode (plan §Failure Modes) 기반.
"""
# LLM 자가 refused → 모델이 준 사유 그대로
if sr.refused and sr.refuse_reason:
return sr.refuse_reason
# synthesis 상태 우선
if sr.status == "no_evidence":
if not pr.results:
return "검색 결과가 없습니다."
return "관련도 높은 근거를 찾지 못했습니다."
if sr.status == "skipped":
return "검색 결과가 없습니다."
if sr.status == "timeout":
return "답변 생성이 지연되어 생략했습니다. 검색 결과를 확인해 주세요."
if sr.status == "parse_failed":
return "답변 형식 오류로 생략했습니다."
if sr.status == "llm_error":
return "AI 서버에 일시적 문제가 있습니다."
# evidence 단계 실패는 fallback 을 탔더라도 notes 용
if ev_skip == "all_low_rerank":
return "관련도 높은 근거를 찾지 못했습니다."
if ev_skip == "empty_retrieval":
return "검색 결과가 없습니다."
return None
def _build_citations(
evidence: list[EvidenceItem], used_citations: list[int]
) -> list[Citation]:
"""answer 본문에 실제로 등장한 n 만 Citation 으로 변환."""
by_n = {e.n: e for e in evidence}
out: list[Citation] = []
for n in used_citations:
e = by_n.get(n)
if e is None:
continue
out.append(
Citation(
n=e.n,
chunk_id=e.chunk_id,
doc_id=e.doc_id,
title=e.title,
section_title=e.section_title,
span_text=e.span_text,
full_snippet=e.full_snippet,
relevance=e.relevance,
rerank_score=e.rerank_score,
)
)
return out
def _build_ask_debug(
pr: PipelineResult,
evidence: list[EvidenceItem],
ev_skip: str | None,
sr: SynthesisResult,
ev_ms: float,
synth_ms: float,
total_ms: float,
) -> AskDebug:
timing: dict[str, float] = dict(pr.timing_ms)
timing["evidence_ms"] = ev_ms
timing["synthesis_ms"] = synth_ms
timing["ask_total_ms"] = total_ms
# candidate count 는 rule filter 통과한 수 (recomputable from results)
# 엄밀히는 evidence_service 내부 숫자인데, evidence 길이 ≈ kept, candidate
# 는 관측이 어려움 → kept 는 evidence 길이, candidate 는 별도 필드 없음.
# 단순화: candidate_count = len(evidence) 를 상한 근사로 둠 (debug 전용).
return AskDebug(
timing_ms=timing,
search_notes=pr.notes,
query_analysis=pr.query_analysis,
confidence_signal=pr.confidence_signal,
evidence_candidate_count=len(evidence),
evidence_kept_count=len(evidence),
evidence_skip_reason=ev_skip,
synthesis_cache_hit=sr.cache_hit,
synthesis_prompt_preview=None, # 현재 synthesis_service 에서 노출 안 함
synthesis_raw_preview=sr.raw_preview,
hallucination_flags=sr.hallucination_flags,
)
def _detect_synthesis_failure(sr: SynthesisResult) -> str | None:
"""Synthesis 가 유효한 답을 못 냈으면 re_gate 라벨, 아니면 None.
판정 우선순위 (Phase 3.5 fix3):
1) sr.refused LLM self-refuse (status="completed") 또는 mechanical fail refused 전파
- status=="completed" + refused=True "synthesis_self_refuse"
- f"synthesis_failed({status})"
2) sr.status {timeout, parse_failed, llm_error} f"synthesis_failed({status})"
3) answer 공백 f"synthesis_failed({status})"
4) 유효 None
"""
if sr.refused:
if sr.status == "completed":
return "synthesis_self_refuse"
return f"synthesis_failed({sr.status})"
if sr.status in ("timeout", "parse_failed", "llm_error"):
return f"synthesis_failed({sr.status})"
if not (sr.answer or "").strip():
return f"synthesis_failed({sr.status})"
return None
def _resolve_eval_identity(
x_source: str | None,
x_eval_case_id: str | None,
x_eval_token: str | None,
) -> tuple[str, str | None]:
"""X-Source/X-Eval-Case-Id 신뢰 검증 (Phase 3.5 fix2).
규칙:
- 기본값: source='document_server', eval_case_id=None
- X-Source=eval 또는 X-Eval-Case-Id 들어왔다면 eval claim 으로 간주
- eval claim X-Eval-Token == settings.eval_runner_token 때만 수용
(constant-time compare, env 미설정 항상 거부)
- 거부 : 헤더 무시 + warning log + source=sanitize(non-eval) / eval_case_id=None
- 통과 : source='eval', eval_case_id=x_eval_case_id
반환: (source, eval_case_id)
"""
claimed_source = sanitize_source(x_source)
is_eval_claim = (claimed_source == "eval") or bool(x_eval_case_id)
if not is_eval_claim:
# 일반 호출 — eval_case_id 강제 None (source != 'eval' 이면 case_id 의미 없음)
return claimed_source, None
# eval claim — token 검증
expected = settings.eval_runner_token
presented = x_eval_token or ""
token_valid = bool(expected) and hmac.compare_digest(presented, expected)
if not token_valid:
logger.warning(
"eval header rejected: source=%s case_id=%s token_present=%s expected_set=%s",
x_source, x_eval_case_id, bool(x_eval_token), bool(expected),
)
# 일반 호출로 강등 — source='eval' 주장은 무시, case_id 도 무시
# claimed_source 가 'eval' 이면 default 'document_server' 로
if claimed_source == "eval":
return "document_server", None
return claimed_source, None
# token OK — eval 라벨 수용
return "eval", x_eval_case_id
@router.get("/ask", response_model=AskResponse)
async def ask(
q: str,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
background_tasks: BackgroundTasks,
limit: int = Query(10, ge=1, le=20, description="synthesis 입력 상한"),
debug: bool = Query(False, description="evidence/synthesis 중간 상태 노출"),
backend: Annotated[
str | None,
Query(
pattern="^(qwen-macbook|gemma-macmini|mac-mini-default|claude-cloud|auto)$",
description=(
"PR-2 of DS AI routing policy (2026-05-23) — 명시 backend opt-in via llm-router. "
"미지정 = mac-mini-default (gemma-macmini alias, default). "
"'mac-mini-default' = router 가 tier_b (Mac mini gemma-4-26b). "
"'qwen-macbook' = router 가 named upstream (M5 Max Qwen 3.6 27B). "
"'claude-cloud' = router 가 503 provider_not_configured (활성화 별 PR). "
"'auto' = router 의 rule + LLM triage. "
"backend unavailable 시 503 + error_reason=macbook_unavailable / router_* "
"(자동 fallback 없음 — 다시 호출하거나 backend 인자 제거 후 재시도)."
),
),
] = None,
corpus_variant: str | None = Query(
None,
pattern=r"^(prehier|hier_sim_raw|hier_sim_clean)$",
description=(
"⚠️ EVAL-ONLY (Hier-PassageRAG-Diagnose-1). evidence retrieval 의 chunk leg 를 측정 뷰로 "
"교체 — prehier(legacy) | hier_sim_raw | hier_sim_clean. 운영 UI 미사용. "
"미지정 = production corpus_chunks (기존 /ask 동작 동일)."
),
),
exact_knn: bool = Query(
False,
description=(
"⚠️ EVAL-ONLY (Hier-PassageRAG-Diagnose-1). vector leg exact KNN (ivfflat 근사 제거). "
"passage 변종 공정 비교용. 운영 미사용. 미지정(false) = 기존 /ask 동작 동일."
),
),
x_source: Annotated[str | None, Header(alias="X-Source")] = None,
x_eval_case_id: Annotated[str | None, Header(alias="X-Eval-Case-Id")] = None,
x_eval_token: Annotated[str | None, Header(alias="X-Eval-Token")] = None,
):
"""근거 기반 AI 답변 (Phase 3.5a).
Phase 3.3 기반 + classifier parallel + refusal gate + grounding re-gate.
실패 경로에서도 `results` 항상 반환.
Phase 3.5 calibration trust boundary (fix2):
- X-Source / X-Eval-Case-Id X-Eval-Token EVAL_RUNNER_TOKEN 일치하는
trusted internal eval runner 에서만 수용된다.
- 일반 client X-Source=eval 시도는 무시되고 source='document_server' 강제.
- source != 'eval' 이면 eval_case_id 항상 None.
"""
t_total = time.perf_counter()
defense_log: dict = {} # per-layer flag snapshot
source, eval_case_id = _resolve_eval_identity(x_source, x_eval_case_id, x_eval_token)
# 1. 검색 파이프라인 (corpus_variant/exact_knn = EVAL-ONLY, 미지정 시 기존 동작 동일)
pr = await run_search(
session, q, mode="hybrid", limit=limit,
fusion=DEFAULT_FUSION, rerank=True, analyze=True,
corpus_variant=corpus_variant, exact_knn=exact_knn,
)
# 1.5. ask_includable=false 문서를 evidence 입력에서 제외
# 검색 결과 자체는 유지 (사용자에게 보여줌), evidence만 필터
if pr.results:
from sqlalchemy import select as sa_select
from models.document import Document as DocModel
ask_doc_ids = set()
excluded_ids = {r.id for r in pr.results}
rows = await session.execute(
sa_select(DocModel.id, DocModel.ask_includable).where(
DocModel.id.in_(excluded_ids)
)
)
for doc_id, includable in rows:
if includable is False:
ask_doc_ids.add(doc_id)
evidence_results = [r for r in pr.results if r.id not in ask_doc_ids]
else:
evidence_results = pr.results
# 2. Evidence + Classifier 병렬
t_ev = time.perf_counter()
evidence_task = asyncio.create_task(extract_evidence(q, evidence_results))
# classifier input: top 3 chunks meta + rerank scores
top_chunks = [
{
"title": r.title or "",
"section": r.section_title or "",
"snippet": (r.snippet or "")[:200],
}
for r in pr.results[:3]
]
rerank_scores_top = [
r.rerank_score if r.rerank_score is not None else r.score
for r in pr.results[:3]
]
classifier_task = asyncio.create_task(
classify(q, top_chunks, rerank_scores_top)
)
evidence, ev_skip = await evidence_task
ev_ms = (time.perf_counter() - t_ev) * 1000
# classifier await (timeout 보호 — classifier_service 내부에도 있지만 여기서 이중 보호)
# 2026-05-17: 6s outer wrapper 가 classifier_service.LLM_TIMEOUT_MS (30s) 를 override → 동시 부하 시
# 거의 모든 classifier 호출 timeout → conservative_refuse(no_classifier) 경로. 15s 로 상향 — classifier
# 가 실제 작동하도록 (단, ask 전체 응답 시간 상한 영향: ev_ms + max(classifier_wait, evidence_extract) +
# synth_ms + verifier 누적).
# 2026-05-17 B-3: 15s 도 동시 부하 시 부족 (classifier_service LLM_TIMEOUT_MS 30s 와 misalign).
# 30s 로 align → classifier 동작 안정. ask 응답 latency 상한 ↑ 의도.
try:
classifier_result = await asyncio.wait_for(classifier_task, timeout=30.0)
except asyncio.CancelledError:
raise # 요청 취소는 전파 — broad except 가 삼키지 않게 명시 (R3)
except Exception:
classifier_result = ClassifierResult("timeout", None, [], [], 0.0)
defense_log["classifier"] = {
"status": classifier_result.status,
"verdict": classifier_result.verdict,
"covered_aspects": classifier_result.covered_aspects,
"missing_aspects": classifier_result.missing_aspects,
"elapsed_ms": classifier_result.elapsed_ms,
}
# 3. Refusal gate (multi-signal fusion)
all_rerank_scores = [
e.rerank_score for e in evidence
] if evidence else rerank_scores_top
decision = refusal_decide(all_rerank_scores, classifier_result)
defense_log["score_gate"] = {
"max": max(all_rerank_scores) if all_rerank_scores else 0.0,
"agg_top3": sum(sorted(all_rerank_scores, reverse=True)[:3]),
}
defense_log["refusal"] = {
"refused": decision.refused,
"rule_triggered": decision.rule_triggered,
}
if decision.refused:
total_ms = (time.perf_counter() - t_total) * 1000
no_reason = "관련 근거를 찾지 못했습니다."
if not pr.results:
no_reason = "검색 결과가 없습니다."
logger.info(
"ask REFUSED query=%r rule=%s max_score=%.2f total=%.0f",
q[:80], decision.rule_triggered,
max(all_rerank_scores) if all_rerank_scores else 0.0, total_ms,
)
# telemetry — search + ask_events 두 경로 동시
background_tasks.add_task(
record_search_event, q, user.id, pr.results, "hybrid",
pr.confidence_signal, pr.analyzer_confidence,
)
# input_snapshot (디버깅/재현용)
defense_log["input_snapshot"] = {
"query": q,
"top_chunks_preview": [
{"title": c.get("title", ""), "snippet": c.get("snippet", "")[:100]}
for c in top_chunks[:3]
],
"answer_preview": None,
}
background_tasks.add_task(
record_ask_event,
q, user.id, "insufficient", "skipped", None,
True, classifier_result.verdict,
max(all_rerank_scores) if all_rerank_scores else 0.0,
sum(sorted(all_rerank_scores, reverse=True)[:3]),
[], len(evidence), 0,
defense_log, int(total_ms),
# Phase E.1 측정 필드
answer_length=0,
covered_aspects=classifier_result.covered_aspects or None,
missing_aspects=classifier_result.missing_aspects or None,
model_name=resolve_primary_model(),
prompt_version=ASK_PROMPT_VERSION,
# Phase 3.5 calibration
source=source,
eval_case_id=eval_case_id,
)
debug_obj = None
if debug:
debug_obj = AskDebug(
timing_ms={**pr.timing_ms, "evidence_ms": ev_ms, "ask_total_ms": total_ms},
search_notes=pr.notes,
confidence_signal=pr.confidence_signal,
evidence_candidate_count=len(evidence),
evidence_kept_count=len(evidence),
evidence_skip_reason=ev_skip,
synthesis_cache_hit=False,
hallucination_flags=[],
defense_layers=defense_log,
)
return AskResponse(
results=pr.results,
ai_answer=None,
citations=[],
synthesis_status="skipped",
synthesis_ms=0.0,
confidence=None,
refused=True,
no_results_reason=no_reason,
query=q,
total=len(pr.results),
completeness="insufficient",
covered_aspects=classifier_result.covered_aspects or None,
missing_aspects=classifier_result.missing_aspects or None,
# refusal gate 단계에서는 backend 호출 자체가 일어나지 않음 →
# backend_used = None. backend_requested 는 호출자 의도 표시용.
backend_requested=backend,
backend_used=None,
debug=debug_obj,
)
# 4. Synthesis (backend dispatcher 적용 — PR-MacBook-RAG-Backend-1)
t_synth = time.perf_counter()
sr = await synthesize(q, evidence, debug=debug, backend=backend)
synth_ms = (time.perf_counter() - t_synth) * 1000
# 4.1. backend_unavailable → 503 fail-fast (자동 fallback 금지)
# 명시 opt-in backend (예: qwen-macbook) 가 비가용일 때만 발생. /ask wrapper 는
# 절대 다른 backend 로 재시도하지 않음. 사용자가 backend 인자 제거 또는 wake 후 재시도.
if sr.status == "backend_unavailable":
backend_requested_val = backend or "gemma-macmini"
total_ms = (time.perf_counter() - t_total) * 1000
logger.warning(
"ask backend_unavailable backend=%s query=%r total_ms=%.0f flags=%s",
backend_requested_val, q[:80], total_ms,
",".join(sr.hallucination_flags) if sr.hallucination_flags else "-",
)
# error_reason 명명 — macbook_unavailable 만 정착 (자동 fallback 부재).
error_reason = (
"macbook_unavailable"
if backend_requested_val == "qwen-macbook"
else "backend_unavailable"
)
# telemetry — search 만 기록 (ask_events 는 200 응답 path 전용)
background_tasks.add_task(
record_search_event, q, user.id, pr.results, "hybrid",
pr.confidence_signal, pr.analyzer_confidence,
)
return JSONResponse(
status_code=503,
content={
"error": "backend_unavailable",
"error_reason": error_reason,
"backend_requested": backend_requested_val,
"backend_used": None,
"query": q,
"detail": (
"명시 선택한 backend 가 일시적으로 응답할 수 없습니다. "
"MacBook 깨우거나 backend 인자를 제거하고 (기본 Gemma) 다시 호출하세요."
),
},
)
# 5. Grounding check + Verifier (조건부 병렬) + re-gate (Phase 3.5b)
grounding = grounding_check(q, sr.answer or "", evidence)
# verifier skip: grounding strong 2+ OR retrieval 자체가 망함
grounding_only_strong = [
f for f in grounding.strong_flags if not f.startswith("verifier_")
]
max_rerank = max(all_rerank_scores, default=0.0)
if len(grounding_only_strong) >= 2 or max_rerank < 0.2:
verifier_result = VerifierResult("skipped", [], 0.0)
else:
verifier_task = asyncio.create_task(
verify(q, sr.answer or "", evidence)
)
# 2026-05-17 B-3: 4s outer wait_for 가 verifier_service LLM_TIMEOUT_MS (10s) 를 override
# → classifier 와 동일 패턴 (search.py:522 가 6s→15s swap 했던 case). 10s 로 align.
try:
verifier_result = await asyncio.wait_for(verifier_task, timeout=10.0)
except asyncio.CancelledError:
raise # 요청 취소는 전파 — broad except 가 삼키지 않게 명시 (R3)
except Exception:
verifier_result = VerifierResult("timeout", [], 0.0)
# Verifier contradictions → grounding flags 머지 (prefix 로 구분, severity 3단계)
for c in verifier_result.contradictions:
if c.severity == "strong":
grounding.strong_flags.append(f"verifier_{c.type}:{c.claim[:30]}")
elif c.severity == "medium":
grounding.weak_flags.append(f"verifier_{c.type}_medium:{c.claim[:30]}")
else:
grounding.weak_flags.append(f"verifier_{c.type}:{c.claim[:30]}")
defense_log["evidence"] = {
"skip_reason": ev_skip,
"kept_count": len(evidence),
}
defense_log["grounding"] = {
"strong": grounding.strong_flags,
"weak": grounding.weak_flags,
}
defense_log["verifier"] = {
"status": verifier_result.status,
"contradictions_count": len(verifier_result.contradictions),
"strong_count": sum(1 for c in verifier_result.contradictions if c.severity == "strong"),
"medium_count": sum(1 for c in verifier_result.contradictions if c.severity == "medium"),
"elapsed_ms": verifier_result.elapsed_ms,
}
# ── Re-gate: 7-tier completeness 결정 (Phase 3.5 B2 — Tier 4 신규 삽입, 재번호) ──
# 기존 6-tier (3.5b 4차 리뷰) + Tier 4(g_strong + v_strong_numeric + low_conf → refuse).
# 호환성: defense_layers["re_gate"] 의 string literal 들은 기존 그대로 유지.
# 신규 "refuse(grounding+verifier_numeric)" 만 추가.
completeness: Literal["full", "partial", "insufficient"] = "full"
covered_aspects = classifier_result.covered_aspects or None
missing_aspects = classifier_result.missing_aspects or None
confirmed_items: list[ConfirmedItem] | None = None
# verifier/grounding strong 구분
g_strong = [f for f in grounding.strong_flags if not f.startswith("verifier_")]
v_strong = [f for f in grounding.strong_flags if f.startswith("verifier_")]
v_medium = [f for f in grounding.weak_flags if f.startswith("verifier_") and "_medium:" in f]
has_direct_negation = any("direct_negation" in f for f in v_strong)
# Phase 3.5 B2: verifier strong flags 중 numeric_conflict 만 카운트.
# promote(VERIFIER_NUMERIC_PROMOTE=1) 활성 시 critical numeric_conflict 가 strong 으로 승격되며
# 여기 카운트에 잡힘. promote off 면 항상 0 → Tier 4 활성 안 됨 (기존 동작 유지).
v_strong_numeric = sum(
1 for f in v_strong if f.startswith("verifier_numeric_conflict")
)
# ── Tier 0 (Phase 3.5 fix3): synthesis 자체 실패 처리 ──
# LLM self-refuse, 메커니즘 실패(timeout/parse_failed/llm_error), answer 공백.
# 빈 답에 대해 grounding/verifier flag 가 0건이라 기존 체인이 "else clean" 으로 빠지며
# completeness="full" 초기값이 보존되던 모순을 여기서 일관되게 차단.
# 과거 baseline(v1-400char) 에서 20(self-refuse)+4(timeout) = 24/223 (10.8%) 해당.
tier0_label = _detect_synthesis_failure(sr)
if tier0_label:
completeness = "insufficient"
sr.answer = None
sr.refused = True
sr.confidence = None
defense_log["re_gate"] = tier0_label
elif len(g_strong) >= 2:
# Tier 1: grounding strong 2+ → refuse
completeness = "insufficient"
sr.answer = None
sr.refused = True
sr.confidence = None
defense_log["re_gate"] = "refuse(grounding_2+strong)"
elif g_strong and has_direct_negation:
# Tier 2: grounding strong + verifier direct_negation → refuse
completeness = "insufficient"
sr.answer = None
sr.refused = True
sr.confidence = None
defense_log["re_gate"] = "refuse(grounding+direct_negation)"
elif g_strong and sr.confidence == "low" and max_rerank < 0.25:
# Tier 3: grounding strong 1 + (low confidence AND weak evidence) → refuse
completeness = "insufficient"
sr.answer = None
sr.refused = True
sr.confidence = None
defense_log["re_gate"] = "refuse(grounding+low_conf+weak_ev)"
elif g_strong and v_strong_numeric >= 1 and sr.confidence == "low":
# Tier 4 (B2 신규): grounding strong + verifier numeric_conflict strong + low conf → refuse.
# verifier strong 단독 refuse 금지 원칙 유지 — g_strong 교차 필수.
completeness = "insufficient"
sr.answer = None
sr.refused = True
sr.confidence = None
defense_log["re_gate"] = "refuse(grounding+verifier_numeric)"
elif g_strong or has_direct_negation:
# Tier 5 (기존 4): grounding strong 1 또는 verifier direct_negation 단독 → partial
completeness = "partial"
sr.confidence = "low"
defense_log["re_gate"] = "partial(strong_or_negation)"
elif v_medium:
# Tier 6 (기존 5): verifier medium 누적 → count 기반 confidence 하향
medium_count = len(v_medium)
if medium_count >= 3:
sr.confidence = "low"
defense_log["re_gate"] = f"conf_low(medium_x{medium_count})"
elif medium_count == 2 and sr.confidence == "high":
sr.confidence = "medium"
defense_log["re_gate"] = "conf_cap_medium(medium_x2)"
else:
defense_log["re_gate"] = f"medium_x{medium_count}(no_action)"
elif grounding.weak_flags:
# Tier 7 (기존 6): weak → confidence 한 단계 하향
if sr.confidence == "high":
sr.confidence = "medium"
defense_log["re_gate"] = "conf_lower(weak)"
else:
defense_log["re_gate"] = "clean"
# Confidence cap from refusal gate (classifier 부재 시 conservative)
if decision.confidence_cap and sr.confidence:
conf_rank = {"low": 0, "medium": 1, "high": 2}
if conf_rank.get(sr.confidence, 0) > conf_rank.get(decision.confidence_cap, 2):
sr.confidence = decision.confidence_cap
# Partial 이면 max confidence = medium
if completeness == "partial" and sr.confidence == "high":
sr.confidence = "medium"
sr.hallucination_flags.extend(
[f"strong:{f}" for f in grounding.strong_flags]
+ [f"weak:{f}" for f in grounding.weak_flags]
)
total_ms = (time.perf_counter() - t_total) * 1000
# 6. 응답 구성
citations = _build_citations(evidence, sr.used_citations)
no_reason = _map_no_results_reason(pr, evidence, ev_skip, sr)
if completeness == "insufficient" and not no_reason:
# Tier 0 경로: synthesis self-refuse 는 LLM 이 준 사유가 가장 정확.
if sr.refused and sr.refuse_reason:
no_reason = sr.refuse_reason
else:
no_reason = "답변 검증에서 복수 오류 감지"
logger.info(
"ask query=%r results=%d evidence=%d cite=%d synth=%s conf=%s completeness=%s "
"refused=%s grounding_strong=%d grounding_weak=%d ev_ms=%.0f synth_ms=%.0f total=%.0f",
q[:80], len(pr.results), len(evidence), len(citations),
sr.status, sr.confidence or "-", completeness,
sr.refused, len(grounding.strong_flags), len(grounding.weak_flags),
ev_ms, synth_ms, total_ms,
)
# 7. telemetry — search + ask_events 두 경로 동시
background_tasks.add_task(
record_search_event, q, user.id, pr.results, "hybrid",
pr.confidence_signal, pr.analyzer_confidence,
)
# input_snapshot (디버깅/재현용)
defense_log["input_snapshot"] = {
"query": q,
"top_chunks_preview": [
{"title": (r.title or "")[:50], "snippet": (r.snippet or "")[:100]}
for r in pr.results[:3]
],
"answer_preview": (sr.answer or "")[:200],
}
background_tasks.add_task(
record_ask_event,
q, user.id, completeness, sr.status, sr.confidence,
sr.refused, classifier_result.verdict,
max(all_rerank_scores) if all_rerank_scores else 0.0,
sum(sorted(all_rerank_scores, reverse=True)[:3]),
sr.hallucination_flags, len(evidence), len(citations),
defense_log, int(total_ms),
# Phase E.1 측정 필드
answer_length=len(sr.answer or ""),
covered_aspects=covered_aspects,
missing_aspects=missing_aspects,
model_name=resolve_primary_model(),
prompt_version=ASK_PROMPT_VERSION,
# Phase 3.5 calibration
source=source,
eval_case_id=eval_case_id,
)
debug_obj = None
if debug:
timing = dict(pr.timing_ms)
timing["evidence_ms"] = ev_ms
timing["synthesis_ms"] = synth_ms
timing["ask_total_ms"] = total_ms
debug_obj = AskDebug(
timing_ms=timing,
search_notes=pr.notes,
query_analysis=pr.query_analysis,
confidence_signal=pr.confidence_signal,
evidence_candidate_count=len(evidence),
evidence_kept_count=len(evidence),
evidence_skip_reason=ev_skip,
synthesis_cache_hit=sr.cache_hit,
synthesis_raw_preview=sr.raw_preview,
hallucination_flags=sr.hallucination_flags,
defense_layers=defense_log,
)
# backend_used: synthesize 가 실제 호출한 backend (backend 인자 그대로 신뢰 OK —
# backend_unavailable 은 위 503 분기에서 이미 return 됨).
backend_used_val = backend or "gemma-macmini"
return AskResponse(
results=pr.results,
ai_answer=sr.answer,
citations=citations,
synthesis_status=sr.status,
synthesis_ms=sr.elapsed_ms,
confidence=sr.confidence,
refused=sr.refused,
no_results_reason=no_reason,
query=q,
total=len(pr.results),
completeness=completeness,
covered_aspects=covered_aspects,
missing_aspects=missing_aspects,
confirmed_items=confirmed_items,
backend_requested=backend,
backend_used=backend_used_val,
debug=debug_obj,
)
# ─── PR-DocSrv-Ask-ToolCalling-ReAct-1 ────────────────────────────────────
# /api/search/ask/react — Qwen native tool calling 로 ReAct loop.
# 본 endpoint 는 qwen-macbook only (endpoint 자체가 implicit opt-in).
# MacBook unavailable 시 503 + error_reason=macbook_unavailable. Gemma 자동 fallback X.
# G0-2 counter semantics: max_tool_rounds=2, max LLM calls=3, search exec ≤ 2.
# G0-3 trace exposure: default response 의 debug_trace=None, debug=True 시만 채움.
class AskReactRequest(BaseModel):
query: str
debug: bool = False
class AskReactResponse(BaseModel):
final_answer: str
iterations: int
partial: bool
sources: list[dict]
debug_trace: list[dict] | None = None
@router.post("/ask/react", response_model=AskReactResponse)
async def ask_react(
payload: AskReactRequest,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""ReAct loop endpoint (qwen-macbook only, no fallback).
호출자가 명시 opt-in endpoint. MacBook sleep / unreachable / 5xx
HTTP 503 + body `{error_reason: "macbook_unavailable", backend: "qwen-macbook"}`
반환한다. Gemma Mac mini 자동 fallback 하지 않는다 (정정 4 연장).
request body:
- query: str (사용자 원본 질의)
- debug: bool (default false; true 응답 `debug_trace` 채움)
response body (성공 200):
- final_answer: str (Qwen 종합문, partial 있음)
- iterations: int (실제 진행된 tool round )
- partial: bool (max_tool_rounds 도달 LLM content 비었을 true)
- sources: list[dict] (검색에서 모인 evidence 메타, id-기준 dedup)
- debug_trace: list[dict] | null (debug=true round trace)
"""
# 지연 import — 순환 의존성 회피 (react_loop 가 api.search.SearchResult 사용 안 함)
from services.llm.backends import BackendUnavailable, get_backend
from services.search.react_loop import agentic_ask_loop
backend_inst = get_backend("qwen-macbook")
# PR-2 of DS AI routing policy: backend_inst may be RouterBackend (default)
# or QwenMacBookBackend (DS_BACKENDS_VIA_ROUTER=false rollback). Both
# implement generate_with_tools so the ReAct loop is identical.
assert hasattr(backend_inst, "generate_with_tools")
try:
result = await agentic_ask_loop(
session,
payload.query,
backend=backend_inst,
debug=payload.debug,
)
except BackendUnavailable as exc:
logger.warning(
"ask_react backend unavailable backend=%s reason=%s",
exc.backend_name, exc.reason,
)
return JSONResponse(
status_code=503,
content={
"error_reason": "macbook_unavailable",
"backend_requested": "qwen-macbook",
"backend_used": None,
"detail": exc.reason,
},
)
return AskReactResponse(
final_answer=result.final_answer,
iterations=result.iterations,
partial=result.partial,
sources=result.sources,
debug_trace=result.debug_trace,
)
+94
View File
@@ -0,0 +1,94 @@
"""study_concepts API — 이론공부 홈(오늘의 개념 · 진도 · 회독 SR). prefix = /api/study.
문제풀이 표면 무접촉. 개념문서(가스기사 태그) 읽기 집계 + 회독 SR write . 단일 토픽(가스기사=4).
경로: GET /curriculum · GET /today-concepts · POST /concepts/{doc_id}/read.
"""
from __future__ import annotations
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import get_current_user
from core.database import get_session
from models.user import User
from services.study import concept_curriculum as cc
from services.study import concept_links as cl
router = APIRouter()
# 가스기사 단일 토픽 운영(현행). 다토픽 확장 시 쿼리 파라미터로 승격.
DEFAULT_TOPIC_ID = 4
@router.get("/curriculum")
async def get_curriculum(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
topic_id: int = DEFAULT_TOPIC_ID,
):
"""과목별 회독 진도 + 개념/문항 복습 due 요약."""
return await cc.curriculum(session, user.id, topic_id)
@router.get("/today-concepts")
async def get_today_concepts(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
topic_id: int = DEFAULT_TOPIC_ID,
limit: int = 6,
):
"""오늘 공부할 개념(재복습 → 미독 빈출순)."""
return await cc.today_concepts(session, user.id, topic_id, limit)
@router.get("/concepts/weakness-map")
async def get_weakness_map(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
topic_id: int = DEFAULT_TOPIC_ID,
limit: int = 12,
):
"""개념 약점 지도 — 링크된 기출 정답률로 약점 개념(정답률<60%) 우선(이론↔문제)."""
name = await cc._topic_name(session, topic_id)
if not name:
return {"weak": [], "weak_total": 0, "evaluated_total": 0}
return await cl.weakness_map(session, user.id, name, limit)
@router.get("/concepts/{doc_id}")
async def get_concept_detail(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
topic_id: int = DEFAULT_TOPIC_ID,
):
"""개념 리더 재료 — 구조 파싱(요약/본문/빈출/관련) + 백링크 해소 + 회독/SR + 이전/다음."""
detail = await cc.concept_detail(session, user.id, topic_id, doc_id)
if detail is None:
raise HTTPException(status_code=404, detail="concept not found")
return detail
@router.get("/concepts/{doc_id}/questions")
async def get_concept_questions(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
limit: int = 20,
):
"""개념 관련 기출 + 내 정답률 (이론↔문제 브리지)."""
return await cl.related_questions(session, user.id, doc_id, limit)
@router.post("/concepts/{doc_id}/read")
async def post_concept_read(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
topic_id: int = DEFAULT_TOPIC_ID,
):
"""개념 회독 처리 → 회독 플래그 + SR 입고/전진."""
return await cc.mark_read(session, user.id, topic_id, doc_id)
+11 -2
View File
@@ -31,11 +31,11 @@ def hash_password(password: str) -> str:
return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
def create_access_token(subject: str, expires_minutes: int | None = None) -> str:
def create_access_token(subject: str, expires_minutes: int | None = None, egress: str = "local") -> str:
minutes = expires_minutes if expires_minutes is not None else ACCESS_TOKEN_EXPIRE_MINUTES
now = datetime.now(timezone.utc)
expire = now + timedelta(minutes=minutes)
payload = {"sub": subject, "exp": expire, "iat": int(now.timestamp()), "type": "access"}
payload = {"sub": subject, "exp": expire, "iat": int(now.timestamp()), "type": "access", "egress": egress}
return jwt.encode(payload, settings.jwt_secret, algorithm=ALGORITHM)
@@ -100,6 +100,15 @@ def verify_totp(code: str, secret: str | None = None) -> bool:
return totp.verify(code)
async def get_egress_class(
credentials: Annotated[HTTPAuthorizationCredentials, Depends(security)],
) -> str:
"""토큰 egress claim -> 'cloud'|'local' (갭2 cloud-egress allowlist). claim 부재=local
(비파괴; 기존 토큰=신뢰/로컬). 쿼리파라미터 아님 -> 호출자가 없음(우회 차단)."""
payload = decode_token(credentials.credentials)
return (payload or {}).get("egress", "local")
async def get_current_user(
credentials: Annotated[HTTPAuthorizationCredentials, Depends(security)],
session: Annotated[AsyncSession, Depends(get_session)],
+21 -15
View File
@@ -57,12 +57,12 @@ def _parse_migration_files(migrations_dir: Path) -> list[tuple[int, str, Path]]:
def _validate_sql_content(name: str, sql: str) -> None:
"""migration SQL에 BEGIN/COMMIT이 포함되어 있으면 에러 (외부 트랜잭션 깨짐 방지)"""
# 주석(-- ...) 라인 제거 후 검사
lines = [
line for line in sql.splitlines()
if not line.strip().startswith("--")
]
stripped = "\n".join(lines).upper()
# 주석(전체 줄 + 인라인 `-- ...`) 제거 후 검사. ★인라인 주석을 안 지우면 설명 주석의
# 'commit/begin' 단어(예 365_scan_jobs 의 `-- commit 시 documents.title 로 전파`)를
# 트랜잭션 제어문으로 false-positive 로 잡아 fresh DB/DR 부트스트랩이 깨진다(verification
# 실측 2026-06). 줄별로 `--` 이후를 잘라 주석 텍스트를 검사에서 제외.
cleaned = [re.sub(r"--.*$", "", line) for line in sql.splitlines()]
stripped = "\n".join(cleaned).upper()
for keyword in ("BEGIN", "COMMIT", "ROLLBACK"):
# 단어 경계로 매칭 (예: BEGIN_SOMETHING은 제외)
if re.search(rf"\b{keyword}\b", stripped):
@@ -70,6 +70,13 @@ def _validate_sql_content(name: str, sql: str) -> None:
f"migration {name}{keyword} 포함됨 — "
f"migration SQL에는 트랜잭션 제어문을 넣지 마세요"
)
# schema_migrations 수정 금지 (runner 가 스탬프 관리) — 주석 제외(stripped) 검사.
# (구: _run_migrations 의 raw `"schema_migrations" in sql.lower()` 가 주석 미제외라
# 365 의 '-- ... schema_migrations 를 건드리지 않음' 주석을 false-positive 로 잡았음.)
if "SCHEMA_MIGRATIONS" in stripped:
raise RuntimeError(
f"Migration {name} must not modify schema_migrations table"
)
# R1: baseline 스냅샷이 대표하는 마지막 마이그레이션 버전 (이하 버전은 baseline 에 포함).
@@ -167,16 +174,15 @@ async def _run_migrations(conn) -> None:
for version, name, path in pending:
sql = path.read_text(encoding="utf-8")
_validate_sql_content(name, sql)
if "schema_migrations" in sql.lower():
raise ValueError(
f"Migration {name} must not modify schema_migrations table"
)
_validate_sql_content(name, sql) # BEGIN/COMMIT + schema_migrations 검사(주석 제외)
logger.info(f"[migration] {name} 실행 중...")
# raw driver SQL 사용 — text() 의 :name bind parameter 해석으로
# SQL 주석/literal 에 콜론이 들어가면 InvalidRequestError 발생.
# exec_driver_sql 은 SQL 을 driver(asyncpg) 에 그대로 전달.
await conn.exec_driver_sql(sql)
# raw asyncpg simple 프로토콜로 실행 — baseline 적재(_load_baseline_if_fresh)와 동일.
# ★exec_driver_sql 은 prepared 프로토콜이라 multi-statement 불허("cannot insert multiple
# commands into a prepared statement"). 365_scan_jobs 처럼 테이블+시드+인덱스를 한 파일에
# 담은 마이그(컨벤션상 1-statement 권장이나 이미 prod 적재)도 fresh DB/DR replay 되게
# simple execute 사용. text() :name 콜론-binding 이슈도 동일하게 회피(raw 전달).
raw = await conn.get_raw_connection()
await raw.driver_connection.execute(sql)
await conn.execute(
text("INSERT INTO schema_migrations (version, name) VALUES (:v, :n)"),
{"v": version, "n": name},
+3
View File
@@ -33,6 +33,7 @@ from api.study_sessions import router as study_sessions_router
from api.study_topics import router as study_topics_router
from api.study_reminders import router as study_reminders_router
from api.study_cards import router as study_cards_router
from api.study_concepts import router as study_concepts_router
from api.video import router as video_router
from core.config import settings
from core.database import async_session, engine, init_db
@@ -249,6 +250,8 @@ app.include_router(study_reminders_router, prefix="/api/study-reminders", tags=[
app.include_router(study_cards_router, prefix="/api/study-cards", tags=["study-cards"])
# Phase 1: 학습 진행 상태 (review-complete + review-queue). prefix=/api/study-topics 안에 정의됨.
app.include_router(study_question_progress_router, prefix="/api", tags=["study-progress"])
# 이론공부 홈: 오늘의 개념·진도·회독 SR (개념문서 소비 표면, 문제풀이 무접촉).
app.include_router(study_concepts_router, prefix="/api/study", tags=["study-theory"])
# TODO: Phase 5에서 추가
# app.include_router(tasks.router, prefix="/api/tasks", tags=["tasks"])
+4
View File
@@ -56,5 +56,9 @@ class PublishOutbox(Base):
DateTime(timezone=True), default=datetime.now, nullable=False
)
processed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# mig378: 행별 격리 재시도/terminal. attempts=savepoint 실패 누적, failed_at=MAX 초과 terminal
# (set 시 워커 select 에서 제외 → head-of-line block 방지).
attempts: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)
failed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# 미처리 부분 인덱스 idx(id) WHERE processed_at IS NULL = mig372.
+46
View File
@@ -0,0 +1,46 @@
"""study_concept_progress — 사용자 × 개념문서 단위 간격반복(SR) 진행 (이론공부 홈).
문제 SR(study_question_progress) 개념(이론). '개념문서' = documents (가스기사 태그).
회독( read) 복습 진입, 이후 회독마다 sr_schedule 산술(1·3·7·14·졸업) 공용 전진.
concept_doc_id documents.id 가리키나 FK 미설정 hot 테이블(documents) 회피(clause_study 선례).
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import BigInteger, DateTime, ForeignKey, SmallInteger, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from core.database import Base
class StudyConceptProgress(Base):
__tablename__ = "study_concept_progress"
__table_args__ = (
UniqueConstraint(
"user_id", "concept_doc_id", name="uq_concept_progress_user_doc"
),
)
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
user_id: Mapped[int] = mapped_column(
BigInteger, ForeignKey("users.id", ondelete="CASCADE"), nullable=False
)
study_topic_id: Mapped[int] = mapped_column(
BigInteger, ForeignKey("study_topics.id", ondelete="CASCADE"), nullable=False
)
# documents.id 참조 — FK 없음(락 회피). 개념문서 삭제 시 고아 행은 read 집계에서 자연 제외.
concept_doc_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
# 복습 큐 (sr_schedule 공용): stage 0~3 = 1·3·7·14일, 4 = 졸업(due_at NULL)
review_stage: Mapped[int | None] = mapped_column(SmallInteger)
due_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
last_read_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.now, nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.now, onupdate=datetime.now, nullable=False
)
-33
View File
@@ -1,33 +0,0 @@
You are an answerability judge. Given a query and evidence chunks, determine if the evidence can answer the query. Respond ONLY in JSON.
## CALIBRATION (CRITICAL)
- verdict=full: evidence is SUFFICIENT to answer the CORE of the query. Missing minor details does NOT make it insufficient.
- verdict=partial: evidence covers SOME major aspects but CLEARLY MISSES others the user explicitly asked about.
- verdict=insufficient: evidence has NO relevant information for the query, or is completely off-topic.
Example: Query="제6장 주요 내용", Evidence covers 제6장 definition+scope → verdict=full (core is covered).
Example: Query="제6장 처벌 조항", Evidence covers 제6장 definition but NOT 처벌 → verdict=partial.
Example: Query="감귤 출하량", Evidence about 산업안전보건법 → verdict=insufficient.
## Rules
1. Your "verdict" must be based ONLY on whether the CONTENT semantically answers the query. Ignore retrieval scores for this field.
2. "covered_aspects": query aspects that evidence covers. Korean labels for Korean queries.
3. "missing_aspects": query aspects that evidence does NOT cover. Korean labels.
4. Keep aspects concise (2-5 words each), non-overlapping.
## Output Schema
{
"verdict": "full" | "partial" | "insufficient",
"covered_aspects": ["aspect1"],
"missing_aspects": ["aspect2"],
"confidence": "high" | "medium" | "low"
}
## Query
{query}
## Evidence chunks:
{chunks}
## Retrieval scores (for reference only, NOT for verdict):
[{scores}]
+3 -1
View File
@@ -1,5 +1,5 @@
[System]
너는 한국어 문서 태거 + 짧은 요약기다. 입력 본문을 읽고 TL;DR + 핵심 bullets + tags 생성한다. **상세 문단·entities 는 생성하지 않는다** (깊은 요약은 26B, entity 는 P3b 담당).
너는 한국어 문서 태거 + 요약기다. 입력 본문을 읽고 짧은 요약(ai_summary 2~3문장) + TL;DR + 핵심 bullets + tags 생성한다. **여러 문단의 상세 심층요약·entities 는 생성하지 않는다** (깊은 요약은 26B, entity 는 P3b 담당).
subject_description: {subject_description}
@@ -13,6 +13,7 @@ subject_description: {subject_description}
- pii 감지 시 "pii" 추가 + confidence 감점.
요약 규칙:
- **ai_summary**: 2~3문장 문단. 문서의 핵심 내용·목적을 서술 (검색·표시용 요약).
- **TL;DR**: 1문장, 최대 60자.
- **Bullets**: 정확히 5개, 각 30~60자.
- 본문에 없는 정보 추가 금지 (hallucination 금지).
@@ -20,6 +21,7 @@ subject_description: {subject_description}
출력 (JSON only):
{{
"ai_summary": "2~3문장 문단 요약",
"tldr": "1문장 최대 60자",
"bullets": ["...", "...", "...", "...", "..."],
"tags": ["..."],
-42
View File
@@ -1,42 +0,0 @@
You are a grounding verifier. Given an answer and its evidence sources, check if the answer contradicts or fabricates information. Respond ONLY in JSON.
## Contradiction Types (IMPORTANT — severity depends on type)
- **direct_negation** (CRITICAL): Answer directly contradicts evidence. Examples: evidence "의무" but answer "권고"; evidence "금지" but answer "허용"; negation reversal ("~해야 한다" vs "~할 필요 없다").
- **numeric_conflict**: Answer states a number different from evidence. "50명" in evidence but "100명" in answer. Only flag if the same concept is referenced. severity=critical when the number is the CORE answered quantity (amount/count/rate/date/duration that the query asked for); severity=minor when the number is peripheral (e.g., example/footnote).
- **intent_core_mismatch**: Answer addresses a fundamentally different topic than the query asked about.
- **nuance**: Answer overgeneralizes or adds qualifiers not in evidence (e.g., "모든" when evidence says "일부").
- **unsupported_claim**: Answer makes a factual claim with no basis in any evidence.
## Rules
1. Compare each claim in the answer against the cited evidence. A claim with [n] citation should be checked against evidence [n].
2. NOT a contradiction: Paraphrasing, summarizing, or restating the same fact in different words. Korean formal/informal style (합니다/한다) differences.
3. Numbers must match exactly after normalization (1,000 = 1000). Range values (e.g., "100~200명") satisfy any answer within range.
4. Legal/regulatory terms must preserve original meaning (의무 ≠ 권고, 금지 ≠ 제한, 허용 ≠ 금지).
5. Maximum 5 contradictions (most severe first: direct_negation > numeric_conflict > intent_core_mismatch > nuance > unsupported_claim).
## Output Schema
{
"contradictions": [
{
"type": "direct_negation" | "numeric_conflict" | "intent_core_mismatch" | "nuance" | "unsupported_claim",
"severity": "critical" | "minor",
"claim": "answer 내 해당 구절 (50자 이내)",
"evidence_ref": "대응 근거 내용 (50자 이내, [n] 포함)",
"explanation": "모순 이유 (한국어, 30자 이내)"
}
],
"verdict": "clean" | "minor_issues" | "major_issues"
}
severity mapping:
- direct_negation → "critical"
- numeric_conflict → "critical" if the number is the CORE answered quantity, else "minor"
- All other types → "minor"
If no contradictions: {"contradictions": [], "verdict": "clean"}
## Answer
{answer}
## Evidence
{numbered_evidence}
-156
View File
@@ -1,156 +0,0 @@
"""Answerability classifier (Phase 3.5a).
Mac mini 26B MLX 기반 (config.yaml ai.models.classifier PR #20 이후 triage/primary/classifier 동일 endpoint). MLX gate 밖 — evidence extraction 과 병렬 실행 (concurrent 안전성 별 검토).
P1 실측 결과: ternary (full/partial/insufficient) 불안정 **binary (sufficient/insufficient)**.
"full" vs "partial" 구분은 grounding_check intent alignment 담당.
Classifier verdict "relevant evidence 가 있나" binary 판단.
covered_aspects / missing_aspects 로깅용으로 유지 (refusal gate 에서 사용 ).
"""
from __future__ import annotations
import asyncio
import time
from dataclasses import dataclass
from typing import Literal
from ai.client import AIClient, _load_prompt, parse_json_response
from core.config import settings
from core.utils import setup_logger
from .llm_gate import Priority, acquire_mlx_gate
logger = setup_logger("classifier")
LLM_TIMEOUT_MS = 30000
CIRCUIT_THRESHOLD = 5
CIRCUIT_RECOVERY_SEC = 60
_failure_count = 0
_circuit_open_until: float | None = None
@dataclass(slots=True)
class ClassifierResult:
status: Literal["ok", "timeout", "error", "circuit_open", "skipped"]
verdict: Literal["sufficient", "insufficient"] | None
covered_aspects: list[str]
missing_aspects: list[str]
elapsed_ms: float
try:
CLASSIFIER_PROMPT = _load_prompt("classifier.txt")
except FileNotFoundError:
CLASSIFIER_PROMPT = ""
logger.warning("classifier.txt not found — classifier will always skip")
def _build_input(
query: str,
top_chunks: list[dict],
rerank_scores: list[float],
) -> str:
"""Y+ input (content + scores with role separation)."""
chunk_block = "\n".join(
f"[{i+1}] title: {c.get('title','')}\n"
f" section: {c.get('section','')}\n"
f" snippet: {c.get('snippet','')}"
for i, c in enumerate(top_chunks[:3])
)
scores_str = ", ".join(f"{s:.2f}" for s in rerank_scores[:3])
return (
CLASSIFIER_PROMPT
.replace("{query}", query)
.replace("{chunks}", chunk_block)
.replace("{scores}", scores_str)
)
async def classify(
query: str,
top_chunks: list[dict],
rerank_scores: list[float],
) -> ClassifierResult:
"""Always-on binary classifier. Parallel with evidence extraction.
Returns:
ClassifierResult with verdict=sufficient|insufficient.
Status "ok" 아니면 verdict=None (caller fallback 처리).
"""
global _failure_count, _circuit_open_until
t_start = time.perf_counter()
# Circuit breaker
if _circuit_open_until and time.time() < _circuit_open_until:
return ClassifierResult("circuit_open", None, [], [], 0.0)
if not CLASSIFIER_PROMPT:
return ClassifierResult("skipped", None, [], [], 0.0)
if not hasattr(settings.ai, "classifier") or settings.ai.classifier is None:
return ClassifierResult("skipped", None, [], [], 0.0)
prompt = _build_input(query, top_chunks, rerank_scores)
client = AIClient()
try:
# 2026-05-17: PR #20 이후 endpoint 가 Mac mini 26B → llm_gate Semaphore(1) 필수.
# Gate 미사용 시 classifier + evidence + synthesis 가 동시에 single-inference
# MLX 에 race → 거의 모두 timeout (실측: 8/10 fixture query). docstring 영구 룰:
# "MLX primary 호출 경로는 예외 없이 gate 획득 필수".
async with acquire_mlx_gate(Priority.FOREGROUND):
async with asyncio.timeout(LLM_TIMEOUT_MS / 1000):
raw = await client.call_classifier(prompt)
_failure_count = 0
except asyncio.TimeoutError:
_failure_count += 1
if _failure_count >= CIRCUIT_THRESHOLD:
_circuit_open_until = time.time() + CIRCUIT_RECOVERY_SEC
logger.error(f"classifier circuit OPEN for {CIRCUIT_RECOVERY_SEC}s")
logger.warning("classifier timeout")
return ClassifierResult(
"timeout", None, [], [],
(time.perf_counter() - t_start) * 1000,
)
except Exception as e:
_failure_count += 1
if _failure_count >= CIRCUIT_THRESHOLD:
_circuit_open_until = time.time() + CIRCUIT_RECOVERY_SEC
logger.error(f"classifier circuit OPEN for {CIRCUIT_RECOVERY_SEC}s")
logger.warning("classifier error: type=%s repr=%r", type(e).__name__, e)
return ClassifierResult(
"error", None, [], [],
(time.perf_counter() - t_start) * 1000,
)
finally:
await client.close()
elapsed_ms = (time.perf_counter() - t_start) * 1000
parsed = parse_json_response(raw)
if not isinstance(parsed, dict):
logger.warning("classifier parse failed raw=%r", (raw or "")[:200])
return ClassifierResult("error", None, [], [], elapsed_ms)
# ternary → binary 매핑
raw_verdict = parsed.get("verdict", "")
if raw_verdict == "insufficient":
verdict: Literal["sufficient", "insufficient"] | None = "insufficient"
elif raw_verdict in ("full", "partial", "sufficient"):
verdict = "sufficient"
else:
verdict = None
covered = parsed.get("covered_aspects") or []
missing = parsed.get("missing_aspects") or []
if not isinstance(covered, list):
covered = []
if not isinstance(missing, list):
missing = []
logger.info(
"classifier ok query=%r verdict=%s (raw=%s) covered=%d missing=%d elapsed_ms=%.0f",
query[:60], verdict, raw_verdict, len(covered), len(missing), elapsed_ms,
)
return ClassifierResult("ok", verdict, covered, missing, elapsed_ms)
-505
View File
@@ -1,505 +0,0 @@
"""Grounding check — post-synthesis 검증 (Phase 3.5a).
Strong/weak flag 분리:
- **Strong** ( partial 강등 or refuse): fabricated_number, intent_misalignment(important)
- **Weak** ( confidence lower only): uncited_claim, low_overlap, intent_misalignment(generic)
Re-gate 로직 (Phase 3.5a 9라운드 토론 결과):
- strong 1 partial 강등
- strong 2 이상 refuse
- weak confidence "low"
Intent alignment (rule-based):
- query 핵심 명사가 answer 등장하는지 확인
- "처벌" 같은 중요 키워드 누락은 strong
- "주요", "관련" 같은 generic 무시
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from typing import TYPE_CHECKING
from core.utils import setup_logger
if TYPE_CHECKING:
from .evidence_service import EvidenceItem
logger = setup_logger("grounding")
# "주요", "관련" 등 intent alignment 에서 제외할 generic 단어
GENERIC_TERMS = frozenset({
"주요", "관련", "내용", "정의", "기준", "방법", "설명", "개요",
"대한", "위한", "대해", "무엇", "어떤", "어떻게", "있는",
"하는", "되는", "이런", "그런", "이것", "그것",
})
@dataclass(slots=True)
class GroundingResult:
strong_flags: list[str]
weak_flags: list[str]
_UNIT_CHARS = r'명인개%년월일조항호세건원회'
# "이상/이하/초과/미만" — threshold 표현 (numeric conflict 에서 skip 대상)
_THRESHOLD_SUFFIXES = re.compile(r'이상|이하|초과|미만')
# 약칭/근사치 prefix — 매칭 전 제거 (Phase 3.5 B1).
# ⚠ 최대/최소 는 의도적으로 제외 — 이들은 bound operator 라 의미가 다름 (Phase 3.5 B1 fix3).
# 약/대략/거의/얼추 만 노이즈 prefix 로 strip.
_APPROX_PREFIX_RE = re.compile(r'(약|대략|거의|얼추)\s*')
# 단위 동의어 dict — 추출 직후 정규화 (Phase 3.5 B1)
# 의미가 동일한 단위는 같은 표기로 통일해서 set 비교/range overlap 안정화.
_UNIT_SYNONYMS: dict[str, str] = {
"": "",
"사람": "",
"퍼센트": "%",
"프로": "%",
"KRW": "",
"krw": "",
}
# tolerance(±1%) 허용 단위 — 양적 측정값 (Phase 3.5 B1)
_TOLERANCE_UNITS: frozenset[str] = frozenset({"", "", "%", "", ""})
# tolerance 미적용 단위 — 식별자성 숫자 (연도/조문/횟수)
_EXACT_ONLY_UNITS: frozenset[str] = frozenset({"", "", "", "", "", "", ""})
# 최대/최소 prefix 패턴 — bound operator (Phase 3.5 B1 fix3).
# 매칭된 숫자는 exact pool 에서 제외하고 one-sided range 로 변환.
# 경계값 자체는 clear 대상 아님 (Codex 권장: "최대 100명" + answer "100명" → flag 유지).
_BOUND_PATTERN_RE = re.compile(
rf'(최대|최소)\s*(\d[\d,.]*)\s*([{_UNIT_CHARS}]|인|사람|퍼센트|프로|KRW|krw)'
)
_RANGE_INF = 10**18 # one-sided range 상한 sentinel
def _normalize_unit(unit: str) -> str:
"""단위 동의어 → 대표 표기."""
return _UNIT_SYNONYMS.get(unit, unit)
def _extract_unit(literal: str) -> str | None:
"""리터럴에서 숫자 뒤 단위(한 글자 또는 동의어) 추출 + 정규화."""
# 천단위 콤마 + 옵션 소수 + 한글 단위 한 글자 또는 동의어
m = re.match(rf'[\d,.]+\s*([{_UNIT_CHARS}]|인|사람|퍼센트|프로|KRW|krw)', literal)
if not m:
return None
return _normalize_unit(m.group(1))
def _extract_numeric_corpus(text: str) -> dict:
"""단위별 숫자 + 범위 + bound 통합 추출 (Phase 3.5 B1 fix1+fix3).
Returns:
{
"exact_by_unit": {unit_or_None: set(digits)}, # 평범한 숫자 (bound 제외)
"ranges_by_unit": {unit: [(lo, hi), ...]}, # 양방향(A~B) + 단방향(최대/최소)
}
None 키는 단위 없는 bare 숫자.
`최대 N <unit>` ranges[(0, N-1)] (경계값 자체는 cleared 대상 아님)
`최소 N <unit>` ranges[(N+1, INF)]
"""
cleaned = _APPROX_PREFIX_RE.sub('', text)
exact_by_unit: dict[str | None, set[str]] = {None: set()}
ranges_by_unit: dict[str, list[tuple[int, int]]] = {}
# 1) 최대/최소 — bound. exact pool 에서 제외, one-sided range 로 변환.
bound_spans: list[tuple[int, int]] = [] # 매칭 substring 위치 — 이후 단계에서 skip
for m in _BOUND_PATTERN_RE.finditer(cleaned):
bound_kind = m.group(1)
try:
n = int(m.group(2).replace(',', '').split('.')[0])
except ValueError:
continue
unit = _normalize_unit(m.group(3))
if bound_kind == "최대":
ranges_by_unit.setdefault(unit, []).append((0, max(0, n - 1)))
else: # 최소
ranges_by_unit.setdefault(unit, []).append((n + 1, _RANGE_INF))
bound_spans.append((m.start(), m.end()))
def _in_bound_span(pos: int) -> bool:
return any(s <= pos < e for s, e in bound_spans)
# 2) 천단위 콤마 bare number
for m in re.finditer(r'\d{1,3}(?:,\d{3})+(?:\.\d+)?', cleaned):
if _in_bound_span(m.start()):
continue
exact_by_unit[None].add(m.group().replace(',', ''))
# 3) 단위 있는 숫자 (단위 동의어 포함)
for m in re.finditer(
rf'(\d[\d,.]*)\s*([{_UNIT_CHARS}]|인|사람|퍼센트|프로|KRW|krw)',
cleaned,
):
if _in_bound_span(m.start()):
continue
digits = m.group(1).replace(',', '').split('.')[0]
if not digits:
continue
unit = _normalize_unit(m.group(2))
exact_by_unit.setdefault(unit, set()).add(digits)
# 4) 양방향 범위 표현 (A~B / A 부터 B)
for m in re.finditer(
rf'(\d[\d,.]*)\s*(?:[~\-]|부터)\s*(\d[\d,.]*)\s*([{_UNIT_CHARS}]|인|사람|퍼센트|프로)',
cleaned,
):
if _in_bound_span(m.start()):
continue
try:
lo = int(m.group(1).replace(',', '').split('.')[0])
hi = int(m.group(2).replace(',', '').split('.')[0])
except ValueError:
continue
unit = _normalize_unit(m.group(3))
ranges_by_unit.setdefault(unit, []).append((min(lo, hi), max(lo, hi)))
# 5) bare 2자리+ 단독 숫자
for m in re.finditer(r'\b(\d{2,})\b', cleaned):
if _in_bound_span(m.start()):
continue
exact_by_unit[None].add(m.group())
return {
"exact_by_unit": exact_by_unit,
"ranges_by_unit": ranges_by_unit,
}
def _within_unit_range(
n: int, unit: str | None, ranges_by_unit: dict[str, list[tuple[int, int]]]
) -> bool:
"""unit-matching range 검증.
answer unit None (bare 숫자) 보수적으로 False bare 답변은 range clear 대상 아님.
"""
if unit is None:
return False
return any(lo <= n <= hi for lo, hi in ranges_by_unit.get(unit, []))
def _close_to_unit_pool(
n: int, unit: str | None, exact_by_unit: dict[str | None, set[str]], tol: float
) -> bool:
"""unit-matching tolerance 검증.
answer unit None 이면 False bare 답변은 tolerance 대상 아님.
같은 unit bucket 안의 후보만 비교.
"""
if unit is None:
return False
candidates = exact_by_unit.get(unit, set())
for c in candidates:
try:
cn = int(c)
except ValueError:
continue
if cn == 0:
continue
if abs(n - cn) / cn <= tol:
return True
return False
def _extract_number_literals(text: str) -> set[str]:
"""숫자 + 단위 추출 + normalize (Phase 3.5 B1: 6단계 확장).
1) 약칭 prefix 제거 ("약 100명" "100명")
2) 천단위 콤마 bare number 우선 ("1,000" "1000" set 등록)
3) 한국어 단위 접미사 매칭 (기존)
4) 범위 표현 양쪽 숫자 추출 (separator: ~, -, , 부터)
5) 단위 동의어 정규화 (, 퍼센트%, KRW)
6) bare 2자리+ 추출 (기존)
"""
# 1. 약칭 prefix 제거 (전체 텍스트에서)
cleaned = _APPROX_PREFIX_RE.sub('', text)
# 2. 천단위 콤마 bare number — normalize 된 값을 set 에 선등록
normalized: set[str] = set()
for m in re.finditer(r'\d{1,3}(?:,\d{3})+(?:\.\d+)?', cleaned):
normalized.add(m.group().replace(',', ''))
# 3. 숫자 + 한국어 단위 접미사 (동의어 포함)
raw: set[str] = set(re.findall(
rf'\d[\d,.]*\s*(?:[{_UNIT_CHARS}]|인|사람|퍼센트|프로|KRW|krw)\w{{0,2}}',
cleaned,
))
# 4. 범위 표현 — separator 에 "부터" 추가
for m in re.finditer(
rf'(\d[\d,.]*)\s*(?:[~\-]|부터)\s*(\d[\d,.]*)\s*([{_UNIT_CHARS}]|인|사람|퍼센트|프로)',
cleaned,
):
unit_norm = _normalize_unit(m.group(3))
raw.add(m.group(1) + unit_norm)
raw.add(m.group(2) + unit_norm)
# 5. normalize: 단위 동의어 통일 + 콤마 제거
for r in raw:
# 단위 부분 정규화
m = re.match(r'([\d,.]+)\s*([^\d\s]+)', r)
if m:
digits_part = m.group(1)
unit_part = _normalize_unit(m.group(2))
normalized.add(digits_part + unit_part)
normalized.add(digits_part.replace(',', '') + unit_part)
normalized.add(r.strip())
num_only = re.match(r'[\d,.]+', r)
if num_only:
normalized.add(num_only.group().replace(',', ''))
# 6. 단독 숫자 (2자리+ 만)
for d in re.findall(r'\b(\d{2,})\b', cleaned):
normalized.add(d)
return normalized
def _within_evidence_range(digits: str, raw: str, evidence_text: str) -> bool:
"""evidence 에 'A~B 단위' 가 있고 answer 의 숫자가 그 범위 안이면 True.
범위 단위는 무시 (단위 비교는 호출 단계). digits = 정수 문자열.
"""
try:
n = int(digits)
except ValueError:
return False
cleaned_ev = _APPROX_PREFIX_RE.sub('', evidence_text)
for m in re.finditer(
rf'(\d[\d,.]*)\s*(?:[~\-]|부터)\s*(\d[\d,.]*)\s*[{_UNIT_CHARS}]',
cleaned_ev,
):
try:
lo = int(m.group(1).replace(',', '').split('.')[0])
hi = int(m.group(2).replace(',', '').split('.')[0])
if min(lo, hi) <= n <= max(lo, hi):
return True
except ValueError:
continue
return False
def _close_to_any(n: int, candidates: set[str], tol: float) -> bool:
"""candidates 중 하나라도 (1±tol) 배율 안에 들어오면 True.
n 정수, candidates digits-only 문자열 집합.
"""
for c in candidates:
try:
cn = int(c)
except ValueError:
continue
if cn == 0:
continue
if abs(n - cn) / cn <= tol:
return True
return False
def _extract_content_tokens(text: str) -> set[str]:
"""한국어 2자 이상 명사 + 영어 3자 이상 단어."""
return set(re.findall(r'[가-힣]{2,}|[a-zA-Z]{3,}', text))
def _parse_number_with_unit(literal: str) -> tuple[str, str] | None:
"""숫자 리터럴에서 (digits_only, unit) 분리. 단위 없으면 None."""
m = re.match(rf'([\d,.]+)\s*([{_UNIT_CHARS}])', literal)
if not m:
return None
digits = m.group(1).replace(',', '')
unit = m.group(2)
return (digits, unit)
def _check_evidence_numeric_conflicts(evidence: list["EvidenceItem"]) -> list[str]:
"""evidence 간 숫자 충돌 감지 (Phase 3.5b). evidence >= 2 일 때만 활성.
동일 단위, 다른 숫자 weak flag. "이상/이하/초과/미만" 포함 skip.
bare number 비교 (조항 번호 false positive 방지).
"""
if len(evidence) < 2:
return []
# 각 evidence 에서 단위 있는 숫자 + threshold 여부 추출
# {evidence_idx: [(digits, unit, has_threshold), ...]}
per_evidence: dict[int, list[tuple[str, str, bool]]] = {}
for idx, ev in enumerate(evidence):
nums = re.findall(
rf'\d[\d,.]*\s*[{_UNIT_CHARS}]\w{{0,4}}',
ev.span_text,
)
entries = []
for raw in nums:
parsed = _parse_number_with_unit(raw)
if not parsed:
continue
has_thr = bool(_THRESHOLD_SUFFIXES.search(raw))
entries.append((parsed[0], parsed[1], has_thr))
if entries:
per_evidence[idx] = entries
if len(per_evidence) < 2:
return []
# 단위별로 evidence 간 숫자 비교
# {unit: {digits: [evidence_idx, ...]}}
unit_map: dict[str, dict[str, list[int]]] = {}
for idx, entries in per_evidence.items():
for digits, unit, has_thr in entries:
if has_thr:
continue # threshold 표현은 skip
if unit not in unit_map:
unit_map[unit] = {}
if digits not in unit_map[unit]:
unit_map[unit][digits] = []
if idx not in unit_map[unit][digits]:
unit_map[unit][digits].append(idx)
flags: list[str] = []
for unit, digits_map in unit_map.items():
distinct_values = list(digits_map.keys())
if len(distinct_values) >= 2:
# 가장 많이 등장하는 2개 비교
top2 = sorted(distinct_values, key=lambda d: len(digits_map[d]), reverse=True)[:2]
flags.append(
f"evidence_numeric_conflict:{top2[0]}{unit}_vs_{top2[1]}{unit}"
)
return flags
def check(
query: str,
answer: str,
evidence: list[EvidenceItem],
) -> GroundingResult:
"""답변 vs evidence grounding 검증 + query intent alignment."""
strong: list[str] = []
weak: list[str] = []
if not answer or not evidence:
return GroundingResult([], [])
# ⚠ citation marker [n] 양측 제거 (대칭성 — Phase 3.5 B1)
evidence_text = re.sub(r'\[\d+\]', '', " ".join(e.span_text for e in evidence))
# ── Strong 1: fabricated number (unit-aware 3단계 — Phase 3.5 B1 fix1+fix3) ──
# Codex 지적 반영:
# - fix1: range/tolerance/exact 모두 단위 일치 시에만 clear
# (예: "150원" vs "100~200명" → flag 유지)
# - fix3: 최대/최소 prefix 는 bound 의미 보존
# (예: "최대 100명" + answer "100명" → flag 유지, "최대 100명" + answer "50명" → cleared)
answer_clean = re.sub(r'\[\d+\]', '', answer)
answer_corpus = _extract_numeric_corpus(answer_clean)
evidence_corpus = _extract_numeric_corpus(evidence_text)
ev_exact_by_unit = evidence_corpus["exact_by_unit"]
ev_ranges_by_unit = evidence_corpus["ranges_by_unit"]
# cleared 는 (unit, digits) 쌍 단위로 추적 — 단위 충돌 케이스 방어
cleared_pairs: set[tuple[str | None, str]] = set()
# Pass 1: 각 (unit, digits) 가 evidence 에서 정당화되는지 판정
for unit, digits_set in answer_corpus["exact_by_unit"].items():
for d in digits_set:
# 1) exact match — 같은 unit bucket 내에서만
if d in ev_exact_by_unit.get(unit, set()):
cleared_pairs.add((unit, d))
continue
# bare answer (unit=None) 는 evidence bare bucket 도 보조 매칭
if unit is None and d in ev_exact_by_unit.get(None, set()):
cleared_pairs.add((unit, d))
continue
try:
n = int(d)
except ValueError:
continue
# 2) range — same-unit 만 (bare answer 는 range clear 대상 아님)
if _within_unit_range(n, unit, ev_ranges_by_unit):
cleared_pairs.add((unit, d))
continue
# 3) ±1% tolerance — 단위가 양적(_TOLERANCE_UNITS) + 4자리+ + same-unit
if (
unit in _TOLERANCE_UNITS
and len(d) >= 4
and _close_to_unit_pool(n, unit, ev_exact_by_unit, tol=0.01)
):
cleared_pairs.add((unit, d))
continue
# 식별자성 단위(_EXACT_ONLY_UNITS) 는 tolerance 패스 X.
# Pass 2: cleared 되지 않은 (unit, digits) 를 strong flag.
# 1자리 무시는 unit 이 식별자성(_EXACT_ONLY_UNITS: 년/월/일/조/항/호/회) 이 아닐 때만 적용.
# bare(None) 답변 숫자는 같은 digit 이 다른 unit 에서 cleared 됐으면 skip — 추출 부산물 방어.
# ⚠ 단위 cross-clear (예: "원" cleared → "명" 도 skip) 은 금지: Codex unit-mismatch 케이스가 깨짐.
unit_anchored_cleared: set[str] = {d for (u, d) in cleared_pairs if u is not None}
flagged_keys: set[tuple[str | None, str]] = set()
for unit, digits_set in answer_corpus["exact_by_unit"].items():
for d in digits_set:
if (unit, d) in cleared_pairs or (unit, d) in flagged_keys:
continue
# bare(None) 답변 숫자가 임의의 단위 bucket 에서 cleared 됐으면 duplicate 로 처리.
# 사례: "1,000명" → unit bucket "명" 에 1000 + bare bucket None 에 1000 (comma normalize 부산물).
# 이미 ("명", "1000") 가 cleared 라면 (None, "1000") 도 같은 사실을 가리키므로 skip.
if unit is None and d in unit_anchored_cleared:
continue
if len(d) < 2 and unit not in _EXACT_ONLY_UNITS:
continue
flagged_keys.add((unit, d))
# 사람이 읽기 좋게 "{digits}{unit}" 또는 bare 형태로 표기
label = f"{d}{unit}" if unit else d
strong.append(f"fabricated_number:{label}")
# ── Strong/Weak 2: query-answer intent alignment ──
query_content = _extract_content_tokens(query)
answer_content = _extract_content_tokens(answer)
if query_content:
missing_terms = query_content - answer_content
important_missing = [
t for t in missing_terms
if t not in GENERIC_TERMS and len(t) >= 2
]
if important_missing:
strong.append(
f"intent_misalignment:{','.join(important_missing[:3])}"
)
elif len(missing_terms) > len(query_content) * 0.5:
weak.append(
f"intent_misalignment_generic:"
f"missing({','.join(list(missing_terms)[:5])})"
)
# ── Weak 1: uncited claim ──
sentences = re.split(r'(?<=[.!?。])\s+', answer)
for s in sentences:
if len(s.strip()) > 20 and not re.search(r'\[\d+\]', s):
weak.append(f"uncited_claim:{s[:40]}")
# ── Weak: evidence 간 숫자 충돌 (Phase 3.5b) ──
conflicts = _check_evidence_numeric_conflicts(evidence)
weak.extend(conflicts)
# ── Weak 2: token overlap ──
answer_tokens = _extract_content_tokens(answer)
evidence_tokens = _extract_content_tokens(evidence_text)
if answer_tokens:
overlap = len(answer_tokens & evidence_tokens) / len(answer_tokens)
if overlap < 0.4:
weak.append(f"low_overlap:{overlap:.2f}")
if strong or weak:
logger.info(
"grounding query=%r strong=%d weak=%d flags=%s",
query[:60],
len(strong),
len(weak),
",".join(strong[:3] + weak[:3]),
)
return GroundingResult(strong, weak)
-105
View File
@@ -1,105 +0,0 @@
"""Refusal gate — multi-signal fusion (Phase 3.5a).
Score gate (deterministic) + classifier verdict (semantic, binary) 독립 평가 합성.
Classifier 부재 3-tier conservative fallback.
P1 실측 결과: exaone ternary 불안정 binary (sufficient/insufficient) 축소.
"full" vs "partial" 구분은 grounding check (intent alignment) 담당.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Literal
from core.utils import setup_logger
if TYPE_CHECKING:
from .classifier_service import ClassifierResult
logger = setup_logger("refusal_gate")
# Placeholder thresholds — Phase 3.5b 에서 실측 기반 tuning
# AND 조건이라 false refusal 방어됨 (둘 다 만족해야 refuse)
SCORE_MAX_REFUSE = 0.25
SCORE_AGG_REFUSE = 0.70
# Conservative fallback tiers (classifier 부재 시)
CONSERVATIVE_WEAK = 0.35
CONSERVATIVE_MID = 0.55
@dataclass(slots=True)
class RefusalDecision:
refused: bool
confidence_cap: Literal["high", "medium", "low"] | None # None = no cap
rule_triggered: str | None # 디버깅: 어느 signal 이 결정에 기여?
def decide(
rerank_scores: list[float],
classifier: ClassifierResult | None,
) -> RefusalDecision:
"""Multi-signal fusion. Binary classifier verdict 기반.
Returns:
RefusalDecision. refused=True 이면 synthesis skip.
confidence_cap synthesis 결과의 confidence upper bound 적용.
"""
max_score = max(rerank_scores) if rerank_scores else 0.0
agg_top3 = sum(sorted(rerank_scores, reverse=True)[:3])
score_gate_fails = (
max_score < SCORE_MAX_REFUSE and agg_top3 < SCORE_AGG_REFUSE
)
# ── Classifier 사용 가능 (정상 경로) ──
if classifier and classifier.verdict is not None:
if classifier.verdict == "insufficient":
# Evidence quality override: classifier 가 insufficient 라 해도
# evidence 가 충분히 좋으면 override (토론 8라운드 합의)
# (evidence quality 는 이 함수 밖에서 별도 체크 — caller 에서 처리)
logger.info(
"refusal gate: classifier=insufficient max=%.2f agg=%.2f",
max_score, agg_top3,
)
return RefusalDecision(
refused=True,
confidence_cap=None,
rule_triggered="classifier_insufficient",
)
if score_gate_fails:
logger.info(
"refusal gate: score_low max=%.2f agg=%.2f classifier=%s",
max_score, agg_top3, classifier.verdict,
)
return RefusalDecision(
refused=True,
confidence_cap=None,
rule_triggered="score_low",
)
# Classifier says sufficient → proceed
return RefusalDecision(
refused=False,
confidence_cap=None,
rule_triggered=None,
)
# ── Classifier 부재 → 3-tier conservative ──
if max_score < CONSERVATIVE_WEAK:
return RefusalDecision(
refused=True,
confidence_cap=None,
rule_triggered="conservative_refuse(no_classifier)",
)
if max_score < CONSERVATIVE_MID:
return RefusalDecision(
refused=False,
confidence_cap="low",
rule_triggered="conservative_low(no_classifier)",
)
return RefusalDecision(
refused=False,
confidence_cap="medium",
rule_triggered="conservative_medium(no_classifier)",
)
+37 -2
View File
@@ -76,10 +76,15 @@ class AxisFilter:
jurisdiction: str | None = None
year_from: int | None = None
year_to: int | None = None
domain_buckets: list[str] | None = None # 377: domain_bucket = ANY (도메인 스코프)
exclude_buckets: list[str] | None = None # 377: domain_bucket <> ALL (예: News 제외)
cloud_egress: bool = False # 갭2: 클라우드 소비자 cloud-eligibility allowlist 강제(토큰 claim 유래)
def active(self) -> bool:
return bool(self.material_types or self.jurisdiction
or self.year_from is not None or self.year_to is not None)
or self.year_from is not None or self.year_to is not None
or self.domain_buckets or self.exclude_buckets
or self.cloud_egress)
def _axis_sql(alias: str, af: "AxisFilter | None", params: dict) -> str:
@@ -104,6 +109,22 @@ def _axis_sql(alias: str, af: "AxisFilter | None", params: dict) -> str:
if af.year_to is not None:
cl.append(f"COALESCE({p}published_date, {p}created_at::date) <= make_date(:af_yt, 12, 31)")
params["af_yt"] = af.year_to
if af.domain_buckets:
cl.append(f"{p}domain_bucket = ANY(:af_db)")
params["af_db"] = af.domain_buckets
if af.exclude_buckets:
cl.append(f"{p}domain_bucket <> ALL(:af_xdb)")
params["af_xdb"] = af.exclude_buckets
if af.cloud_egress:
# 갭2 클라우드 egress allowlist(default-deny). restricted 는 _license_sql 별도 차단.
cl.append(
f"({p}data_origin = 'external' OR ("
f"{p}data_origin = 'work' "
f"AND {p}domain_bucket IN ('Engineering','Safety','Law') "
f"AND ({p}source_channel IS NULL OR {p}source_channel::text NOT IN ('voice','chat','memo')) "
f"AND {p}category::text IS DISTINCT FROM 'memo' "
f"AND ({p}user_note IS NULL OR {p}user_note = '')))"
)
return " AND " + " AND ".join(cl)
@@ -121,7 +142,21 @@ def _license_sql(alias: str) -> str:
술어 정의 = license_filter.restricted_exclude_sql 공유(digest/briefing/study 풀이와 단일 source).
"""
from services.search.license_filter import restricted_exclude_sql
return " AND " + restricted_exclude_sql(alias)
_p = (alias + ".") if alias else ""
# ASME clause-KB(379): clause docs (doc_kind='clause') = read/nav/backlink only, excluded from retrieval/digest legs.
return " AND " + restricted_exclude_sql(alias) + f" AND {_p}doc_kind = 'standard'"
def cloud_eligible_doc_sql(alias: str = "") -> str:
"""단일 문서가 cloud 소비자(예: Claude/MCP)에게 노출 가능한가 = search retrieval 과
동일한 egress allowlist(갭2) + license 제한(B-4) 결합 술어. fetch_document(cloud)
search byte-동일 게이트를 공유하도록 단일 source([[feedback_structural_integrity_over_path_discipline]]).
cloud_egress·license leg 모두 bind 파라미터 없는 리터럴 술어라 호출측 추가 params 불요.
주의: _license_sql 소유자 단건 다운로드엔 미적용(a안)이지만, cloud 노출은 구매 전자책
verbatim 누출을 막아야 하므로 여기선 항상 적용 = search 동일(local 토큰은 게이트 미발동).
반환 ' AND (egress allowlist) AND (license)' (alias='' = 컬럼 직접 참조). default-deny."""
return _axis_sql(alias, AxisFilter(cloud_egress=True), {}) + _license_sql(alias)
# 2단계 gate (R2-B1) — SQL string interpolation 직전 final allowlist.
-196
View File
@@ -1,196 +0,0 @@
"""Exaone semantic verifier (Phase 3.5b).
답변-근거 의미적 모순(contradiction) 감지. rule-based grounding_check 잡는
미묘한 모순 포착. classifier 동일 패턴: circuit breaker + timeout + fail open.
## Severity 3단계
- strong: direct_negation (완전 모순) re-gate 교차 자격
- medium: numeric_conflict, intent_core_mismatch confidence 하향 (누적 강제 low)
- weak: nuance, unsupported_claim 로깅 + mild confidence 하향
## 핵심 원칙
- **Verifier strong 단독 refuse 금지** grounding strong 교차해야 refuse
- **Timeout 3s** 느리면 없는 낫다 (fail open)
- MLX gate 사용 (Mac mini 26B endpoint classifier/evidence 동일 gate 공유, 동시 race 방지)
"""
from __future__ import annotations
import asyncio
import os
import time
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Literal
from ai.client import AIClient, _load_prompt, parse_json_response
from core.config import settings
from core.utils import setup_logger
from .llm_gate import Priority, acquire_mlx_gate
if TYPE_CHECKING:
from .evidence_service import EvidenceItem
logger = setup_logger("verifier")
LLM_TIMEOUT_MS = 10000 # 2026-05-17 B-3: 3s 시 동시 부하 시 verifier 빈발 skip → grounding 약화. Mac mini 26B 가 verifier-style 짧은 LLM call 도 concurrent 호출 시 3s 초과 빈번 — 10s 로 raise
CIRCUIT_THRESHOLD = 5
CIRCUIT_RECOVERY_SEC = 60
_failure_count = 0
_circuit_open_until: float | None = None
# Phase 3.5 B2: numeric_conflict severity promote 실험.
# import time 평가 — env 변경 후 process restart 필수 (docker compose restart fastapi).
# default=0 (off). production 적용은 B3 FP 검증 통과 후만.
_NUMERIC_PROMOTE = os.getenv("VERIFIER_NUMERIC_PROMOTE", "0") == "1"
# severity 매핑 (프롬프트 "critical"/"minor" → 코드 strong/medium/weak)
# Tier 4 (B2): _NUMERIC_PROMOTE=1 일 때 numeric_conflict critical → strong 으로 격상.
# minor 는 medium 유지 (FP 위험 분리).
_SEVERITY_MAP: dict[str, dict[str, Literal["strong", "medium", "weak"]]] = {
"direct_negation": {"critical": "strong", "minor": "strong"},
"numeric_conflict": (
{"critical": "strong", "minor": "medium"} if _NUMERIC_PROMOTE
else {"critical": "medium", "minor": "medium"}
),
"intent_core_mismatch": {"critical": "medium", "minor": "medium"},
"nuance": {"critical": "weak", "minor": "weak"},
"unsupported_claim": {"critical": "weak", "minor": "weak"},
}
@dataclass(slots=True)
class Contradiction:
"""개별 모순 발견."""
type: str # direct_negation / numeric_conflict / intent_core_mismatch / nuance / unsupported_claim
severity: Literal["strong", "medium", "weak"]
claim: str
evidence_ref: str
explanation: str
@dataclass(slots=True)
class VerifierResult:
status: Literal["ok", "timeout", "error", "circuit_open", "skipped"]
contradictions: list[Contradiction]
elapsed_ms: float
try:
VERIFIER_PROMPT = _load_prompt("verifier.txt")
except FileNotFoundError:
VERIFIER_PROMPT = ""
logger.warning("verifier.txt not found — verifier will always skip")
def _build_input(
answer: str,
evidence: list[EvidenceItem],
) -> str:
"""답변 + evidence spans → 프롬프트."""
spans = "\n\n".join(
f"[{e.n}] {(e.title or '').strip()}\n{e.span_text}"
for e in evidence
)
return (
VERIFIER_PROMPT
.replace("{answer}", answer)
.replace("{numbered_evidence}", spans)
)
def _map_severity(ctype: str, raw_severity: str) -> Literal["strong", "medium", "weak"]:
"""type + raw severity → 코드 severity 3단계."""
type_map = _SEVERITY_MAP.get(ctype, {"critical": "weak", "minor": "weak"})
return type_map.get(raw_severity, "weak")
async def verify(
query: str,
answer: str,
evidence: list[EvidenceItem],
) -> VerifierResult:
"""답변-근거 semantic 검증. Parallel with grounding_check.
Returns:
VerifierResult. status "ok" 아니면 contradictions 리스트 (fail open).
"""
global _failure_count, _circuit_open_until
t_start = time.perf_counter()
if _circuit_open_until and time.time() < _circuit_open_until:
return VerifierResult("circuit_open", [], 0.0)
if not VERIFIER_PROMPT:
return VerifierResult("skipped", [], 0.0)
if not hasattr(settings.ai, "verifier") or settings.ai.verifier is None:
return VerifierResult("skipped", [], 0.0)
if not answer or not evidence:
return VerifierResult("skipped", [], 0.0)
prompt = _build_input(answer, evidence)
client = AIClient()
try:
async with acquire_mlx_gate(Priority.FOREGROUND):
async with asyncio.timeout(LLM_TIMEOUT_MS / 1000):
raw = await client.call_verifier(prompt)
_failure_count = 0
except asyncio.TimeoutError:
_failure_count += 1
if _failure_count >= CIRCUIT_THRESHOLD:
_circuit_open_until = time.time() + CIRCUIT_RECOVERY_SEC
logger.error(f"verifier circuit OPEN for {CIRCUIT_RECOVERY_SEC}s")
logger.warning("verifier timeout")
return VerifierResult(
"timeout", [],
(time.perf_counter() - t_start) * 1000,
)
except Exception as e:
_failure_count += 1
if _failure_count >= CIRCUIT_THRESHOLD:
_circuit_open_until = time.time() + CIRCUIT_RECOVERY_SEC
logger.error(f"verifier circuit OPEN for {CIRCUIT_RECOVERY_SEC}s")
logger.warning(f"verifier error: {e}")
return VerifierResult(
"error", [],
(time.perf_counter() - t_start) * 1000,
)
finally:
await client.close()
elapsed_ms = (time.perf_counter() - t_start) * 1000
parsed = parse_json_response(raw)
if not isinstance(parsed, dict):
logger.warning("verifier parse failed raw=%r", (raw or "")[:200])
return VerifierResult("error", [], elapsed_ms)
# contradiction 파싱
raw_items = parsed.get("contradictions") or []
if not isinstance(raw_items, list):
raw_items = []
results: list[Contradiction] = []
for item in raw_items[:5]:
if not isinstance(item, dict):
continue
ctype = item.get("type", "")
if ctype not in _SEVERITY_MAP:
ctype = "unsupported_claim"
raw_sev = item.get("severity", "minor")
severity = _map_severity(ctype, raw_sev)
claim = str(item.get("claim", ""))[:50]
ev_ref = str(item.get("evidence_ref", ""))[:50]
explanation = str(item.get("explanation", ""))[:30]
results.append(Contradiction(ctype, severity, claim, ev_ref, explanation))
logger.info(
"verifier ok query=%r contradictions=%d strong=%d medium=%d elapsed_ms=%.0f",
query[:60],
len(results),
sum(1 for c in results if c.severity == "strong"),
sum(1 for c in results if c.severity == "medium"),
elapsed_ms,
)
return VerifierResult("ok", results, elapsed_ms)
+284
View File
@@ -0,0 +1,284 @@
"""concept_curriculum — 이론공부 홈 재료 (오늘의 개념 · 진도 · 회독 SR).
개념문서 = documents (user_tags = @library/{topic}/{과목}/... , 가스기사). is_read = 회독,
md_content 개수 = 빈출 tier(=3 / =2 / else 1). 회독 SR = study_concept_progress
+ sr_schedule(문제 SR 공용 산술). 읽기 전용 집계 + mark_read(회독+SR 입고) write. LLM 0.
문제풀이 표면 무접촉 여기서 읽는 study_question_progress '문항 due 카운트'( 표시용).
"""
from __future__ import annotations
from datetime import datetime, timezone
from sqlalchemy import func, or_, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from models.document_read import DocumentRead
from models.study_concept_progress import StudyConceptProgress
from models.study_question_progress import StudyQuestionProgress
from models.study_topic import StudyTopic
from services.study.concept_parser import parse_concept, resolve_related
from services.study.sr_schedule import advance, first_due
# 개념 행 조회 — 태그로 개념문서 필터 + 회독 진행 LEFT JOIN. md_content 는 전송 안 하고
# ★ 유무만 서버측 boolean 으로(홈이 자주 호출돼도 페이로드 최소).
# is_read = document_reads(회독 정본, is_read 컬럼 아님) EXISTS. library unread 와 동일 기준.
_CONCEPT_ROWS_SQL = text(
"""
SELECT d.id AS doc_id,
d.title AS title,
EXISTS (
SELECT 1 FROM document_reads r
WHERE r.document_id = d.id AND r.user_id = :uid
) AS is_read,
(d.md_content LIKE '%★★★%') AS f3,
(d.md_content LIKE '%★★%') AS f2,
split_part(replace(d.user_tags::text, '"', ''), '/', 3) AS subject,
p.review_stage AS review_stage,
p.due_at AS due_at,
p.last_read_at AS last_read_at
FROM documents d
LEFT JOIN study_concept_progress p
ON p.concept_doc_id = d.id AND p.user_id = :uid
WHERE d.user_tags::text LIKE :like
AND d.deleted_at IS NULL
"""
)
async def _topic_name(session: AsyncSession, topic_id: int) -> str | None:
return (
await session.execute(select(StudyTopic.name).where(StudyTopic.id == topic_id))
).scalar_one_or_none()
async def _concept_rows(session: AsyncSession, user_id: int, topic_name: str):
like = f"%@library/{topic_name}/%"
return (
await session.execute(_CONCEPT_ROWS_SQL, {"uid": user_id, "like": like})
).mappings().all()
def _freq(row) -> int:
if row["f3"]:
return 3
if row["f2"]:
return 2
return 1
def _is_due(row, now: datetime) -> bool:
return (
row["due_at"] is not None
and row["due_at"] <= now
and (row["review_stage"] or 0) < 4
)
def _item(row) -> dict:
return {
"doc_id": row["doc_id"],
"title": row["title"],
"subject": row["subject"],
"freq": _freq(row),
"review_stage": row["review_stage"],
"due_at": row["due_at"],
}
async def _question_due_count(session: AsyncSession, user_id: int, topic_id: int, now: datetime) -> int:
"""문항 복습 due (기존 study_question_progress 엔진 재사용, 홈 표시용)."""
return (
await session.execute(
select(func.count())
.select_from(StudyQuestionProgress)
.where(
StudyQuestionProgress.user_id == user_id,
StudyQuestionProgress.study_topic_id == topic_id,
StudyQuestionProgress.due_at.is_not(None),
StudyQuestionProgress.due_at <= now,
or_(
StudyQuestionProgress.review_stage.is_(None),
StudyQuestionProgress.review_stage < 4,
),
)
)
).scalar_one()
async def curriculum(session: AsyncSession, user_id: int, topic_id: int) -> dict:
"""과목별 회독 진도 + 개념/문항 복습 due 요약 (진도 대시보드)."""
name = await _topic_name(session, topic_id)
rows = await _concept_rows(session, user_id, name) if name else []
now = datetime.now(timezone.utc)
subj: dict[str, dict] = {}
for r in rows:
s = subj.setdefault(r["subject"], {"subject": r["subject"], "total": 0, "read": 0})
s["total"] += 1
if r["is_read"]:
s["read"] += 1
total = len(rows)
read = sum(1 for r in rows if r["is_read"])
concept_due = sum(1 for r in rows if _is_due(r, now))
question_due = await _question_due_count(session, user_id, topic_id, now)
return {
"topic_id": topic_id,
"topic_name": name,
"subjects": sorted(subj.values(), key=lambda x: x["subject"]),
"total": total,
"read": read,
"concept_due": concept_due,
"question_due": question_due,
}
async def today_concepts(
session: AsyncSession, user_id: int, topic_id: int, limit: int = 6
) -> dict:
"""오늘 공부할 개념 = 재복습(SR due) 먼저 → 미독(빈출 우선). 졸업/재복습대기 제외."""
name = await _topic_name(session, topic_id)
rows = await _concept_rows(session, user_id, name) if name else []
now = datetime.now(timezone.utc)
due = [r for r in rows if _is_due(r, now)]
due.sort(key=lambda r: r["due_at"])
# 미독 & 아직 SR 큐 진입 전(due_at NULL) → 빈출 높은 순
unread = [r for r in rows if not r["is_read"] and r["due_at"] is None]
unread.sort(key=lambda r: (-_freq(r), r["subject"], r["title"]))
picked = [{**_item(r), "reason": "재복습"} for r in due]
picked += [{**_item(r), "reason": "신규"} for r in unread]
return {
"concepts": picked[:limit],
"due_total": len(due),
"unread_total": len(unread),
}
async def mark_read(
session: AsyncSession, user_id: int, topic_id: int, doc_id: int, now: datetime | None = None
) -> dict:
"""개념 회독 처리 = document_reads(+1) + 회독 SR 입고/전진.
회독 정본 = document_reads(append-only), documents.is_read 컬럼 아님(library unread 정합).
회독 first_due(stage 0, 내일). 이후 회독은 'due 도래(due_at<=now)' 때만 correct 전진
(이른 재열람/다중클릭 과전진 방지). stage 4 졸업 후엔 due_at NULL 이라 전진 없음.
"""
now = now or datetime.now(timezone.utc)
# 회독 로그 append (+1) — 사용자 명시 회독. 자동 아님(엔드포인트 = 명시 POST).
session.add(DocumentRead(user_id=user_id, document_id=doc_id, read_at=now))
prog = (
await session.execute(
select(StudyConceptProgress).where(
StudyConceptProgress.user_id == user_id,
StudyConceptProgress.concept_doc_id == doc_id,
)
)
).scalar_one_or_none()
if prog is None:
stage, due = first_due(now)
prog = StudyConceptProgress(
user_id=user_id,
study_topic_id=topic_id,
concept_doc_id=doc_id,
review_stage=stage,
due_at=due,
last_read_at=now,
)
session.add(prog)
else:
# due 도래 시에만 전진 — 미래 due(재열람 이른 클릭)는 stage 불변, last_read_at 만 갱신.
if prog.due_at is not None and prog.due_at <= now:
res = advance(prog.review_stage, "correct", now)
if res is not None:
prog.review_stage, prog.due_at = res
prog.last_read_at = now
await session.commit()
await session.refresh(prog)
return {"ok": True, "review_stage": prog.review_stage, "due_at": prog.due_at}
_CONCEPT_ONE_SQL = text(
"""
SELECT d.id AS doc_id, d.title AS title, d.md_content AS md_content,
split_part(replace(d.user_tags::text, '"', ''), '/', 3) AS subject,
(d.md_content LIKE '%★★★%') AS f3,
(d.md_content LIKE '%★★%') AS f2,
EXISTS (
SELECT 1 FROM document_reads r
WHERE r.document_id = d.id AND r.user_id = :uid
) AS is_read,
p.review_stage AS review_stage,
p.due_at AS due_at
FROM documents d
LEFT JOIN study_concept_progress p ON p.concept_doc_id = d.id AND p.user_id = :uid
WHERE d.id = :doc_id AND d.deleted_at IS NULL AND d.user_tags::text LIKE :like
"""
)
async def concept_detail(
session: AsyncSession, user_id: int, topic_id: int, doc_id: int
) -> dict | None:
"""개념 리더 재료 — md 구조 파싱 + 관련개념 백링크 해소 + 회독/SR 상태 + 같은 과목 이전/다음."""
name = await _topic_name(session, topic_id)
if not name:
return None
like = f"%@library/{name}/%"
row = (
await session.execute(
_CONCEPT_ONE_SQL, {"uid": user_id, "doc_id": doc_id, "like": like}
)
).mappings().first()
if row is None:
return None
parsed = parse_concept(row["md_content"] or "")
# 백링크 해소 + 이전/다음 = 같은 토픽 개념 title 인덱스(회독 rows 재사용)
idx = await _concept_rows(session, user_id, name)
title_index = [(r["doc_id"], r["title"], r["subject"]) for r in idx]
resolved = resolve_related(parsed["related"], title_index)
# 이전/다음 = 같은 과목, title 순
same = sorted(
[(r["doc_id"], r["title"]) for r in idx if r["subject"] == row["subject"]],
key=lambda x: (x[1] or "", x[0]),
)
ids = [d for d, _ in same]
prev_id = next_id = None
if doc_id in ids:
pos = ids.index(doc_id)
if pos > 0:
prev_id = ids[pos - 1]
if pos < len(ids) - 1:
next_id = ids[pos + 1]
freq = 3 if row["f3"] else (2 if row["f2"] else 1)
return {
"doc_id": row["doc_id"],
"db_title": row["title"],
"title": parsed["title"] or row["title"],
"subject": row["subject"],
"freq": freq,
"summary": parsed["summary"],
"body": parsed["body"],
"bincheol": parsed["bincheol"],
"related": resolved,
"is_read": row["is_read"],
"review_stage": row["review_stage"],
"due_at": row["due_at"],
"prev_id": prev_id,
"next_id": next_id,
}
+139
View File
@@ -0,0 +1,139 @@
"""concept_links — 이론↔문제 브리지 롤업 (Stage B).
study_concept_links(개념 doc 기출문항, 임베딩 코사인) + study_question_progress( 풀이상태)
조인해 (a) 개념별 관련 기출 + 정답률(related_questions), (b) 개념 약점 지도(weakness_map) 산출.
읽기 전용 집계 · LLM 0. 링크 적재는 scripts/concept_links_backfill.sql(임베딩) 배치.
정답률 = 링크된 문항 progress.last_outcome 기준(attempted=풀이이력 보유, correct=최근정답).
"""
from __future__ import annotations
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
_ACCURACY_WEAK_PCT = 60 # 정답률 < 60% = 약점(attempted>0 일 때만)
_AGG_SQL = text(
"""
SELECT count(*) AS linked,
count(pr.study_question_id) FILTER (WHERE pr.last_outcome IS NOT NULL) AS attempted,
count(*) FILTER (WHERE pr.last_outcome = 'correct') AS correct
FROM study_concept_links l
LEFT JOIN study_question_progress pr
ON pr.study_question_id = l.question_id AND pr.user_id = :uid
WHERE l.concept_doc_id = :doc_id AND l.link_source = 'embedding'
"""
)
_QROWS_SQL = text(
"""
SELECT q.id AS id, q.subject AS subject, q.exam_round AS exam_round,
q.exam_question_number AS qnum, l.score AS score,
pr.last_outcome AS last_outcome, pr.review_stage AS review_stage
FROM study_concept_links l
JOIN study_questions q ON q.id = l.question_id AND q.deleted_at IS NULL AND q.is_active
LEFT JOIN study_question_progress pr
ON pr.study_question_id = q.id AND pr.user_id = :uid
WHERE l.concept_doc_id = :doc_id AND l.link_source = 'embedding'
ORDER BY l.score DESC
LIMIT :limit
"""
)
_WEAKNESS_SQL = text(
"""
SELECT d.id AS doc_id, d.title AS title,
split_part(replace(d.user_tags::text, '"', ''), '/', 3) AS subject,
count(l.id) AS linked,
count(pr.study_question_id) FILTER (WHERE pr.last_outcome IS NOT NULL) AS attempted,
count(*) FILTER (WHERE pr.last_outcome = 'correct') AS correct
FROM documents d
JOIN study_concept_links l ON l.concept_doc_id = d.id AND l.link_source = 'embedding'
LEFT JOIN study_question_progress pr
ON pr.study_question_id = l.question_id AND pr.user_id = :uid
WHERE d.user_tags::text LIKE :like AND d.deleted_at IS NULL
GROUP BY d.id, d.title, subject
"""
)
async def related_questions(
session: AsyncSession, user_id: int, doc_id: int, limit: int = 20
) -> dict:
"""개념 doc 의 관련 기출 + 내 정답률(전체 링크 기준 집계 + 상위 N 표시용)."""
agg = (
await session.execute(_AGG_SQL, {"uid": user_id, "doc_id": doc_id})
).mappings().first()
rows = (
await session.execute(
_QROWS_SQL, {"uid": user_id, "doc_id": doc_id, "limit": limit}
)
).mappings().all()
linked = (agg["linked"] if agg else 0) or 0
attempted = (agg["attempted"] if agg else 0) or 0
correct = (agg["correct"] if agg else 0) or 0
accuracy = round(100 * correct / attempted) if attempted else None
return {
"linked": linked,
"attempted": attempted,
"correct": correct,
"accuracy": accuracy,
"questions": [
{
"id": r["id"],
"subject": r["subject"],
"exam_round": r["exam_round"],
"qnum": r["qnum"],
"score": round(r["score"], 3) if r["score"] is not None else None,
"last_outcome": r["last_outcome"],
"review_stage": r["review_stage"],
}
for r in rows
],
}
async def weakness_map(
session: AsyncSession, user_id: int, topic_name: str, limit: int = 12
) -> dict:
"""개념 약점 지도 — 링크된 기출 정답률로 개념 채색. 약점(attempted>0·정답률<60%) 우선 정렬."""
like = f"%@library/{topic_name}/%"
rows = (
await session.execute(_WEAKNESS_SQL, {"uid": user_id, "like": like})
).mappings().all()
concepts = []
for r in rows:
attempted = r["attempted"] or 0
correct = r["correct"] or 0
accuracy = round(100 * correct / attempted) if attempted else None
if accuracy is None:
state = "unattempted"
elif accuracy < _ACCURACY_WEAK_PCT:
state = "weak"
else:
state = "ok"
concepts.append(
{
"doc_id": r["doc_id"],
"title": r["title"],
"subject": r["subject"],
"linked": r["linked"] or 0,
"attempted": attempted,
"accuracy": accuracy,
"state": state,
}
)
# 약점 우선(정답률 오름차순) → 미평가는 뒤로. 홈 위젯용 상위 N.
weak = sorted(
[c for c in concepts if c["state"] == "weak"],
key=lambda c: (c["accuracy"], -c["attempted"], c["doc_id"]),
)
return {
"weak": weak[:limit],
"weak_total": len(weak),
"evaluated_total": sum(1 for c in concepts if c["state"] != "unattempted"),
}
+175
View File
@@ -0,0 +1,175 @@
"""concept_parser — 개념노트 markdown 구조 파서 + 관련개념 백링크 해소 (이론 리더용).
정찰 실측 불변식(273/273): 개념노트는 고정 골격을 100% 따름
# {H1 제목} (첫 줄, DB title 과 다른 표시용 제목)
> ** 요약**: {요약} (blockquote, 라벨 고정)
## {본문 라벨} ... (BODY, 자유 라벨 H2 0~N, 트레일 ★ 가능)
## 빈출 포인트 (항상, 관련개념 직전)
## 관련 개념 (항상, 문서 최종 섹션)
코드펜스(``` ASCII 도식) 내부의 ##/- 는 무시. 헤딩 트레일 ★ 는 스트립(라벨 정규화).
'빈출 포인트'/'관련 개념' 앵커만 이름으로 잡고 나머지 BODY 순서·위치로 처리(라벨 화이트리스트 금지).
순수 함수 · LLM 0.
"""
from __future__ import annotations
import re
_FENCE = re.compile(r"^\s*```")
_H1 = re.compile(r"^#\s+(.+?)\s*$")
_H2 = re.compile(r"^##\s+(.+?)\s*$") # ### 는 매칭 안 됨(## 뒤 \s 요구)
_SUMMARY = re.compile(r"^>\s*\*\*한 줄 요약\*\*:\s*(.+)$")
_STAR_SUFFIX = re.compile(r"\s*★+\s*$")
_TRAIL_STARS = re.compile(r"★+\s*$")
_BINCHEOL_ITEM = re.compile(r"^\s*-\s+(★*)\s*(.+)$")
_RELATED_ITEM = re.compile(r"^\s*-\s+(.+)$")
_PAREN = re.compile(r"\s*\(.*$") # 괄호부터 끝(clarifier 힌트 절단)
_NUM_PREFIX = re.compile(r"^\d+_")
_STRIP_SYM = re.compile(r"[\s_·,./()\-]")
_ANCHOR_BINCHEOL = "빈출 포인트"
_ANCHOR_RELATED = "관련 개념"
def parse_concept(md: str) -> dict:
"""개념노트 md → {title, summary, body[{label,stars,md}], bincheol[{tier,text}], related[{raw,phrase,hint}]}."""
lines = (md or "").split("\n")
title: str | None = None
summary: str | None = None
body: list[dict] = []
bincheol_lines: list[str] = []
related_lines: list[str] = []
in_fence = False
zone = "pre" # pre | body | bincheol | related
body_cur: dict | None = None
def emit(line: str) -> None:
if body_cur is not None:
body_cur["_lines"].append(line)
elif zone == "bincheol":
bincheol_lines.append(line)
elif zone == "related":
related_lines.append(line)
# pre-zone 내용(요약 앞 잡음)은 버림
for ln in lines:
if _FENCE.match(ln):
in_fence = not in_fence
emit(ln)
continue
if in_fence:
emit(ln)
continue
if title is None:
m = _H1.match(ln)
if m:
title = m.group(1).strip()
continue
if summary is None:
m = _SUMMARY.match(ln)
if m:
summary = m.group(1).strip()
continue
m2 = _H2.match(ln)
if m2:
raw_label = m2.group(1).strip()
star_m = _TRAIL_STARS.search(raw_label)
stars = len(star_m.group(0).strip()) if star_m else 0
label = _STAR_SUFFIX.sub("", raw_label).strip()
if label == _ANCHOR_BINCHEOL:
zone = "bincheol"
body_cur = None
continue
if label == _ANCHOR_RELATED:
zone = "related"
body_cur = None
continue
body_cur = {"label": label, "stars": stars, "_lines": []}
body.append(body_cur)
zone = "body"
continue
emit(ln)
body_out = []
for s in body:
text = "\n".join(s["_lines"]).strip()
if text or s["label"]:
body_out.append({"label": s["label"], "stars": s["stars"], "md": text})
bincheol = []
for ln in bincheol_lines:
m = _BINCHEOL_ITEM.match(ln)
if m:
bincheol.append({"tier": len(m.group(1)), "text": m.group(2).strip()})
related = []
for ln in related_lines:
m = _RELATED_ITEM.match(ln)
if m:
raw = m.group(1).strip()
phrase = _PAREN.sub("", raw).strip()
hint = raw[len(phrase):].strip() if len(raw) > len(phrase) else ""
if phrase:
related.append({"raw": raw, "phrase": phrase, "hint": hint})
return {
"title": title,
"summary": summary,
"body": body_out,
"bincheol": bincheol,
"related": related,
}
def _normalize(s: str) -> str:
"""해소용 정규화: NN_ 접두 제거 → 소문자 → 공백/기호 제거. 영문은 lowercase 유지."""
s = _NUM_PREFIX.sub("", s or "")
s = s.lower()
s = _STRIP_SYM.sub("", s)
return s
def resolve_related(related: list[dict], title_index: list[tuple]) -> list[dict]:
"""관련개념 구절 → 개념 doc 해소. title_index = [(doc_id, title, subject), ...].
다단 fallback(정찰 ~79%): 정규화 exact 양방향 substring(2 가드) 미해소=dangling(doc_id None).
"""
norm_exact: dict[str, int] = {}
norm_list: list[tuple[str, int, str]] = []
for did, ttl, _subj in title_index:
n = _normalize(ttl)
if n:
norm_exact.setdefault(n, did)
norm_list.append((n, did, ttl))
out = []
for it in related:
pn = _normalize(it["phrase"])
did: int | None = None
rtitle: str | None = None
if pn and len(pn) >= 2:
if pn in norm_exact:
did = norm_exact[pn]
else:
# substring 폴백: title-norm ⊆ phrase-norm 방향만(짧은 phrase 가 더 큰 title 을
# 삼키는 오결선 방지, 예: '염산'→'염산나트륨' X) + 길이차 최소(가장 구체적) +
# doc_id tiebreak(순서 무관 결정성). 후보 없으면 dangling(doc_id None).
cands = [
(abs(len(n) - len(pn)), cand, ttl)
for n, cand, ttl in norm_list
if len(n) >= 2 and n in pn
]
if cands:
cands.sort(key=lambda c: (c[0], c[1]))
_, did, rtitle = cands[0]
if did is not None and rtitle is None:
rtitle = next((t for d, t, _ in title_index if d == did), None)
out.append(
{"phrase": it["phrase"], "hint": it["hint"], "doc_id": did, "title": rtitle}
)
return out
+12 -12
View File
@@ -68,10 +68,10 @@ async def enqueue_question_publish(session: AsyncSession, q: Any) -> None:
await enqueue_publish(session, kind=KIND_EXPLANATION, source_id=q.id, payload=expl)
async def backfill_publish_questions(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> int:
async def backfill_publish_questions(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> tuple[int, int]:
"""active(미삭제) 문항을 id>after_id 부터 bounded 로 outbox 적재.
반환 = enqueue 문항 (0 이면 ). 셋은 마지막 id 페이지 반복. caller commit.
반환 = (enqueue , 마지막 처리 id). caller ==limit last_id 다음 페이지. caller commit.
"""
rows = (
await session.execute(
@@ -83,7 +83,7 @@ async def backfill_publish_questions(session: AsyncSession, *, after_id: int = 0
).scalars().all()
for q in rows:
await enqueue_question_publish(session, q)
return len(rows)
return len(rows), (rows[-1].id if rows else after_id)
async def enqueue_topic_publish(session: AsyncSession, topic: Any) -> None:
@@ -91,10 +91,10 @@ async def enqueue_topic_publish(session: AsyncSession, topic: Any) -> None:
await enqueue_publish(session, kind=KIND_TOPIC, source_id=topic.id, payload=project_topic(topic))
async def backfill_publish_topics(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> int:
async def backfill_publish_topics(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> tuple[int, int]:
"""active(미삭제) 주제를 id>after_id 부터 bounded 로 outbox 적재(S-1 초기 백필).
반환 = enqueue 주제 (0 이면 ). 셋은 마지막 id 페이지 반복. caller commit.
반환 = (enqueue , 마지막 처리 id). caller ==limit last_id 다음 페이지. caller commit.
멱등 = 발행 워커의 (payload_hash, deleted) 디둡이 no-op 재투영 흡수(중복 enqueue 무해).
"""
rows = (
@@ -107,7 +107,7 @@ async def backfill_publish_topics(session: AsyncSession, *, after_id: int = 0, l
).scalars().all()
for t in rows:
await enqueue_topic_publish(session, t)
return len(rows)
return len(rows), (rows[-1].id if rows else after_id)
async def enqueue_card_publish(session: AsyncSession, card: Any) -> None:
@@ -123,10 +123,10 @@ async def enqueue_card_publish(session: AsyncSession, card: Any) -> None:
await enqueue_publish(session, kind=KIND_CARD, source_id=card.id, payload=project_card(card))
async def backfill_publish_cards(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> int:
async def backfill_publish_cards(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> tuple[int, int]:
"""검수완료(needs_review=False)·미삭제 카드를 id>after_id 부터 bounded 로 outbox 적재(S-2 초기 백필).
반환 = enqueue 카드 (0 이면 ). 멱등 = 워커 (payload_hash, deleted) 디둡. caller commit.
반환 = (enqueue , 마지막 처리 id). caller ==limit last_id 다음 페이지. 멱등 = 워커 디둡. caller commit.
"""
rows = (
await session.execute(
@@ -142,7 +142,7 @@ async def backfill_publish_cards(session: AsyncSession, *, after_id: int = 0, li
).scalars().all()
for c in rows:
await enqueue_card_publish(session, c)
return len(rows)
return len(rows), (rows[-1].id if rows else after_id)
async def enqueue_card_progress_publish(session: AsyncSession, progress: Any) -> None:
@@ -155,11 +155,11 @@ async def enqueue_card_progress_publish(session: AsyncSession, progress: Any) ->
)
async def backfill_publish_card_progress(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> int:
async def backfill_publish_card_progress(session: AsyncSession, *, after_id: int = 0, limit: int = 200) -> tuple[int, int]:
"""모든 card progress row 를 id>after_id 부터 bounded 로 outbox 적재(S-4 초기 백필).
필터 없음 = ALL row(due_at NULL sentinel·terminal 포함) due-only 백필은 sentinel 누락.
반환 = enqueue row (0 ). 멱등 = 워커 디둡. caller commit.
반환 = (enqueue , 마지막 처리 id). caller ==limit last_id 다음 페이지. 멱등 = 워커 디둡. caller commit.
"""
rows = (
await session.execute(
@@ -171,4 +171,4 @@ async def backfill_publish_card_progress(session: AsyncSession, *, after_id: int
).scalars().all()
for p in rows:
await enqueue_card_progress_publish(session, p)
return len(rows)
return len(rows), (rows[-1].id if rows else after_id)
+4
View File
@@ -20,6 +20,7 @@ from models.chunk import DocumentChunk
from models.document import Document
from models.study_question import StudyQuestion
from models.study_topic import StudyTopicDocument
from services.search.license_filter import restricted_exclude_orm
logger = logging.getLogger(__name__)
@@ -113,6 +114,9 @@ async def _gather_document_evidence(
select(Document.id, Document.title, Document.ai_summary).where(
Document.id.in_(doc_ids),
Document.deleted_at.is_(None),
# B-4: licensed_restricted 제외 — explanation_rag 와 동일 술어(a안 U-2①). 누락 시
# 구매 자료 verbatim 이 분야노트 RAG 로 새던 보안 drift(복제 과정 누락).
restricted_exclude_orm(),
)
)
).all()
+21 -12
View File
@@ -90,6 +90,7 @@ HARD_ESCALATE_REASONS = {
class TriageOutput(BaseModel):
"""p3a_short_summary (4B) 응답 스키마. 파싱 실패 시 기본값 + escalate=True fallback."""
ai_summary: str = "" # B-1 3→2: triage 가 ai_summary 도 생산 (별 summarize 콜 대체)
tldr: str = ""
bullets: list[str] = Field(default_factory=list)
tags: list[str] = Field(default_factory=list)
@@ -579,16 +580,7 @@ async def process(
"reason": "classify pipeline",
}
# ─── 2. Legacy 요약 (primary 또는 deep) ───
try:
summary = await client.summarize(doc.extracted_text[:50000], cfg=legacy_cfg)
except Exception as exc:
if legacy_cfg is not None and is_deferrable_error(exc):
raise StageDeferred(f"macbook_unavailable:{type(exc).__name__}") from exc
raise
doc.ai_summary = strip_thinking(summary)
# ─── 메타데이터 (legacy 완료) — 실제 처리 머신 귀속 (drain=qwen-macbook) ───
# ─── 메타데이터 (classify 완료) — 실제 처리 머신 귀속 (drain=qwen-macbook) ───
doc.ai_model_version = (legacy_cfg or settings.ai.primary).model
doc.ai_processed_at = datetime.now(timezone.utc)
@@ -598,13 +590,27 @@ async def process(
f"confidence={doc.ai_confidence:.2f}, tags={doc.ai_tags}"
)
# ─── 3. PR-B B-1 — tier triage (4B, 실패는 legacy 결과 보존) ───
# ─── 2+3 통합 (B-1 3→2): tier triage 가 tldr/bullets/tier + ai_summary 생산.
# 기존 별도 summarize 콜 제거 → 본문 prefill 1회 절감 (Mac mini 부하). 실패는 fallback.
try:
await _run_tier_triage(client, doc, session, use_deep=use_deep)
except StageDeferred:
raise # 보류는 실패가 아님 — drain/consumer 가 attempts 미소모 처리
except Exception as exc:
logger.exception(f"[triage] id={document_id} 전체 실패 — legacy 유지: {exc}")
logger.exception(f"[triage] id={document_id} 전체 실패: {exc}")
# ─── ai_summary fallback: triage 가 못 채운 경우만 summarize ───
# (>120K long_context 는 triage 가 LLM skip, 또는 triage 파싱실패). 정상 경로는 미발동.
if not doc.ai_summary:
try:
summary = await client.summarize(doc.extracted_text[:50000], cfg=legacy_cfg)
doc.ai_summary = strip_thinking(summary)
except Exception as exc:
if legacy_cfg is not None and is_deferrable_error(exc):
raise StageDeferred(f"macbook_unavailable:{type(exc).__name__}") from exc
# ai_summary=NULL 로 완료되면 digest/briefing 이 조용히 제외 → ERROR 로 가시화
# (best-effort 강등 자체는 유지, 운영 추적성만 보강).
logger.error(f"[summary-fallback] id={document_id} ai_summary 미생성: {exc}")
finally:
await client.close()
@@ -774,6 +780,9 @@ async def _apply_triage_result(
if not parse_error:
doc.ai_tldr = (triage_out.tldr or "").strip() or None
doc.ai_bullets = triage_out.bullets or []
# B-1 3→2: triage 가 ai_summary 도 생산(summarize 콜 대체). 비면 process() 가 fallback.
if triage_out.ai_summary.strip():
doc.ai_summary = triage_out.ai_summary.strip()
# Memo Intake Upgrade PR-2B — event kind hint (4B 가 출력했을 때만)
# 허용 enum 외 값이면 무시 (DB enum 제약). AI worker 는 events row 직접 생성 X.
valid_kinds = {"note", "task", "calendar_event", "activity_log", "reference"}
+5 -2
View File
@@ -140,7 +140,8 @@ async def _download_pdf(url: str, dest: Path) -> int:
if len(resp.content) > _MAX_PDF_BYTES:
raise FeedError(f"PDF 크기 초과 ({len(resp.content)} bytes): {url}")
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_bytes(resp.content)
# 최대 50MB PDF write 는 동기 blocking — 이벤트루프 점유 회피 to_thread (R5 동형).
await asyncio.to_thread(dest.write_bytes, resp.content)
return len(resp.content)
@@ -190,9 +191,11 @@ async def _ingest_pdf(session, page_slug: str, pdf_url: str) -> bool:
dest = Path(settings.nas_mount_path) / rel_path
size = await _download_pdf(pdf_url, dest)
# 50MB PDF read + sha256 는 동기 blocking(I/O+CPU) — 이벤트루프 점유 회피 to_thread (R5 동형).
file_hash = await asyncio.to_thread(lambda: hashlib.sha256(dest.read_bytes()).hexdigest())
doc = Document(
file_path=rel_path,
file_hash=hashlib.sha256(dest.read_bytes()).hexdigest(),
file_hash=file_hash,
file_format="pdf",
file_size=size,
file_type="immutable",
+96 -82
View File
@@ -251,104 +251,118 @@ async def watch_inbox():
for extra_path in settings.additional_watch_targets:
targets.append((extra_path, "library"))
async with async_session() as session:
# ─── Web/ 트랙 (devonagent) — DEVONthink Smart Rule 이 떨군 .html 만 진입 ───
if web_root.exists():
# rglob NFS 디렉토리 walk(blocking stat 다발)를 off-thread 로 수집 (R5).
for file_path in await asyncio.to_thread(lambda: list(web_root.rglob("*.html"))):
if not file_path.is_file() or should_skip(file_path):
continue
rel_path = str(file_path.relative_to(nas_root))
added, _ = await _ingest_web_file(session, file_path, rel_path)
# 파일별 독립 세션+commit 으로 격리 — 한 파일 실패(예: rglob↔stat 사이 삭제로 FileNotFoundError,
# flush 오류)가 watch_inbox 전체를 raise·롤백해 그 사이클 등록분을 모두 잃거나, 결정적 poison
# 파일이 매 사이클 같은 지점에서 중단시키는 것을 차단 (news_collector/csb_collector 와 동형).
# ─── Web/ 트랙 (devonagent) — DEVONthink Smart Rule 이 떨군 .html 만 진입 ───
if web_root.exists():
# rglob NFS 디렉토리 walk(blocking stat 다발)를 off-thread 로 수집 (R5).
for file_path in await asyncio.to_thread(lambda: list(web_root.rglob("*.html"))):
if not file_path.is_file() or should_skip(file_path):
continue
rel_path = str(file_path.relative_to(nas_root))
try:
async with async_session() as session:
added, _ = await _ingest_web_file(session, file_path, rel_path)
await session.commit()
new_count += added
# ─── PKM 트랙 (기존 drive_sync) ─────────────────────────────────────────
for sub, expected_category in targets:
scan_root = pkm_root / sub
if not scan_root.exists():
except Exception as e:
logger.warning("[Web] 파일 처리 실패 skip path=%s: %s", rel_path, e)
continue
# 안전 자료실 A-2/B-4 — 타깃 폴더 기반 (material, jurisdiction, license)
target_mt, target_jur, target_license = _TARGET_AXIS.get(
Path(sub).name, (None, None, None)
# ─── PKM 트랙 (기존 drive_sync) ─────────────────────────────────────────
for sub, expected_category in targets:
scan_root = pkm_root / sub
if not scan_root.exists():
continue
# 안전 자료실 A-2/B-4 — 타깃 폴더 기반 (material, jurisdiction, license)
target_mt, target_jur, target_license = _TARGET_AXIS.get(
Path(sub).name, (None, None, None)
)
# NFS 디렉토리 walk(blocking) off-thread 수집 (R5).
for file_path in await asyncio.to_thread(lambda: list(scan_root.rglob("*"))):
if not file_path.is_file() or should_skip(file_path):
continue
category, needs_conversion, next_stage = _route_media(
file_path, expected_category
)
# NFS 디렉토리 walk(blocking) off-thread 수집 (R5).
for file_path in await asyncio.to_thread(lambda: list(scan_root.rglob("*"))):
if not file_path.is_file() or should_skip(file_path):
continue
# audio/video 폴더에 엉뚱한 확장자가 들어왔거나 Inbox 에
# audio/video 가 잘못 떨어진 경우 — 이 라운드에서 아예 skip
if category is None and next_stage is None:
continue
category, needs_conversion, next_stage = _route_media(
file_path, expected_category
)
# audio/video 폴더에 엉뚱한 확장자가 들어왔거나 Inbox 에
# audio/video 가 잘못 떨어진 경우 — 이 라운드에서 아예 skip
if category is None and next_stage is None:
continue
rel_path = str(file_path.relative_to(nas_root))
rel_path = str(file_path.relative_to(nas_root))
try:
# GB 파일 SHA-256 은 이벤트 루프를 점유 → 같은 루프의 모든 1분 주기 consumer
# + FastAPI 요청이 수십초~분 동시 정지. to_thread 오프로드. 스캔 루프가 이미
# 순차라 file_hash 는 한 번에 하나만 실행(직렬화) — 병렬 해싱 X = NFS 2.5GbE
# 대역폭·버퍼 메모리 blowup 방지 (R5).
# 대역폭·버퍼 메모리 blowup 방지 (R5). 세션 밖에서 계산(커넥션 미점유).
fhash = await asyncio.to_thread(file_hash, file_path)
result = await session.execute(
select(Document).where(Document.file_path == rel_path)
)
existing = result.scalar_one_or_none()
if existing is None:
ext = file_path.suffix.lstrip(".").lower() or "unknown"
doc = Document(
file_path=rel_path,
file_hash=fhash,
file_format=ext,
file_size=file_path.stat().st_size,
file_type="immutable",
title=file_path.stem,
source_channel="drive_sync",
category=category,
needs_conversion=needs_conversion,
# 안전 자료실 A-2/B-4 — watch 타깃 매핑 (KGS=law/KR 등, 비대상=NULL)
material_type=target_mt,
jurisdiction=target_jur,
async with async_session() as session:
result = await session.execute(
select(Document).where(Document.file_path == rel_path)
)
# B-4 — 타깃 폴더 license 주입(restricted 포함, 비대상=미주입). classify 는
# material_type IS NULL 일 때만 제안 + extract_meta 미기록이라 주입 보존.
if target_license:
doc.extract_meta = {"license": dict(target_license)}
session.add(doc)
await session.flush()
existing = result.scalar_one_or_none()
if next_stage:
await enqueue_stage(session, doc.id, next_stage)
new_count += 1
if existing is None:
ext = file_path.suffix.lstrip(".").lower() or "unknown"
doc = Document(
file_path=rel_path,
file_hash=fhash,
file_format=ext,
file_size=file_path.stat().st_size,
file_type="immutable",
title=file_path.stem,
source_channel="drive_sync",
category=category,
needs_conversion=needs_conversion,
# 안전 자료실 A-2/B-4 — watch 타깃 매핑 (KGS=law/KR 등, 비대상=NULL)
material_type=target_mt,
jurisdiction=target_jur,
)
# B-4 — 타깃 폴더 license 주입(restricted 포함, 비대상=미주입). classify 는
# material_type IS NULL 일 때만 제안 + extract_meta 미기록이라 주입 보존.
if target_license:
doc.extract_meta = {"license": dict(target_license)}
session.add(doc)
await session.flush()
elif existing.file_hash != fhash:
existing.file_hash = fhash
existing.file_size = file_path.stat().st_size
# 기존 문서에 category/quarantine flag 가 비어있으면 보정
if existing.category is None and category is not None:
existing.category = category
if needs_conversion and not getattr(existing, "needs_conversion", False):
existing.needs_conversion = True
# B-4 — 축/license 보정(B-4 이전 적재분이 재변경 시): material 미설정 시 주입,
# license 부재 시에만 merge 주입(clobber 회피 — 기존 extract_meta 키 보존).
if existing.material_type is None and target_mt is not None:
existing.material_type = target_mt
existing.jurisdiction = target_jur
if target_license and not (existing.extract_meta or {}).get("license"):
meta = dict(existing.extract_meta or {})
meta["license"] = dict(target_license)
existing.extract_meta = meta
if next_stage:
await enqueue_stage(session, doc.id, next_stage)
await session.commit()
new_count += 1
if next_stage:
await enqueue_stage(session, existing.id, next_stage)
changed_count += 1
elif existing.file_hash != fhash:
existing.file_hash = fhash
existing.file_size = file_path.stat().st_size
# 기존 문서에 category/quarantine flag 가 비어있으면 보정
if existing.category is None and category is not None:
existing.category = category
if needs_conversion and not getattr(existing, "needs_conversion", False):
existing.needs_conversion = True
# B-4 — 축/license 보정(B-4 이전 적재분이 재변경 시): material 미설정 시 주입,
# license 부재 시에만 merge 주입(clobber 회피 — 기존 extract_meta 키 보존).
if existing.material_type is None and target_mt is not None:
existing.material_type = target_mt
existing.jurisdiction = target_jur
if target_license and not (existing.extract_meta or {}).get("license"):
meta = dict(existing.extract_meta or {})
meta["license"] = dict(target_license)
existing.extract_meta = meta
await session.commit()
if next_stage:
await enqueue_stage(session, existing.id, next_stage)
await session.commit()
changed_count += 1
# else: 무변경 → 쓰기 없음 (세션 자동 닫힘, commit 불요)
except Exception as e:
logger.warning("[PKM] 파일 처리 실패 skip path=%s: %s", rel_path, e)
continue
if new_count or changed_count:
logger.info(f"[Inbox+§3] 새 파일 {new_count}건, 변경 파일 {changed_count}건 등록")
+5
View File
@@ -300,6 +300,11 @@ async def _process_single(
f"[marker] transient error id={document_id} kind={type(exc).__name__}: {exc}"
)
raise
except json.JSONDecodeError as exc:
# 200 응답의 truncated/malformed body — 연결 흔들림 등 transient. _fail(non-retryable)
# 로 박지 말고 raise → queue retry (max_attempts 까지). 진짜 손상이면 재시도 후 failed.
logger.warning(f"[marker] malformed json body (200) id={document_id}: {exc}")
raise
except Exception as exc:
logger.exception(f"[marker] unexpected error id={document_id}: {exc}")
await _fail(session, document_id, str(exc)[:1000])
+9 -3
View File
@@ -497,12 +497,18 @@ async def process(document_id: int, session: AsyncSession) -> None:
logger.warning(f"[presegment] id={document_id} file not found ({raw}) → extract")
return
import asyncio
import fitz # PyMuPDF — extract_worker/marker_worker 와 동일 의존
def _read_toc(path: str):
# fitz open/get_toc 는 동기 blocking — live 스테이지라 이벤트루프(같은 루프의 1분 consumer +
# FastAPI 요청) 점유 회피 위해 to_thread 오프로드(거대/손상 PDF 파싱 수백 ms~초).
with fitz.open(path) as pdf:
return pdf.page_count, (pdf.get_toc(simple=True) or [])
try:
with fitz.open(str(source)) as pdf:
page_count = pdf.page_count
toc = pdf.get_toc(simple=True) or []
page_count, toc = await asyncio.to_thread(_read_toc, str(source))
except Exception as exc:
# PDF 손상 등 — 분할 불가. 단일문서로 통과(extract 가 PyMuPDF/OCR 로 재시도하며 가시화).
logger.warning(
+79 -46
View File
@@ -28,6 +28,8 @@ logger = setup_logger("study_publish_worker")
BATCH_SIZE = 500
# pg_advisory_xact_lock 전역 단일 라이터 키(발행 워커 전용 임의 상수, 타 advisory 락과 비충돌).
ADVISORY_LOCK_KEY = 838201
# 행별 격리 재시도 상한 — 초과 시 failed_at 스탬프(terminal)로 select 에서 제외.
MAX_OUTBOX_ATTEMPTS = 5
async def consume_publish_outbox() -> None:
@@ -46,11 +48,15 @@ async def consume_publish_outbox() -> None:
max_rev = int(
(await session.execute(select(func.coalesce(func.max(Published.rev), 0)))).scalar() or 0
)
# 3) 미처리 outbox 를 커밋순(id)으로.
# 3) 미처리 outbox 를 커밋순(id)으로. failed_at(terminal) 은 제외 — poison 행이
# head-of-line 을 영구 점유하지 않게 함.
rows = (
await session.execute(
select(PublishOutbox)
.where(PublishOutbox.processed_at.is_(None))
.where(
PublishOutbox.processed_at.is_(None),
PublishOutbox.failed_at.is_(None),
)
.order_by(PublishOutbox.id.asc())
.limit(BATCH_SIZE)
)
@@ -60,59 +66,86 @@ async def consume_publish_outbox() -> None:
now = datetime.now(timezone.utc)
published_count = 0
failed_count = 0
for ob in rows:
existing = (
await session.execute(
select(Published).where(
Published.kind == ob.kind,
Published.source_id == ob.source_id,
)
)
).scalar_one_or_none()
try:
# 행 단위 savepoint 격리 — 한 행의 예외가 배치 전체(앞 행 processed_at 포함)를
# 롤백해 poison 행이 다음 사이클에 다시 최저 id 로 선택되는 무한 재선택을 차단.
async with session.begin_nested():
existing = (
await session.execute(
select(Published).where(
Published.kind == ob.kind,
Published.source_id == ob.source_id,
)
)
).scalar_one_or_none()
# (payload_hash, deleted) 디둡 — no-op 재투영은 rev 안 올림.
if (
existing is not None
and existing.payload_hash == ob.payload_hash
and existing.deleted == ob.deleted
):
ob.processed_at = now
# (payload_hash, deleted) 디둡 — no-op 재투영은 rev 안 올림.
is_noop = (
existing is not None
and existing.payload_hash == ob.payload_hash
and existing.deleted == ob.deleted
)
if is_noop:
ob.processed_at = now
else:
new_rev = max_rev + 1
if existing is None:
session.add(
Published(
kind=ob.kind,
source_id=ob.source_id,
pub_id=uuid.uuid4().hex,
payload=ob.payload,
payload_hash=ob.payload_hash,
schema_version=ob.schema_version,
rev=new_rev,
deleted=ob.deleted,
created_at=now,
updated_at=now,
)
)
else:
existing.payload = ob.payload
existing.payload_hash = ob.payload_hash
existing.schema_version = ob.schema_version
existing.deleted = ob.deleted
existing.rev = new_rev
existing.updated_at = now
ob.processed_at = now
# 배치 내 동일 (kind, source_id) 후속 행이 직전 반영을 보도록 flush(최신 승).
await session.flush()
except Exception as row_err:
# savepoint 롤백 = 이 행의 쓰기(processed_at 포함) 취소. attempts/failed_at 만
# 바깥 트랜잭션에 누적돼 최종 commit 으로 영속(영구 재선택 방지).
ob.attempts = (ob.attempts or 0) + 1
if ob.attempts >= MAX_OUTBOX_ATTEMPTS:
ob.failed_at = now
failed_count += 1
logger.error(
"publish_outbox_row_terminal id=%s kind=%s source_id=%s attempts=%s: %s",
ob.id, ob.kind, ob.source_id, ob.attempts, row_err,
)
else:
logger.warning(
"publish_outbox_row_retry id=%s kind=%s source_id=%s attempts=%s: %s",
ob.id, ob.kind, ob.source_id, ob.attempts, row_err,
)
continue
max_rev += 1
if existing is None:
session.add(
Published(
kind=ob.kind,
source_id=ob.source_id,
pub_id=uuid.uuid4().hex,
payload=ob.payload,
payload_hash=ob.payload_hash,
schema_version=ob.schema_version,
rev=max_rev,
deleted=ob.deleted,
created_at=now,
updated_at=now,
)
)
else:
existing.payload = ob.payload
existing.payload_hash = ob.payload_hash
existing.schema_version = ob.schema_version
existing.deleted = ob.deleted
existing.rev = max_rev
existing.updated_at = now
ob.processed_at = now
# 배치 내 동일 (kind, source_id) 후속 행이 직전 반영을 보도록 flush(최신 승).
await session.flush()
published_count += 1
# savepoint 커밋 성공 시에만 rev 카운터 전진(실패 행은 rev 미소모 → 드물게 gap,
# 단일 라이터·커밋순 부여라 viewer since-rev 증분 동기 정합엔 무해).
if not is_noop:
max_rev = new_rev
published_count += 1
await session.commit()
logger.info(
"publish_outbox_drained scanned=%s published=%s max_rev=%s",
"publish_outbox_drained scanned=%s published=%s failed=%s max_rev=%s",
len(rows),
published_count,
failed_count,
max_rev,
)
except Exception as e:
+5 -15
View File
@@ -30,21 +30,11 @@ ai:
repetition_penalty: 1.05 # 한국어 장문 반복/코드스위칭(CJK·라틴 누수) 억제 (보수적 시작값)
top_k: 20 # Qwen3 권장
# deep: 야간 night-drain 전용 — 맥북 M5 Max Qwen3.6-27B-6bit (llm-router :8890 경유,
# model=qwen-macbook alias). 2026-06-11 재도입 (사용자: 자기 전 night-drain 으로 백로그 분담).
# 맥북 불가(503/연결/절단) = StageDeferred 보류 — 맥미니/cloud 강등 없음, attempts 미소모.
# consumer 의 deep_summary 도 슬롯 존재 시 맥북 경유 (잠들어 있으면 30분 백오프 보류 = 무해).
# 슬롯 제거 시 deep_summary 는 primary(맥미니) 경로 복귀.
deep:
endpoint: "http://100.76.254.116:8890/v1/chat/completions"
model: "qwen-macbook"
max_tokens: 8192
timeout: 900
context_char_limit: 260000
temperature: 0.3
top_p: 0.9
repetition_penalty: 1.05 # 한국어 장문 반복/코드스위칭 억제 (보수적 시작값)
top_k: 20
# deep: ★2026-06-29 잠정 보류 (사용자 "맥북 night-drain 의미없어 → 맥미니 일원화").
# 슬롯 제거 → deep_summary 가 primary(맥미니) 경로 복귀 + use_deep/drain 도 맥미니 폴백
# (맥북 라우팅 0). drain-keeper(GPU cron)도 비활성. 맥북 mlx-vlm-server 는 OpenCode 로컬용 보존.
# 복원(night-drain 재개 시): git history 에서 deep 슬롯(qwen-macbook :8890, max_tokens 8192,
# timeout 900, context_char_limit 260000, temp 0.3 / top_p 0.9 / rep 1.05 / top_k 20) 부활 + drain-keeper 재활성.
# fallback: primary 장애 시 최후 방어선. Claude Sonnet 4 API (소액 한도, 자동 trigger).
# 호출 빈도 낮음 가정 (Mac mini 가 거의 항상 up) → premium 과 budget 공유 OK.
+7 -1
View File
@@ -3,7 +3,13 @@ services:
image: pgvector/pgvector:pg16
volumes:
- pgdata:/var/lib/postgresql/data
- ./migrations:/docker-entrypoint-initdb.d
# ★ 2026-06-29 fresh-DB/DR 부팅 fix: initdb.d 마운트 제거(기존 `./migrations:/docker-entrypoint-initdb.d`).
# 빈 볼륨 첫 기동 시 postgres 엔트리포인트가 migrations/*.sql(001~) 을 psql autocommit 으로 실행해
# 스키마는 만들되 schema_migrations 스탬프는 안 남김(runner 만 생성) → fastapi init_db 가 documents
# 존재로 'fresh' 를 오판해 baseline(_load_baseline_if_fresh) 로드를 건너뛰고, 빈 schema_migrations
# 로 001 부터 재replay → `CREATE TABLE users`(IF NOT EXISTS 없음) 충돌 → 부팅 크래시(DR/신규환경).
# fresh-boot 은 init_db 의 baseline 적재 + migration runner 단일 경로로 일원화(설계 의도). 기존 prod
# 볼륨은 비어있지 않아 init scripts 가 애초에 미발동 → 무영향.
environment:
POSTGRES_DB: pkm
POSTGRES_USER: pkm
+1 -1
View File
@@ -1094,7 +1094,7 @@ services:
image: pgvector/pgvector:pg16
volumes:
- pgdata:/var/lib/postgresql/data
- ./migrations:/docker-entrypoint-initdb.d
# initdb.d 마운트 제거(2026-06-29): fresh-boot 은 fastapi init_db+baseline 단일 경로.
environment:
POSTGRES_DB: pkm
POSTGRES_USER: pkm
+2 -2
View File
@@ -71,7 +71,7 @@ GPU 서버의 NFS mount (`/proc/mounts` 실측):
| 컨테이너 | 마운트 | 모드 | 비고 |
|---|---|---|---|
| postgres | `pgdata:/var/lib/postgresql/data` + `./migrations:/docker-entrypoint-initdb.d` | rw | DB 본체 named volume |
| postgres | `pgdata:/var/lib/postgresql/data` | rw | DB 본체 named volume (initdb.d 마운트는 2026-06-29 제거 — 아래 관찰) |
| kordoc-service | `${NAS}/Document_Server:/documents` | **ro** | PDF/HWP parse |
| ocr-service | `${NAS}/Document_Server:/documents` + `ocr_models:/root/.cache` | **ro** + rw | |
| marker-service | `${NAS}/Document_Server:/documents` + `marker_models:/models` | **ro** + rw | PDF→markdown |
@@ -84,7 +84,7 @@ GPU 서버의 NFS mount (`/proc/mounts` 실측):
**관찰**:
- worker 컨테이너 (kordoc/ocr/marker/stt) 는 모두 NAS **read-only** 마운트 → 원본 안전.
- fastapi 만 NAS **rw** → 업로드/preview/extracted_images 쓰기 단일 책임.
- `./migrations` 이 postgres 의 `docker-entrypoint-initdb.d` fastapi 의 `/app/migrations` 양쪽에 마운트. 단 실제 migration runner 는 fastapi `init_db()` 만 사용 (postgres init scripts 는 첫 생성 시만 실행 → 효과 X, 안전).
- `./migrations` fastapi 의 `/app/migrations` 마운트. migration runner 는 fastapi `init_db()` 단일 경로. (~2026-06-29: postgres `docker-entrypoint-initdb.d` 마운트 제거. 기존엔 "첫 생성 시만 실행 → 효과 X" 로 봤으나, 빈 볼륨 첫 기동 시 postgres 가 migrations/*.sql 을 실제 실행해 스키마는 만들되 schema_migrations 스탬프를 안 남겨 → init_db 의 baseline fresh 판정을 깨고 부팅 크래시 유발. fresh-DB/DR 부팅을 init_db+baseline 단일 경로로 일원화.)
## 정책 정리
@@ -1,160 +0,0 @@
<!--
AskAnswerCard.svelte — 검색 결과 페이지 상단 AI 답변 카드 (컴팩트).
/ask 페이지의 AskAnswer.svelte와 달리, 검색 결과를 가리지 않는
보조 영역으로 설계. 출처 목록 클릭이 must-have, 본문 [n] 클릭은 nice-to-have.
-->
<script lang="ts">
import Badge from '$lib/components/ui/Badge.svelte';
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import { Sparkles, X, FileText } from 'lucide-svelte';
import type { AskResponse, Confidence } from '$lib/types/ask';
interface Props {
data: AskResponse | null;
loading: boolean;
error: boolean;
onCitationClick: (docId: number) => void;
onDismiss: () => void;
}
let { data, loading, error, onCitationClick, onDismiss }: Props = $props();
// [n] 파싱 (AskAnswer.svelte에서 가져옴)
type Token =
| { type: 'text'; value: string }
| { type: 'cite'; n: number; raw: string };
function splitAnswer(text: string): Token[] {
return text
.split(/(\[\d+\])/g)
.filter(Boolean)
.map((tok): Token => {
const m = tok.match(/^\[(\d+)\]$/);
return m
? { type: 'cite', n: Number(m[1]), raw: tok }
: { type: 'text', value: tok };
});
}
function confidenceTone(c: Confidence | null): 'success' | 'warning' | 'error' | 'neutral' {
if (c === 'high') return 'success';
if (c === 'medium') return 'warning';
if (c === 'low') return 'error';
return 'neutral';
}
function confidenceLabel(c: Confidence | null): string {
if (c === 'high') return '높음';
if (c === 'medium') return '중간';
if (c === 'low') return '낮음';
return '';
}
let tokens = $derived(data?.ai_answer ? splitAnswer(data.ai_answer) : []);
// 출처 중복 제거 (같은 doc_id)
let uniqueCitations = $derived.by(() => {
if (!data?.citations?.length) return [];
const seen = new Set<number>();
return data.citations.filter((c) => {
if (seen.has(c.doc_id)) return false;
seen.add(c.doc_id);
return true;
});
});
</script>
<div class="bg-surface border border-default rounded-card p-4">
<!-- 헤더 -->
<div class="flex items-center justify-between gap-2 mb-2">
<div class="flex items-center gap-1.5">
<Sparkles size={12} class="text-accent" />
<span class="text-[10px] font-semibold tracking-wider uppercase text-dim">
내 자료 기준 답변
</span>
{#if data?.confidence}
<Badge tone={confidenceTone(data.confidence)} size="sm">
신뢰도 {confidenceLabel(data.confidence)}
</Badge>
{/if}
{#if data?.completeness === 'partial'}
<Badge tone="warning" size="sm">일부 답변</Badge>
{/if}
</div>
<button
type="button"
onclick={onDismiss}
class="p-0.5 rounded text-dim hover:text-text transition-colors"
aria-label="답변 카드 접기"
>
<X size={14} />
</button>
</div>
<!-- 본문 -->
{#if loading}
<div class="space-y-2">
<Skeleton w="w-full" h="h-3" />
<Skeleton w="w-4/5" h="h-3" />
</div>
<p class="mt-3 text-[10px] text-dim flex items-center gap-1.5">
<span class="inline-block w-2.5 h-2.5 rounded-full border-2 border-dim border-t-accent animate-spin"></span>
근거 기반 답변 생성 중…
</p>
{:else if error}
<p class="text-xs text-dim">답변을 가져오지 못했습니다.</p>
{:else if data?.ai_answer}
<!-- 답변 텍스트 -->
<div class="text-sm leading-6 text-text">
{#each tokens as tok}
{#if tok.type === 'cite'}
{@const citation = data?.citations?.find((c) => c.n === tok.n)}
{#if citation}
<button
type="button"
class="inline text-accent font-semibold hover:underline px-0.5"
onclick={() => onCitationClick(citation.doc_id)}
title={citation.title || `문서 #${citation.doc_id}`}
>
{tok.raw}
</button>
{:else}
<span class="text-dim">{tok.raw}</span>
{/if}
{:else}
<span>{tok.value}</span>
{/if}
{/each}
</div>
<!-- partial: 누락 측면 -->
{#if data.completeness === 'partial' && data.missing_aspects?.length}
<p class="mt-2 text-[10px] text-dim">
다루지 못한 부분: {data.missing_aspects.join(', ')}
</p>
{/if}
<!-- 출처 목록 (must-have) -->
{#if uniqueCitations.length > 0}
<div class="mt-3 pt-2 border-t border-default">
<p class="text-[10px] font-medium text-dim mb-1.5">출처</p>
<div class="flex flex-wrap gap-1.5">
{#each uniqueCitations as citation}
<button
type="button"
onclick={() => onCitationClick(citation.doc_id)}
class="inline-flex items-center gap-1 text-[11px] px-2 py-0.5 rounded bg-surface text-text border border-default hover:border-accent hover:text-accent transition-colors"
title={citation.span_text}
>
<FileText size={10} />
<span class="max-w-[200px] truncate">
{citation.title || `문서 #${citation.doc_id}`}
</span>
</button>
{/each}
</div>
</div>
{/if}
{/if}
</div>
@@ -0,0 +1,45 @@
<script>
// 관련 문서 (유사도) — 문서 레벨 임베딩 KNN. 자기완결: docId 받아 /related 조회.
import { onMount } from 'svelte';
import { api } from '$lib/api';
let { documentId } = $props();
let items = $state([]);
let loaded = $state(false);
const KIND = { law: '법령', guide: '지침', paper: '논문', standard: '표준', incident: '사례' };
onMount(async () => {
try {
const r = await api(`/documents/${documentId}/related?limit=6`);
items = r?.related ?? [];
} catch (e) { /* silent */ }
finally { loaded = true; }
});
</script>
{#if items.length}
<div class="rel">
<div class="lab">관련 문서</div>
{#each items as it (it.id)}
<a class="ri" href={`/documents/${it.id}`}>
<span class="rt">{it.title}</span>
<span class="rm">
{#if it.material_type && KIND[it.material_type]}<span class="kind">{KIND[it.material_type]}</span>{/if}
<span class="rs">{Math.round((it.sim ?? 0) * 100)}</span>
</span>
</a>
{/each}
</div>
{/if}
<style>
.rel { background: var(--surface); border: 1px solid var(--border); border-radius: 14px; padding: 13px; }
.lab { font-size: 10.5px; font-weight: 700; color: var(--text-dim); letter-spacing: .4px; margin-bottom: 8px; }
.ri { display: flex; align-items: baseline; gap: 8px; padding: 5px 6px; border-radius: 7px; text-decoration: none; }
.ri:hover { background: var(--surface-hover, #ecf0e8); }
.rt { flex: 1; font-size: 12px; line-height: 1.4; color: var(--text); overflow: hidden; display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; }
.rm { flex-shrink: 0; display: flex; align-items: center; gap: 5px; }
.kind { font-size: 9px; font-weight: 700; color: var(--accent-hover, #3d7256); background: #e3efe2; border: 1px solid #cfe3cd; border-radius: 4px; padding: 0 4px; }
.rs { font-size: 10.5px; font-family: ui-monospace, Menlo, monospace; color: var(--faint, #9aa090); }
</style>
@@ -0,0 +1,33 @@
<script>
// 시안 B — 글로벌 네비 슬림 아이콘 레일 (분류 사이드바 접힘 상태). 앱 토큰 사용.
import { page } from '$app/stores';
import { Home, FolderTree, Newspaper, StickyNote, Hash, GraduationCap, MessageCircle, Inbox, CalendarCheck } from 'lucide-svelte';
const items = [
{ href: '/', icon: Home, label: '홈', exact: true },
{ href: '/library', icon: FolderTree, label: '문서' },
{ href: '/news', icon: Newspaper, label: '뉴스' },
{ href: '/memos', icon: StickyNote, label: '메모' },
{ href: '/clause', icon: Hash, label: '절' },
{ href: '/events', icon: CalendarCheck, label: '일정' },
{ href: '/study', icon: GraduationCap, label: '공부' },
{ href: '/chat', icon: MessageCircle, label: '이드' },
{ href: '/inbox', icon: Inbox, label: '편지함' },
];
let path = $derived($page.url.pathname);
const active = (it) => (it.exact ? path === it.href : path.startsWith(it.href));
</script>
<nav class="flex flex-col items-center gap-1 py-2 h-full overflow-y-auto bg-sidebar">
{#each items as it (it.href)}
{@const Icon = it.icon}
<a
href={it.href}
title={it.label}
class="flex flex-col items-center justify-center gap-0.5 w-12 h-[46px] rounded-lg text-dim hover:bg-surface-hover hover:text-accent transition-colors {active(it) ? 'bg-surface-active text-accent font-semibold' : ''}"
>
<Icon size={17} strokeWidth={1.75} />
<span class="text-[8.5px] leading-none tracking-tight">{it.label}</span>
</a>
{/each}
</nav>
@@ -1,228 +0,0 @@
<!--
AskAnswer.svelte — /ask 페이지 상단 패널.
Answer 본문 + clickable [n] citations + 신뢰도/상태 Badge.
status != completed 또는 refused=true → warning empty state +
no_results_reason + "검색 결과 확인하기" 역링크.
-->
<script lang="ts">
import Badge from '$lib/components/ui/Badge.svelte';
import Button from '$lib/components/ui/Button.svelte';
import EmptyState from '$lib/components/ui/EmptyState.svelte';
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import { AlertTriangle, Sparkles } from 'lucide-svelte';
import type { AskResponse, Confidence, SynthesisStatus } from '$lib/types/ask';
interface Props {
data: AskResponse | null;
loading: boolean;
onCitationClick: (n: number) => void;
}
let { data, loading, onCitationClick }: Props = $props();
type Token =
| { type: 'text'; value: string }
| { type: 'cite'; n: number; raw: string };
function splitAnswer(text: string): Token[] {
return text
.split(/(\[\d+\])/g)
.filter(Boolean)
.map((tok): Token => {
const m = tok.match(/^\[(\d+)\]$/);
return m
? { type: 'cite', n: Number(m[1]), raw: tok }
: { type: 'text', value: tok };
});
}
function confidenceTone(
c: Confidence | null,
): 'success' | 'warning' | 'error' | 'neutral' {
if (c === 'high') return 'success';
if (c === 'medium') return 'warning';
if (c === 'low') return 'error';
return 'neutral';
}
function confidenceLabel(c: Confidence | null): string {
if (c === 'high') return '높음';
if (c === 'medium') return '중간';
if (c === 'low') return '낮음';
return '없음';
}
const STATUS_LABEL: Record<SynthesisStatus, string> = {
completed: '답변 완료',
timeout: '답변 지연',
skipped: '답변 생략',
no_evidence: '근거 없음',
parse_failed: '형식 오류',
llm_error: 'AI 오류',
backend_unavailable: 'Backend 비가용',
};
/**
* backend chip label — `backend_requested` 가 명시 opt-in 인 경우만 표시.
* 미지정 (null/undefined) default 호출은 chip 없음 (시각 noise 회피).
*/
function backendChipLabel(backend: string | null | undefined): string | null {
if (!backend) return null;
if (backend === 'qwen-macbook') return 'Qwen 27B (MacBook)';
if (backend === 'gemma-macmini') return 'Gemma 26B (Mac mini)';
return backend;
}
let tokens = $derived(data?.ai_answer ? splitAnswer(data.ai_answer) : []);
let showFullAnswer = $derived(
!!data && !!data.ai_answer && data.completeness === 'full'
&& data.synthesis_status === 'completed' && !data.refused,
);
let showPartial = $derived(
!!data && data.completeness === 'partial' && !data.refused,
);
let showWarning = $derived(!!data && !showFullAnswer && !showPartial);
</script>
<section class="bg-surface border border-default rounded-card p-5">
<!-- 헤더 -->
<div class="flex items-start justify-between gap-3 mb-4">
<div>
<p class="text-[10px] font-semibold tracking-wider uppercase text-dim flex items-center gap-1.5">
<Sparkles size={12} /> AI Answer
</p>
<h2 class="mt-1 text-base font-semibold text-text">근거 기반 답변</h2>
</div>
{#if data && !loading}
<div class="flex flex-wrap gap-1.5">
<Badge tone={confidenceTone(data.confidence)} size="sm">
신뢰도 {confidenceLabel(data.confidence)}
</Badge>
{#if backendChipLabel(data.backend_requested)}
<span title={`backend_requested=${data.backend_requested} / backend_used=${data.backend_used ?? 'null'}`}>
<Badge tone="neutral" size="sm">
{backendChipLabel(data.backend_requested)}
</Badge>
</span>
{/if}
<Badge tone="neutral" size="sm">
{STATUS_LABEL[data.synthesis_status]}
</Badge>
{#if data.synthesis_ms > 0}
<Badge tone="neutral" size="sm">
{Math.round(data.synthesis_ms)}ms
</Badge>
{/if}
</div>
{/if}
</div>
<!-- 본문 -->
{#if loading}
<div class="space-y-3">
<Skeleton w="w-3/4" h="h-4" />
<Skeleton w="w-full" h="h-4" />
<Skeleton w="w-5/6" h="h-4" />
<p class="mt-4 text-xs text-dim flex items-center gap-2">
<span class="inline-block w-3 h-3 rounded-full border-2 border-dim border-t-accent animate-spin"></span>
근거 기반 답변 생성 중… 약 15초 소요
</p>
</div>
{:else if showFullAnswer && data}
<div class="text-sm leading-7 text-text">
{#each tokens as tok}
{#if tok.type === 'cite'}
<button
type="button"
class="inline-block align-baseline text-accent font-semibold hover:underline focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-accent-ring rounded px-0.5"
onclick={() => onCitationClick(tok.n)}
aria-label={`인용 ${tok.n}번 보기`}
>
{tok.raw}
</button>
{:else}
<span>{tok.value}</span>
{/if}
{/each}
</div>
{:else if showPartial && data}
<!-- Phase 3.5a: question-aligned partial structure -->
<div>
<Badge tone="warning" size="sm">일부 답변</Badge>
{#if data.ai_answer}
<div class="mt-3 text-sm leading-7 text-text">
{#each tokens as tok}
{#if tok.type === 'cite'}
<button
type="button"
class="inline-block align-baseline text-accent font-semibold hover:underline rounded px-0.5"
onclick={() => onCitationClick(tok.n)}
>{tok.raw}</button>
{:else}
<span>{tok.value}</span>
{/if}
{/each}
</div>
{:else if data.confirmed_items?.length}
<div class="mt-3">
<h4 class="text-xs font-semibold text-dim uppercase tracking-wider">✓ 답변 가능</h4>
<ul class="mt-2 space-y-2">
{#each data.confirmed_items as item}
<li class="text-sm text-text">
<strong class="text-accent">{item.aspect}:</strong>
<span>{item.text}</span>
{#each item.citations as n}
<button
type="button"
class="text-accent font-semibold hover:underline px-0.5"
onclick={() => onCitationClick(n)}
>[{n}]</button>
{/each}
</li>
{/each}
</ul>
</div>
{/if}
{#if data.missing_aspects?.length}
<div class="mt-4 border-t border-default pt-3">
<h4 class="text-xs font-semibold text-dim uppercase tracking-wider">✗ 답변 불가</h4>
<ul class="mt-2 space-y-1">
{#each data.missing_aspects as aspect}
<li class="text-sm text-dim">{aspect} <span class="text-[10px]">(근거 없음)</span></li>
{/each}
</ul>
</div>
{/if}
<div class="mt-4">
<Button
variant="secondary"
size="sm"
href={`/documents?q=${encodeURIComponent(data.query)}`}
>
검색 결과 확인하기
</Button>
</div>
</div>
{:else if showWarning && data}
<EmptyState
icon={AlertTriangle}
title={data.refused && data.no_results_reason
? data.no_results_reason
: (data.no_results_reason ?? '관련 근거를 찾지 못했습니다.')}
description="검색 결과를 직접 확인해 보세요."
>
<Button
variant="secondary"
size="sm"
href={`/documents?q=${encodeURIComponent(data.query)}`}
>
검색 결과 확인하기
</Button>
</EmptyState>
{/if}
</section>
@@ -1,91 +0,0 @@
<!--
AskEvidence.svelte — /ask 페이지 우측 sticky 패널.
⚠ 영구 룰 (Phase 3.4 plan):
`citation.full_snippet` 은 UI 에 직접 렌더 금지. debug 모드(`?debug=1`)
에서 hover tooltip 으로만 조건부 노출 가능.
이 규칙이 깨지면 backend span-precision UX 가치가 사라진다. 코드 리뷰에서
반드시 reject. span_text 만 본문으로 노출한다.
-->
<script lang="ts">
import Badge from '$lib/components/ui/Badge.svelte';
import EmptyState from '$lib/components/ui/EmptyState.svelte';
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import { BookOpen } from 'lucide-svelte';
import type { AskResponse } from '$lib/types/ask';
interface Props {
data: AskResponse | null;
loading: boolean;
activeCitation: number | null;
registerCitation: (n: number, node: HTMLElement) => { destroy: () => void };
}
let { data, loading, activeCitation, registerCitation }: Props = $props();
let citations = $derived(data?.citations ?? []);
</script>
<section class="bg-surface border border-default rounded-card p-5">
<div class="flex items-start justify-between gap-3 mb-4">
<div>
<p class="text-[10px] font-semibold tracking-wider uppercase text-dim flex items-center gap-1.5">
<BookOpen size={12} /> Evidence Highlights
</p>
<h3 class="mt-1 text-sm font-semibold text-text">인용 근거</h3>
</div>
{#if data && !loading}
<Badge tone="neutral" size="sm">{citations.length}</Badge>
{/if}
</div>
{#if loading}
<div class="space-y-3">
{#each Array(2) as _}
<div class="border border-default rounded-card p-4 space-y-2">
<Skeleton w="w-24" h="h-3" />
<Skeleton w="w-full" h="h-3" />
<Skeleton w="w-5/6" h="h-3" />
<Skeleton w="w-3/4" h="h-3" />
</div>
{/each}
</div>
{:else if citations.length === 0}
<EmptyState title="표시할 근거가 없습니다." class="py-6" />
{:else}
<div class="space-y-3">
{#each citations as citation (citation.n)}
{@const isActive = activeCitation === citation.n}
<article
class="border rounded-card p-4 transition-colors {isActive
? 'border-accent ring-2 ring-accent/20 bg-accent/5'
: 'border-default'}"
use:registerCitation={citation.n}
>
<div class="flex items-start gap-2">
<span class="text-accent font-bold text-sm shrink-0">[{citation.n}]</span>
<div class="flex-1 min-w-0">
<strong class="block text-sm text-text truncate">
{citation.title ?? `문서 ${citation.doc_id}`}
</strong>
{#if citation.section_title}
<p class="mt-0.5 text-xs text-dim truncate">{citation.section_title}</p>
{/if}
</div>
</div>
<!-- ⚠ span_text 만 렌더. full_snippet 금지 -->
<p class="mt-3 text-sm leading-relaxed text-text whitespace-pre-wrap">
{citation.span_text}
</p>
<div class="mt-3 flex gap-2 text-[10px] text-dim">
<span>relevance {citation.relevance.toFixed(2)}</span>
<span>rerank {citation.rerank_score.toFixed(2)}</span>
</div>
</article>
{/each}
</div>
{/if}
</section>
@@ -1,78 +0,0 @@
<!--
AskResults.svelte — /ask 페이지 하단 패널.
검색 결과 리스트. DocumentCard 재사용 X — SearchResult 필드 셋이 달라서
의존성 리스크 회피. inline 간단 카드로 title/score/snippet/section_title 표시.
클릭 시 `/documents/{id}` 로 이동.
-->
<script lang="ts">
import Badge from '$lib/components/ui/Badge.svelte';
import EmptyState from '$lib/components/ui/EmptyState.svelte';
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import { FileText } from 'lucide-svelte';
import type { AskResponse } from '$lib/types/ask';
interface Props {
data: AskResponse | null;
loading: boolean;
}
let { data, loading }: Props = $props();
let results = $derived(data?.results ?? []);
</script>
<section class="bg-surface border border-default rounded-card p-5">
<div class="flex items-start justify-between gap-3 mb-4">
<div>
<p class="text-[10px] font-semibold tracking-wider uppercase text-dim flex items-center gap-1.5">
<FileText size={12} /> Search Results
</p>
<h3 class="mt-1 text-sm font-semibold text-text">검색 결과</h3>
</div>
{#if data && !loading}
<Badge tone="neutral" size="sm">{data.total}</Badge>
{/if}
</div>
{#if loading}
<div class="space-y-3">
{#each Array(5) as _}
<div class="border border-default rounded-card p-4 space-y-2">
<Skeleton w="w-2/3" h="h-4" />
<Skeleton w="w-full" h="h-3" />
<Skeleton w="w-4/5" h="h-3" />
</div>
{/each}
</div>
{:else if results.length === 0}
<EmptyState title="검색 결과가 없습니다." class="py-6" />
{:else}
<div class="space-y-3">
{#each results as result (result.id)}
<a
href={`/documents/${result.id}`}
class="block border border-default rounded-card p-4 hover:border-accent hover:bg-surface-hover transition-colors"
>
<div class="flex items-start justify-between gap-3">
<strong class="text-sm text-text flex-1 min-w-0 truncate">
{result.title ?? `문서 ${result.id}`}
</strong>
<div class="flex gap-1.5 text-[10px] text-dim shrink-0">
<span>score {result.score.toFixed(2)}</span>
{#if result.rerank_score != null}
<span>rerank {result.rerank_score.toFixed(2)}</span>
{/if}
</div>
</div>
{#if result.section_title}
<p class="mt-1 text-xs text-dim truncate">{result.section_title}</p>
{/if}
{#if result.snippet}
<p class="mt-2 text-xs text-dim line-clamp-2">{result.snippet}</p>
{/if}
</a>
{/each}
</div>
{/if}
</section>
-84
View File
@@ -1,84 +0,0 @@
/**
* Phase 3.4: `/api/search/ask` .
*
* Backend Pydantic (`app/api/search.py::AskResponse`) 1:1 .
* .
*/
export type SynthesisStatus =
| 'completed'
| 'timeout'
| 'skipped'
| 'no_evidence'
| 'parse_failed'
| 'llm_error'
| 'backend_unavailable';
export type Confidence = 'high' | 'medium' | 'low';
export interface Citation {
n: number;
chunk_id: number | null;
doc_id: number;
title: string | null;
section_title: string | null;
/** LLM이 추출한 50~300자 핵심 span. UI에서 이것만 노출. */
span_text: string;
/**
* 800 window.
*
* UI . debug hover tooltip
* . full_snippet을 backend span-precision UX
* (plan §Evidence ).
*/
full_snippet: string;
relevance: number;
rerank_score: number;
}
export interface SearchResult {
id: number;
title: string | null;
ai_domain: string | null;
ai_summary: string | null;
file_format: string;
score: number;
snippet: string | null;
match_reason: string | null;
chunk_id: number | null;
chunk_index: number | null;
section_title: string | null;
rerank_score: number | null;
}
export type Completeness = 'full' | 'partial' | 'insufficient';
export interface ConfirmedItem {
aspect: string;
text: string;
citations: number[];
}
export interface AskResponse {
results: SearchResult[];
ai_answer: string | null;
citations: Citation[];
synthesis_status: SynthesisStatus;
synthesis_ms: number;
confidence: Confidence | null;
refused: boolean;
no_results_reason: string | null;
query: string;
total: number;
/** Phase 3.5a */
completeness: Completeness;
covered_aspects: string[] | null;
missing_aspects: string[] | null;
confirmed_items: ConfirmedItem[] | null;
/**
* PR-MacBook-RAG-Backend-1: backend dispatcher metadata.
* backend null ( ). opt-in .
*/
backend_requested?: string | null;
backend_used?: string | null;
}
-61
View File
@@ -1,61 +0,0 @@
/**
* "질문형" ( ).
* true이면 /api/search/ask .
*
* false positive: /ask refused=true .
* false negative: 기존 .
*/
export function isQuestion(q: string): boolean {
const trimmed = q.trim();
if (trimmed.length === 0) return false;
// 1. ?로 끝나면 단일 단어라도 허용 (왜?, 절차?)
if (trimmed.endsWith('?')) return true;
// ? 없으면 단일 단어 / 4자 미만 제외 (키워드 보호)
if (trimmed.length < 4) return false;
if (trimmed.split(/\s+/).length < 2) return false;
// 2. 한국어 질문 어미
const KO_ENDINGS = [
'인가요', '인가', '인지', '있나요', '있나',
'할까요', '할까', '될까요', '될까',
'뭐야', '뭔가', '뭘까', '뭔지', '뭐지', '뭐죠',
'는지', '은지', '일까',
'어때', '어떤가',
'됩니까', '합니까', '입니까', '나요', '까요',
'주세요', '알려줘', '설명해', '비교해',
];
for (const ending of KO_ENDINGS) {
if (trimmed.endsWith(ending)) return true;
}
// 3. 한국어 질문 시작어
const KO_STARTS = [
'어떻게', '왜', '무엇', '무슨', '뭐가', '누가',
'어디', '언제', '몇', '얼마나', '어떤', '어느',
];
for (const start of KO_STARTS) {
if (trimmed.startsWith(start)) return true;
}
// 4. 영어 질문 시작어 (대소문자 무시)
const EN_STARTS = [
'what', 'how', 'why', 'when', 'where', 'who', 'which',
'is', 'are', 'do', 'does', 'can', 'could', 'should', 'would',
'explain', 'describe', 'compare', 'tell me',
];
const lower = trimmed.toLowerCase();
for (const start of EN_STARTS) {
if (lower.startsWith(start + ' ')) return true;
}
// 5. 의미 패턴 (끝 단어)
const SEMANTIC_ENDINGS = ['차이', '비교', '설명', '요약', '정리', '방법', '절차'];
const lastWord = trimmed.split(/\s+/).pop() || '';
for (const pat of SEMANTIC_ENDINGS) {
if (lastWord === pat || lastWord.endsWith(pat)) return true;
}
return false;
}
+4 -3
View File
@@ -11,6 +11,7 @@
import { queueOverview } from '$lib/stores/queueOverview';
import { MACHINE_STATE_LABEL, machineChipClass } from '$lib/utils/queueDisplay';
import Sidebar from '$lib/components/Sidebar.svelte';
import SlimRail from '$lib/components/SlimRail.svelte';
import SystemStatusDot from '$lib/components/SystemStatusDot.svelte';
import QueueDrawer from '$lib/components/QueueDrawer.svelte';
import QuickMemoButton from '$lib/components/QuickMemoButton.svelte';
@@ -21,7 +22,7 @@
const PUBLIC_PATHS = ['/login', '/setup', '/__styleguide'];
const NO_CHROME_PATHS = ['/login', '/setup', '/__styleguide'];
// /news = 풀스크린 브리핑 → 데스크탑 상시 사이드바 없음
const NO_SIDEBAR_PATHS = ['/news'];
const NO_SIDEBAR_PATHS = ['/news', '/book']; // /book = 책 몰입(글로벌 분류 트리 숨김, 상단 네비 유지)
// toast 의미 토큰 매핑 (A-8 B3)
const TOAST_CLASS = {
@@ -198,8 +199,8 @@
<!-- 메인: 데스크탑 상시 사이드바 + 콘텐츠 -->
<div class="flex-1 min-h-0 flex">
{#if showSidebar}
<aside class="hidden lg:block shrink-0 overflow-hidden transition-[width] duration-200 ease-out {sidebarCollapsed ? 'w-0 border-r-0' : 'w-sidebar border-r border-default'}">
<Sidebar />
<aside class="hidden lg:block shrink-0 overflow-hidden transition-[width] duration-200 ease-out {sidebarCollapsed ? 'w-14 border-r border-default' : 'w-sidebar border-r border-default'}">
{#if sidebarCollapsed}<SlimRail />{:else}<Sidebar />{/if}
</aside>
{/if}
<main class="flex-1 min-w-0 overflow-auto">
-305
View File
@@ -1,305 +0,0 @@
<!--
/ask — Phase 3.4 Ask Pipeline Frontend.
URL-driven: `/ask?q=<encoded>[&backend=<alias>]` 가 진입점.
$effect 로 (q, backend) 변화 감지 → `/api/search/ask` 호출 →
3-panel 렌더 (Answer / Evidence / Results).
중복 호출 방지: lastKey (q+backend) 가드.
Backend selector (PR-3 of DS AI routing policy, 2026-05-23,
[[document-server-ai-routing-policy]]) — PR-DocSrv-Web-Ask-Selector-1 확장:
- `auto` (기본, URL param 없음 → router 의 rule + LLM triage)
- `mac-mini-default` (명시, Mac mini gemma-4-26b)
- `qwen-macbook` (명시, M5 Max Qwen3.6-27B; "This device" 토글 on 시만)
- `claude-cloud` (명시, 503 scaffold; VITE_ENABLE_CLOUD_BACKEND_DEV=true 시만)
- "This device" 토글: localStorage `ds_device_self_label = 'macbook-m5-max' | null`.
source IP 의존 0 (PR-0 round 2 발견: caddy 2-hop + X-Forwarded-For 미설정 →
DS 가 보는 source IP = LAN gateway, 신뢰 불가).
- 503 + error_reason ∈ {macbook_unavailable, provider_not_configured, router_*}
시 자동 fallback 금지. UI 가 친절한 메시지 + "Mac mini default 로 재요청" 버튼.
- legacy URL `?backend=default|gemma-macmini` 는 그대로 받아서 mac-mini-default 와 동등 처리.
-->
<script lang="ts">
import { page } from '$app/stores';
import { goto } from '$app/navigation';
import { api, type ApiError } from '$lib/api';
import { addToast } from '$lib/stores/toast';
import EmptyState from '$lib/components/ui/EmptyState.svelte';
import Button from '$lib/components/ui/Button.svelte';
import AskAnswer from '$lib/components/ask/AskAnswer.svelte';
import AskEvidence from '$lib/components/ask/AskEvidence.svelte';
import AskResults from '$lib/components/ask/AskResults.svelte';
import { Sparkles, Search, AlertCircle } from 'lucide-svelte';
import type { AskResponse } from '$lib/types/ask';
type BackendChoice = 'auto' | 'mac-mini-default' | 'qwen-macbook' | 'claude-cloud';
function parseBackend(raw: string | null): BackendChoice {
if (raw === 'qwen-macbook') return 'qwen-macbook';
if (raw === 'mac-mini-default') return 'mac-mini-default';
if (raw === 'claude-cloud') return 'claude-cloud';
// legacy aliases (?backend=default | gemma-macmini) → mac-mini-default 와 동등
if (raw === 'default' || raw === 'gemma-macmini') return 'mac-mini-default';
return 'auto';
}
// Build-time feature flag — Claude Cloud UI 노출 여부 (default false).
// VITE_ENABLE_CLOUD_BACKEND_DEV=true npm run build 시만 활성. 운영 토글 X
// (build-time 한계). DS runtime feature flag API migrate 는 후속 PR.
const CLOUD_DEV_ENABLED = import.meta.env.VITE_ENABLE_CLOUD_BACKEND_DEV === 'true';
const DEVICE_LABEL_KEY = 'ds_device_self_label';
const DEVICE_LABEL_M5_MAX = 'macbook-m5-max';
// ── state ───────────────────────────────────────────
let queryInput = $state('');
let selectedBackend = $state<BackendChoice>('auto');
let data = $state<AskResponse | null>(null);
let loading = $state(false);
let backendUnavailable = $state(false);
let backendUnavailableMessage = $state('');
// "I am on MacBook M5 Max" 토글. mount 시 localStorage 에서 복원.
let isMacBookM5Max = $state(false);
$effect(() => {
if (typeof window === 'undefined') return;
const stored = window.localStorage.getItem(DEVICE_LABEL_KEY);
isMacBookM5Max = stored === DEVICE_LABEL_M5_MAX;
});
function toggleMacBookM5Max() {
isMacBookM5Max = !isMacBookM5Max;
if (typeof window === 'undefined') return;
if (isMacBookM5Max) {
window.localStorage.setItem(DEVICE_LABEL_KEY, DEVICE_LABEL_M5_MAX);
} else {
window.localStorage.removeItem(DEVICE_LABEL_KEY);
// 토글 off 시 qwen-macbook 선택돼 있었으면 auto 로 복귀 (선택권 박탈 X 명시 신호).
if (selectedBackend === 'qwen-macbook') {
selectedBackend = 'auto';
}
}
}
// 중복 호출 방지 가드 (hydration + reactive trigger 이중 발동 방지)
let lastKey = '';
// citation scroll 연동: Answer 가 [n] 클릭 → Evidence 카드로 이동 + highlight
const citationNodes = new Map<number, HTMLElement>();
let activeCitation = $state<number | null>(null);
function registerCitation(n: number, node: HTMLElement) {
citationNodes.set(n, node);
return {
destroy() {
citationNodes.delete(n);
},
};
}
function scrollToCitation(n: number) {
activeCitation = n;
const node = citationNodes.get(n);
node?.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
// ── URL 빌더: backend !== 'auto' 일 때만 param 추가 ─────
function buildAskUrl(q: string, backend: BackendChoice): string {
const params = new URLSearchParams({ q });
if (backend !== 'auto') params.set('backend', backend);
return `/ask?${params.toString()}`;
}
// ── submit (URL-driven, back 자동) ──────────────────
function submit() {
const q = queryInput.trim();
if (!q) return;
goto(buildAskUrl(q, selectedBackend));
}
function handleKeydown(e: KeyboardEvent) {
if (e.key === 'Enter' && !e.isComposing) {
e.preventDefault();
submit();
}
}
// 503 후 "Mac mini default 로 재요청" — auto 로 reset (param 명시 제거).
function retryWithMacMiniDefault() {
const q = $page.url.searchParams.get('q') ?? queryInput.trim();
if (!q) return;
goto(`/ask?q=${encodeURIComponent(q)}`);
}
// PR-3 of routing policy — error_reason → 친절 메시지 매핑.
// silent fallback 금지 ([[feedback_no_silent_fallback_explicit_opt_in]]):
// 모든 503 case 는 명시 표시, 다른 backend 자동 재호출 X.
function friendlyErrorMessage(reason: string | undefined, detail: string): string {
switch (reason) {
case 'macbook_unavailable':
return 'MacBook M5 Max 가 응답하지 않습니다. 깨우거나 (launchctl start) Mac mini default 로 다시 요청하세요.';
case 'provider_not_configured':
return 'Claude Cloud 백엔드는 아직 구성되지 않았습니다 (2026-06-15 이후 별 PR 활성화 예정).';
default:
if (reason && reason.startsWith('router_')) {
return `라우터 호출 실패 (${reason}). Mac mini default 로 다시 요청하거나 잠시 후 재시도하세요.`;
}
if (reason && reason.startsWith('upstream_')) {
return `Upstream backend 가 응답하지 않습니다 (${reason}). 잠시 후 재시도하세요.`;
}
return detail || '답변 생성 실패';
}
}
// ── API 호출 ───────────────────────────────────────
async function runAsk(q: string, backend: BackendChoice) {
loading = true;
activeCitation = null;
backendUnavailable = false;
backendUnavailableMessage = '';
const path = backend !== 'auto'
? `/search/ask?q=${encodeURIComponent(q)}&backend=${encodeURIComponent(backend)}&limit=10`
: `/search/ask?q=${encodeURIComponent(q)}&limit=10`;
try {
data = await api<AskResponse>(path);
} catch (err) {
const apiErr = err as ApiError;
if (apiErr.status === 503) {
backendUnavailable = true;
backendUnavailableMessage = friendlyErrorMessage(apiErr.errorReason, apiErr.detail);
data = null;
} else {
addToast('error', apiErr.detail || '답변 생성 실패');
data = null;
}
} finally {
loading = false;
}
}
// ── URL → runAsk (중복 가드) ────────────────────────
$effect(() => {
const q = $page.url.searchParams.get('q') ?? '';
const backend = parseBackend($page.url.searchParams.get('backend'));
queryInput = q;
selectedBackend = backend;
if (!q) {
data = null;
loading = false;
backendUnavailable = false;
lastKey = '';
return;
}
const key = `${q}|${backend}`;
if (key === lastKey) return;
lastKey = key;
runAsk(q, backend);
});
</script>
<svelte:head>
<title>질문 - PKM</title>
</svelte:head>
<div class="h-full overflow-auto">
<!-- 상단 검색바 (sticky) -->
<div class="sticky top-0 z-10 bg-bg/80 backdrop-blur border-b border-default px-4 py-3">
<div class="flex flex-wrap items-center gap-2 max-w-[1680px] mx-auto">
<div class="relative flex-1 min-w-0">
<Search
size={14}
class="absolute left-3 top-1/2 -translate-y-1/2 text-dim pointer-events-none"
/>
<input
data-search-input
type="text"
bind:value={queryInput}
onkeydown={handleKeydown}
placeholder="질문을 입력하세요 (/ 키로 포커스)"
class="w-full pl-9 pr-3 py-2 bg-surface border border-default rounded-lg text-text text-sm focus:border-accent outline-none"
/>
</div>
<label
class="flex items-center gap-1.5 text-xs text-dim cursor-pointer select-none"
title="이 디바이스가 MacBook M5 Max 인 경우 체크 — This device (qwen-macbook) 옵션 활성화됩니다. localStorage 저장."
>
<input
type="checkbox"
checked={isMacBookM5Max}
onchange={toggleMacBookM5Max}
class="accent-accent"
/>
<span>This is M5 Max</span>
</label>
<select
bind:value={selectedBackend}
title="Backend 선택 — silent fallback 0 정책 (선택한 backend 만 시도, 실패 시 503)."
class="py-2 px-2 bg-surface border border-default rounded-lg text-text text-xs focus:border-accent outline-none min-w-0 max-w-[42vw] truncate"
>
<option value="auto">Auto (router)</option>
<option value="mac-mini-default">Mac mini (default)</option>
<option
value="qwen-macbook"
disabled={!isMacBookM5Max}
title={isMacBookM5Max
? 'MacBook M5 Max Qwen3.6-27B (직접 호출)'
: 'Check "This is M5 Max" toggle to enable'}
>
{isMacBookM5Max ? 'This device (Qwen MacBook)' : 'This device (unavailable)'}
</option>
<option
value="claude-cloud"
disabled={!CLOUD_DEV_ENABLED}
title={CLOUD_DEV_ENABLED
? 'Claude Cloud (dev mode — returns 503 until activation PR)'
: 'Cloud backend not configured yet'}
>
Claude Cloud {CLOUD_DEV_ENABLED ? '(dev)' : '(unavailable)'}
</option>
</select>
</div>
</div>
<!-- 본문 -->
<div class="max-w-[1680px] mx-auto p-4">
{#if backendUnavailable}
<div class="py-16">
<EmptyState
icon={AlertCircle}
title="Backend 가 응답하지 않습니다"
description={backendUnavailableMessage}
>
<Button variant="primary" size="sm" onclick={retryWithMacMiniDefault}>
Mac mini (default) 로 재요청
</Button>
</EmptyState>
</div>
{:else if !queryInput && !loading && !data}
<div class="py-16">
<EmptyState
icon={Sparkles}
title="근거 기반 답변을 받아보세요"
description="질문을 입력하면 문서에서 근거를 찾아 인용 기반 답변을 생성합니다."
/>
</div>
{:else}
<div class="grid gap-4 lg:grid-cols-[1.2fr_0.9fr] items-start">
<!-- 좌: Answer + Results -->
<div class="flex flex-col gap-4 min-w-0">
<AskAnswer {data} {loading} onCitationClick={scrollToCitation} />
<AskResults {data} {loading} />
</div>
<!-- 우: Evidence (lg 이상 sticky) -->
<div class="lg:sticky lg:top-20 lg:self-start min-w-0">
<AskEvidence
{data}
{loading}
{activeCitation}
{registerCitation}
/>
</div>
</div>
{/if}
</div>
</div>
+281
View File
@@ -0,0 +1,281 @@
<script>
// ASME/법령 절-KB — 코드북·공부 리더 (r2). parent 표준/법령을 한 권의 책처럼.
// 좌 인덱스(Part/章→절/조) · 중 본문(MarkdownDoc=공식·표·이미지) · breadcrumb·이전다음·양방향 백링크.
import { onMount, tick } from 'svelte';
import { page } from '$app/stores';
import { goto } from '$app/navigation';
import { api } from '$lib/api';
import MarkdownDoc from '$lib/components/MarkdownDoc.svelte';
let parentId = $state(null);
let parentTitle = $state('');
let clauses = $state([]);
let selectedId = $state(null);
let clauseDoc = $state(null);
let links = $state(null);
let expanded = $state({});
let loading = $state(false);
let q = $state('');
// 공부도구 (노트/형광펜/암기카드) — clause_study
let studyItems = $state([]);
let studyOpen = $state(false);
let noteDraft = $state('');
const KLABEL = { note: '노트', highlight: '형광펜', card: '암기카드' };
async function loadStudy(id) {
try { const r = await api(`/documents/${id}/study`); studyItems = r?.items ?? []; }
catch { studyItems = []; }
}
async function addStudy(kind, payload) {
if (!selectedId) return;
try { await api(`/documents/${selectedId}/study`, { method: 'POST', body: JSON.stringify({ kind, payload }) }); await loadStudy(selectedId); }
catch (e) { console.warn(e); }
}
function selText() { return (typeof window !== 'undefined' && window.getSelection ? window.getSelection().toString() : '').trim(); }
function addNote() { const t = noteDraft.trim(); if (!t) return; addStudy('note', { text: t }); noteDraft = ''; }
function addHighlight() { const s = selText(); if (!s) { studyOpen = true; alert('본문에서 형광펜 칠할 부분을 먼저 드래그하세요'); return; } addStudy('highlight', { text: s }); studyOpen = true; }
function addCard() {
const s = selText();
const code = links?.clause_code ?? selMeta?.clause_code ?? '';
addStudy('card', { cue: `${code} ${strip(clauseDoc?.title, code)}`.trim(), fact: s || (clauseDoc?.md_content ?? clauseDoc?.extracted_text ?? '').replace(/[#*>]/g, '').slice(0, 280).trim() });
studyOpen = true;
}
async function delStudy(id) {
try { await api(`/documents/${selectedId}/study/${id}`, { method: 'DELETE' }); await loadStudy(selectedId); } catch {}
}
let parts = $derived.by(() => {
const out = [], idx = {};
for (const c of clauses) {
const p = c.clause_part || '·';
if (!(p in idx)) { idx[p] = out.length; out.push({ part: p, items: [] }); }
out[idx[p]].items.push(c);
}
return out;
});
let visibleParts = $derived.by(() => {
const term = q.trim().toLowerCase();
if (!term) return parts;
return parts.map(g => ({ part: g.part, items: g.items.filter(c =>
(c.clause_code || '').toLowerCase().includes(term) || (c.title || '').toLowerCase().includes(term)) }))
.filter(g => g.items.length);
});
let selMeta = $derived(clauses.find((c) => c.id === selectedId) || null);
const strip = (t, c) => (t || '').replace(c || '', '').replace(/^[(\s)]+|[(\s)]+$/g, '').trim();
async function loadBook() {
const r = await api(`/documents/${parentId}/clauses`);
parentTitle = r?.parent_title ?? '';
clauses = r?.clauses ?? [];
const e = {};
for (const c of clauses) e[c.clause_part || '·'] = true;
expanded = e;
}
async function loadClause(id) {
if (!id) return;
loading = true;
selectedId = id;
try {
const [d, l] = await Promise.all([api(`/documents/${id}`), api(`/documents/${id}/backlinks`)]);
clauseDoc = d; links = l;
loadStudy(id);
const sel = clauses.find((c) => c.id === id);
if (sel) expanded = { ...expanded, [sel.clause_part || '·']: true };
goto(`/book/${parentId}?c=${id}`, { replaceState: true, keepFocus: true, noScroll: true });
await tick(); window.scrollTo({ top: 0 });
} finally { loading = false; }
}
onMount(async () => {
parentId = Number($page.params.id);
await loadBook();
const c = Number($page.url.searchParams.get('c'));
await loadClause(c && clauses.find((x) => x.id === c) ? c : clauses[0]?.id);
});
</script>
<div class="book">
<!-- top bar -->
<div class="bar">
<span class="brand">절-KB</span>
<span class="crumbs">{parentTitle} {#if selMeta}<b class="sep"></b> {selMeta.clause_part} <b class="sep"></b> <b>{links?.clause_code ?? selMeta.clause_code}</b>{/if}</span>
<div class="search"><input placeholder="절·조 번호 또는 키워드" bind:value={q} /></div>
<div class="tools"><span class="tool on">읽기</span><span class="tool">형광펜</span><span class="tool">노트</span><span class="tool">암기카드</span></div>
</div>
<div class="main">
<!-- left index -->
<aside class="idx">
<a class="btitle" href={`/documents/${parentId}`}>{parentTitle || '표준'}</a>
<div class="bmeta">{clauses.length} · 한 권의 책처럼 탐색</div>
{#each visibleParts as g (g.part)}
<div class="parttab" role="button" tabindex="0" onclick={() => (expanded = { ...expanded, [g.part]: !expanded[g.part] })}>
<span class="bar2"></span><span class="pname">{g.part}</span><span class="ct">{g.items.length}</span>
</div>
{#if expanded[g.part] || q.trim()}
{#each g.items as c (c.id)}
<div class="ci" class:on={c.id === selectedId} role="button" tabindex="0" onclick={() => loadClause(c.id)}>
<span class="no">{c.clause_code}</span><span class="tt">{strip(c.title, c.clause_code)}</span>
</div>
{/each}
{/if}
{/each}
</aside>
<!-- reader -->
<section class="read">
<div class="col">
{#if clauseDoc}
<div class="studybar">
<button class="sbtn" title="선택 형광펜" onclick={addHighlight}>▰</button>
<button class="sbtn" class:on={studyOpen} title="노트/공부" onclick={() => (studyOpen = !studyOpen)}>✎</button>
<button class="sbtn" title="암기카드 추가" onclick={addCard}></button>
{#if studyItems.length}<span class="scount">{studyItems.length}</span>{/if}
</div>
<div class="kicker"><span class="pth">{selMeta?.clause_part}</span></div>
<div class="h-no">{links?.clause_code ?? selMeta?.clause_code}</div>
<h1 class="h-title">{strip(clauseDoc.title, links?.clause_code ?? '')}</h1>
<div class="flow">
<button class="fl" disabled={!links?.prev} onclick={() => loadClause(links?.prev?.id)}>{links?.prev?.clause_code ?? ''}</button>
<button class="fl next" disabled={!links?.next} onclick={() => loadClause(links?.next?.id)}>{links?.next?.clause_code ?? ''}</button>
</div>
{#key clauseDoc.id}
<div class="docbody">
<MarkdownDoc documentId={clauseDoc.id} mdContent={clauseDoc.md_content ?? clauseDoc.extracted_text} mdStatus={null} class="prose prose-base max-w-none" />
</div>
{/key}
{#if links && (links.forward.length || links.back.length)}
<section class="conn">
{#if links.forward.length}
<div><h4>이 절이 참조 <span>{links.forward.length}</span></h4>
<div class="chiprow">{#each links.forward as f}
{#if f.doc_id}<button class="ref" onclick={() => loadClause(f.doc_id)}>{f.code}</button>
{:else}<span class="ref dg" title="외부/미분해">{f.code}</span>{/if}
{/each}</div></div>
{/if}
{#if links.back.length}
<div><h4>이 절을 참조 <span>{links.back.length}</span></h4>
<div class="chiprow">{#each links.back as b}<button class="ref" onclick={() => loadClause(b.doc_id)}>{b.code}</button>{/each}</div></div>
{/if}
</section>
{/if}
{#if studyOpen}
<section class="study">
<div class="slab">공부 — 노트 · 형광펜 · 암기카드{#if studyItems.length} <span>{studyItems.length}</span>{/if}</div>
<div class="noteadd">
<textarea bind:value={noteDraft} placeholder="이 절에 노트…" rows="2"></textarea>
<button class="nbtn" onclick={addNote}>노트 저장</button>
</div>
{#if studyItems.length}
<ul class="slist">
{#each studyItems as it (it.id)}
<li class="sitem">
<span class="skind k-{it.kind}">{KLABEL[it.kind] ?? it.kind}</span>
<span class="stext">{it.payload?.text ?? it.payload?.cue ?? ''}</span>
<button class="sdel" title="삭제" onclick={() => delStudy(it.id)}>×</button>
</li>
{/each}
</ul>
{:else}
<p class="shint">본문을 드래그한 뒤 형광펜(▰)/암기카드(+), 또는 위에 노트를 적으세요.</p>
{/if}
</section>
{/if}
<div class="pager">
<button class="pg" disabled={!links?.prev} onclick={() => loadClause(links?.prev?.id)}>
<div class="d">← 이전</div><div class="t"><span class="pno">{links?.prev?.clause_code ?? '—'}</span> {strip(links?.prev?.title, links?.prev?.clause_code)}</div></button>
<button class="pg next" disabled={!links?.next} onclick={() => loadClause(links?.next?.id)}>
<div class="d">다음 →</div><div class="t"><span class="pno">{links?.next?.clause_code ?? '—'}</span> {strip(links?.next?.title, links?.next?.clause_code)}</div></button>
</div>
{:else}
<p class="empty">{loading ? '불러오는 중…' : '왼쪽에서 절을 선택하세요'}</p>
{/if}
</div>
</section>
</div>
</div>
<style>
:global(body) { background: var(--bg); }
.book { --paper:#fbfcf9; --serif:"Iowan Old Style","Palatino Linotype","Noto Serif KR",Georgia,serif;
display:flex; flex-direction:column; min-height:100vh; }
.bar { display:flex; align-items:center; gap:14px; height:50px; padding:0 18px; background:var(--paper); border-bottom:1px solid var(--border); }
.brand { font-weight:700; font-size:13.5px; color:var(--text); }
.crumbs { color:var(--text-dim); font-size:12.5px; overflow:hidden; text-overflow:ellipsis; white-space:nowrap; max-width:46%; }
.crumbs b { color:var(--text); font-weight:600; } .crumbs .sep { color:#c8d6c0; margin:0 5px; }
.search { margin-left:auto; }
.search input { width:280px; background:var(--surface); border:1px solid var(--border); border-radius:9px; padding:7px 12px; font-size:13px; color:var(--text); outline:none; }
.search input:focus { border-color:var(--accent); }
.tools { display:flex; gap:2px; }
.tool { font-size:12px; color:var(--text-dim); padding:6px 10px; border-radius:8px; border:1px solid transparent; cursor:pointer; }
.tool:hover { background:var(--surface); } .tool.on { background:#ecf0e8; border-color:var(--border); color:var(--accent-hover); font-weight:600; }
.main { display:flex; align-items:flex-start; flex:1; }
.idx { width:264px; flex-shrink:0; align-self:stretch; border-right:1px solid var(--border);
background:linear-gradient(180deg,#f6f8f3,#f1f4ec); padding:16px 10px 30px 16px; position:sticky; top:0; max-height:100vh; overflow:auto; }
.btitle { display:block; font-family:var(--serif); font-size:15.5px; font-weight:600; color:var(--text); text-decoration:none; line-height:1.32; }
.btitle:hover { text-decoration:underline; }
.bmeta { font-size:11px; color:#9aa090; margin:3px 0 14px; }
.parttab { display:flex; align-items:center; gap:8px; margin:11px 0 4px; padding:3px 4px; border-radius:6px; cursor:pointer;
font-size:11px; font-weight:700; letter-spacing:.5px; color:var(--text-dim); text-transform:uppercase; }
.parttab:hover { background:#fff; } .parttab .bar2 { width:3px; height:12px; border-radius:2px; background:var(--domain-engineering); }
.parttab .pname { flex:1; overflow:hidden; text-overflow:ellipsis; white-space:nowrap; } .parttab .ct { color:#9aa090; font-weight:600; letter-spacing:0; }
.ci { display:flex; gap:9px; align-items:baseline; padding:4px 9px; border-radius:7px; cursor:pointer; line-height:1.4; }
.ci .no { font-family:ui-monospace,Menlo,monospace; font-size:11px; color:var(--accent); font-weight:600; min-width:52px; white-space:nowrap; }
.ci .tt { font-size:12.5px; color:var(--text-dim); overflow:hidden; text-overflow:ellipsis; }
.ci:hover { background:#fff; }
.ci.on { background:#fff; box-shadow:inset 3px 0 0 var(--accent), 0 1px 2px rgba(35,41,31,.05); }
.ci.on .no { color:var(--accent-hover); font-weight:700; } .ci.on .tt { color:var(--text); font-weight:600; }
.read { flex:1; min-width:0; padding:34px 40px 80px; }
.col { max-width:680px; margin:0 auto; position:relative; }
.studybar { position:absolute; right:-30px; top:4px; display:flex; flex-direction:column; gap:6px; }
.sbtn { width:34px; height:34px; border-radius:9px; border:1px solid var(--border); background:var(--paper); color:var(--text-dim); font-size:13px; cursor:pointer; }
.sbtn:hover { background:var(--surface); color:var(--accent-hover); }
.kicker { margin-bottom:5px; } .kicker .pth { font-size:11.5px; color:#9aa090; font-weight:600; letter-spacing:.3px; }
.h-no { font-family:ui-monospace,Menlo,monospace; font-size:13px; color:var(--accent); font-weight:700; letter-spacing:.5px; }
.h-title { font-family:var(--serif); font-size:26px; line-height:1.24; font-weight:600; margin:2px 0 14px; letter-spacing:-.2px; color:var(--text); }
.flow { display:flex; justify-content:space-between; gap:8px; margin-bottom:18px; }
.flow .fl { font-size:11.5px; color:var(--text-dim); background:var(--surface); border:1px solid var(--border); border-radius:8px; padding:5px 11px; cursor:pointer; }
.flow .fl:hover:not(:disabled) { background:#ecf0e8; } .flow .fl:disabled { opacity:.35; cursor:default; }
.docbody { font-size:15.5px; }
.docbody :global(.prose) { color:#2a3024; line-height:1.78; }
.docbody :global(.prose h1), .docbody :global(.prose h2), .docbody :global(.prose h3) { font-family:var(--serif); }
.docbody :global(a) { color:var(--accent-hover); }
.conn { margin-top:34px; padding-top:18px; border-top:1px solid var(--border); display:grid; grid-template-columns:1fr 1fr; gap:22px; }
.conn h4 { font-size:11px; font-weight:700; color:var(--text-dim); letter-spacing:.4px; margin:0 0 9px; } .conn h4 span { color:#9aa090; font-weight:500; }
.chiprow { display:flex; flex-wrap:wrap; gap:5px; }
.ref { font-family:ui-monospace,Menlo,monospace; font-size:11.5px; font-weight:600; color:var(--accent-hover); background:#eef4ec; border:1px solid #d9e6d8; border-radius:6px; padding:2px 8px; cursor:pointer; }
.ref:hover { background:#e2efe0; } .ref.dg { color:#9aa090; background:var(--surface); border-color:var(--border); cursor:default; }
.pager { display:flex; gap:10px; margin-top:30px; }
.pg { flex:1; text-align:left; border:1px solid var(--border); border-radius:11px; padding:11px 14px; background:var(--paper); cursor:pointer; }
.pg.next { text-align:right; } .pg:hover:not(:disabled) { border-color:#cfd7c6; background:#fff; } .pg:disabled { opacity:.4; cursor:default; }
.pg .d { font-size:10.5px; color:#9aa090; } .pg .t { font-size:12.5px; color:var(--text-dim); font-weight:600; margin-top:1px; overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
.pg .pno { font-family:ui-monospace,Menlo,monospace; color:var(--accent); }
.empty { color:#9aa090; text-align:center; padding:80px 0; }
.sbtn.on { background:#ecf0e8; color:var(--accent-hover,#3d7256); border-color:var(--border); }
.scount { font-size:9px; font-weight:700; color:#fff; background:var(--accent,#4f8a6b); border-radius:8px; padding:1px 5px; text-align:center; }
.study { margin-top:24px; padding:14px; border:1px solid var(--border); border-radius:12px; background:var(--surface); }
.slab { font-size:11px; font-weight:700; color:var(--text-dim); letter-spacing:.3px; margin-bottom:9px; }
.slab span { color:var(--accent-hover,#3d7256); }
.noteadd { display:flex; gap:8px; align-items:flex-end; margin-bottom:10px; }
.noteadd textarea { flex:1; resize:vertical; border:1px solid var(--border); border-radius:8px; padding:7px 9px; font-size:12.5px; font-family:inherit; color:var(--text); background:var(--paper,#fbfcf9); outline:none; }
.noteadd textarea:focus { border-color:var(--accent); }
.nbtn { flex-shrink:0; font-size:12px; color:#fff; background:var(--accent,#4f8a6b); border:0; border-radius:8px; padding:8px 12px; cursor:pointer; }
.nbtn:hover { background:var(--accent-hover,#3d7256); }
.slist { list-style:none; margin:0; padding:0; display:flex; flex-direction:column; gap:5px; }
.sitem { display:flex; align-items:baseline; gap:8px; padding:6px 8px; border-radius:8px; background:var(--paper,#fbfcf9); border:1px solid var(--border); }
.skind { flex-shrink:0; font-size:9.5px; font-weight:700; border-radius:4px; padding:1px 6px; }
.k-note { color:#3d7256; background:#e3efe2; border:1px solid #cfe3cd; }
.k-highlight { color:#8a6306; background:#faf3e2; border:1px solid #ecdca3; }
.k-card { color:#1d4ed8; background:#eef4fc; border:1px solid #d7e4f7; }
.stext { flex:1; font-size:12px; line-height:1.5; color:var(--text); white-space:pre-wrap; word-break:break-word; }
.sdel { flex-shrink:0; background:none; border:0; color:var(--faint,#9aa090); cursor:pointer; font-size:14px; }
.sdel:hover { color:var(--error,#c0392b); }
.shint { font-size:11.5px; color:var(--faint,#9aa090); margin:0; }
@media(max-width:820px){ .idx{display:none} .read{padding:24px 18px} .conn{grid-template-columns:1fr} .studybar{position:static;flex-direction:row} .crumbs{max-width:30%} .search input{width:150px} }
</style>
+1 -49
View File
@@ -8,7 +8,7 @@
import { goto } from '$app/navigation';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/toast';
import { X, Plus, Trash2, Tag, FolderTree, Sparkles, ArrowUpDown } from 'lucide-svelte';
import { X, Plus, Trash2, Tag, FolderTree, ArrowUpDown } from 'lucide-svelte';
import MarkdownStatusBadge from '$lib/components/MarkdownStatusBadge.svelte';
import { isMdStatusVisible } from '$lib/utils/mdStatus';
import UploadDropzone from '$lib/components/UploadDropzone.svelte';
@@ -22,9 +22,7 @@
import { useIsXl } from '$lib/composables/useMedia.svelte';
import { useListKeyboardNav } from '$lib/composables/useListKeyboardNav.svelte';
import { pLimit } from '$lib/utils/pLimit';
import { isQuestion } from '$lib/utils/isQuestion';
import { domainBgClass, domainLabel } from '$lib/utils/domainSlug';
import AskAnswerCard from '$lib/components/AskAnswerCard.svelte';
const FORMATS = ['pdf', 'hwp', 'hwpx', 'md', 'docx', 'xlsx', 'png', 'jpg'];
@@ -48,30 +46,6 @@
let searchResults = $state(null);
let selectedDoc = $state(null);
// 이드 답변 상태 (질문형 검색)
let askData = $state(null);
let askLoading = $state(false);
let askError = $state(false);
let askDismissed = $state(false);
async function askSearch(q) {
askLoading = true; askError = false; askData = null;
try {
askData = await api(`/search/ask?q=${encodeURIComponent(q)}&limit=10`);
} catch {
askError = true; askData = null;
} finally {
askLoading = false;
}
}
let showAskCard = $derived(
!askDismissed && (
askLoading ||
(askData && !askData.refused && askData.ai_answer && askData.synthesis_status === 'completed')
)
);
// 인스펙터(우측) 토글 — xl+ inline, < xl Drawer.
const isXl = useIsXl();
let inspectorOpen = $state(
@@ -145,7 +119,6 @@
selectedDoc = null;
selectedIds = new Set();
kbIndex = 0;
askData = null; askLoading = false; askError = false; askDismissed = false;
if (ui.isDrawerOpen('meta')) ui.closeDrawer();
if (urlQ) doSearch(urlQ, urlMode);
else loadDocuments();
@@ -191,7 +164,6 @@
async function doSearch(q, mode) {
loading = true;
if (isQuestion(q)) askSearch(q);
try {
const data = await api(`/search/?q=${encodeURIComponent(q)}&mode=${mode}&limit=50`);
searchResults = data.results;
@@ -406,13 +378,6 @@
<option value="trgm">부분매칭</option>
<option value="vector">의미검색</option>
</select>
{#if searchQuery.trim()}
<a
href={`/ask?q=${encodeURIComponent(searchQuery.trim())}`}
class="flex items-center px-2 py-1.5 rounded-lg border border-default text-dim hover:text-accent hover:border-accent transition-colors"
title="이 쿼리로 AI 답변"
><Sparkles size={14} /></a>
{/if}
</div>
<!-- 필터 칩 row -->
@@ -483,19 +448,6 @@
{/if}
</div>
<!-- AI 답변 (질문형 검색) — 목록 상단 고정, 아래로 목록 스크롤 -->
{#if showAskCard}
<div class="px-3 py-2 shrink-0 border-b border-default max-h-[55vh] overflow-y-auto">
<AskAnswerCard
data={askData}
loading={askLoading}
error={askError}
onCitationClick={(docId) => goto(`/documents/${docId}`)}
onDismiss={() => { askDismissed = true; }}
/>
</div>
{/if}
<!-- 선택 toolbar -->
{#if selectionCount > 0}
<div class="flex flex-wrap items-center gap-2 px-3 py-2 shrink-0 bg-accent/10 border-y border-accent/30">
@@ -16,6 +16,7 @@
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import HandwriteCanvas from '$lib/components/HandwriteCanvas.svelte';
import MarkdownDoc from '$lib/components/MarkdownDoc.svelte';
import RelatedDocs from '$lib/components/RelatedDocs.svelte';
import { renderDocMarkdown } from '$lib/utils/docMarkdown';
import MarkdownStatusBadge from '$lib/components/MarkdownStatusBadge.svelte';
import NoteEditor from '$lib/components/editors/NoteEditor.svelte';
@@ -321,6 +322,7 @@
<!-- ════ 우 슬림 레일 (시안 카드 스타일) ════ -->
{#snippet rail()}
<div style="display:flex;flex-direction:column;gap:11px;font-size:14px;">
<RelatedDocs documentId={doc.id} />
{#if doc.ai_tldr || doc.ai_summary}
<div style="background:#f4f7f1;border:1px solid #dde3d6;border-radius:14px;padding:13px;">
<div style="font-size:10.5px;font-weight:700;color:#697061;letter-spacing:.4px;margin-bottom:7px;">TL;DR</div>
+124 -4
View File
@@ -1,13 +1,58 @@
<script>
// /study — 학습 hub.
// 주제로 보기(퀴즈·복습·통계) / 자료 학습 / 필사 세션 / 암기카드 검수.
// /study — 학습 hub + 데일리 랜딩('오늘의 공부' 대시보드).
// 상단 = 이론 홈(진도·오늘의 개념·복습 due, 재노출 트리거). 하단 = 기존 모드 진입.
import { onMount } from 'svelte';
import { api } from '$lib/api';
import { BookOpen, PenLine, GraduationCap, FolderKanban, Layers, Repeat, Flag, Inbox, Activity } from 'lucide-svelte';
import { addToast } from '$lib/stores/toast';
import { BookOpen, PenLine, GraduationCap, FolderKanban, Layers, Repeat, Flag, Inbox, Activity, CalendarCheck, Target } from 'lucide-svelte';
let cardReviewCount = $state(0);
let questionFlagCount = $state(0);
// 오늘의 공부 (이론 홈)
let curriculum = $state(null);
let todayConcepts = $state([]);
let weakConcepts = $state([]); // 약점 개념(관련 기출 정답률 낮음)
let dashLoading = $state(true);
let readPct = $derived(
curriculum && curriculum.total ? Math.round((curriculum.read / curriculum.total) * 100) : 0
);
async function loadDashboard() {
dashLoading = true;
try {
const [cur, today] = await Promise.all([
api('/study/curriculum'),
api('/study/today-concepts?limit=6'),
]);
curriculum = cur;
todayConcepts = today?.concepts ?? [];
} catch {
// 코어 대시보드 실패해도 허브 나머지는 동작 (조용히)
} finally {
dashLoading = false;
}
// 약점 개념 = 비차단(신규 엔드포인트 실패해도 코어 대시보드 블랙아웃 방지)
try {
const weak = await api('/study/concepts/weakness-map?limit=5');
weakConcepts = weak?.weak ?? [];
} catch {}
}
async function markRead(doc) {
try {
await api(`/study/concepts/${doc.doc_id}/read`, { method: 'POST' });
todayConcepts = todayConcepts.filter((c) => c.doc_id !== doc.doc_id);
addToast('success', `회독: ${doc.title}`);
loadDashboard(); // 진도 갱신
} catch {
addToast('error', '회독 처리 실패');
}
}
onMount(async () => {
loadDashboard();
try {
const r = await api('/study-cards/needs-review/count');
cardReviewCount = r?.count ?? 0;
@@ -27,6 +72,80 @@
<p class="text-sm text-dim mt-1">주제별 퀴즈·복습(SRS)·통계 / 학습 자료 회독 / 손글씨 필사 세션.</p>
</header>
<!-- 오늘의 공부 (이론 홈 대시보드 = 데일리 트리거) -->
<section class="mb-5 rounded-lg border border-default bg-surface p-4 md:p-5">
<div class="flex items-center gap-2 mb-3">
<CalendarCheck size={18} class="text-accent" />
<h2 class="text-base font-semibold text-text">오늘의 공부</h2>
{#if curriculum}
<span class="ml-auto text-xs text-dim">이론 회독 <span class="text-text font-medium">{curriculum.read}</span> / {curriculum.total} ({readPct}%)</span>
{/if}
</div>
{#if dashLoading}
<p class="text-xs text-dim">불러오는 중…</p>
{:else}
{#if curriculum}
<div class="h-2 rounded-full bg-bg overflow-hidden mb-3">
<div class="h-full bg-accent" style="width: {readPct}%"></div>
</div>
<div class="flex flex-wrap gap-x-4 gap-y-1 mb-4 text-xs text-dim">
{#each curriculum.subjects as s}
<span>{s.subject} <span class="text-text">{s.read}/{s.total}</span></span>
{/each}
</div>
<div class="flex flex-wrap gap-2 mb-4">
<a
href="/study/topics/{curriculum.topic_id}/review-queue"
class="flex items-center gap-1.5 rounded border border-default px-3 py-1.5 text-xs text-dim hover:border-accent hover:text-text transition-colors"
>
<Repeat size={13} /> 문항 복습 <span class="font-semibold text-text">{curriculum.question_due}</span>
</a>
<span class="flex items-center gap-1.5 rounded border border-default px-3 py-1.5 text-xs text-dim">
<BookOpen size={13} /> 개념 재복습 <span class="font-semibold text-text">{curriculum.concept_due}</span>
</span>
</div>
{/if}
<div class="text-xs text-dim mb-2">오늘의 개념</div>
{#if todayConcepts.length === 0}
<p class="text-xs text-dim">오늘 볼 개념이 없습니다. 잘 하고 있어요.</p>
{:else}
<ul class="space-y-1.5">
{#each todayConcepts as c (c.doc_id)}
<li class="flex items-center gap-2 rounded border border-default px-3 py-2">
<span class="text-accent shrink-0 text-xs" title="빈출">{#each Array(c.freq) as _}{/each}</span>
<a href="/study/read/{c.doc_id}" class="text-sm text-text hover:text-accent truncate flex-1">{c.title}</a>
<span class="shrink-0 text-[10px] rounded-full px-2 py-0.5 {c.reason === '재복습' ? 'bg-accent/15 text-accent' : 'bg-surface border border-default text-dim'}">{c.reason}</span>
<button
type="button"
onclick={() => markRead(c)}
class="shrink-0 text-xs rounded border border-default px-2 py-1 text-dim hover:border-accent hover:text-accent transition-colors"
>읽음</button>
</li>
{/each}
</ul>
{/if}
{#if weakConcepts.length > 0}
<div class="mt-4 pt-3 border-t border-default">
<div class="text-xs text-dim mb-2 flex items-center gap-1.5">
<Target size={13} class="text-error" /> 약점 개념 <span class="text-faint">(관련 기출 정답률 낮음)</span>
</div>
<div class="flex flex-wrap gap-2">
{#each weakConcepts as w (w.doc_id)}
<a href="/study/read/{w.doc_id}"
class="text-xs rounded-full border border-error/40 bg-error/10 text-error px-3 py-1 hover:bg-error/20 transition-colors">
{w.title.replace(/^\d+_/, '')} <span class="font-semibold">{w.accuracy}%</span>
</a>
{/each}
</div>
</div>
{/if}
{/if}
</section>
<a
href="/study/topics"
class="block mb-3 p-5 rounded-lg border border-default bg-surface hover:border-accent hover:bg-accent/5 transition-colors"
@@ -126,7 +245,8 @@
<div class="mt-6 p-4 rounded-lg border border-dashed border-default/60 text-xs text-dim">
<div class="font-medium text-dim mb-1">예정</div>
<ul class="list-disc list-inside space-y-0.5">
<li>애플워치 빠른복습 + 공부 알람(push)</li>
<li>개념 학습 리더 (가리고 떠올리기 · 빈출★ · 관련개념 백링크)</li>
<li>이론↔문제 연결 (개념별 정답률 · 약점 개념 지도)</li>
</ul>
</div>
</div>
@@ -0,0 +1,254 @@
<script>
/**
* /study/read/[docId] — 개념 학습 리더.
* 개념노트(가스기사 documents)를 구조(요약/본문/빈출★/관련개념)로 렌더 +
* '떠올리기' 능동 회상 토글 + 회독 SR(POST read) + 관련개념 백링크 + 이전/다음.
* 본문 렌더 = MarkdownDoc(KaTeX + docimg 내장). 서버 파싱 = /api/study/concepts/{id}.
*/
import { page } from '$app/stores';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/toast';
import { renderMathMarkdownInline } from '$lib/utils/mathMarkdown';
import MarkdownDoc from '$lib/components/MarkdownDoc.svelte';
import Button from '$lib/components/ui/Button.svelte';
import EmptyState from '$lib/components/ui/EmptyState.svelte';
import Skeleton from '$lib/components/ui/Skeleton.svelte';
import { BookOpen, ArrowLeft, Eye, EyeOff, Check, ChevronLeft, ChevronRight, FileQuestion } from 'lucide-svelte';
let docId = $derived($page.params.docId);
let concept = $state(null);
let relatedQ = $state(null); // 관련 기출(이론↔문제, 비차단)
let loading = $state(true);
let notFound = $state(false);
let mode = $state('read'); // 'read' | 'recall'(떠올리기)
let revealed = $state({}); // {sectionIndex: true}
let marking = $state(false);
const STAGE_LABEL = { 0: '복습 시작', 1: '복습 1단계', 2: '복습 2단계', 3: '복습 3단계', 4: '학습 완료' };
const OUTCOME_MARK = { correct: '○', wrong: '✕', unsure: '?' };
const OUTCOME_CLASS = { correct: 'text-success', wrong: 'text-error', unsure: 'text-warning' };
const outcomeMark = (o) => OUTCOME_MARK[o] ?? '';
const outcomeClass = (o) => OUTCOME_CLASS[o] ?? 'text-faint';
async function load() {
const reqId = docId; // in-flight 가드: 백링크 연타 시 stale 응답 무시
loading = true;
notFound = false;
concept = null;
relatedQ = null;
revealed = {};
mode = 'read';
try {
const data = await api(`/study/concepts/${reqId}`);
if (reqId !== docId) return; // 그새 다른 개념으로 이동 → 폐기
concept = data;
} catch (e) {
if (reqId !== docId) return;
if (e?.status === 404) notFound = true;
else addToast('error', '개념을 불러오지 못했습니다');
return; // 본문 실패 → 관련기출 스킵
} finally {
if (reqId === docId) loading = false;
}
// 관련 기출(비차단 — 실패해도 본문 표시엔 영향 없음)
try {
const rq = await api(`/study/concepts/${reqId}/questions?limit=6`);
if (reqId === docId) relatedQ = rq;
} catch {}
}
// $effect 가 마운트 1회 + docId 변경(백링크/이전·다음) 재로드를 모두 커버 (onMount 불필요)
$effect(() => {
void docId;
load();
});
function toggleMode() {
mode = mode === 'read' ? 'recall' : 'read';
revealed = {};
}
function reveal(i) {
revealed = { ...revealed, [i]: true };
}
function shown(i) {
return mode === 'read' || revealed[i];
}
async function markRead() {
marking = true;
try {
const r = await api(`/study/concepts/${docId}/read`, { method: 'POST' });
if (concept) {
concept.is_read = true;
concept.review_stage = r?.review_stage ?? concept.review_stage;
concept.due_at = r?.due_at ?? concept.due_at;
}
addToast('success', '회독 완료 — 다음 복습에 다시 나옵니다');
} catch {
addToast('error', '회독 처리 실패');
} finally {
marking = false;
}
}
</script>
<svelte:head><title>{concept?.title ?? '개념'} — 공부</title></svelte:head>
<div class="p-4 md:p-6 max-w-3xl mx-auto">
<!-- 상단 네비 -->
<div class="flex items-center gap-2 text-xs md:text-sm mb-4 min-w-0">
<a href="/study" class="text-dim hover:text-text flex items-center gap-1 shrink-0">
<ArrowLeft size={14} /> 공부
</a>
{#if concept?.subject}
<span class="text-faint shrink-0">/</span>
<span class="text-dim truncate">{concept.subject}</span>
{/if}
</div>
{#if loading}
<Skeleton h="h-10" rounded="card" />
<div class="mt-3 space-y-2">
{#each Array(4) as _}<Skeleton h="h-24" rounded="card" />{/each}
</div>
{:else if notFound}
<EmptyState icon={BookOpen} title="개념을 찾을 없습니다" description="삭제되었거나 잘못된 주소입니다." />
{:else if concept}
<!-- 제목 + 빈출 tier -->
<header class="mb-3">
<div class="flex items-start gap-2">
<h1 class="text-xl md:text-2xl font-semibold text-text flex-1">{concept.title}</h1>
<span class="text-accent text-sm shrink-0 mt-1" title="빈출도">
{#each Array(concept.freq) as _}{/each}
</span>
</div>
{#if concept.is_read || (concept.review_stage !== null && concept.review_stage !== undefined)}
<div class="mt-1 text-xs text-dim">
{#if concept.review_stage !== null && concept.review_stage !== undefined}
{STAGE_LABEL[concept.review_stage] ?? '복습 중'}
{:else}회독함{/if}
</div>
{/if}
</header>
<!-- 한 줄 요약 (고정 표시) -->
{#if concept.summary}
<div class="mb-4 rounded-lg border-l-4 border-accent bg-accent/10 px-4 py-3 markdown-body text-sm text-text">
{@html renderMathMarkdownInline(concept.summary)}
</div>
{/if}
<!-- 모드 토글 -->
<div class="flex items-center gap-2 mb-4">
<Button variant={mode === 'recall' ? 'primary' : 'secondary'} size="sm" icon={mode === 'recall' ? EyeOff : Eye} onclick={toggleMode}>
{mode === 'recall' ? '떠올리기 모드' : '읽기 모드'}
</Button>
{#if mode === 'recall'}
<span class="text-xs text-dim">각 섹션을 떠올린 뒤 확인하세요</span>
{/if}
</div>
<!-- 본문 섹션 -->
{#if concept.body.length > 0}
<div class="space-y-3 mb-5">
{#each concept.body as sec, i (i)}
<section class="rounded-lg border border-default bg-surface overflow-hidden">
<div class="flex items-center gap-2 px-4 py-2.5 border-b border-default bg-surface-hover">
<h2 class="text-sm font-semibold text-text flex-1">{sec.label}</h2>
{#if sec.stars > 0}
<span class="text-accent text-xs shrink-0">{#each Array(sec.stars) as _}{/each}</span>
{/if}
</div>
{#if shown(i)}
<div class="px-4 py-3">
<MarkdownDoc documentId={concept.doc_id} mdContent={sec.md} mdStatus={null}
class="markdown-body max-w-none text-text" />
</div>
{:else}
<button type="button" onclick={() => reveal(i)}
class="w-full px-4 py-6 text-center text-sm text-dim hover:text-accent hover:bg-accent/5 transition-colors">
<Eye size={16} class="inline mr-1" /> 떠올린 뒤 확인
</button>
{/if}
</section>
{/each}
</div>
{/if}
<!-- 빈출 포인트 -->
{#if concept.bincheol.length > 0}
<section class="mb-5 rounded-lg border border-default bg-surface p-4">
<h2 class="text-sm font-semibold text-text mb-2 flex items-center gap-1.5">
<span class="text-accent"></span> 빈출 포인트
</h2>
<ul class="space-y-1.5">
{#each concept.bincheol as item}
<li class="flex gap-2 text-sm text-text">
<span class="text-accent shrink-0 text-xs mt-0.5">{#each Array(item.tier || 1) as _}{/each}</span>
<span class="markdown-body flex-1">{@html renderMathMarkdownInline(item.text)}</span>
</li>
{/each}
</ul>
</section>
{/if}
<!-- 관련 개념 (백링크) -->
{#if concept.related.length > 0}
<section class="mb-5">
<h2 class="text-xs text-dim mb-2">관련 개념</h2>
<div class="flex flex-wrap gap-2">
{#each concept.related as rel}
{#if rel.doc_id}
<a href="/study/read/{rel.doc_id}"
class="text-xs rounded-full border border-accent/40 bg-accent/10 text-accent px-3 py-1 hover:bg-accent/20 transition-colors">
{rel.phrase}
</a>
{:else}
<span class="text-xs rounded-full border border-default bg-surface text-faint px-3 py-1" title="아직 없는 개념">
{rel.phrase}
</span>
{/if}
{/each}
</div>
</section>
{/if}
<!-- 관련 기출 (이론↔문제 브리지) -->
{#if relatedQ && relatedQ.linked > 0}
<section class="mb-5 rounded-lg border border-default bg-surface p-4">
<h2 class="text-sm font-semibold text-text mb-2 flex items-center gap-1.5">
<FileQuestion size={15} class="text-accent" /> 관련 기출
<span class="ml-1 text-xs font-normal text-dim">
{relatedQ.linked}문항{#if relatedQ.accuracy !== null} · 정답률 <span class="{relatedQ.accuracy < 60 ? 'text-error' : 'text-text'} font-medium">{relatedQ.accuracy}%</span>{:else} · 아직 안 풂{/if}
</span>
</h2>
<ul class="space-y-0.5">
{#each relatedQ.questions as q (q.id)}
<li>
<a href="/study/topics/4/questions/{q.id}"
class="flex items-center gap-2 text-xs py-1 text-dim hover:text-accent transition-colors">
<span class="{outcomeClass(q.last_outcome)} shrink-0 w-4 text-center font-bold">{outcomeMark(q.last_outcome)}</span>
<span class="truncate">{q.subject ?? '기출'}{#if q.exam_round} · {q.exam_round}{/if}</span>
</a>
</li>
{/each}
</ul>
</section>
{/if}
<!-- 액션바 -->
<div class="flex items-center gap-2 border-t border-default pt-4 mt-2">
{#if concept.prev_id}
<Button variant="ghost" size="sm" icon={ChevronLeft} href="/study/read/{concept.prev_id}">이전</Button>
{/if}
<div class="flex-1"></div>
<Button variant="primary" size="sm" icon={Check} onclick={markRead} loading={marking}>
{concept.is_read ? '다시 회독' : '회독 완료'}
</Button>
{#if concept.next_id}
<Button variant="secondary" size="sm" icon={ChevronRight} href="/study/read/{concept.next_id}">다음 개념</Button>
{/if}
</div>
{/if}
</div>
+23
View File
@@ -0,0 +1,23 @@
-- 377_domain_bucket.sql
-- ai_domain(반자유 AI 분류, 드리프트 존재)을 검색 스코프용 7버킷으로 결정적 롤업.
-- 축: ai_domain(routing/해석 축)의 coarsening — category(UI축) 아님 (feedback_category_vs_ai_domain_axis 준수).
-- 버킷: News / Safety / Law / Engineering / General / Philosophy / Programming.
-- STORED generated → 신규/재분류 문서도 ai_domain 붙으면 자동 버킷. ai_domain 원본 보존(하위 검색 유지).
-- 롤백: ALTER TABLE documents DROP COLUMN domain_bucket;
ALTER TABLE documents ADD COLUMN IF NOT EXISTS domain_bucket text
GENERATED ALWAYS AS (
CASE
WHEN ai_domain LIKE 'News%' THEN 'News'
WHEN ai_domain = '법령' OR ai_domain LIKE 'Industrial_Safety/Legislation%' THEN 'Law'
WHEN ai_domain = 'Safety' OR ai_domain LIKE 'Safety/%'
OR ai_domain LIKE 'Industrial_Safety%'
OR ai_domain = 'Knowledge/Industrial_Safety' THEN 'Safety'
WHEN ai_domain LIKE 'Engineering%' OR ai_domain = 'Knowledge/Engineering' THEN 'Engineering'
WHEN ai_domain LIKE 'Philosophy%' THEN 'Philosophy'
WHEN ai_domain LIKE 'Programming%' THEN 'Programming'
ELSE 'General'
END
) STORED;
CREATE INDEX IF NOT EXISTS documents_domain_bucket_idx
ON documents (domain_bucket) WHERE deleted_at IS NULL;
@@ -0,0 +1,9 @@
-- 378_publish_outbox_attempts_failed.sql
-- (번호: 멀티세션 중 prod 가 377_domain_bucket 을 선점 → 378 로 리넘버.)
-- publish_outbox poison row head-of-line block 차단. 발행 워커가 행별 savepoint 격리 후
-- 예외 시 attempts++ 하고 MAX 초과 시 failed_at 스탬프(terminal) → 그 행을 select 에서 제외해
-- 후속 발행이 막히지 않게 함. 기존 미처리 행은 attempts=0 / failed_at=NULL 로 정상 재처리.
-- (단일 ALTER = 1 statement = asyncpg prepared 호환.)
ALTER TABLE publish_outbox
ADD COLUMN IF NOT EXISTS attempts SMALLINT NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS failed_at TIMESTAMPTZ;
+37
View File
@@ -0,0 +1,37 @@
-- 379_asme_clause_kb.sql
-- ASME 절-지식베이스: 절 = 개별 documents 행(parent_id) + 절↔절 백링크 + 태깅 (additive, idempotent)
-- 검색 무접촉: 절 doc 은 embedding NULL(벡터 제외) + doc_kind='clause'(retrieval doc-leg 필터로 제외).
ALTER TABLE documents
ADD COLUMN IF NOT EXISTS parent_id bigint REFERENCES documents(id),
ADD COLUMN IF NOT EXISTS doc_kind text NOT NULL DEFAULT 'standard',
ADD COLUMN IF NOT EXISTS clause_code text,
ADD COLUMN IF NOT EXISTS clause_part text,
ADD COLUMN IF NOT EXISTS clause_order int;
CREATE INDEX IF NOT EXISTS idx_documents_parent_id ON documents(parent_id) WHERE parent_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_documents_doc_kind ON documents(doc_kind);
CREATE INDEX IF NOT EXISTS idx_documents_clause_code ON documents(clause_code) WHERE clause_code IS NOT NULL;
-- 절↔절 백링크 (dangling 허용: dst_doc_id nullable)
CREATE TABLE IF NOT EXISTS clause_links (
id bigserial PRIMARY KEY,
src_doc_id bigint NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
dst_code text NOT NULL,
dst_doc_id bigint REFERENCES documents(id) ON DELETE SET NULL,
anchor text,
ctx text,
char_off int
);
CREATE INDEX IF NOT EXISTS idx_clause_links_src ON clause_links(src_doc_id);
CREATE INDEX IF NOT EXISTS idx_clause_links_dst ON clause_links(dst_doc_id) WHERE dst_doc_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_clause_links_dstcode ON clause_links(dst_code);
-- 태깅 (Part 자동 + 주제)
CREATE TABLE IF NOT EXISTS document_tags (
doc_id bigint NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
tag text NOT NULL,
tag_kind text NOT NULL DEFAULT 'topic',
PRIMARY KEY (doc_id, tag)
);
CREATE INDEX IF NOT EXISTS idx_document_tags_tag ON document_tags(tag);
+9
View File
@@ -0,0 +1,9 @@
-- 380_clause_study.sql — 절-문서 공부도구(노트/형광펜/암기카드) 저장. FK 없음(documents 락 회피).
CREATE TABLE IF NOT EXISTS clause_study (
id bigserial PRIMARY KEY,
doc_id bigint NOT NULL,
kind text NOT NULL, -- 'note' | 'highlight' | 'card'
payload jsonb NOT NULL DEFAULT '{}',
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_clause_study_doc ON clause_study(doc_id, kind);
+16
View File
@@ -0,0 +1,16 @@
-- 381_study_concept_progress.sql — 이론 개념(문서) 간격반복(SR) 진행. 이론공부 홈 트리거.
-- concept_doc_id 는 documents.id 를 가리키나 FK 미설정(hot 테이블 락 회피, clause_study 380 선례).
-- SR 산술은 study_question_progress 와 동일(sr_schedule 공용): stage 0→1→2→3(1·3·7·14일)→4 졸업.
CREATE TABLE IF NOT EXISTS study_concept_progress (
id bigserial PRIMARY KEY,
user_id bigint NOT NULL REFERENCES users(id) ON DELETE CASCADE,
study_topic_id bigint NOT NULL REFERENCES study_topics(id) ON DELETE CASCADE,
concept_doc_id bigint NOT NULL,
review_stage smallint,
due_at timestamptz,
last_read_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT uq_concept_progress_user_doc UNIQUE (user_id, concept_doc_id)
);
CREATE INDEX IF NOT EXISTS idx_concept_progress_due ON study_concept_progress(user_id, due_at) WHERE due_at IS NOT NULL;
+15
View File
@@ -0,0 +1,15 @@
-- 382_study_concept_links.sql — 개념문서 ↔ 기출문항 링크 (이론↔문제 브리지, Stage B).
-- concept_doc_id=documents.id, question_id=study_questions.id — FK 없음(hot 테이블 락 회피, 선례).
-- link_source: 'embedding'(bge-m3 코사인 top-k, 주력) | 'ref'(해설 .md 참조, 후속 enrichment).
-- score=코사인 유사도(0~1). UNIQUE(doc,question,source) — source별 공존 허용(재튜닝=source 전삭제 후 재삽입).
CREATE TABLE IF NOT EXISTS study_concept_links (
id bigserial PRIMARY KEY,
concept_doc_id bigint NOT NULL,
question_id bigint NOT NULL,
link_source text NOT NULL,
score double precision,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT uq_concept_link UNIQUE (concept_doc_id, question_id, link_source)
);
CREATE INDEX IF NOT EXISTS idx_concept_links_doc ON study_concept_links(concept_doc_id);
CREATE INDEX IF NOT EXISTS idx_concept_links_q ON study_concept_links(question_id);
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env python3
"""ASME clause-KB backlinks: resolve clause-id mentions in each clause doc -> clause_links.
dst resolved to the clause doc of the same parent (top-level code); sub-code mention -> anchor;
unresolved (cross-standard / material spec not split) -> dangling (dst_doc_id NULL).
Idempotent per parent. Usage: python3 asme_backlinks_persist.py <parent_id> [--commit]
"""
import asyncio, os, re, sys
MENTION_RE = re.compile(r'(?<![A-Za-z0-9])([A-Z]{1,4}-\d+(?:\.\d+)*[A-Za-z]?)(?![A-Za-z0-9])')
def top(code): return re.match(r'^[A-Z]{1,4}-\d+', code).group(0)
async def main():
parent = int(sys.argv[1]); commit = '--commit' in sys.argv
import asyncpg
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
docs = await conn.fetch("SELECT id, clause_code, md_content FROM documents "
"WHERE parent_id=$1 AND doc_kind='clause' ORDER BY clause_order", parent)
code2id = {d['clause_code']: d['id'] for d in docs}
edges = [] # (src_id, dst_code, dst_doc_id, anchor, ctx, char_off)
resolved = dangling = 0
for d in docs:
body = d['md_content']; src_top = d['clause_code']
seen = set()
for m in MENTION_RE.finditer(body):
code = m.group(1); t = top(code)
if t == src_top: continue # self-reference
if (d['id'], code) in seen: continue # dedup per (src,dst_code)
seen.add((d['id'], code))
dst_id = code2id.get(t) # resolve to same-parent clause doc
anchor = code.lower().replace('.', '-') if code != t else None
off = m.start()
ctx = re.sub(r'\s+', ' ', body[max(0, off-50):off+50]).strip()
edges.append((d['id'], code, dst_id, anchor, ctx, off))
if dst_id: resolved += 1
else: dangling += 1
print(f"parent={parent} clause_docs={len(docs)} edges={len(edges)} resolved={resolved} dangling={dangling}")
# top referenced clauses
from collections import Counter
tgt = Counter(top(e[1]) for e in edges if e[2])
print("most-referenced:", tgt.most_common(8))
if not commit:
print("DRY-RUN. pass --commit to persist."); await conn.close(); return
async with conn.transaction():
ids = [d['id'] for d in docs]
await conn.execute("DELETE FROM clause_links WHERE src_doc_id = ANY($1::bigint[])", ids)
await conn.executemany(
"INSERT INTO clause_links(src_doc_id,dst_code,dst_doc_id,anchor,ctx,char_off) "
"VALUES ($1,$2,$3,$4,$5,$6)", edges)
n = await conn.fetchval("SELECT count(*) FROM clause_links WHERE src_doc_id = ANY($1::bigint[])", ids)
print(f"COMMITTED: {n} clause_links for parent {parent}")
await conn.close()
asyncio.run(main())
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""ASME clause-KB persist (v2: over-CAP pagination). Split a parent standard into per-clause
documents (A-granularity); over-CAP clause bodies are paginated into readable page-docs.
Idempotent per parent. doc_kind='clause', embedding NULL (search-excluded), parent_id=<parent>.
Usage: python3 asme_clause_persist.py <parent_id> [--commit]
"""
import asyncio, os, re, sys, hashlib, statistics
CAP = 12000; PAGE_TOK = 11000
EN, KO = 0.217, 0.529
LINE_RE = re.compile(r'^([ \t#>*]{0,8})([A-Z]{2,4}-\d+(?:\.\d+)*[A-Za-z]?)(.*)$')
MENTION_RE = re.compile(r'(?<![A-Za-z0-9])([A-Z]{1,4}-\d+(?:\.\d+)*[A-Za-z]?)(?![A-Za-z0-9])')
EXACT_TOP = re.compile(r'^[A-Z]{2,4}-\d+$')
TITLE_AFTER = re.compile(r'^[\s.]*[A-Z(]')
REF_LEAD = re.compile(r'^[\s.]*(and|or|to|of|in|on|the|as|is|are|shall|through|per|see|with|'
r'for|by|that|which|such|또는|및|등|의|은|는|에|을|를|과|와)\b', re.I)
def tok(s):
ko = sum(1 for c in s if '' <= c <= ''); return int((len(s)-ko)*EN + ko*KO)
def clean_title(rest):
t = re.sub(r'<sup>ð</sup>\s*\**\d*\**\s*<sup>Þ</sup>', '', rest)
t = re.sub(r'ð\**\d*\**Þ', '', t)
t = t.replace('**', '').replace('#', '')
return re.sub(r'\s+', ' ', t).strip(' *:—-')
def is_header(markup, rest):
if '#' in markup or '*' in markup: return True
rs = rest.strip()
if rs == '': return True
if REF_LEAD.match(rest): return False
if rs[0] in ',;.)': return False
if '' <= rs[0] <= '': return False
if rs[0].islower(): return False
return bool(TITLE_AFTER.match(rs))
def paginate(body):
"""split an over-CAP body into <=MAX_PAGES line-aligned pages of ~PAGE_TOK tokens."""
pages, cur, ct = [], [], 0
for ln in body.split('\n'):
lt = tok(ln) + 1
if ct + lt > PAGE_TOK and cur:
pages.append('\n'.join(cur)); cur, ct = [ln], lt
else:
cur.append(ln); ct += lt
if cur: pages.append('\n'.join(cur))
return pages
def build_clauses(text):
lines = text.split('\n'); off = []; a = 0
for ln in lines: off.append(a); a += len(ln) + 1
bounds = []; seen = set()
for i, ln in enumerate(lines):
m = LINE_RE.match(ln)
if not m: continue
markup, code, rest = m.group(1), m.group(2), m.group(3)
if not EXACT_TOP.match(code): continue
if not is_header(markup, rest): continue
if code in seen: continue
seen.add(code); bounds.append((off[i], code, clean_title(rest)))
raw = []
for idx, (start, code, title) in enumerate(bounds):
end = bounds[idx+1][0] if idx+1 < len(bounds) else len(text)
body = text[start:end]
part = re.match(r'^[A-Z]{2,4}', code).group(0)
links = sorted(set(re.match(r'^[A-Z]{1,4}-\d+', mm).group(0)
for mm in MENTION_RE.findall(body)) - {code})
raw.append(dict(code=code, part=part, title=(code + (' ' + title if title else '')),
body=body, tok=tok(body), links=links))
# expand over-CAP into pages; assign running clause_order
final, order = [], 0
for c in raw:
if c['tok'] <= CAP:
final.append({**c, 'order': order}); order += 1; continue
pages = paginate(c['body'])
for pi, pb in enumerate(pages):
code = c['code'] if pi == 0 else f"{c['code']}·p{pi+1}"
title = c['title'] if pi == 0 else f"{c['title']} (페이지 {pi+1}/{len(pages)})"
final.append(dict(code=code, part=c['part'], order=order, title=title,
body=pb, tok=tok(pb), links=c['links'] if pi == 0 else []))
order += 1
return final
async def main():
parent = int(sys.argv[1]); commit = '--commit' in sys.argv
import asyncpg
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
row = await conn.fetchrow("SELECT md_content, ai_domain, data_origin FROM documents WHERE id=$1", parent)
if not row: print(f"parent {parent} not found"); return
clauses = build_clauses(row['md_content'])
toks = [c['tok'] for c in clauses]
over = [c for c in clauses if c['tok'] > CAP]
print(f"parent={parent} clause_docs={len(clauses)} median_tok={int(statistics.median(toks))} "
f"max_tok={max(toks)} over_cap_remaining={len(over)}")
if over: print("still over-CAP:", [f"{c['code']}:{c['tok']}t" for c in over])
if not commit:
print("DRY-RUN. pass --commit to persist."); await conn.close(); return
async with conn.transaction():
deld = await conn.execute("DELETE FROM documents WHERE parent_id=$1 AND doc_kind='clause'", parent)
print("deleted prior:", deld)
for c in clauses:
fh = hashlib.sha256(f"{parent}:{c['code']}:{c['body']}".encode()).hexdigest()
cid = await conn.fetchval("""
INSERT INTO documents
(file_format, file_hash, title, md_content, parent_id, doc_kind,
clause_code, clause_part, clause_order, ai_domain, data_origin,
md_status, review_status, conversion_status, preview_status)
VALUES ('md',$1,$2,$3,$4,'clause',$5,$6,$7,$8,$9,'success','approved','none','none')
RETURNING id
""", fh, c['title'], c['body'], parent, c['code'], c['part'], c['order'],
row['ai_domain'], row['data_origin'] or 'external')
await conn.execute("INSERT INTO document_tags(doc_id,tag,tag_kind) VALUES ($1,$2,'part') "
"ON CONFLICT DO NOTHING", cid, c['part'])
n = await conn.fetchval("SELECT count(*) FROM documents WHERE parent_id=$1 AND doc_kind='clause'", parent)
print(f"COMMITTED: {n} clause docs for parent {parent}")
await conn.close()
asyncio.run(main())
+12 -8
View File
@@ -31,8 +31,8 @@ from core.database import async_session
from models.study_memo_card_progress import StudyMemoCardProgress
from services.study.publish_enqueue import backfill_publish_card_progress
# 개인 학습툴 progress row 대비 넉넉. 도달 시 가드 경보.
PAGE = 100000
# 페이지 배치 크기 — after_id 루프로 전량 처리(bounded tx).
PAGE = 5000
async def run(dry_run: bool) -> None:
@@ -50,13 +50,17 @@ async def run(dry_run: bool) -> None:
print("[dry-run] 적재 안 함. 실제 실행은 --dry-run 제거.")
return
async with async_session() as session:
n = await backfill_publish_card_progress(session, after_id=0, limit=PAGE)
await session.commit()
total = 0
after = 0
while True:
async with async_session() as session:
n, after = await backfill_publish_card_progress(session, after_id=after, limit=PAGE)
await session.commit()
total += n
if n < PAGE:
break
print(f"\n[ok] outbox 적재 {n}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
if n >= PAGE:
print(f"[warn] PAGE({PAGE}) 도달 — progress 가 더 있을 수 있음. after_id 페이징 추가 필요.")
print(f"\n[ok] outbox 적재 {total}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
def main() -> None:
+12 -8
View File
@@ -31,8 +31,8 @@ from core.database import async_session
from models.study_memo_card import StudyMemoCard
from services.study.publish_enqueue import backfill_publish_cards
# 개인 학습툴 카드 수 대비 넉넉(단일 outbox 적재 tx, 워커는 BATCH_SIZE 로 drain). 도달 시 가드 경보.
PAGE = 100000
# 페이지 배치 크기 — after_id 루프로 전량 처리(bounded tx). 워커는 BATCH_SIZE 로 drain.
PAGE = 5000
async def run(dry_run: bool) -> None:
@@ -55,13 +55,17 @@ async def run(dry_run: bool) -> None:
print("[dry-run] 적재 안 함. 실제 실행은 --dry-run 제거.")
return
async with async_session() as session:
n = await backfill_publish_cards(session, after_id=0, limit=PAGE)
await session.commit()
total = 0
after = 0
while True:
async with async_session() as session:
n, after = await backfill_publish_cards(session, after_id=after, limit=PAGE)
await session.commit()
total += n
if n < PAGE:
break
print(f"\n[ok] outbox 적재 {n}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
if n >= PAGE:
print(f"[warn] PAGE({PAGE}) 도달 — 카드가 더 있을 수 있음. after_id 페이징 추가 필요.")
print(f"\n[ok] outbox 적재 {total}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
def main() -> None:
+11 -7
View File
@@ -37,7 +37,7 @@ from core.database import async_session
from models.study_topic import StudyTopic
from services.study.publish_enqueue import backfill_publish_topics
# 개인 학습툴 주제 수 대비 넉넉. 도달 시 overflow 가드가 경보.
# 페이지 배치 크기 — after_id 루프로 전량 처리(bounded tx).
PAGE = 5000
@@ -58,13 +58,17 @@ async def run(dry_run: bool) -> None:
print("[dry-run] 적재 안 함. 실제 실행은 --dry-run 제거.")
return
async with async_session() as session:
n = await backfill_publish_topics(session, after_id=0, limit=PAGE)
await session.commit()
total = 0
after = 0
while True:
async with async_session() as session:
n, after = await backfill_publish_topics(session, after_id=after, limit=PAGE)
await session.commit()
total += n
if n < PAGE:
break
print(f"\n[ok] outbox 적재 {n}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
if n >= PAGE:
print(f"[warn] PAGE({PAGE}) 도달 — 주제가 더 있을 수 있음. after_id 페이징 추가 필요.")
print(f"\n[ok] outbox 적재 {total}건 — 발행 워커가 drain(flag on 시) 하며 rev 부여.")
def main() -> None:
+1 -1
View File
@@ -106,7 +106,7 @@ async def main() -> None:
"SELECT count(*) FROM pg_indexes WHERE indexname='uq_attempt_session_question'"))).scalar()
mx = (await s.execute(text("SELECT max(version) FROM schema_migrations"))).scalar()
print(f"SCHEMA OK — max_migration={mx} documents={docs} purge_col={purge} cand_qwen={cand} attempt_uq={uq}")
assert docs and purge == 1 and cand == 0 and uq == 1 and mx == 361, "FAIL: 기대 스키마 상태 불일치"
assert docs and purge == 1 and cand == 0 and uq == 1 and mx == 378, "FAIL: 기대 스키마 상태 불일치"
# ── 5) /health 직접 호출 ──────────────────────────────────────────────
health = await main.health_check()
+23
View File
@@ -0,0 +1,23 @@
-- concept_links_backfill.sql — 개념↔문항 임베딩 링크 재생성 (Stage B, 멱등·재실행 안전).
-- 정찰 확정: bge-m3 1024d 코사인, per-concept top-k=10, threshold 0.62 → ~2362링크·284/289개념·964문항.
-- 재튜닝 시 DELETE(embedding 소스만) 후 재삽입 = ref 링크(후속) 불변. 개념 doc = 가스기사 태그.
DELETE FROM study_concept_links WHERE link_source = 'embedding';
INSERT INTO study_concept_links (concept_doc_id, question_id, link_source, score)
WITH cd AS (
SELECT id, embedding FROM documents
WHERE user_tags::text LIKE '%@library/가스기사/%'
AND deleted_at IS NULL AND embedding IS NOT NULL
),
ranked AS (
SELECT cd.id AS concept_doc_id, q.id AS question_id,
1 - (q.embedding <=> cd.embedding) AS score,
row_number() OVER (PARTITION BY cd.id ORDER BY q.embedding <=> cd.embedding) AS rn
FROM cd
JOIN study_questions q
ON q.study_topic_id = 4 AND q.embedding IS NOT NULL
AND q.deleted_at IS NULL AND q.is_active
)
SELECT concept_doc_id, question_id, 'embedding', score
FROM ranked
WHERE rn <= 10 AND score >= 0.62
ON CONFLICT (concept_doc_id, question_id, link_source) DO NOTHING;
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""기술지침(KOSHA guide) 절-KB persist: 번호섹션(# 1. 목적 / ## 4.1) 단위 분해 + 제본.
ASME/법령과 동일 clause-KB 모델(doc_kind='clause', parent_id=지침, 검색제외, /book 리더 공용).
Usage: python3 guide_clause_persist.py <id|all> [--commit]
"""
import asyncio, os, re, sys, hashlib, statistics
CAP = 12000; PAGE_TOK = 11000
EN, KO = 0.217, 0.529
# 번호섹션 헤더: '# 1. 목 적', '## 4.1 누출...' (번호 1~3자리=연도(4자리) 배제)
ART_RE = re.compile(r'^#{1,6}\s*(\d{1,3}(?:\.\d{1,3})*)\.?\s+(\S.*)$')
TOP_RE = re.compile(r'^\d{1,3}$')
# 외부 표준/법규 참조(대부분 dangling): ASME B16.5 · KS B 1501 · 규칙 제N조
EXT_RE = re.compile(r'(ASME\s+[A-Z][0-9.]+|KS\s+[A-Z]\s*[0-9]+|ISO\s+[0-9]+|제\d+조)')
def tok(s):
ko = sum(1 for c in s if '' <= c <= ''); return int((len(s)-ko)*EN + ko*KO)
def build_sections(text):
lines = text.split('\n'); off = []; a = 0
for ln in lines: off.append(a); a += len(ln) + 1
bounds = []; seen = set()
for i, ln in enumerate(lines):
m = ART_RE.match(ln)
if not m: continue
code, name = m.group(1), m.group(2).strip()
if not TOP_RE.match(code): continue # top-level 번호섹션만 경계
if code in seen: continue
if len(name) < 1: continue
seen.add(code); bounds.append((off[i], code, name))
out = []
for idx, (start, code, name) in enumerate(bounds):
end = bounds[idx+1][0] if idx+1 < len(bounds) else len(text)
body = text[start:end].strip()
ext = sorted(set(EXT_RE.findall(body)))[:8]
out.append(dict(code=code, part='본문', order=0, title=f"{code}. {name}"[:120],
body=body, tok=tok(body), links=[], ext=ext))
# over-CAP 페이지네이션 + 순번
final, order = [], 0
for c in out:
if c['tok'] <= CAP:
final.append({**c, 'order': order}); order += 1; continue
pages, cur, ct = [], [], 0
for ln in c['body'].split('\n'):
lt = tok(ln)+1
if ct+lt > PAGE_TOK and cur: pages.append('\n'.join(cur)); cur=[ln]; ct=lt
else: cur.append(ln); ct+=lt
if cur: pages.append('\n'.join(cur))
for pi, pb in enumerate(pages):
final.append(dict(code=c['code'] if pi==0 else f"{c['code']}·p{pi+1}", part='본문',
order=order, title=c['title'] if pi==0 else f"{c['title']} (p{pi+1})",
body=pb, tok=tok(pb), links=[], ext=[]))
order += 1
return final
async def process_one(conn, gid, commit, verbose=True):
row = await conn.fetchrow("SELECT title, md_content, ai_domain, data_origin FROM documents WHERE id=$1", gid)
if not row: return ('notfound', 0)
if not row['md_content']: return ('nullmd', 0)
secs = build_sections(row['md_content'])
if len(secs) < 2: return ('few', len(secs)) # 섹션 2 미만 = 번호구조 아님
toks = [c['tok'] for c in secs]
if verbose:
print(f"guide={gid} «{(row['title'] or '')[:40]}» 섹션={len(secs)} median={int(statistics.median(toks))} max={max(toks)}")
print(" 샘플:", [c['title'][:26] for c in secs[:7]])
if not commit: return ('dry', len(secs))
async with conn.transaction():
await conn.execute("DELETE FROM clause_links WHERE src_doc_id IN (SELECT id FROM documents WHERE parent_id=$1 AND doc_kind='clause')", gid)
await conn.execute("DELETE FROM documents WHERE parent_id=$1 AND doc_kind='clause'", gid)
for c in secs:
fh = hashlib.sha256(f"{gid}:{c['code']}:{c['body']}".encode()).hexdigest()
cid = await conn.fetchval("""
INSERT INTO documents (file_format,file_hash,title,md_content,parent_id,doc_kind,
clause_code,clause_part,clause_order,ai_domain,data_origin,
md_status,review_status,conversion_status,preview_status)
VALUES ('md',$1,$2,$3,$4,'clause',$5,$6,$7,$8,$9,'success','approved','none','none') RETURNING id
""", fh, c['title'], c['body'], gid, c['code'], c['part'], c['order'], row['ai_domain'], row['data_origin'] or 'external')
await conn.execute("INSERT INTO document_tags(doc_id,tag,tag_kind) VALUES ($1,'기술지침','kind') ON CONFLICT DO NOTHING", cid)
n = await conn.fetchval("SELECT count(*) FROM documents WHERE parent_id=$1 AND doc_kind='clause'", gid)
print(f" COMMITTED: {n} 섹션 for guide {gid}")
return ('committed', len(secs))
async def main():
import asyncpg
arg = sys.argv[1]; commit = '--commit' in sys.argv
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
if arg == 'all':
gs = await conn.fetch("SELECT id FROM documents WHERE material_type='guide' AND doc_kind='standard' "
"AND deleted_at IS NULL AND md_content IS NOT NULL ORDER BY id")
agg = {}; tot = 0
for i, r in enumerate(gs):
st, n = await process_one(conn, r['id'], commit, verbose=False)
agg[st] = agg.get(st, 0)+1; tot += n if st in ('dry','committed') else 0
if commit and (i+1) % 40 == 0: print(f"{i+1}/{len(gs)} (누적섹션 {tot})")
print(f"BATCH {'COMMIT' if commit else 'DRY'} guides={len(gs)} status={agg} 총섹션={tot}")
else:
await process_one(conn, int(arg), commit, verbose=True)
await conn.close()
asyncio.run(main())
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/env python3
"""법령 조-KB persist: 법령을 조(條) 단위 개별 문서로 분해 + 조↔조 백링크 + 장(章) 태그.
ASME clause-KB와 동일 모델(doc_kind='clause', parent_id=법령, embedding NULL, 검색제외).
법령 추출 노이즈( ### 메타 반복) 트림. Usage: python3 law_clause_persist.py <law_id> [--commit]
"""
import asyncio, os, re, sys, hashlib, statistics
CAP = 12000; PAGE_TOK = 11000
EN, KO = 0.217, 0.529
# 조 헤더: '### 제3조의2(가스안전관리...) 본문'
ART_RE = re.compile(r'^#{0,6}\s*(제\d+조(?:의\d+)?)\s*\(([^)]*)\)\s*(.*)$')
CHAP_RE = re.compile(r'^#{1,6}\s*(제\d+장(?:의\d+)?)\s*(.*)$') # 장 = part
# 같은-법 조 멘션(백링크)
MENTION_RE = re.compile(r'\d+조(?:의\d+)?')
# 타법 참조: 「법명」 ... 제N조
EXTLAW_RE = re.compile(r'「([^」]+)」')
def tok(s):
ko = sum(1 for c in s if '' <= c <= ''); return int((len(s)-ko)*EN + ko*KO)
def art_code(c): return c # '제3조의2'
def build_articles(text):
lines = text.split('\n'); off = []; a = 0
for ln in lines: off.append(a); a += len(ln) + 1
arts = [] # (line_idx, code, name, part)
cur_part = None
for i, ln in enumerate(lines):
ch = CHAP_RE.match(ln)
if ch and not ART_RE.match(ln):
cur_part = (ch.group(1) + (' ' + ch.group(2).strip() if ch.group(2).strip() else '')).strip()
continue
m = ART_RE.match(ln)
if m:
arts.append((i, m.group(1), m.group(2).strip(), cur_part))
# 본문 슬라이스 + 다음 조 앞 메타 노이즈 트림
out = []
for idx, (li, code, name, part) in enumerate(arts):
end_li = arts[idx+1][0] if idx+1 < len(arts) else len(lines)
body_lines = lines[li:end_li]
# 트림: 끝에서부터 '### {짧은 메타}' (조번호/조문/날짜/제목, [개정] 제N조 아님) 제거
while len(body_lines) > 1:
last = body_lines[-1].strip()
if last == '':
body_lines.pop(); continue
mh = re.match(r'^#{1,6}\s+(.*)$', last)
if mh:
c = mh.group(1).strip()
if not c.startswith('[') and not c.startswith('') and (
c in ('조문', 'N') or re.fullmatch(r'\d+', c) or re.fullmatch(r'\d{8}', c) or len(c) <= 30):
body_lines.pop(); continue
break
body = '\n'.join(body_lines).strip()
links = sorted(set(MENTION_RE.findall(body)) - {code})
ext = sorted(set(EXTLAW_RE.findall(body)))[:6]
out.append(dict(code=code, part=part or '본칙', order=0,
title=f"{code}({name})" if name else code,
body=body, tok=tok(body), links=links, ext=ext))
# 페이지네이션(over-CAP) + 순번
final, order = [], 0
for c in out:
if c['tok'] <= CAP:
final.append({**c, 'order': order}); order += 1; continue
# 11K 토큰 라인 단위 분할
pages, cur, ct = [], [], 0
for ln in c['body'].split('\n'):
lt = tok(ln)+1
if ct+lt > PAGE_TOK and cur: pages.append('\n'.join(cur)); cur=[ln]; ct=lt
else: cur.append(ln); ct+=lt
if cur: pages.append('\n'.join(cur))
for pi, pb in enumerate(pages):
final.append(dict(code=c['code'] if pi==0 else f"{c['code']}·p{pi+1}", part=c['part'],
order=order, title=c['title'] if pi==0 else f"{c['title']} (p{pi+1}/{len(pages)})",
body=pb, tok=tok(pb), links=c['links'] if pi==0 else [], ext=[]))
order += 1
return final
async def process_one(conn, law, commit, verbose=True):
row = await conn.fetchrow("SELECT title, coalesce(md_content, extracted_text) AS md_content, ai_domain, data_origin FROM documents WHERE id=$1", law)
if not row: return ('notfound', 0, 0)
if not row['md_content']: return ('nullmd', 0, 0)
arts = build_articles(row['md_content'])
if not arts: return ('noart', 0, 0)
toks = [c['tok'] for c in arts]
nlink = sum(len(c['links']) for c in arts)
if verbose:
parts = {}
for c in arts: parts[c['part']] = parts.get(c['part'], 0)+1
print(f"law={law} «{(row['title'] or '')[:34]}» 조문={len(arts)} median={int(statistics.median(toks))} "
f"max={max(toks)} 장={len(parts)} 백링크={nlink}")
print(" 샘플:", [c['title'][:22] for c in arts[:6]])
if not commit:
return ('dry', len(arts), nlink)
async with conn.transaction():
await conn.execute(
"DELETE FROM clause_links WHERE src_doc_id IN (SELECT id FROM documents WHERE parent_id=$1 AND doc_kind='clause')", law)
await conn.execute("DELETE FROM documents WHERE parent_id=$1 AND doc_kind='clause'", law)
code2id = {}
for c in arts:
fh = hashlib.sha256(f"{law}:{c['code']}:{c['body']}".encode()).hexdigest()
cid = await conn.fetchval("""
INSERT INTO documents (file_format,file_hash,title,md_content,parent_id,doc_kind,
clause_code,clause_part,clause_order,ai_domain,data_origin,
md_status,review_status,conversion_status,preview_status)
VALUES ('md',$1,$2,$3,$4,'clause',$5,$6,$7,$8,$9,'success','approved','none','none') RETURNING id
""", fh, c['title'], c['body'], law, c['code'], c['part'], c['order'],
row['ai_domain'], row['data_origin'] or 'external')
code2id[c['code']] = cid
await conn.execute("INSERT INTO document_tags(doc_id,tag,tag_kind) VALUES ($1,$2,'chapter') ON CONFLICT DO NOTHING", cid, c['part'])
# 조↔조 백링크 (같은 법 내부; 타법 참조는 dangling)
edges = []
for c in arts:
src = code2id[c['code']]
for dst in c['links']:
edges.append((src, dst, code2id.get(dst), None, None, None))
if edges:
await conn.executemany(
"INSERT INTO clause_links(src_doc_id,dst_code,dst_doc_id,anchor,ctx,char_off) VALUES ($1,$2,$3,$4,$5,$6)", edges)
n = await conn.fetchval("SELECT count(*) FROM documents WHERE parent_id=$1 AND doc_kind='clause'", law)
print(f" COMMITTED: {n} 조문 + {len(edges)} 백링크 for law {law}")
return ('committed', n, len(edges))
async def main():
import asyncpg
arg = sys.argv[1]; commit = '--commit' in sys.argv
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
if arg == 'all':
laws = await conn.fetch("SELECT lm.document_id AS id FROM legal_meta lm "
"JOIN documents d ON d.id=lm.document_id "
"WHERE lm.law_doc_kind='primary' AND lm.version_status='current' "
"AND coalesce(d.md_content, d.extracted_text) IS NOT NULL "
"ORDER BY lm.document_id")
agg = {}; tot_art = tot_link = 0; zero = []
for i, r in enumerate(laws):
st, na, nl = await process_one(conn, r['id'], commit, verbose=False)
agg[st] = agg.get(st, 0) + 1
tot_art += na; tot_link += nl
if st == 'noart': zero.append(r['id'])
if commit and (i + 1) % 30 == 0: print(f"{i+1}/{len(laws)} (누적 조 {tot_art})")
print(f"BATCH {'COMMIT' if commit else 'DRY'} laws={len(laws)} status={agg} 총조문={tot_art} 총백링크={tot_link}")
if zero: print(f" 0-조(추출구조 이질) {len(zero)}건: {zero[:20]}")
else:
await process_one(conn, int(arg), commit, verbose=True)
await conn.close()
asyncio.run(main())
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""논문 인용그래프 가능성 측정(read-only) — 본문 DOI로 코퍼스내 인용 엣지 추정.
own_doi = 헤더( 2500) DOI / cited = References 이후(또는 전체) DOI. owner 엣지.
"""
import asyncio, os, re, sys
DOI_RE = re.compile(r'10\.\d{4,9}/[^\s"<>)\]\},;]+')
REF_RE = re.compile(r'(references|참고문헌|bibliography|reference\s*list)', re.I)
def norm(d): return d.rstrip('.').lower()
async def main():
import asyncpg
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
rows = await conn.fetch("SELECT id, title, coalesce(md_content, extracted_text) AS txt FROM documents "
"WHERE material_type='paper' AND doc_kind='standard' AND deleted_at IS NULL "
"AND coalesce(md_content, extracted_text) IS NOT NULL")
owner = {} # doi -> paper id (헤더 DOI = 그 논문 소유)
cited = {} # paper id -> set(cited doi)
n_own = n_refsec = 0
for r in rows:
txt = r['txt']
head = txt[:2500]
hdois = [norm(d) for d in DOI_RE.findall(head)]
if hdois:
owner.setdefault(hdois[0], r['id']); n_own += 1
m = REF_RE.search(txt)
body = txt[m.start():] if m else ''
if m: n_refsec += 1
cds = set(norm(d) for d in DOI_RE.findall(body))
if cds: cited[r['id']] = cds
# 엣지: paper -> owner(cited doi)
edges = []
for pid, cds in cited.items():
for d in cds:
o = owner.get(d)
if o and o != pid: edges.append((pid, o, d))
cited_papers = set(e[0] for e in edges)
target_papers = set(e[1] for e in edges)
print(f"papers={len(rows)} 헤더DOI보유={n_own} References보유={n_refsec} owner_map={len(owner)}")
print(f"인용엣지(코퍼스내)={len(edges)} 인용하는논문={len(cited_papers)} 피인용논문={len(target_papers)}")
# 피인용 top
from collections import Counter
top = Counter(e[1] for e in edges).most_common(6)
if top:
idmap = {r['id']: r['title'] for r in rows}
print("피인용 top:")
for pid, c in top: print(f" {c}회 ← {(idmap.get(pid) or '')[:48]}")
await conn.close()
asyncio.run(main())
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""OpenAlex 고신뢰 매치율 측정 — References 보유 논문(학술 추정) 표본."""
import asyncio, os, re
def toks(s):
return set(re.findall(r'[a-z0-9]+', (s or '').lower()))
def sim(a, b):
ta, tb = toks(a), toks(b)
if not ta or not tb: return 0.0
return len(ta & tb) / len(ta | tb)
async def main():
import asyncpg, httpx
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
rows = await conn.fetch("SELECT id, title FROM documents WHERE material_type='paper' "
"AND doc_kind='standard' AND deleted_at IS NULL AND title IS NOT NULL "
"AND coalesce(md_content,extracted_text) ~* 'references|참고문헌' "
"ORDER BY id LIMIT 40")
hi = mid = lo = 0; hits = []
async with httpx.AsyncClient(timeout=20) as client:
for r in rows:
title = re.sub(r'\s+', ' ', r['title']).strip()
try:
resp = await client.get("https://api.openalex.org/works",
params={"search": title[:200], "per_page": 1, "mailto": "hyun49196@gmail.com"})
res = (resp.json().get("results") or [])
if not res: lo += 1; continue
s = sim(title, res[0].get("title"))
if s >= 0.6: hi += 1; hits.append((s, title[:40], (res[0].get('title') or '')[:40], res[0].get('cited_by_count'), len(res[0].get('referenced_works') or [])))
elif s >= 0.4: mid += 1
else: lo += 1
except Exception: lo += 1
print(f"표본={len(rows)} 고신뢰(≥0.6)={hi} 중간(0.4~0.6)={mid} 저신뢰/무매치={lo}")
print("고신뢰 매치 샘플:")
for s, a, b, cb, rf in hits[:8]:
print(f" sim={s:.2f} cited={cb} refs={rf} | {a}{b}")
await conn.close()
asyncio.run(main())
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
"""OpenAlex 보강 타당성 테스트 — 소수 논문 제목으로 매칭/메타 확인 (외부 API)."""
import asyncio, os, re
async def main():
import asyncpg, httpx
conn = await asyncpg.connect(os.environ['DATABASE_URL'].replace('+asyncpg', ''))
rows = await conn.fetch("SELECT id, title FROM documents WHERE material_type='paper' "
"AND doc_kind='standard' AND deleted_at IS NULL AND title IS NOT NULL "
"AND length(title) > 15 ORDER BY id LIMIT 6")
async with httpx.AsyncClient(timeout=20) as client:
for r in rows:
title = re.sub(r'\s+', ' ', r['title']).strip()
try:
resp = await client.get("https://api.openalex.org/works",
params={"search": title[:200], "per_page": 1, "mailto": "hyun49196@gmail.com"})
js = resp.json()
res = (js.get("results") or [])
if not res:
print(f"[{r['id']}] NO MATCH | {title[:50]}"); continue
w = res[0]
oid = (w.get("id") or "").split("/")[-1]
print(f"[{r['id']}] {title[:46]}")
print(f" → OA {oid} | {(w.get('title') or '')[:46]} | {w.get('publication_year')} | "
f"cited_by={w.get('cited_by_count')} | refs={len(w.get('referenced_works') or [])} | doi={w.get('doi')}")
except Exception as e:
print(f"[{r['id']}] ERROR {type(e).__name__}: {e}")
await conn.close()
asyncio.run(main())
+25 -2
View File
@@ -59,6 +59,11 @@ MAX_IMAGES_PER_DOC = int(os.getenv("MINERU_MAX_IMAGES_PER_DOC", "200"))
MAX_BYTES_PER_IMAGE = int(os.getenv("MINERU_MAX_BYTES_PER_IMAGE", str(10 * 1024 * 1024)))
MAX_PAGES_HARD = int(os.getenv("MINERU_MAX_PAGES_HARD", "200")) # 1-shot max_pages 안전장치
# self-timeout — 변환/워밍이 vLLM 행으로 _engine_lock 을 영구 점유해 서비스가 wedge 되는 것을 차단.
# (클라이언트 marker_worker 는 300s 로 포기하나 서버측 inflight 는 자동 취소 안 됨 → 서버 자체 상한 필요.)
PARSE_TIMEOUT_S = float(os.getenv("MINERU_PARSE_TIMEOUT_S", "600"))
WARMUP_TIMEOUT_S = float(os.getenv("MINERU_WARMUP_TIMEOUT_S", "1200")) # 최초 모델 다운로드(~2.4GB) 여유
_PRELOAD = os.getenv("MINERU_PRELOAD", "1") != "0"
# ---- 엔진 상태 ---------------------------------------------------------------
@@ -68,6 +73,15 @@ _warmup_error: str | None = None
_engine_lock = asyncio.Lock()
def _is_engine_fatal(exc: BaseException) -> bool:
"""OOM/CUDA 류 = 엔진 상태 오염 가능 → 재워밍 강제 대상(타임아웃은 호출측에서 별도 판정)."""
s = f"{type(exc).__name__} {exc}".lower()
return any(
k in s
for k in ("out of memory", "oom", "cuda", "cublas", "device-side", "illegal memory")
)
async def _run_mineru(pdf_bytes: bytes, lang: str) -> tuple[str, list[dict]]:
"""슬라이스된 PDF bytes → (markdown, 이미지 dict 리스트). **async 엔진 경로.**
@@ -148,7 +162,7 @@ async def _ensure_warmup() -> None:
page.insert_text((72, 72), "MinerU warmup.")
warmup_bytes = doc.tobytes()
doc.close()
await _run_mineru(warmup_bytes, MINERU_LANG)
await asyncio.wait_for(_run_mineru(warmup_bytes, MINERU_LANG), timeout=WARMUP_TIMEOUT_S)
_warmup_done = True
_warmup_error = None
logger.info(f"[mineru-service] warmup done engine_version={_engine_version}")
@@ -274,6 +288,7 @@ def _serialize_images(images: list[dict], src_path: str) -> tuple[list[ConvertIm
@app.post("/convert", response_model=ConvertResponse)
async def convert(req: ConvertRequest):
global _warmup_done
p = _resolve_path(req.file_path)
if p is None or not p.is_file():
raise HTTPException(404, detail={"code": "file_not_found", "message": req.file_path})
@@ -288,10 +303,18 @@ async def convert(req: ConvertRequest):
async with _engine_lock: # 실제 변환 직렬화(단일 GPU)
start = time.monotonic()
try:
md_text, raw_images = await _run_mineru(pdf_bytes, MINERU_LANG)
md_text, raw_images = await asyncio.wait_for(
_run_mineru(pdf_bytes, MINERU_LANG), timeout=PARSE_TIMEOUT_S
)
except HTTPException:
raise
except Exception as exc:
# 타임아웃(엔진 행) 또는 OOM/CUDA 류면 엔진 오염 가능 → 다음 요청이 재워밍하도록 리셋.
# 재워밍까지 실패하면 _ensure_warmup 이 _warmup_error 설정 → /ready 503 → healthcheck
# 재시작으로 escalate(영구 degradation 차단). 일시 OOM 이면 재워밍 성공 후 정상화.
if isinstance(exc, (asyncio.TimeoutError, TimeoutError)) or _is_engine_fatal(exc):
_warmup_done = False
logger.error("[mineru-service] engine reset (timeout/fatal) path=%s: %s", p, exc)
logger.exception(f"[mineru-service] conversion failed path={p}: {exc}")
raise HTTPException(422, detail={"code": "conversion_failed",
"message": f"{type(exc).__name__}: {exc}"}) from exc
-291
View File
@@ -1,291 +0,0 @@
"""PR-MacBook-RAG-Backend-1 정정 4 핵심 테스트.
검증 invariant (synthesize 함수 레벨 /ask wrapper 503 매핑은 search.py
status="backend_unavailable" 분기로 1:1 deterministic):
1. backend="qwen-macbook" + MacBook URL 죽은 포트
synthesize() SynthesisResult(status="backend_unavailable", ...) 반환
Gemma backend generate() ** 1번도 호출되지 않음** (자동 fallback 부재)
2. backend 미지정 (None)
Gemma backend.generate() 호출, Qwen backend.generate() 호출 0
기존 호출자 (Hermes docsrv_ask / voice-memo-bot) 회귀 0
3. backend="qwen-macbook" + MacBook 정상 응답
status="completed" + answer 채워짐, Gemma backend 호출 0
테스트 전략:
- synthesize() 호출하는 backend dispatcher (services.llm.get_backend)
monkeypatch 해서 mock backend 주입.
- Gemma backend generate AsyncMock 호출 횟수를 추적.
- 정정 4 핵심 가드: `gemma_backend.generate.assert_not_called()`
"""
from __future__ import annotations
import asyncio
import os
import sys
from dataclasses import dataclass
from unittest.mock import AsyncMock
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "app"))
# ── 가짜 evidence (synthesize 의 no_evidence 분기 회피용 최소 객체) ─────────
@dataclass
class _FakeEvidence:
n: int = 1
doc_id: int = 100
chunk_id: int | None = 200
title: str | None = "fake doc"
span_text: str = "이것은 짧은 근거 텍스트입니다."
source: str = "llm"
def _make_evidence():
return [_FakeEvidence()]
# ── backend mock ───────────────────────────────────────────────────────────
def _gemma_mock(content: str = "GEMMA_SHOULD_NEVER_BE_CALLED"):
m = AsyncMock()
m.name = "gemma-macmini"
m.generate = AsyncMock(return_value=content)
return m
def _qwen_mock_success(content: str):
m = AsyncMock()
m.name = "qwen-macbook"
m.generate = AsyncMock(return_value=content)
return m
def _qwen_mock_unavailable():
from services.llm import BackendUnavailable
m = AsyncMock()
m.name = "qwen-macbook"
m.generate = AsyncMock(
side_effect=BackendUnavailable("qwen-macbook", "ConnectError")
)
return m
# ── 공통 fixture: synthesis_service 에 mock backend 주입 ───────────────────
@pytest.fixture
def patched_backends(monkeypatch):
"""services.llm.get_backend 를 mock dispatcher 로 치환.
Returns (gemma_mock, qwen_mock, set_qwen_unavailable_fn).
"""
from services.search import synthesis_service
gemma = _gemma_mock()
qwen_holder = {"backend": _qwen_mock_success(
'{"answer":"Qwen ok [1]","confidence":"high","refused":false}'
)}
def _fake_get_backend(name: str | None):
key = (name or "").strip().lower() or "gemma-macmini"
if key == "gemma-macmini":
return gemma
if key == "qwen-macbook":
return qwen_holder["backend"]
raise ValueError(f"unknown backend: {name!r}")
monkeypatch.setattr(synthesis_service, "get_backend", _fake_get_backend)
# synthesis_service 캐시 비움 (qwen vs gemma 캐시 분리 invariant)
synthesis_service._CACHE.clear()
def _swap_qwen_unavailable():
qwen_holder["backend"] = _qwen_mock_unavailable()
return gemma, qwen_holder, _swap_qwen_unavailable
# ── 정정 4 핵심: backend=qwen-macbook + MacBook 비가용 → Gemma 호출 0 ─────
def test_qwen_unavailable_yields_backend_unavailable_status_and_gemma_not_called(
patched_backends,
):
"""**정정 4 의 핵심 invariant**.
backend="qwen-macbook" 명시 + Qwen 호출이 BackendUnavailable 실패
synthesize() status="backend_unavailable" 반환. Gemma backend
generate() ** 번도 호출되지 않음** (silent fallback 금지).
"""
from services.search.synthesis_service import synthesize
gemma, qwen_holder, swap_qwen_unavailable = patched_backends
swap_qwen_unavailable()
qwen = qwen_holder["backend"]
result = asyncio.run(
synthesize(
query="압력용기 최대허용응력은?",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
# 1. status
assert result.status == "backend_unavailable"
assert result.answer is None
assert result.confidence is None
assert result.refused is False
# 2. flag 에 backend 비가용 사유 기록
assert any(
f.startswith("backend_unavailable:qwen-macbook:") for f in result.hallucination_flags
), f"expected backend_unavailable flag, got {result.hallucination_flags}"
# 3. ★ 핵심 가드 ★ — Gemma backend 자동 fallback 금지
gemma.generate.assert_not_called()
# 4. Qwen 은 1회만 호출 (재시도 없음)
assert qwen.generate.call_count == 1
def test_qwen_unavailable_result_not_cached(patched_backends):
"""비가용 결과는 캐시 X — 다음 호출이 다시 Qwen 시도해야 함."""
from services.search.synthesis_service import synthesize
gemma, qwen_holder, swap_qwen_unavailable = patched_backends
swap_qwen_unavailable()
qwen = qwen_holder["backend"]
asyncio.run(
synthesize(
query="동일 쿼리",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
asyncio.run(
synthesize(
query="동일 쿼리",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
# 두 번 모두 실제 호출 (캐시 적중 X) — Gemma 는 여전히 0
assert qwen.generate.call_count == 2
gemma.generate.assert_not_called()
# ── 정정 4: backend 미지정 → 기존 Gemma path (회귀 0) ─────────────────────
def test_default_backend_calls_gemma_not_qwen(patched_backends):
"""backend 미지정 = 기본 Gemma. Qwen 호출 0."""
from services.search.synthesis_service import synthesize
gemma, qwen_holder, _ = patched_backends
qwen = qwen_holder["backend"]
gemma.generate.return_value = (
'{"answer":"Gemma 답변 [1]","confidence":"high","refused":false}'
)
result = asyncio.run(
synthesize(
query="기본 호출",
evidence=_make_evidence(),
backend=None, # 명시 None = default
)
)
assert result.status == "completed"
assert result.answer is not None and "Gemma" in result.answer
# Qwen 은 호출 0
qwen.generate.assert_not_called()
# Gemma 는 1회
assert gemma.generate.call_count == 1
# ── backend="qwen-macbook" + 정상 응답 ──────────────────────────────────────
def test_qwen_success_does_not_call_gemma(patched_backends):
"""Qwen 정상 응답 시 Gemma 는 호출되지 않음 (대칭 invariant)."""
from services.search.synthesis_service import synthesize
gemma, qwen_holder, _ = patched_backends
qwen = qwen_holder["backend"]
result = asyncio.run(
synthesize(
query="정상 호출",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
assert result.status == "completed"
assert result.answer is not None and "Qwen" in result.answer
# Gemma 는 0회
gemma.generate.assert_not_called()
# Qwen 은 1회
assert qwen.generate.call_count == 1
# ── 캐시 분리 (qwen vs gemma 키 충돌 없음) ─────────────────────────────────
def test_qwen_and_gemma_have_separate_caches(patched_backends):
"""같은 query 라도 backend 다르면 캐시 분리 — Qwen 결과가 Gemma 호출 답으로 둔갑하지 않음."""
from services.search.synthesis_service import synthesize
gemma, qwen_holder, _ = patched_backends
qwen = qwen_holder["backend"]
gemma.generate.return_value = (
'{"answer":"GEMMA_ANSWER [1]","confidence":"high","refused":false}'
)
qwen.generate.return_value = (
'{"answer":"QWEN_ANSWER [1]","confidence":"high","refused":false}'
)
r_qwen_1 = asyncio.run(
synthesize(
query="같은 query",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
r_gemma_1 = asyncio.run(
synthesize(
query="같은 query",
evidence=_make_evidence(),
backend=None,
)
)
r_qwen_2 = asyncio.run(
synthesize(
query="같은 query",
evidence=_make_evidence(),
backend="qwen-macbook",
)
)
assert "QWEN_ANSWER" in (r_qwen_1.answer or "")
assert "GEMMA_ANSWER" in (r_gemma_1.answer or "")
# 두 번째 Qwen 호출은 캐시 적중 — 결과는 동일하지만 generate 추가 호출 X
assert "QWEN_ANSWER" in (r_qwen_2.answer or "")
assert r_qwen_2.cache_hit is True
# generate 호출 횟수: Qwen 1 (두번째는 캐시), Gemma 1
assert qwen.generate.call_count == 1
assert gemma.generate.call_count == 1
-218
View File
@@ -1,218 +0,0 @@
"""PR-DocSrv-Ask-ToolCalling-ReAct-1: /api/search/ask/react endpoint integration.
검증 항목 (G0-3 trace exposure + 정정 4 invariant):
- backend unavailable HTTP 503 + error_reason=macbook_unavailable
+ `run_search` mock 호출 횟수 == 0 (search 단계 진입 자체 차단)
- 정상 응답 200 + final_answer + sources + debug_trace=null (default)
- debug=true debug_trace 채워짐
- max rounds 도달 iterations=2 + partial=false (final content 정상)
endpoint 함수 (`api.search.ask_react`) 직접 호출하는 lightweight 패턴.
TestClient 없이 FastAPI deps MagicMock 으로 우회. (priority_gate / backend_dispatcher
test 동일 service-layer 패턴.)
"""
from __future__ import annotations
import asyncio
import json
import os
import sys
from unittest.mock import AsyncMock, MagicMock
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "app"))
# ── helpers ────────────────────────────────────────────────────────────────
def _msg_with_tool_call(q: str, tc_id: str = "tc-1") -> dict:
return {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": tc_id,
"type": "function",
"function": {
"name": "search",
"arguments": json.dumps({"q": q}, ensure_ascii=False),
},
}
],
}
def _msg_with_content(text: str) -> dict:
return {"role": "assistant", "content": text, "tool_calls": None}
def _fake_chunk(chunk_id: int, doc_id: int = 100):
m = MagicMock()
m.id = chunk_id
m.chunk_id = chunk_id
m.doc_id = doc_id
m.title = f"doc {doc_id}"
m.score = 0.9
m.snippet = f"snippet {chunk_id}"
m.text = None
return m
def _fake_pr(chunks: list):
pr = MagicMock()
pr.results = chunks
return pr
@pytest.fixture
def patched_backend_and_search(monkeypatch):
"""get_backend + run_search 둘 다 mock. backend 의 generate_with_tools 는
테스트가 side_effect 설정.
Returns: (backend_mock, run_search_mock, set_backend_unavailable_fn).
"""
from services.llm.backends import BackendUnavailable, QwenMacBookBackend
from services.llm import backends as backends_mod
from services.search import react_loop
backend = MagicMock(spec=QwenMacBookBackend)
backend.name = "qwen-macbook"
backend.generate_with_tools = AsyncMock()
def _fake_get_backend(name):
# endpoint 가 qwen-macbook 만 호출하므로 단일 backend 반환
return backend
monkeypatch.setattr(backends_mod, "get_backend", _fake_get_backend)
# search.py 의 ask_react 안에서 `from services.llm.backends import ... get_backend`
# 로 import 하므로 module-level patch 만으로 충분 (지연 import 라 매번 fresh).
run_search_mock = AsyncMock(return_value=_fake_pr([_fake_chunk(1)]))
monkeypatch.setattr(react_loop, "run_search", run_search_mock)
def _make_unavailable():
backend.generate_with_tools.side_effect = BackendUnavailable(
"qwen-macbook", "ConnectError"
)
return backend, run_search_mock, _make_unavailable
def _call_endpoint(payload):
"""ask_react 를 직접 호출. user/session 은 MagicMock 으로 우회."""
from api.search import ask_react
user = MagicMock()
session = MagicMock()
return asyncio.run(ask_react(payload, user=user, session=session))
# ── ★ 정정 4 invariant: backend unavailable → 503 + run_search 호출 0 ──────
def test_qwen_unavailable_returns_503(patched_backend_and_search):
"""backend BackendUnavailable → HTTP 503 + error_reason=macbook_unavailable."""
from api.search import AskReactRequest
backend, run_search_mock, make_unavailable = patched_backend_and_search
make_unavailable()
response = _call_endpoint(AskReactRequest(query="Q"))
# JSONResponse instance
assert response.status_code == 503
body = json.loads(response.body)
assert body["error_reason"] == "macbook_unavailable"
assert body["backend_used"] is None
assert body["backend_requested"] == "qwen-macbook"
# ★ run_search 호출 0 (search 진입 자체 차단)
assert run_search_mock.call_count == 0
# ── 정상 200 + G0-3 default debug_trace=null ──────────────────────────────
def test_successful_response_default_no_debug_trace(patched_backend_and_search):
"""debug 미지정 (default false) → 200 + debug_trace == null."""
from api.search import AskReactRequest, AskReactResponse
backend, run_search_mock, _ = patched_backend_and_search
backend.generate_with_tools.side_effect = [
_msg_with_tool_call("q1"),
_msg_with_content("최종 답입니다"),
]
response = _call_endpoint(AskReactRequest(query="Q"))
# Pydantic instance (FastAPI response_model 적용 전 raw return)
assert isinstance(response, AskReactResponse)
assert response.final_answer == "최종 답입니다"
assert response.iterations == 2
assert response.partial is False
assert response.debug_trace is None # ★ G0-3
assert len(response.sources) == 1
# ── G0-3: debug=true → debug_trace 채워짐 ──────────────────────────────────
def test_debug_true_populates_trace(patched_backend_and_search):
from api.search import AskReactRequest
backend, run_search_mock, _ = patched_backend_and_search
backend.generate_with_tools.side_effect = [
_msg_with_content("바로 답"),
]
response = _call_endpoint(AskReactRequest(query="Q", debug=True))
assert response.debug_trace is not None
assert isinstance(response.debug_trace, list)
assert len(response.debug_trace) >= 1
# ── max rounds → final content 정상 → partial=false ──────────────────────
def test_max_rounds_with_final_content(patched_backend_and_search):
from api.search import AskReactRequest
backend, run_search_mock, _ = patched_backend_and_search
backend.generate_with_tools.side_effect = [
_msg_with_tool_call("q1"),
_msg_with_tool_call("q2", tc_id="tc-2"),
_msg_with_content("정리된 최종 답"),
]
response = _call_endpoint(AskReactRequest(query="Q"))
assert response.iterations == 2
assert response.partial is False
assert response.final_answer == "정리된 최종 답"
# LLM 호출 3회, search 2회 (G0-2 cap)
assert backend.generate_with_tools.call_count == 3
assert run_search_mock.call_count == 2
# ── max rounds + final content 빈 string → partial=true ──────────────────
def test_max_rounds_with_empty_final_partial(patched_backend_and_search):
from api.search import AskReactRequest
backend, run_search_mock, _ = patched_backend_and_search
backend.generate_with_tools.side_effect = [
_msg_with_tool_call("q1"),
_msg_with_tool_call("q2", tc_id="tc-2"),
_msg_with_content(""),
]
response = _call_endpoint(AskReactRequest(query="Q"))
assert response.iterations == 2
assert response.partial is True
assert response.final_answer == ""
-92
View File
@@ -1,92 +0,0 @@
"""Phase 3.5 fix2: /ask 의 X-Source / X-Eval-Case-Id trust boundary.
`_resolve_eval_identity()` 단위 테스트.
- token 없음/틀림 + X-Source=eval source='document_server', eval_case_id=None
- token 일치 + X-Source=eval + X-Eval-Case-Id=case_xxx ('eval', 'case_xxx')
- token 틀림 + X-Eval-Case-Id (X-Source 미지정) eval_case_id=None
- 일반 호출 (X-Source=ui_search, no eval headers) ('ui_search', None)
- env 미설정 (eval_runner_token='') 모든 eval claim 거부
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
import pytest
@pytest.fixture
def resolve_with_token(monkeypatch):
"""settings.eval_runner_token 을 monkey-patch 해서 _resolve_eval_identity 테스트."""
def _make(token: str):
from core import config as cfg_mod
from api import search as search_mod
# 두 모듈 모두에서 settings 객체 참조하므로 직접 attr 변경
monkeypatch.setattr(search_mod.settings, "eval_runner_token", token)
return search_mod._resolve_eval_identity
return _make
def test_no_token_no_eval_headers_default(resolve_with_token):
"""일반 호출 — eval 헤더 없음, source 기본값."""
resolve = resolve_with_token("secret123")
assert resolve(None, None, None) == ("document_server", None)
def test_normal_source_with_token(resolve_with_token):
"""ui_search 호출 — eval 클레임 아님이라 token 무관."""
resolve = resolve_with_token("secret123")
assert resolve("ui_search", None, None) == ("ui_search", None)
def test_eval_claim_no_token_rejected(resolve_with_token):
"""X-Source=eval 인데 token 없음 → 거부, source='document_server'."""
resolve = resolve_with_token("secret123")
assert resolve("eval", "case_001", None) == ("document_server", None)
def test_eval_claim_wrong_token_rejected(resolve_with_token):
"""token 틀림 → 거부."""
resolve = resolve_with_token("secret123")
assert resolve("eval", "case_001", "wrong_token") == ("document_server", None)
def test_eval_claim_correct_token_accepted(resolve_with_token):
"""token 일치 → 'eval' source + case_id 적재."""
resolve = resolve_with_token("secret123")
assert resolve("eval", "case_001", "secret123") == ("eval", "case_001")
def test_eval_case_id_only_no_source_no_token(resolve_with_token):
"""X-Eval-Case-Id 만 있고 token 없음 → 거부, case_id=None."""
resolve = resolve_with_token("secret123")
assert resolve(None, "case_001", None) == ("document_server", None)
def test_eval_case_id_only_wrong_token(resolve_with_token):
"""X-Eval-Case-Id 만 + token 틀림 → 거부."""
resolve = resolve_with_token("secret123")
assert resolve(None, "case_001", "wrong") == ("document_server", None)
def test_env_unset_rejects_even_correct_format(resolve_with_token):
"""settings.eval_runner_token='' 인 환경 → 모든 eval 클레임 거부."""
resolve = resolve_with_token("")
# token 헤더가 와도 server side 가 비어있으면 거부 (constant-time False)
assert resolve("eval", "case_001", "") == ("document_server", None)
assert resolve("eval", "case_001", "anything") == ("document_server", None)
def test_non_eval_source_forces_case_id_none(resolve_with_token):
"""X-Source=ui_detail + X-Eval-Case-Id (실수로 같이 보냄) → case_id=None.
eval claim 아님 (source != 'eval' 이고 case_id fallback 으로 eval claim 트리거)
이지만 source claim 명시적으로 non-eval 이라 token 검증 case_id None.
"""
resolve = resolve_with_token("secret123")
# case_id 가 있으면 eval claim 으로 처리됨 → token 없으면 거부 → ('ui_detail' 클레임,
# 하지만 거부 분기에서 claimed_source != 'eval' 이라 그대로 'ui_detail' 반환, case_id=None)
assert resolve("ui_detail", "case_001", None) == ("ui_detail", None)
-188
View File
@@ -1,188 +0,0 @@
"""Phase 3.5 B1 (fix1+fix3): unit-aware fabricated_number + bound semantics.
기준:
- 단위 일치 시에만 exact/range/tolerance clear (fix1: Codex unit-mismatch regression 방지)
- /대략/거의/얼추 approx prefix strip; 최대/최소 bound operator 보존 (fix3)
- tolerance 양적 단위(_TOLERANCE_UNITS) + 4자리+ ; 식별자성(_EXACT_ONLY_UNITS) strict
"""
from __future__ import annotations
import os
import sys
# tests/ → 프로젝트 루트 → app/
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
import pytest
from services.search.evidence_service import EvidenceItem
from services.search.grounding_check import check
def _ev(text: str, n: int = 1) -> EvidenceItem:
return EvidenceItem(
n=n,
chunk_id=None,
doc_id=100 + n,
title=f"doc{n}",
section_title=None,
span_text=text,
relevance=0.9,
rerank_score=0.85,
full_snippet=text,
source="llm",
)
def _has_fabricated(result, sub: str | None = None) -> bool:
for f in result.strong_flags:
if not f.startswith("fabricated_number:"):
continue
if sub is None or sub in f:
return True
return False
# ─── 콤마/prefix/range/단위 동의어/citation (기존 17 케이스) ──────
def test_comma_thousand_match():
r = check("질문", "총 1,000명 [1]", [_ev("총원은 1000명입니다.")])
assert not _has_fabricated(r, "1000")
def test_comma_thousand_reverse():
r = check("질문", "총 1000명 [1]", [_ev("총원은 1,000명입니다.")])
assert not _has_fabricated(r)
def test_approx_prefix_in_answer():
r = check("질문", "약 100명이 참여 [1]", [_ev("100명이 참여")])
assert not _has_fabricated(r)
def test_approx_prefix_in_evidence():
r = check("질문", "100명이 참여 [1]", [_ev("약 100명이 참여")])
assert not _has_fabricated(r)
def test_range_inner_value_passes():
r = check("질문", "약 150명 [1]", [_ev("100~200명 사이 추정")])
assert not _has_fabricated(r, "150")
def test_range_outer_value_flagged():
r = check("질문", "300명 [1]", [_ev("100~200명 사이 추정")])
assert _has_fabricated(r, "300")
def test_unit_synonym_in_to_myeong():
r = check("질문", "총 50인이 모임 [1]", [_ev("총 50명이 모임.")])
assert not _has_fabricated(r)
def test_unit_synonym_percent_to_pct():
r = check("질문", "비율 30퍼센트 [1]", [_ev("비율 30%이다.")])
assert not _has_fabricated(r)
def test_citation_marker_both_sides():
"""bug fix: evidence 측 [n] 미제거로 디지트 합쳐지던 케이스."""
r = check("질문", "가격 [1] 5,000원", [_ev("[2] 5,000원이 정확")])
assert not _has_fabricated(r)
def test_genuine_fabricated_number():
r = check("질문", "결과 777명 [1]", [_ev("500명, 300명을 받음.")])
assert _has_fabricated(r, "777")
def test_amount_4digit_tolerance_passes():
r = check("질문", "9,990원 [1]", [_ev("10,000원입니다.")])
assert not _has_fabricated(r)
def test_year_no_tolerance_flagged():
r = check("질문", "2024년 [1]", [_ev("2026년에 발효")])
assert _has_fabricated(r, "2024")
def test_article_no_tolerance_flagged():
r = check("질문", "제5조에 명시 [1]", [_ev("제6조에 따라")])
assert _has_fabricated(r)
def test_count_no_tolerance_flagged():
r = check("질문", "총 3회 위반 [1]", [_ev("총 4회 적발")])
assert _has_fabricated(r)
def test_three_digit_strict():
r = check("질문", "총 15개 [1]", [_ev("총 10개")])
assert _has_fabricated(r, "15")
def test_single_digit_ignored():
"""1자리 + 양적 단위 → 무시 (오탐 방지)."""
r = check("질문", "총 3개 발생 [1]", [_ev("관련 통계 별도")])
assert not _has_fabricated(r, "3개")
def test_range_korean_butter_separator():
r = check("질문", "약 150명 [1]", [_ev("100부터 200명까지 대상.")])
assert not _has_fabricated(r, "150")
# ─── fix1: unit-mismatch (Codex no-ship) ──────────────────
def test_won_vs_myeong_range_flagged():
"""answer '150원' vs evidence '100~200명' → 단위 불일치, flag 유지."""
r = check("질문", "약 150원이 든다 [1]", [_ev("대상은 100~200명")])
assert _has_fabricated(r, "150")
def test_won_vs_myeong_tolerance_flagged():
"""answer '9,990원' vs evidence '10,000명' → tolerance pool 단위 다름, flag 유지."""
r = check("질문", "9,990원 [1]", [_ev("10,000명입니다.")])
assert _has_fabricated(r, "9990")
def test_pct_vs_myeong_range_flagged():
"""answer '15%' vs evidence '10~20명' → 단위 불일치, flag 유지."""
r = check("질문", "약 15% [1]", [_ev("대상 10~20명")])
assert _has_fabricated(r, "15")
# ─── fix3: 최대/최소 bound semantics ───────────────────────
def test_choedae_exact_boundary_flagged():
"""evidence '최대 100명' + answer '100명' → 경계값 자체는 cleared 아님."""
r = check("질문", "100명이다 [1]", [_ev("최대 100명까지 가능")])
assert _has_fabricated(r, "100")
def test_choeso_exact_boundary_flagged():
"""evidence '최소 100명' + answer '100명' → 경계값 자체는 cleared 아님."""
r = check("질문", "100명이다 [1]", [_ev("최소 100명 이상 필요")])
assert _has_fabricated(r, "100")
def test_choedae_inner_value_passes():
"""evidence '최대 100명' + answer '50명' → bound 안, cleared."""
r = check("질문", "50명이다 [1]", [_ev("최대 100명까지 가능")])
assert not _has_fabricated(r, "50")
def test_choeso_above_value_passes():
"""evidence '최소 100명' + answer '150명' → bound 안, cleared."""
r = check("질문", "150명이다 [1]", [_ev("최소 100명 이상 필요")])
assert not _has_fabricated(r, "150")
def test_choedae_outer_value_flagged():
"""evidence '최대 100명' + answer '200명' → bound 밖, flag."""
r = check("질문", "200명이다 [1]", [_ev("최대 100명까지 가능")])
assert _has_fabricated(r, "200")
-123
View File
@@ -1,123 +0,0 @@
"""Phase 3.5 fix3: re-gate Tier 0 — synthesis 자체 실패 처리.
`_detect_synthesis_failure()` 단위 테스트.
기존 버그:
synthesis LLM self-refuse (`sr.refused=True, status="completed"`) 또는
timeout/parse_failed/llm_error grounding/verifier flag 0 re-gate else clean
분기로 빠져 `completeness="full"` 초기값이 남아 `full + refused=True` 모순.
baseline v1-400char 에서 24/223 (10.8%) 해당.
Tier 0 판정:
- LLM self-refuse (completed + refused) "synthesis_self_refuse"
- mechanical fail (timeout/parse_failed/llm_error) "synthesis_failed({status})"
- answer 공백 "synthesis_failed({status})"
- 유효 답변 None (기존 tier 1~7 경로)
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
from api.search import _detect_synthesis_failure
from services.search.synthesis_service import SynthesisResult
def _sr(
status: str = "completed",
answer: str | None = "ok",
refused: bool = False,
refuse_reason: str | None = None,
) -> SynthesisResult:
return SynthesisResult(
status=status, # type: ignore[arg-type]
answer=answer,
used_citations=[],
confidence="low",
refused=refused,
refuse_reason=refuse_reason,
elapsed_ms=100.0,
cache_hit=False,
)
# ─── self-refuse 케이스 ──────────────────────────────────
def test_llm_self_refuse_completed():
"""LLM 이 JSON 에 refused=true 반환 → synthesis_self_refuse."""
sr = _sr(status="completed", answer=None, refused=True, refuse_reason="범위 밖")
assert _detect_synthesis_failure(sr) == "synthesis_self_refuse"
def test_llm_self_refuse_with_answer_still_refused():
"""refused=True 면 answer 있어도 Tier 0 처리 (일관성)."""
sr = _sr(status="completed", answer="왜 답변함", refused=True)
assert _detect_synthesis_failure(sr) == "synthesis_self_refuse"
# ─── mechanical failure 케이스 ──────────────────────────
def test_timeout():
sr = _sr(status="timeout", answer=None, refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(timeout)"
def test_parse_failed():
sr = _sr(status="parse_failed", answer=None, refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(parse_failed)"
def test_llm_error():
sr = _sr(status="llm_error", answer=None, refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(llm_error)"
def test_refused_with_mechanical_fail_propagates_status():
"""refused=True + status!=completed → synthesis_failed({status}) 형식."""
sr = _sr(status="timeout", answer=None, refused=True)
assert _detect_synthesis_failure(sr) == "synthesis_failed(timeout)"
# ─── empty answer 케이스 ───────────────────────────────
def test_empty_answer_completed():
"""status=completed 인데 answer 공백 → synthesis_failed(completed)."""
sr = _sr(status="completed", answer="", refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(completed)"
def test_whitespace_only_answer():
"""공백/탭/개행만 있어도 empty 로 간주."""
sr = _sr(status="completed", answer=" \n\t ", refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(completed)"
def test_none_answer_completed():
"""answer=None + status=completed → failed."""
sr = _sr(status="completed", answer=None, refused=False)
assert _detect_synthesis_failure(sr) == "synthesis_failed(completed)"
# ─── 유효 답변 케이스 (None 반환) ──────────────────────
def test_valid_answer_returns_none():
"""status=completed + answer 있고 refused=False → Tier 0 통과 (None)."""
sr = _sr(status="completed", answer="교육 시간은 매년 6시간 이상이다 [1].", refused=False)
assert _detect_synthesis_failure(sr) is None
def test_skipped_status_with_answer_passes():
"""status=skipped 는 Tier 0 대상 아님 — 초기 refusal gate 에서 이미 early-return 처리됨.
(skipped 여기까지 도달하지 않는다는 전제. 만약 도달하더라도 refused True .)
"""
sr = _sr(status="skipped", answer="abc", refused=False)
# 이 경우 Tier 0 미발동 (answer 있고 refused 아님) — 정상 경로로 나감.
assert _detect_synthesis_failure(sr) is None
-58
View File
@@ -1,58 +0,0 @@
"""Phase 3.5 B2: verifier _SEVERITY_MAP env flag 테스트.
VERIFIER_NUMERIC_PROMOTE 환경변수에 따른 _SEVERITY_MAP 변화 검증.
모듈은 import time env 평가하므로 reload 필요.
"""
from __future__ import annotations
import importlib
import os
import sys
# tests/ → 프로젝트 루트 → app/
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
import pytest
def _reload_verifier(monkeypatch, value: str | None):
"""env 설정 후 verifier_service 를 reload 하여 _SEVERITY_MAP 재평가."""
if value is None:
monkeypatch.delenv("VERIFIER_NUMERIC_PROMOTE", raising=False)
else:
monkeypatch.setenv("VERIFIER_NUMERIC_PROMOTE", value)
from services.search import verifier_service
importlib.reload(verifier_service)
return verifier_service
def test_severity_map_off_default(monkeypatch):
"""env 미설정 → numeric_conflict critical 은 medium (기존 동작)."""
vs = _reload_verifier(monkeypatch, None)
assert vs._SEVERITY_MAP["numeric_conflict"]["critical"] == "medium"
assert vs._SEVERITY_MAP["numeric_conflict"]["minor"] == "medium"
assert vs._NUMERIC_PROMOTE is False
def test_severity_map_on_critical_promoted(monkeypatch):
"""VERIFIER_NUMERIC_PROMOTE=1 → critical 만 strong, minor 는 medium 유지."""
vs = _reload_verifier(monkeypatch, "1")
assert vs._SEVERITY_MAP["numeric_conflict"]["critical"] == "strong"
assert vs._SEVERITY_MAP["numeric_conflict"]["minor"] == "medium"
assert vs._NUMERIC_PROMOTE is True
def test_severity_map_off_explicit_zero(monkeypatch):
"""VERIFIER_NUMERIC_PROMOTE=0 명시 → off (default 와 동일)."""
vs = _reload_verifier(monkeypatch, "0")
assert vs._SEVERITY_MAP["numeric_conflict"]["critical"] == "medium"
assert vs._NUMERIC_PROMOTE is False
def test_direct_negation_invariant(monkeypatch):
"""direct_negation 은 env 무관 항상 strong (불변 — 안전장치)."""
for value in [None, "0", "1"]:
vs = _reload_verifier(monkeypatch, value)
assert vs._SEVERITY_MAP["direct_negation"]["critical"] == "strong"
assert vs._SEVERITY_MAP["direct_negation"]["minor"] == "strong"