6785d53d3d
Phase 4-A 가 wrong/unsure 한 문제씩 풀이 캐시. 4-B 는 세션 전체 wrong/unsure
5~30건을 묶어 200~400자 자연어 요약 1건 생성. 결과 화면 헤더 카드.
큐 인프라는 4-A study_question_jobs 와 분리 — FK 단일 의미 + 운영 SQL 명확성
+ 4-A/4-B 가드/payload/재시도 정책 차이. 신규 study_quiz_session_jobs (큐) +
study_quiz_session_analysis (결과 캐시 PK=session_id, UPSERT) + 전용 consumer.
Backend:
- migrations/233 — study_quiz_session_jobs (FK study_quiz_sessions NOT NULL,
status pending/processing/completed/failed/skipped, max_attempts=2)
- migrations/234 — partial unique idx (session_id) WHERE pending/processing
- migrations/235 — study_quiz_session_analysis (session_id PK, summary_md,
confidence, model_name, generated_at, is_stale)
- models/study_quiz_session_job — ORM + enqueue_session_analysis_job() (멱등)
- models/study_quiz_session_analysis — ORM (PK = session_id)
- services/study/session_summary_guard — GUARD_PATTERN (정규식) +
normalize_confidence() 단일 source, worker + tests 가 import 공유
- services/study/session_summary_rag — gather_session_summary_context()
documents 만 (PR-3 _gather_document_evidence 재사용). evidence 없어도 호출
허용 (4-A 와 다른 정책 — 세션 기록 자체가 evidence)
- services/study/session_analysis_enqueue — auto (finalize/fallback) +
request_session_analysis_regenerate (manual). manual 은 wrong/unsure < 5
즉시 차단, active job 차단, 기존 analysis 있으면 is_stale=true 박기
- prompts/study_session_summary_envelope.txt — envelope JSON
{summary_md, confidence}. 정량 정수만 인용 가능, 비율/추세/범위/날짜 금지
- workers/study_session_analysis_worker — terminal status 분기:
· wrong/unsure < 5 → status=skipped, error_code=insufficient_attempts
· question_text/outcome 부족 → skipped, evidence_missing
· GUARD_PATTERN match → failed, guard_fail
· 800자 hard cap + confidence normalize
· timeout/parse/unknown → 재시도 후보
· UPSERT study_quiz_session_analysis ON CONFLICT DO UPDATE (PK session_id)
- workers/study_session_queue_consumer — 4-A consumer 패턴 복제. BATCH_SIZE=1
+ STALE_MINUTES=10. MLX gate 4-A 와 공유 (Semaphore(1))
- main.py — APScheduler add_job(consume_study_session_queue, ..., 1분 주기)
- session_finalize — 끝에서 enqueue_session_analysis_auto (best-effort)
- api/study_topics:
· QuizSessionAnalysisOut + ai_session_analysis 응답 필드 (analysis row +
최신 job status/error_code)
· GET fallback enqueue (기존 analysis 또는 active job 없으면만, non-blocking)
· POST /quiz-sessions/{sid}/regenerate-summary — manual 트리거
Frontend (quiz-sessions/[sid]/+page.svelte):
- 결과 헤더에 세션 요약 카드 (AI 풀이 indicator 직후, 바로 할 일 직전)
- summary_md 박혔으면 markdown 렌더, 없으면 job_status / error_code 분기:
· pending/processing → "AI 가 세션 분석 중"
· insufficient_attempts → "오답·모르겠음 5건 미만"
· evidence_missing → "자료 부족"
· guard_fail → "환각 검증 차단" + 재생성 링크
- confidence='low' 배지 + is_stale "재생성 중" 배지
- 재생성 버튼 + regenerateSummary() — reason 별 toast 분기
ship gate:
- tests/test_session_summary_guard_pattern.py — 허용 5 + 차단 7 케이스 +
normalize_confidence 표준/비표준 검증. python3 직접 실행 패스.
Plan: ~/.claude/plans/nifty-sparking-spindle.md (Phase 4-B v1)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
203 lines
9.5 KiB
Python
203 lines
9.5 KiB
Python
"""hyungi_Document_Server — FastAPI 엔트리포인트"""
|
|
|
|
from contextlib import asynccontextmanager
|
|
|
|
from fastapi import FastAPI, Request
|
|
from fastapi.responses import RedirectResponse
|
|
from sqlalchemy import func, select, text
|
|
|
|
from api.audio import router as audio_router
|
|
from api.auth import router as auth_router
|
|
from api.config import router as config_router
|
|
from api.dashboard import router as dashboard_router
|
|
from api.digest import router as digest_router
|
|
from api.document_notes import router as document_notes_router
|
|
from api.document_reads import router as document_reads_router
|
|
from api.documents import router as documents_router
|
|
from api.library import router as library_router
|
|
from api.memos import router as memos_router
|
|
from api.news import router as news_router
|
|
from api.search import router as search_router
|
|
from api.setup import router as setup_router
|
|
from api.study_question_progress import router as study_question_progress_router
|
|
from api.study_questions import router as study_questions_router
|
|
from api.study_sessions import router as study_sessions_router
|
|
from api.study_topics import router as study_topics_router
|
|
from api.video import router as video_router
|
|
from core.config import settings
|
|
from core.database import async_session, engine, init_db
|
|
from models.user import User
|
|
|
|
|
|
@asynccontextmanager
|
|
async def lifespan(app: FastAPI):
|
|
"""앱 시작/종료 시 실행되는 lifespan 핸들러"""
|
|
import asyncio
|
|
|
|
from apscheduler.schedulers.asyncio import AsyncIOScheduler
|
|
from apscheduler.triggers.cron import CronTrigger
|
|
from services.search.query_analyzer import prewarm_analyzer
|
|
from workers.daily_digest import run as daily_digest_run
|
|
from workers.digest_worker import run as global_digest_run
|
|
from workers.file_watcher import watch_inbox
|
|
from workers.law_monitor import run as law_monitor_run
|
|
from workers.mailplus_archive import run as mailplus_run
|
|
from workers.news_collector import run as news_collector_run
|
|
from workers.queue_consumer import consume_queue
|
|
from workers.study_queue_consumer import consume_study_queue
|
|
from workers.study_session_queue_consumer import consume_study_session_queue
|
|
from workers.study_question_embed_worker import (
|
|
refresh_stale_related as study_q_related_refresh,
|
|
run as study_q_embed_run,
|
|
)
|
|
from workers.tier_backfill import run as tier_backfill_run
|
|
from workers.upload_cleanup import cleanup_orphan_uploads
|
|
|
|
# 시작: DB 연결 확인
|
|
await init_db()
|
|
|
|
# NAS 마운트 확인 (NFS 미마운트 시 로컬 빈 디렉토리에 쓰는 것 방지)
|
|
from pathlib import Path
|
|
nas_check = Path(settings.nas_mount_path) / "PKM"
|
|
if not nas_check.is_dir():
|
|
raise RuntimeError(
|
|
f"NAS 마운트 확인 실패: {nas_check} 디렉토리 없음. "
|
|
f"NFS 마운트 상태를 확인하세요."
|
|
)
|
|
|
|
# APScheduler: 백그라운드 작업
|
|
scheduler = AsyncIOScheduler(timezone="Asia/Seoul")
|
|
# 상시 실행
|
|
scheduler.add_job(consume_queue, "interval", minutes=1, id="queue_consumer")
|
|
scheduler.add_job(watch_inbox, "interval", minutes=5, id="file_watcher")
|
|
scheduler.add_job(cleanup_orphan_uploads, "interval", minutes=10, id="upload_cleanup")
|
|
# PR-4: study_questions 자동 임베딩 (status='none/failed/stale' 행을 batch=10 처리).
|
|
# 별도 큐 테이블 없이 status 자체가 큐. backfill 도 cron 이 'none' 행을 자연스럽게 처리.
|
|
scheduler.add_job(study_q_embed_run, "interval", minutes=1, id="study_q_embed")
|
|
# PR-12-A 후속: related-types 캐시 stale 행 재계산. 임베딩 워커와 분리한 별도 cron.
|
|
# 새 문제 ready / 같은 토픽 invalidation / 임계값 변경 시 NULL 마킹된 행을 batch=20 처리.
|
|
scheduler.add_job(study_q_related_refresh, "interval", minutes=1, id="study_q_related_refresh")
|
|
# Phase 4-A: study_question_jobs 처리 — wrong/unsure AI 풀이 prefetch.
|
|
# MLX gate 직렬화 + BATCH_SIZE=1 로 GPU 부하 통제. STALE_MINUTES=10 자체 복구.
|
|
scheduler.add_job(consume_study_queue, "interval", minutes=1, id="study_queue_consumer")
|
|
# Phase 4-B v1: study_quiz_session_jobs 처리 — 세션 단위 자유 마크다운 분석.
|
|
# 4-A 와 같은 MLX gate 공유 — 4-A 처리 중이면 직렬 대기.
|
|
scheduler.add_job(consume_study_session_queue, "interval", minutes=1, id="study_session_queue_consumer")
|
|
# PR-B 레거시 tier 백필 — 30분 주기로 호출되지만 KST 00:00~06:00 시간대만 실제 enqueue.
|
|
# safety > law > manual 우선순위로 25건씩. 6720 레거시 → 야간당 ~150건 → 약 45일 소화.
|
|
scheduler.add_job(tier_backfill_run, "interval", minutes=30, id="tier_backfill")
|
|
# 일일 스케줄 (KST)
|
|
scheduler.add_job(law_monitor_run, CronTrigger(hour=7), id="law_monitor")
|
|
scheduler.add_job(mailplus_run, CronTrigger(hour=7), id="mailplus_morning")
|
|
scheduler.add_job(mailplus_run, CronTrigger(hour=18), id="mailplus_evening")
|
|
scheduler.add_job(daily_digest_run, CronTrigger(hour=20), id="daily_digest")
|
|
scheduler.add_job(global_digest_run, CronTrigger(hour=4, minute=0), id="global_digest")
|
|
scheduler.add_job(news_collector_run, "interval", hours=6, id="news_collector")
|
|
scheduler.start()
|
|
|
|
# Phase 2.1 (async 구조): QueryAnalyzer prewarm.
|
|
# 대표 쿼리 15~20개를 background task로 분석해 cache 적재.
|
|
# 첫 사용자 요청부터 cache hit rate 70~80% 목표.
|
|
# 논블로킹 — startup을 막지 않음. MLX 부하 완화 위해 delay_between=0.5.
|
|
prewarm_task = asyncio.create_task(prewarm_analyzer())
|
|
prewarm_task.add_done_callback(
|
|
lambda t: t.exception() and None # 예외는 query_analyzer 내부에서 로깅
|
|
)
|
|
|
|
yield
|
|
|
|
# 종료: 스케줄러 → DB 순서로 정리
|
|
scheduler.shutdown(wait=False)
|
|
await engine.dispose()
|
|
|
|
|
|
app = FastAPI(
|
|
title="hyungi_Document_Server",
|
|
description="Self-hosted PKM 웹 애플리케이션 API",
|
|
version="2.0.0",
|
|
lifespan=lifespan,
|
|
)
|
|
|
|
# ─── 라우터 등록 ───
|
|
app.include_router(setup_router, prefix="/api/setup", tags=["setup"])
|
|
app.include_router(config_router, prefix="/api/config", tags=["config"])
|
|
app.include_router(auth_router, prefix="/api/auth", tags=["auth"])
|
|
app.include_router(documents_router, prefix="/api/documents", tags=["documents"])
|
|
# 회독 카운트 — /api/documents/{id}/read* 경로. documents_router 와 prefix 같아 충돌 없음.
|
|
app.include_router(document_reads_router, prefix="/api/documents", tags=["document-reads"])
|
|
app.include_router(document_notes_router, prefix="/api/documents", tags=["document-notes"])
|
|
app.include_router(search_router, prefix="/api/search", tags=["search"])
|
|
|
|
app.include_router(memos_router, prefix="/api/memos", tags=["memos"])
|
|
app.include_router(dashboard_router, prefix="/api/dashboard", tags=["dashboard"])
|
|
app.include_router(library_router, prefix="/api/library", tags=["library"])
|
|
app.include_router(news_router, prefix="/api/news", tags=["news"])
|
|
app.include_router(digest_router, prefix="/api/digest", tags=["digest"])
|
|
app.include_router(audio_router, prefix="/api/audio", tags=["audio"])
|
|
app.include_router(video_router, prefix="/api/video", tags=["video"])
|
|
app.include_router(study_sessions_router, prefix="/api/study-sessions", tags=["study-sessions"])
|
|
app.include_router(study_topics_router, prefix="/api/study-topics", tags=["study-topics"])
|
|
# study_questions: 라우터 안에서 /study-topics/{id}/questions 와 /study-questions/{id} 두 줄기를 모두 정의하므로 prefix=/api 로 등록
|
|
app.include_router(study_questions_router, prefix="/api", tags=["study-questions"])
|
|
# Phase 1: 학습 진행 상태 (review-complete + review-queue). prefix=/api/study-topics 안에 정의됨.
|
|
app.include_router(study_question_progress_router, prefix="/api", tags=["study-progress"])
|
|
|
|
# TODO: Phase 5에서 추가
|
|
# app.include_router(tasks.router, prefix="/api/tasks", tags=["tasks"])
|
|
# app.include_router(export.router, prefix="/api/export", tags=["export"])
|
|
|
|
|
|
# ─── 셋업 미들웨어: 유저 0명이면 /setup으로 리다이렉트 ───
|
|
SETUP_BYPASS_PREFIXES = (
|
|
"/api/setup", "/api/config", "/setup", "/health", "/docs", "/openapi.json", "/redoc",
|
|
)
|
|
|
|
|
|
@app.middleware("http")
|
|
async def setup_redirect_middleware(request: Request, call_next):
|
|
path = request.url.path
|
|
# 바이패스 경로는 항상 통과
|
|
if any(path.startswith(p) for p in SETUP_BYPASS_PREFIXES):
|
|
return await call_next(request)
|
|
|
|
# 유저 존재 여부 확인
|
|
try:
|
|
async with async_session() as session:
|
|
result = await session.execute(select(func.count(User.id)))
|
|
user_count = result.scalar()
|
|
if user_count == 0:
|
|
return RedirectResponse(url="/setup")
|
|
except Exception:
|
|
pass # DB 연결 실패 시 통과 (health에서 확인 가능)
|
|
|
|
return await call_next(request)
|
|
|
|
|
|
# ─── 셋업 페이지 라우트 (API가 아닌 HTML 페이지) ───
|
|
@app.get("/setup")
|
|
async def setup_page_redirect(request: Request):
|
|
"""셋업 위자드 페이지로 포워딩"""
|
|
from api.setup import setup_page
|
|
from core.database import get_session
|
|
|
|
async for session in get_session():
|
|
return await setup_page(request, session)
|
|
|
|
|
|
@app.get("/health")
|
|
async def health_check():
|
|
"""헬스체크 — DB 연결 상태 포함"""
|
|
db_ok = False
|
|
try:
|
|
async with engine.connect() as conn:
|
|
await conn.execute(text("SELECT 1"))
|
|
db_ok = True
|
|
except Exception:
|
|
pass
|
|
|
|
return {
|
|
"status": "ok" if db_ok else "degraded",
|
|
"version": "2.0.0",
|
|
"database": "connected" if db_ok else "disconnected",
|
|
}
|