Hyungi Ahn
6fdc48e5b6
PR-A policy 레이어를 재사용하여 classify_worker 에 tier triage 경로를 추가.
Legacy ai_summary / ai_domain / ai_suggestion 은 유지 (회귀 0), tldr/bullets/
detail/inconsistencies 는 별도 필드로 분리.
Migrations (156~160):
- 156 documents: ai_tldr, ai_bullets, ai_detail_summary, ai_inconsistencies,
ai_analysis_tier 5컬럼
- 157 process_stage 에 'deep_summary' ADD VALUE 단독 (Postgres 동일 트랜잭션
제약 회피)
- 158 processing_queue.payload JSONB (envelope 전달)
- 159 analyze_events 에 tier + suppressed_reason
- 160 suppressed_reason partial index
Models/ORM:
- Document: 5컬럼 Mapped 추가
- ProcessingQueue: deep_summary enum 확장 + payload 필드, enqueue_stage 에
payload 옵션
- AnalyzeEvent: PR-A shadow 6컬럼 + PR-B tier/suppressed_reason
Workers:
- classify_worker: 기존 legacy 경로 뒤에 _run_tier_triage 추가.
- _match_subject_domain(doc, text): source_channel + 본문 keywords + ai_domain
prefix 로 PR-A policy 의 subject_domain 이름 결정 (category 매칭 금지).
- R1 TriageOutput pydantic + JSON 깨짐 fallback (triage_json_invalid).
- R2 _check_backlog_guard(): 30분 window ratio > threshold OR pending 초과면
soft escalate suppress. hard escalate 는 통과.
- R3 _slice_text_ranges(): 260k 초과 시 head 120k + mid 20k + tail 120k 3조각.
- escalate 시 EscalationEnvelope 구성 + {envelope, subject_domain} payload 로
deep_summary enqueue.
- deep_summary_worker (신규): queue payload 에서 envelope + subject_domain 읽기 →
render_26b("p3c_deep_summary", subject_domain) + MLX 호출 (llm_gate Semaphore(1)
경유) → ai_detail_summary + ai_inconsistencies 저장 + ai_analysis_tier='deep'.
_filter_inconsistencies 로 허용 kind (version_drift / procedure_conflict /
source_conflict / missing_basis) 만 통과 — 구매/계약 kind drop.
- queue_consumer: workers dict 에 deep_summary 추가 + BATCH_SIZE=1. next_stages
는 건드리지 않음 — classify → embed/chunk 는 그대로, deep_summary 는 독립 체인.
Telemetry:
- record_analyze_event: subject_domain / risk_flags / escalation_reasons /
confidence / policy_version / shadow_would_route_to / tier / escalated_to_26b /
suppressed_reason 파라미터 확장. classify/deep worker 가 mode="summary_triage"
또는 "summary_deep" 로 기록.
API:
- DocumentResponse 에 ai_tldr / ai_bullets / ai_detail_summary /
ai_inconsistencies / ai_analysis_tier 5필드 노출.
Prompts:
- classify.txt 에 DEPRECATED 주석만 추가 (파일 유지 — rollback 경로 보존).
- PR-A 의 app/prompts/policy/p3a_short_summary.txt (4B) 와 p3c_deep_summary.txt
(26B) 를 그대로 사용. 내 소유의 summary_triage.txt / summary_deep.txt 는 중복
이라 별도 커밋에서 제거하지 않고 바로 생성 전 삭제.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
59 lines
3.0 KiB
Python
59 lines
3.0 KiB
Python
"""analyze_events 테이블 ORM — POST /documents/{id}/analyze 호출 관측 (Phase E.2)
|
|
|
|
목적: 분석 failure mode 분류 (timeout / parse / llm / missing_summary) +
|
|
source 별 사용 패턴 (document_server / synology_chat / ui_search / ui_detail / eval).
|
|
단계 3 snapshot DB 설계 입력이 됨.
|
|
"""
|
|
|
|
from datetime import datetime
|
|
from typing import Any
|
|
|
|
from sqlalchemy import ARRAY, BigInteger, Boolean, DateTime, Float, ForeignKey, Integer, Text
|
|
from sqlalchemy.dialects.postgresql import JSONB
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from core.database import Base
|
|
|
|
|
|
class AnalyzeEvent(Base):
|
|
__tablename__ = "analyze_events"
|
|
|
|
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
|
|
doc_id: Mapped[int] = mapped_column(
|
|
BigInteger, ForeignKey("documents.id", ondelete="CASCADE"), nullable=False
|
|
)
|
|
user_id: Mapped[int | None] = mapped_column(
|
|
BigInteger, ForeignKey("users.id", ondelete="SET NULL")
|
|
)
|
|
mode: Mapped[str] = mapped_column(Text, default="quick", nullable=False) # quick / full / summary_triage / summary_deep / retrieval_select / synthesis
|
|
text_limit: Mapped[int | None] = mapped_column(Integer)
|
|
truncated: Mapped[bool] = mapped_column(Boolean, default=False)
|
|
layers_returned: Mapped[list[Any] | None] = mapped_column(JSONB, default=list)
|
|
cached: Mapped[bool] = mapped_column(Boolean, default=False)
|
|
latency_ms: Mapped[int | None] = mapped_column(Integer)
|
|
model_name: Mapped[str | None] = mapped_column(Text)
|
|
prompt_version: Mapped[str | None] = mapped_column(Text)
|
|
# None (success) | "timeout" | "llm" | "parse" | "missing_summary" | "no_text"
|
|
error_code: Mapped[str | None] = mapped_column(Text)
|
|
# document_server / synology_chat / ui_search / ui_detail / eval / unknown
|
|
source: Mapped[str] = mapped_column(Text, default="document_server", nullable=False)
|
|
created_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), default=datetime.now, nullable=False
|
|
)
|
|
|
|
# PR-A (migration 153) — routing shadow observability
|
|
subject_domain: Mapped[str | None] = mapped_column(Text)
|
|
risk_flags: Mapped[list[str] | None] = mapped_column(ARRAY(Text))
|
|
high_impact_task: Mapped[bool | None] = mapped_column(Boolean)
|
|
escalated_to_26b: Mapped[bool | None] = mapped_column(Boolean)
|
|
escalation_reasons: Mapped[list[str] | None] = mapped_column(ARRAY(Text))
|
|
confidence: Mapped[float | None] = mapped_column(Float)
|
|
policy_violation: Mapped[bool | None] = mapped_column(Boolean)
|
|
policy_violation_ids: Mapped[list[str] | None] = mapped_column(ARRAY(Text))
|
|
shadow_would_route_to: Mapped[str | None] = mapped_column(Text)
|
|
policy_version: Mapped[str | None] = mapped_column(Text)
|
|
|
|
# PR-B (migration 159) — 실제 호출 tier 와 R2 backlog guard 이벤트
|
|
tier: Mapped[str | None] = mapped_column(Text) # 'triage' | 'primary' | 'fallback'
|
|
suppressed_reason: Mapped[str | None] = mapped_column(Text) # 'backlog_guard(ratio=0.42,pending=7)'
|