hyungi/hyungi_document_server
1
1
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat(category): law 카테고리 분리 — enum + backfill + classify skip

Browse Source 
- migrations/152: ALTER TYPE doc_category ADD VALUE 'law' (DDL only; PG16 단일-트랜잭션 제약상 backfill 은 별도)
- models/document.py: Enum 에 'law' 추가 (7 활성 + 3 유보)
- workers/law_monitor.py: Document(..., category='law') — 신규 유입부터 세팅
- workers/classify_worker.py: source_channel='law_monitor' early-return + 최소 필드 (ai_domain='법령', ai_tags=['법령'], importance='medium'). AI classify skip — 법령 구조 고정/외부 source of truth/자동 재수집
- scripts/backfill_category.py: law 분기 + WHERE re-target ((source_channel='law_monitor' AND category='document')) + VERIFY cat_law/law_source_count + fail 조건
- api/documents.py: default 목록 제외에 law_monitor 추가 (news 와 동일 패턴)
- api/dashboard.py: documents count FILTER 에 law_monitor 제외 (category_counts.law 는 기존 GROUP BY category 로 자동 노출)
- frontend/Sidebar.svelte: '법령 알림' 버튼 ?source=law_monitor → ?category=law (explicit category 경로가 default exclusion 을 skip)

plan: ~/.claude/plans/stateless-churning-raccoon.md
axis 원칙: category=UI 축, policy/telemetry=source_channel+ai_domain 축 (feedback_category_vs_ai_domain_axis.md)

배포 순서: push → GPU pull → compose up --build fastapi frontend → backfill --dry-run → --apply.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hyungi Ahn
2026-04-24 09:14:56 +09:00
parent 91d0fcd023
commit e88640d3d8
8 changed files with 58 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/api/dashboard.py
+1 -1
View File
@@ -136,7 +136,7 @@ async def get_dashboard(
text("""
SELECT
COUNT(*) AS total,
COUNT(*) FILTER (WHERE source_channel != 'news' AND file_type != 'note') AS documents,
COUNT(*) FILTER (WHERE source_channel NOT IN ('news', 'law_monitor') AND file_type != 'note') AS documents,
COUNT(*) FILTER (WHERE source_channel = 'memo' AND file_type = 'note') AS memos,
COUNT(*) FILTER (WHERE source_channel = 'news') AS news
FROM documents WHERE deleted_at IS NULL
app/api/documents.py
+2 -1
View File
@@ -417,9 +417,10 @@ async def list_documents(
# 명시적 카테고리 필터 — 기본 exclude 해제
query = query.where(Document.category == category)
else:
# 기본 목록: 뉴스/메모 제외 (문서함 용도)
# 기본 목록: 뉴스/메모/법령 제외 (문서함 용도)
query = query.where(
Document.source_channel != "news",
Document.source_channel != "law_monitor",
Document.file_type != "note",
)
app/models/document.py
+2 -2
View File
@@ -103,10 +103,10 @@ class Document(Base):
title: Mapped[str | None] = mapped_column(Text)
# 카테고리 (1차 진입점 — UI 탭/라우트 분기)
# 6 활성: document / library / news / memo / audio / video
# 7 활성: document / library / news / memo / audio / video / law
# 3 유보: mail / calendar / plex
category: Mapped[str | None] = mapped_column(
Enum("document", "library", "news", "memo", "audio", "video",
Enum("document", "library", "news", "memo", "audio", "video", "law",
"mail", "calendar", "plex",
name="doc_category", create_type=False)
)
app/workers/classify_worker.py
+14
View File
@@ -71,6 +71,20 @@ async def process(document_id: int, session: AsyncSession) -> None:
if not doc:
raise ValueError(f"문서 ID {document_id}를 찾을 수 없음")
# 법령은 구조 고정 + 외부 source of truth (law.go.kr) + 자동 재수집.
# AI 분류 skip, downstream(embed/chunk) 은 queue_consumer NEXT_STAGES 가 자동 chain.
# ai_domain 단일 "법령" — PR-A policy.domain_policy.yaml 에서 source_channel 기준 세분화.
if doc.source_channel == "law_monitor":
if not doc.ai_domain:
doc.ai_domain = "법령"
if not doc.ai_tags:
doc.ai_tags = ["법령"]
if not doc.importance:
doc.importance = "medium"
await session.commit()
logger.info(f"doc {document_id}: law_monitor → classify skip")
return
if not doc.extracted_text:
raise ValueError(f"문서 ID {document_id}: extracted_text가 비어있음")
app/workers/law_monitor.py
+1
View File
@@ -271,6 +271,7 @@ async def _save_law_split(
title=f"{law_name} ({proclamation_date}) {section_name}",
source_channel="law_monitor",
data_origin="work",
category="law",
user_note=note or None,
)
session.add(doc)
frontend/src/lib/components/Sidebar.svelte
+1 -1
View File
@@ -177,7 +177,7 @@
<Clock size={14} /> 최근 7일
</button>
<button
onclick={() => { const p = new URLSearchParams(); p.set('source', 'law_monitor'); goto(`/documents?${p}`, { noScroll: true }); }}
onclick={() => { const p = new URLSearchParams(); p.set('category', 'law'); goto(`/documents?${p}`, { noScroll: true }); }}
class="w-full flex items-center gap-2 px-3 py-1.5 rounded-md text-sm text-dim hover:bg-surface hover:text-text"
>
<Scale size={14} /> 법령 알림
migrations/152_category_law.sql
+8
View File
@@ -0,0 +1,8 @@
-- 152_category_law.sql
-- doc_category enum 에 'law' 추가 — 법령 문서 전용 카테고리.
-- plan: ~/.claude/plans/stateless-churning-raccoon.md
--
-- 주의: _run_migrations 는 단일 outer 트랜잭션으로 실행됨.
-- PG16 제약상 이 migration 안에서는 'law' 값을 INSERT/UPDATE 에 사용 불가.
-- 기존 206건 law_monitor 행의 category 갱신은 scripts/backfill_category.py 에서 별도 트랜잭션으로 처리.
ALTER TYPE doc_category ADD VALUE IF NOT EXISTS 'law';
scripts/backfill_category.py
+29 -8
View File
@@ -1,13 +1,21 @@
"""§1 백필 — documents.category 전체 행 채우기.
plan: luminous-sprouting-hamster.md §1
확장: stateless-churning-raccoon.md (law 카테고리 분기 + idempotent 재실행)
매핑 규칙 (category IS NULL 인 모든 행 대상):
매핑 규칙:
source_channel='law_monitor' → category='law'
source_channel='news' → category='news'
source_channel='memo' → category='memo'
file_type='note' → category='memo'
user_tags 에 '@library/' 태그 보유 → category='library'
그 외 → category='document'
대상 WHERE:
category IS NULL
OR (source_channel='law_monitor' AND category='document')
— law 신규 enum 도입 전 backfill 로 'document' 배정됐던 206건 재분류.
자동 library 전이 금지 — 기존 @library/ 태그 보유분만 'library' 로 이행.
audio/video 는 §3 이후 생성 (백필 대상 없음).
@@ -36,6 +44,7 @@ WITH classified AS (
SELECT
id,
CASE
WHEN source_channel = 'law_monitor' THEN 'law'
WHEN source_channel = 'news' THEN 'news'
WHEN source_channel = 'memo' THEN 'memo'
WHEN file_type = 'note' THEN 'memo'
@@ -49,6 +58,7 @@ WITH classified AS (
END AS target_category
FROM documents
WHERE category IS NULL
OR (source_channel = 'law_monitor' AND category = 'document')
)
SELECT target_category, COUNT(*) AS n FROM classified GROUP BY 1 ORDER BY 2 DESC;
"""
@@ -56,6 +66,7 @@ SELECT target_category, COUNT(*) AS n FROM classified GROUP BY 1 ORDER BY 2 DESC
APPLY_SQL = """
UPDATE documents
SET category = CASE
WHEN source_channel = 'law_monitor' THEN 'law'::doc_category
WHEN source_channel = 'news' THEN 'news'::doc_category
WHEN source_channel = 'memo' THEN 'memo'::doc_category
WHEN file_type = 'note' THEN 'memo'::doc_category
@@ -67,7 +78,8 @@ UPDATE documents
) THEN 'library'::doc_category
ELSE 'document'::doc_category
END
WHERE category IS NULL;
WHERE category IS NULL
OR (source_channel = 'law_monitor' AND category = 'document');
"""
VERIFY_SQL = """
@@ -82,7 +94,9 @@ SELECT
COALESCE(user_tags, '[]'::jsonb)
) AS t
WHERE t LIKE '@library/%'
)) AS has_library_tag;
)) AS has_library_tag,
(SELECT COUNT(*) FROM documents WHERE category = 'law') AS cat_law,
(SELECT COUNT(*) FROM documents WHERE source_channel = 'law_monitor') AS law_source_count;
"""
DIST_SQL = """
@@ -138,11 +152,13 @@ async def run(apply: bool) -> int:
# 4. verify
print("\n=== 백필 후 검증 ===")
row = (await session.execute(text(VERIFY_SQL))).one()
print(f" all_rows = {row.all_rows}")
print(f" categorized = {row.categorized}")
print(f" uncategorized = {row.uncategorized}")
print(f" cat_library = {row.cat_library}")
print(f" has_library_tag = {row.has_library_tag}")
print(f" all_rows = {row.all_rows}")
print(f" categorized = {row.categorized}")
print(f" uncategorized = {row.uncategorized}")
print(f" cat_library = {row.cat_library}")
print(f" has_library_tag = {row.has_library_tag}")
print(f" cat_law = {row.cat_law}")
print(f" law_source_count = {row.law_source_count}")
fail = []
if row.uncategorized != 0:
@@ -154,6 +170,11 @@ async def run(apply: bool) -> int:
f"cat_library={row.cat_library} != has_library_tag={row.has_library_tag} "
"(자동 전이 없음 정책 위반)"
)
if row.cat_law != row.law_source_count:
fail.append(
f"cat_law={row.cat_law} != law_source_count={row.law_source_count} "
"(law_monitor 전체가 category='law' 여야 함)"
)
if fail:
print("\n!! 검증 실패:")
for f in fail: