Compare commits

88 Commits

Author SHA1 Message Date
Hyungi Ahn
7ca3abf17c feat: 뉴스 전용 페이지 + 분류 격리 + 읽음 상태
- /news 전용 페이지: 신문사 필터, 읽지않음 필터, 시간순 리스트, 미리보기
- 뉴스 분류 격리: ai_domain='News', classify 제거, embed만 등록
- is_read: 클릭 시 자동 읽음, 전체 읽음 API
- documents 목록에서 뉴스 제외 (source_channel != 'news')
- nav에 뉴스 링크 추가
- GET /api/news/articles, POST /api/news/mark-all-read

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 14:16:00 +09:00
Hyungi Ahn
cd5f1c526d fix: 상세 페이지에도 뉴스 전용 뷰어 적용 (source_channel=news → article)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 14:01:39 +09:00
Hyungi Ahn
2b457a8305 feat: 뉴스 전용 뷰어 + 카드 구분 + 설정 UI
- DocumentViewer: source_channel=news → article 전용 뷰어
  (제목/소스/날짜/요약/원문 링크 rel=noopener)
- DocumentCard: 뉴스 카드에 📰 아이콘
- settings: 뉴스 소스 관리 (목록/추가/삭제/토글/수집/마지막 시간)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 13:55:49 +09:00
Hyungi Ahn
d03fa0df37 fix: source_channel enum에 'news' 추가 (ORM 누락)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 13:41:20 +09:00
Hyungi Ahn
a6c19ef76c feat: 뉴스 자동 수집 시스템 — 6개국 신문 RSS/API
- news_sources 테이블 (소스 관리, UI 동적 제어)
- news_collector 워커: RSS(feedparser) + NYT API
  - 중복 체크: hash(title+date+source) + URL normalize
  - category 표준화, summary HTML 정제, timezone UTC
  - 30일 이내만 embed, source별 try/catch
- News API: 소스 CRUD + 수동 수집 트리거
- APScheduler: 6시간 간격 자동 수집
- 대상: 경향/아사히/NYT/르몽드/신화/슈피겔

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 13:38:07 +09:00
Hyungi Ahn
bf8efd1cd3 feat: 임베딩 모델 변경 — nomic-embed-text → bge-m3 (1024차원, 다국어)
- config.yaml: embedding model → bge-m3
- document.py: Vector(768) → Vector(1024)
- embed_worker.py: 모델 버전 업데이트
- migration 011: 벡터 컬럼 재생성 (기존 임베딩 초기화)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:49:45 +09:00
Hyungi Ahn
204c5ca99f fix: AI 요약 마크다운 렌더링 — 상세페이지는 렌더링, 카드는 기호 제거
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:36:29 +09:00
Hyungi Ahn
c885b5be27 fix: 3+4단계 — 반응형/에러분기/a11y/Synology URL
- DocumentCard: window.innerWidth → matchMedia (반응형 정확)
- documents/[id]: 로딩 상태 3분기 (loading/not_found/network)
- documents/[id]: Synology URL 하드코딩 → edit_url fallback
- DocumentCard: aria-label 추가
- Toast: aria-live 이미 적용 (1단계)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:24:32 +09:00
Hyungi Ahn
1b21d9bb53 feat: 2단계 — DEVONthink 스타일 테이블 뷰 + 카드/테이블 토글
- DocumentTable.svelte: 컬럼 정렬(stable sort), domain 색상 바, 포맷 아이콘
- 뷰 모드 토글 버튼 (카드 ↔ 테이블)
- localStorage로 뷰 모드 + 정렬 상태 기억

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:16:45 +09:00
Hyungi Ahn
3374eebfc6 fix: 프론트엔드 1단계 — XSS 수정 + Svelte 5 변환 + 필터/아이콘/a11y
- [critical] DOMPurify 적용 (FORBID_TAGS/ATTR, ALLOW_UNKNOWN_PROTOCOLS)
- [high] $: → $derived 변환 (documents/[id])
- [high] 태그/소스 필터 구현 (filterTag, filterSource)
- FormatIcon: docx/xlsx/pptx/odt/ods/odp/dwg/dxf 추가
- editTab 선언 순서 수정
- debounceTimer 미사용 변수 제거
- Toast role="status" aria-live 추가
- marked 옵션: mangle/headerIds false

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:15:02 +09:00
Hyungi Ahn
24142ea605 fix: Codex 리뷰 5건 수정 (critical 1 + high 4)
1. [critical] config.yaml → settings 객체에서 taxonomy 로드 (import crash 방지)
2. [high] ODF 변환: file_path 유지, derived_path 별도 필드 (무한 중복 방지)
3. [high] 법령 분할: 첫 장 이전 조문을 "서문"으로 보존
4. [high] Inbox: review_status 필드 분리 (pending/approved/rejected)
5. [high] 삭제: soft-delete (deleted_at) + worker 방어 + active_documents 뷰
   - 모든 조회에 deleted_at IS NULL 일관 적용
   - queue_consumer: row 없으면 gracefully skip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 07:15:13 +09:00
Hyungi Ahn
6c92e375c2 feat: Markdown 뷰어/편집기 개선
- startEdit(): extracted_text || rawMarkdown fallback
- split editor → 편집/미리보기 탭 전환 방식
- GitHub Dark 스타일 markdown-body CSS (테이블/코드/인용/리스트)
- prose 클래스 → markdown-body로 교체

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 15:48:41 +09:00
Hyungi Ahn
06da098eab fix: 법령 분할 — 조문키 000 기반 장(章) 단위 분할로 변경
국가법령 XML은 <편>/<장> 태그가 아닌 <조문단위 조문키="xxxx000">에
"제X장 ..." 형태로 장 구분자가 포함됨. 이를 파싱하여 분할.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 15:05:48 +09:00
Hyungi Ahn
749ed51dd7 fix: Markdown 뷰어 — extracted_text 없으면 원본 파일 직접 렌더링
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:55:51 +09:00
Hyungi Ahn
1668be0a75 fix: 법령 저장 후 즉시 commit — 알림 실패가 DB 롤백하지 않도록
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:50:39 +09:00
Hyungi Ahn
93c5805060 feat: 법령 API 전면 개편 — 26개 법령, 분할 저장, 변경 이력 추적
- 모니터링 법령 12개 → 26개 (산업안전/건설/위험물/소방/전기/가스/근로/환경)
- lawSearch.do로 검색, lawService.do로 본문 조회
- 대형 법령 편/장 단위 분할 저장 (fallback: 편→장→전체)
- 저장 경로: PKM/Inbox/ (AI 자동 분류 연계)
- 변경 감지 시 user_note에 이력 자동 기록
- CalDAV + SMTP 알림

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:47:08 +09:00
Hyungi Ahn
b4ca918125 fix: 벡터 검색 asyncpg 캐스트 — ::vector → cast(:embedding AS vector)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:35:14 +09:00
Hyungi Ahn
e23c4feaa0 feat: 검색 전면 개편 — 필드별 가중치 + 벡터 합산 + match reason
검색 대상: title > ai_tags > user_note > ai_summary > extracted_text
- 필드별 가중치: title(3.0), tags(2.5), note(2.0), summary(1.5), text(1.0)
- 벡터 검색: 별도 쿼리로 분리, 결과 합산 (asyncpg 충돌 방지)
- match_reason: 어떤 필드에서 매칭됐는지 반환
- 중복 제거 + 점수 합산

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:33:34 +09:00
Hyungi Ahn
e7cd710e69 fix: hybrid 검색 단순화 — FTS + ILIKE (vector/trgm 복잡 쿼리 제거)
asyncpg 파라미터 바인딩 충돌 문제 근본 해결.
한국어 검색: ILIKE fallback으로 안정 동작.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:16:36 +09:00
Hyungi Ahn
3236b8d812 fix: 검색 500 에러 (ILIKE % 이스케이프) + 한글 조합 중 Enter 방지
- ILIKE '%' → '%%' (SQLAlchemy text() 파라미터 충돌 해결)
- e.isComposing 체크로 한글 조합 완료 전 Enter 무시

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:14:07 +09:00
Hyungi Ahn
4d205b67c2 fix: 검색 UX 개선 — Enter 키 기반 + 한국어 검색 ILIKE fallback
- 프론트: debounce 자동검색 제거 → Enter 키로만 검색 (한글 조합 문제 해결)
- 백엔드: trgm threshold 0.1로 낮춤 + ILIKE '%검색어%' fallback 추가
- hybrid 검색 score threshold 0.01 → 0.001로 낮춤

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:10:47 +09:00
Hyungi Ahn
b54cc25650 fix: 미분류 판단 기준 변경 — file_path 기반 → ai_domain 없음 기준
파일을 물리적으로 이동하지 않으므로 file_path로 미분류 판단 불가.
ai_domain이 NULL 또는 빈 문자열인 문서를 미분류로 취급.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:05:41 +09:00
Hyungi Ahn
d63a6b85e1 feat: 사이드바 3단계 재귀 트리 + 너비 확장 (320px)
- tree API: domain 경로를 파싱하여 계층 구조로 반환
  (Industrial_Safety → Practice → Patrol_Inspection)
- Sidebar: 재귀 snippet으로 N단계 트리 렌더링
- domain 필터: prefix 매칭 (상위 클릭 시 하위 전부 포함)
- 사이드바 너비: 260px → 320px

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:03:36 +09:00
Hyungi Ahn
bf0506023c feat: 정보 패널 — taxonomy 기반 분류 표시 (breadcrumb + type/confidence 배지)
- domain 경로를 breadcrumb으로 표시 (Industrial_Safety › Practice › Patrol_Inspection)
- document_type 배지 (파란색)
- confidence 배지 (85%+ 초록, 60~85% 주황, <60% 빨강)
- importance 배지 (high만 표시)
- 원본 포맷 표시

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:56:37 +09:00
Hyungi Ahn
7f5e09096a fix: 문서 삭제 시 processing_queue FK 제약 해결 + 변환본/preview 함께 삭제
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:51:35 +09:00
Hyungi Ahn
5153169d5d fix: 검색바 상단 고정 — 문서 목록만 스크롤
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:49:42 +09:00
Hyungi Ahn
9b0705b79f config: fallback 모델 qwen3.5:35b → qwen3.5:9b-q8_0 (GPU VRAM 제한)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:40:25 +09:00
Hyungi Ahn
63f75de89d fix: Qwen3.5 thinking 모드 비활성화 (enable_thinking: false)
JSON 응답에 Thinking Process 텍스트가 섞이는 문제 해결.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:38:10 +09:00
Hyungi Ahn
6d73e7ee12 feat: 분류 체계 전면 개편 — taxonomy + document_type + confidence
- config.yaml: 6개 domain × 3단계 taxonomy + 13개 document_types 정의
- classify.txt: 영문 프롬프트, taxonomy 경로 기반 분류 + 분류 규칙 주입
- classify_worker: taxonomy 검증, confidence 기반 분류, document_type 저장
- migration 008: document_type, importance, ai_confidence 컬럼
- API: DocumentResponse에 document_type, importance, ai_confidence 추가

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:32:20 +09:00
Hyungi Ahn
770d38b72c feat: 문서 삭제 기능 — 정보 패널에서 확인 후 삭제 (파일+DB)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:14:20 +09:00
Hyungi Ahn
1b5fa95a9f feat: 오피스 → ODF 변환 + 원본/편집본 분리 아키텍처
- original_path/format/hash + conversion_status 필드 추가 (migration 007)
- extract_worker: 텍스트 추출 후 xlsx→ods, docx→odt 등 ODF 변환
  - 변환본은 .derived/{doc_id}.ods 에 저장
  - 원본 메타 보존 (original_path/format/hash)
- file_watcher: .derived/ .preview/ 디렉토리 제외
- DocumentViewer: ODF 포맷이면 편집 버튼 자동 표시
  - edit_url 있으면 "편집", 없으면 "Synology Drive에서 열기"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:11:43 +09:00
Hyungi Ahn
b937eb948b feat: Noto 다국어 폰트 추가 (fonts-noto-core/extra — 라틴/아랍/태국 등)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:45:02 +09:00
Hyungi Ahn
1030bffc82 fix: LibreOffice 한글/CJK 폰트 추가 (fonts-noto-cjk, fonts-nanum)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:43:13 +09:00
Hyungi Ahn
733f730e16 fix: preview enum 누락 + AI summary thinking 제거 + CLAUDE.md 전면 갱신
- queue.py: process_stage enum에 'preview' 추가
- classify_worker: ai_summary에 strip_thinking() 적용
- CLAUDE.md: 현재 아키텍처 전면 반영 (파이프라인, UI, 인프라, 코딩규칙)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:38:59 +09:00
Hyungi Ahn
6893ea132d refactor: preview 병렬 트리거 + 파일 이동 제거 + domain 색상 바
- queue_consumer: extract 완료 시 classify + preview 동시 등록
- classify_worker: _move_to_knowledge() 제거, 파일 원본 위치 유지
- DocumentCard: 좌측 domain별 색상 바 (4px) 추가

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:31:57 +09:00
Hyungi Ahn
47e9981660 fix: Qwen3.5 Thinking Process 텍스트 제거 — JSON 파싱 개선
첫 번째 { 이전의 모든 비-JSON 텍스트를 제거하여
thinking/reasoning preamble이 있어도 JSON 추출 가능.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:44:21 +09:00
Hyungi Ahn
03b0612aa2 fix: extract_worker OFFICE_FORMATS 블록에 return 누락 수정
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:28:09 +09:00
Hyungi Ahn
a5186bf4aa fix: 스프레드시트 텍스트 추출 — csv 필터 사용 (txt:Text는 Calc 미지원)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:21:29 +09:00
Hyungi Ahn
b37043d651 fix: LibreOffice 한글 파일명 호환 — 영문 임시파일로 복사 후 변환
extract_worker, preview_worker 모두 적용.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:18:06 +09:00
Hyungi Ahn
45448b4036 feat: extract_worker에 LibreOffice 텍스트 추출 추가 (오피스 포맷)
- xlsx, docx, pptx, odt, ods, odp, odoc, osheet 지원
- LibreOffice --convert-to txt로 텍스트 추출 (60s timeout)
- 추가 의존성 없음 (Docker에 이미 설치된 LibreOffice 사용)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:12:19 +09:00
Hyungi Ahn
9fd44ab268 fix: 드래그 앤 드롭 — window 이벤트로 브라우저 기본 동작 차단
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:06:18 +09:00
Hyungi Ahn
87bdd8003c feat: 드래그 앤 드롭 업로드 (UploadDropzone)
- 파일 드래그 시 전체 페이지 오버레이
- 순차 업로드 + 파일별 진행 상태
- 성공/실패 토스트 + 목록 자동 새로고침
- documents 페이지에 통합

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:02:42 +09:00
Hyungi Ahn
41072a2e6d feat: 수동 편집 URL — 정보 패널에서 Synology Drive 링크 입력/관리
- edit_url 컬럼 추가 (migration 006)
- PreviewPanel: 편집 링크 입력/수정/표시 UI
- DocumentViewer: edit_url 있으면 편집 버튼에서 해당 URL로 새 탭
- API: DocumentResponse/DocumentUpdate에 edit_url 필드

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:37:44 +09:00
Hyungi Ahn
4bea408bbd feat: Markdown 편집기 + PDF 변환 파이프라인 + 뷰어 포맷 분기
- Markdown split editor: textarea + marked preview, Ctrl+S 저장
- PUT /api/documents/{id}/content: 원본 파일 저장 + extracted_text 갱신
- GET /api/documents/{id}/preview: PDF 미리보기 캐시 서빙
- preview_worker: LibreOffice headless → PDF 변환 (timeout 60s, retry 1회)
- queue_consumer: preview stage 추가 (embed 후 자동 트리거)
- DocumentViewer: 포맷별 분기 (markdown/pdf/preview-pdf/image/text/cad)
- 오피스/CAD 문서: 새 탭 편집 버튼
- Dockerfile: LibreOffice headless 설치
- migration 005: preview_status, preview_hash, preview_at 컬럼

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:10:03 +09:00
Hyungi Ahn
3546c8cefb refactor: 레이아웃 개선 — 30:70 비율, 사이드바 접힘, 정보 패널 drawer
- 사이드바: 데스크톱도 기본 접힘, ☰로 오버레이, localStorage 상태 기억
- 상단 30%: 문서 목록 + 검색 (문서 미선택 시 100%)
- 하단 70%: 뷰어 전체 너비 (우측 패널 제거)
- 정보 패널: ℹ 버튼 → 우측 전체 높이 drawer (ESC/외부 클릭 닫기)
- nav 높이 축소, 폰트 크기 최적화

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:05:47 +09:00
Hyungi Ahn
17d41a8526 feat: Phase 1D+2 — 모바일 대응, 스마트 그룹, 메모, 태그 편집
- 모바일: 카드 클릭 시 detail 페이지로 이동 (뷰어 패널 미표시)
- 스마트 그룹: 사이드바에 최근 7일/법령 알림/이메일 프리셋 필터
- 메모: user_note 컬럼 추가 (migration 004), PATCH API, PreviewPanel 인라인 편집
- 태그 편집: PreviewPanel에서 태그 추가(+)/삭제(×) 기능
- DB 모델 + API 스키마 user_note 필드 추가

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:27:18 +09:00
Hyungi Ahn
47abf40bf1 feat: 하단 문서 뷰어 + 우측 정보 패널 (DEVONthink 레이아웃)
- DocumentViewer: 문서 선택 시 하단에 본문 미리보기/편집
  (Markdown 렌더링, PDF iframe, 이미지, Synology Office iframe)
- 레이아웃 변경: 상단(목록 45%) + 하단(뷰어+정보 55%)
- 우측 패널은 문서 정보/태그/처리상태 (메모/태그 편집은 Phase 2)
- 문서 선택 해제 시 목록 전체 표시로 복원

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:17:45 +09:00
Hyungi Ahn
9239e9c1d5 fix: DocumentCard svelte:element → button (Svelte 5 호환)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:12:08 +09:00
Hyungi Ahn
a15208f0cf feat: Phase 1C — 프리뷰 패널 (문서 선택 시 우측 표시)
- PreviewPanel: AI 요약, 태그, 메타 정보, 처리 상태 표시
- DocumentCard: 선택 모드 지원 (클릭→프리뷰, 더블클릭 불필요)
- 3-pane 완성: sidebar | document list | preview panel
- 필터 변경 시 선택 자동 해제
- 데스크톱만 표시 (모바일은 detail 페이지로 이동)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:11:13 +09:00
Hyungi Ahn
f4a0229f15 fix: detail 페이지 태그를 TagPill 컴포넌트로 교체 (클릭→필터)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:07:51 +09:00
Hyungi Ahn
cb8a846773 feat: Phase 1B — DocumentCard/TagPill/FormatIcon 컴포넌트
- DocumentCard: 포맷 아이콘, 제목+요약, domain 경로, 태그 pill,
  data_origin 배지, 날짜, 파일 크기
- TagPill: 계층별 색상 (@amber, #blue, $green, !red), 클릭→필터
- FormatIcon: 파일 포맷별 lucide 아이콘 매핑
- documents 페이지에서 DocumentCard 컴포넌트 사용

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:05:40 +09:00
Hyungi Ahn
1a207be261 fix: authChecked를 $state로 변경 (반응성 복원)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:03:06 +09:00
Hyungi Ahn
b04e1de8a6 fix: Svelte 5 runes mode 호환 ($: → $derived/$effect)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:01:00 +09:00
Hyungi Ahn
1a2b3b49af refactor: 사이드바를 전역 레이아웃으로 이동
- +layout.svelte: 사이드바 + 상단 nav 통합 (로그인/셋업 제외)
- 각 페이지 중복 nav 제거 (dashboard, documents, detail, inbox, settings)
- 모바일 drawer + ESC 닫기 전역 처리

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 09:00:20 +09:00
Hyungi Ahn
87747866b6 feat: Phase 1A — 사이드바 트리 네비게이션 + domain/sub_group 필터
- Sidebar.svelte: /api/documents/tree 기반 domain→sub_group 트리,
  접기/펼치기, active highlight, 모바일 drawer
- documents/+page.svelte: 2-pane 레이아웃, URL params 기반 필터,
  빈 상태 개선, 카드 정보 밀도 향상 (domain 경로, 태그, origin 배지)
- documents.py: sub_group 필터 파라미터 추가
- app.css: domain 7색 + sidebar CSS 변수

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:54:09 +09:00
Hyungi Ahn
faf9bda77a fix: set correct Content-Type and inline disposition for file serving
PDF was downloading instead of displaying because media_type was None
(defaulting to octet-stream). Now maps file extensions to proper MIME
types and sets Content-Disposition: inline for in-browser viewing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:38:10 +09:00
Hyungi Ahn
1affcb1afd fix: add query param token auth for file serving (iframe compat)
iframe/img tags can't send Bearer headers. File endpoint now accepts
?token= query parameter for authentication. Frontend passes access
token in URL for PDF/image viewers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:34:45 +09:00
Hyungi Ahn
e14084d5cd feat: add file serving endpoint GET /api/documents/{id}/file
Returns original document file from NAS. Fixes 404 on PDF/image
viewer in frontend. Updated frontend iframe/img src to match.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:32:51 +09:00
Hyungi Ahn
62f5eccb96 fix: isolate each worker call in independent async session
Shared session between queue consumer and workers caused
MissingGreenlet errors in APScheduler context. Each worker
call now gets its own session with explicit commit/rollback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:29:14 +09:00
Hyungi Ahn
87683ca000 security: NAS 마운트 검증 + AI 서비스 포트 제한 + deploy 문서 갱신
- NAS fail-fast: 시작 시 /documents/PKM 존재 확인, NFS 미마운트 방지
- ollama/ai-gateway 포트를 127.0.0.1로 제한 (외부 무인증 접근 차단)
- deploy.md: Caddy HTTPS 자동발급 → 앞단 프록시 HTTPS 종료 구조 반영

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:25:07 +09:00
Hyungi Ahn
7cdeac20cf fix: update migration script to read .dtBase2/Files.noindex directly
Instead of requiring DEVONthink export, reads files directly from
.dtBase2 bundle's Files.noindex/ directory structure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:17:44 +09:00
Hyungi Ahn
3df03134ff fix: bind Caddy to 0.0.0.0:8080 for external proxy access
Mac mini nginx proxies to GPU server Caddy. localhost-only binding
blocked external connections.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 08:14:56 +09:00
Hyungi Ahn
0ca78640ee infra: migrate application from Mac mini to GPU server
- Integrate ollama + ai-gateway into root docker-compose.yml
  (NVIDIA GPU runtime, single compose for all services)
- Change NAS mount from SMB (NAS_SMB_PATH) to NFS (NAS_NFS_PATH)
  Default: /mnt/nas/Document_Server (fstab registered on GPU server)
- Update config.yaml AI endpoints:
  primary → Mac mini MLX via Tailscale (100.76.254.116:8800)
  fallback/embedding/vision/rerank → ollama (same Docker network)
  gateway → ai-gateway (same Docker network)
- Update credentials.env.example (remove GPU_SERVER_IP, add NFS path)
- Mark gpu-server/docker-compose.yml as deprecated
- Update CLAUDE.md network diagram and AI model config
- Update architecture.md, deploy.md, devlog.md for GPU server as main
- Caddyfile: auto_https off, HTTP only (TLS at upstream proxy)
- Caddy port: 127.0.0.1:8080:80 (localhost only)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 07:47:09 +09:00
Hyungi Ahn
8afa3c401f fix: wait for auth refresh check before redirecting to login
The $: reactive statement was firing before onMount's tryRefresh()
completed, immediately redirecting to /login on every page refresh.
Added authChecked flag to gate the redirect logic.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 07:05:20 +09:00
Hyungi Ahn
aebfa14984 fix: don't intercept 401 on login/refresh endpoints for token refresh
Login 401 (TOTP required) was being caught by the refresh interceptor,
masking the actual error detail with "인증이 만료되었습니다".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:58:36 +09:00
Hyungi Ahn
17c1b7cf30 fix: set refresh cookie secure=False, samesite=lax for reverse proxy chain
Nginx terminates TLS and forwards HTTP internally. Secure=True cookies
don't get sent when the backend sees HTTP connections.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:53:59 +09:00
Hyungi Ahn
4ef27fc51c fix: use :80 instead of domain in Caddyfile (nginx handles TLS)
Nginx home-service-proxy terminates TLS and forwards plain HTTP to
Caddy on port 8080. Caddy doesn't need to match the domain name.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:51:03 +09:00
Hyungi Ahn
a872dfc10f fix: guard goto() with browser check to prevent SSR crash
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:47:48 +09:00
Hyungi Ahn
fce9124c28 fix: add type:module to frontend package.json for ESM vite config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:46:59 +09:00
Hyungi Ahn
cfa95ff031 feat: implement Phase 4 SvelteKit frontend + backend enhancements
Backend:
- Add dashboard API (today stats, inbox count, law alerts, pipeline status)
- Add /api/documents/tree endpoint for sidebar domain/sub_group tree
- Migrate auth to HttpOnly cookie for refresh token (XSS defense)
- Add /api/auth/logout endpoint (cookie cleanup)
- Register dashboard router in main.py

Frontend (SvelteKit + Tailwind CSS v4):
- api.ts: fetch wrapper with refresh queue pattern, 401 single retry,
  forced logout on refresh failure
- Auth store: login/logout/refresh with memory-based access token
- UI store: toast system, sidebar state
- Login page with TOTP support
- Dashboard with 4 stat widgets + recent documents
- Document list with hybrid search (debounce, URL query state, mode select)
- Document detail with format-aware viewer (markdown/PDF/HWP/Synology/fallback)
- Metadata panel (AI summary, tags, processing history)
- Inbox triage UI (batch select, confirm dialog, domain override)
- Settings page (password change, TOTP status)

Infrastructure:
- Enable frontend service in docker-compose
- Caddy path routing (/api/* → fastapi, / → frontend) + gzip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 06:46:19 +09:00
Hyungi Ahn
46537ee11a fix: Codex 리뷰 P1/P2 버그 4건 수정
- [P1] migration runner 도입: schema_migrations 추적, advisory lock,
  단일 트랜잭션 실행, SQL 검증 (기존 DB 업그레이드 대응)
- [P1] eml extract 큐 조건 분기: extract_worker 미지원 포맷 큐 스킵
- [P2] iCalendar escape_ical_text() 추가: RFC 5545 준수
- [P2] 이메일 charset 감지: get_content_charset() 사용 + payload None 방어

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:55:38 +09:00
Hyungi Ahn
d93e50b55c security: fix 5 review findings (2 high, 3 medium)
HIGH:
- Lock setup TOTP/NAS endpoints behind _require_setup() guard
  (prevented unauthenticated admin 2FA takeover after setup)
- Sanitize upload filename with Path().name + resolve() validation
  (prevented path traversal writing outside Inbox)

MEDIUM:
- Add score > 0.01 filter to hybrid search via subquery
  (prevented returning irrelevant documents with zero score)
- Implement Inbox → Knowledge file move after classification
  (classify_worker now moves files based on ai_domain)
- Add Anthropic Messages API support in _request()
  (premium/Claude path now sends correct format and parses
  content[0].text instead of choices[0].message.content)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:33:31 +09:00
Hyungi Ahn
31d5498f8d feat: implement Phase 3 automation workers
- Add automation_state table for incremental sync (last UID, last check)
- Add law_monitor worker: 국가법령정보센터 API → NAS/DB/CalDAV VTODO
  (LAW_OC 승인 대기 중, 코드 완성)
- Add mailplus_archive worker: IMAP(993) → .eml NAS save + DB + SMTP
  notification (imaplib via asyncio.to_thread, timeout=30)
- Add daily_digest worker: PostgreSQL/pipeline stats → Markdown + SMTP
  (documents, law changes, email, queue errors, inbox backlog)
- Add CalDAV VTODO helper and SMTP email helper to core/utils.py
- Wire 3 cron jobs in APScheduler (law@07:00, mail@07:00+18:00,
  digest@20:00) with timezone=Asia/Seoul

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:24:50 +09:00
Hyungi Ahn
a5312c044b fix: replace deprecated regex with pattern in search Query param
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:02:44 +09:00
Hyungi Ahn
4b695332b9 feat: implement Phase 2 core features
- Add document CRUD API (list/get/upload/update/delete with auth)
  - Upload saves to Inbox + auto-enqueues processing pipeline
  - Delete defaults to DB-only, explicit flag for file deletion
- Add hybrid search API (FTS 0.4 + trigram 0.2 + vector 0.4 weighted)
  - Modes: fts, trgm, vector, hybrid (default)
  - Vector search gracefully degrades if GPU unavailable
- Add Inbox file watcher (5min interval, new file + hash change detection)
- Register documents/search routers and file_watcher scheduler in main.py
- Add IVFFLAT vector index migration (lists=50, with tuning guide)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:49:12 +09:00
Hyungi Ahn
2dfb05e653 fix: convert kordoc service to ESM (kordoc requires ESM import)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:38:34 +09:00
Hyungi Ahn
299fac3904 feat: implement Phase 1 data pipeline and migration
- Implement kordoc /parse endpoint (HWP/HWPX/PDF via kordoc lib,
  text files direct read, images flagged for OCR)
- Add queue consumer with APScheduler (1min interval, stage chaining
  extract→classify→embed, stale item recovery, retry logic)
- Add extract worker (kordoc HTTP call + direct text read)
- Add classify worker (Qwen3.5 AI classification with think-tag
  stripping and robust JSON extraction from AI responses)
- Add embed worker (GPU server nomic-embed-text, graceful failure)
- Add DEVONthink migration script with folder mapping for 16 DBs,
  dry-run mode, batch commits, and idempotent file_path UNIQUE
- Enhance ai/client.py with strip_thinking() and parse_json_response()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:35:36 +09:00
Hyungi Ahn
23ee055357 fix: replace passlib with bcrypt directly (passlib+bcrypt 5.0 incompatible)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:00:43 +09:00
Hyungi Ahn
e63d2971a9 fix: update TemplateResponse call for Starlette 1.0 API
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:58:16 +09:00
Hyungi Ahn
b7c3040f1a chore: add .env to gitignore (docker-compose variable substitution)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:53:23 +09:00
Hyungi Ahn
d8fbe187bf fix: use port 9443 for Caddy HTTPS (8443 also taken by OrbStack)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:52:35 +09:00
Hyungi Ahn
0290dad923 fix: remap Caddy ports to 8080/8443 to avoid OrbStack conflict
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:52:08 +09:00
Hyungi Ahn
629fe37790 fix: use node fetch for kordoc healthcheck (wget/curl missing in slim)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:51:29 +09:00
Hyungi Ahn
8484389086 fix: change postgres external port to 15432 to avoid OrbStack conflict
Internal container communication still uses 5432.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:50:57 +09:00
Hyungi Ahn
16d99011db fix: disable frontend service until Phase 4, simplify Caddy proxy
Frontend SvelteKit build has dependency conflicts (Svelte 5 + Vite 8).
Phase 0 setup wizard is served by FastAPI/Jinja2, no frontend needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:28:03 +09:00
Hyungi Ahn
99821df5c9 fix: bump vite to ^8.0.0 for @sveltejs/vite-plugin-svelte 7 compat
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:27:04 +09:00
Hyungi Ahn
5a13b83e4d fix: upgrade frontend to Svelte 5 + Vite 6 for dependency compatibility
Svelte 4 conflicts with latest @sveltejs/kit peer requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:26:37 +09:00
Hyungi Ahn
a601991f48 feat: implement Phase 0 auth system, setup wizard, and Docker config
- Add users table to migration, User ORM model
- Implement JWT+TOTP auth API (login, refresh, me, change-password)
- Add first-run setup wizard with rate-limited admin creation,
  TOTP QR enrollment (secret saved only after verification), and
  NAS path verification — served as Jinja2 single-page HTML
- Add setup redirect middleware (bypasses /health, /docs, /openapi.json)
- Mount config.yaml, scripts, logs volumes in docker-compose
- Route API vs frontend traffic in Caddyfile
- Include admin seed script as CLI fallback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 13:21:45 +09:00
83 changed files with 10351 additions and 207 deletions

1
.gitignore vendored
View File

@@ -1,5 +1,6 @@
# 인증 정보 (절대 커밋 금지)
credentials.env
.env
# Python
venv/

157
CLAUDE.md
View File

@@ -4,7 +4,7 @@
Self-hosted PKM(Personal Knowledge Management) 웹 애플리케이션.
FastAPI + PostgreSQL(pgvector) + SvelteKit + Docker Compose 기반.
Mac mini M4 Pro를 애플리케이션 서버, Synology NAS를 파일 저장소, GPU 서버를 AI 추론에 사용한다.
GPU 서버를 메인 서버, Mac mini를 AI 추론, Synology NAS를 파일 저장소 사용.
## 핵심 문서
@@ -18,34 +18,35 @@ Mac mini M4 Pro를 애플리케이션 서버, Synology NAS를 파일 저장소,
|------|------|
| 백엔드 | FastAPI (Python 3.11+) |
| 데이터베이스 | PostgreSQL 16 + pgvector + pg_trgm |
| 프론트엔드 | SvelteKit |
| 문서 파싱 | kordoc (Node.js, HWP/HWPX/PDF → Markdown) |
| 리버스 프록시 | Caddy (자동 HTTPS) |
| 프론트엔드 | SvelteKit 5 (runes mode) + Tailwind CSS 4 |
| 문서 파싱 | kordoc (HWP/HWPX/PDF → Markdown) + LibreOffice (오피스 → 텍스트/PDF) |
| 리버스 프록시 | Caddy (HTTP only, 앞단 프록시에서 HTTPS 처리) |
| 인증 | JWT + TOTP 2FA |
| 컨테이너 | Docker Compose |
## 네트워크 환경
```
Mac mini M4 Pro (애플리케이션 서버):
- Docker Compose: FastAPI(:8000), PostgreSQL(:5432), kordoc(:3100), Caddy(:80,:443)
- MLX Server: http://localhost:8800/v1/chat/completions (Qwen3.5-35B-A3B)
- 외부 접근: pkm.hyungi.net (Caddy 프록시)
GPU 서버 (RTX 4070 Ti Super, Ubuntu, 메인 서버):
- Docker Compose: FastAPI(:8000), PostgreSQL(:5432), kordoc(:3100),
Caddy(:8080 HTTP only), Ollama(127.0.0.1:11434), AI Gateway(127.0.0.1:8081), frontend(:3000)
- NFS 마운트: /mnt/nas/Document_Server → NAS /volume4/Document_Server
- 외부 접근: document.hyungi.net (Mac mini nginx → Caddy)
- 로컬 IP: 192.168.1.186
Mac mini M4 Pro (AI 서버 + 앞단 프록시):
- MLX Server: http://100.76.254.116:8800/v1/chat/completions (Qwen3.5-35B-A3B)
- nginx: HTTPS 종료 → GPU 서버 Caddy(:8080)로 프록시
- Tailscale IP: 100.76.254.116
Synology NAS (DS1525+):
- 도메인: ds1525.hyungi.net
- LAN IP: 192.168.1.227
- Tailscale IP: 100.101.79.37
- 포트: 15001
- 파일 원본: /volume4/Document_Server/PKM/
- Synology Office: 문서 편집/미리보기
- Synology Calendar: CalDAV 태스크 관리 (OmniFocus 대체)
- NFS export → GPU 서버
- Synology Drive: https://link.hyungi.net (문서 편집)
- Synology Calendar: CalDAV 태스크 관리
- MailPlus: IMAP(993) + SMTP(465)
GPU 서버 (RTX 4070 Ti Super):
- AI Gateway: http://gpu-server:8080 (모델 라우팅, 폴백, 비용 제어)
- nomic-embed-text: 벡터 임베딩
- Qwen2.5-VL-7B: 이미지/도면 OCR
- bge-reranker-v2-m3: RAG 리랭킹
```
## 인증 정보
@@ -57,19 +58,19 @@ GPU 서버 (RTX 4070 Ti Super):
## AI 모델 구성
```
Primary (Mac mini MLX, 상시, 무료):
Primary (Mac mini MLX, Tailscale 경유, 상시, 무료):
mlx-community/Qwen3.5-35B-A3B-4bit — 분류, 태그, 요약
→ http://localhost:8800/v1/chat/completions
→ http://100.76.254.116:8800/v1/chat/completions
Fallback (GPU Ollama, MLX 장애 시):
Fallback (GPU Ollama, 같은 Docker 네트워크, MLX 장애 시):
qwen3.5:35b-a3b
→ http://gpu-server:11434/v1/chat/completions
→ http://ollama:11434/v1/chat/completions
Premium (Claude API, 종량제, 수동 트리거만):
claude-sonnet — 복잡한 분석, 장문 처리
→ 일일 한도 $5, require_explicit_trigger: true
Embedding (GPU 서버 전용):
Embedding (GPU Ollama, 같은 Docker 네트워크):
nomic-embed-text → 벡터 임베딩
Qwen2.5-VL-7B → 이미지/도면 OCR
bge-reranker-v2-m3 → RAG 리랭킹
@@ -79,62 +80,112 @@ Embedding (GPU 서버 전용):
```
hyungi_Document_Server/
├── docker-compose.yml ← Mac mini용
├── Caddyfile
├── docker-compose.yml
├── Caddyfile ← HTTP only, auto_https off
├── config.yaml ← AI 엔드포인트, NAS 경로, 스케줄
├── credentials.env.example
├── app/ ← FastAPI 백엔드
│ ├── main.py
│ ├── main.py ← 엔트리포인트 + APScheduler (watcher/consumer 포함)
│ ├── Dockerfile ← LibreOffice headless 포함
│ ├── core/ (config, database, auth, utils)
│ ├── models/ (document, task, queue)
│ ├── api/ (documents, search, tasks, dashboard, export)
│ ├── workers/(file_watcher, extract, classify, embed, law_monitor, mailplus, digest)
│ ├── api/ (documents, search, dashboard, auth, setup)
│ ├── workers/ (file_watcher, extract, classify, embed, preview, law_monitor, mailplus, digest, queue_consumer)
│ ├── prompts/classify.txt
│ └── ai/client.py
├── services/kordoc/ ← Node.js 마이크로서비스
├── gpu-server/ ← GPU 서버용 (별도 배포)
│ ├── docker-compose.yml
│ └── services/ai-gateway/
├── frontend/ ← SvelteKit
├── migrations/ ← PostgreSQL 스키마
├── scripts/migrate_from_devonthink.py
│ └── ai/client.py ← AIClient + parse_json_response (Qwen3.5 thinking 처리)
├── services/kordoc/ ← Node.js 마이크로서비스 (HWP/PDF 파싱)
├── gpu-server/ ← AI Gateway (deprecated, 통합됨)
├── frontend/ ← SvelteKit 5
│ └── src/
├── routes/ ← 페이지 (documents, inbox, settings, login)
│ └── lib/
│ ├── components/ ← Sidebar, DocumentCard, DocumentViewer, PreviewPanel,
│ │ TagPill, FormatIcon, UploadDropzone
│ ├── stores/ ← auth, ui
│ └── api.ts ← fetch wrapper (JWT 토큰 관리)
├── migrations/ ← PostgreSQL 스키마 (schema_migrations로 추적)
├── scripts/
├── docs/
└── tests/
```
## 데이터 3계층
## 문서 처리 파이프라인
1. **원본 파일** (NAS `/volume4/Document_Server/PKM/`) — 유일한 진짜 원본
2. **가공 데이터** (PostgreSQL) — 텍스트 추출, AI 메타데이터, 검색 인덱스
3. **파생물** (pgvector + 캐시) — 벡터 임베딩, 썸네일
```
파일 업로드 (드래그 앤 드롭 or file_watcher)
extract (텍스트 추출)
- kordoc: HWP, HWPX, PDF → Markdown
- LibreOffice: xlsx, docx, pptx, odt 등 → txt/csv
- 직접 읽기: md, txt, csv, json, xml, html
↓ ↓
classify (AI 분류) preview (PDF 미리보기 생성)
- Qwen3.5 → domain - LibreOffice → PDF 변환
- tags, summary - 캐시: PKM/.preview/{id}.pdf
embed (벡터 임베딩)
- nomic-embed-text (768차원)
```
**핵심 원칙:**
- 파일은 업로드 위치에 그대로 유지 (물리적 이동 없음)
- 분류(domain/sub_group/tags)는 DB 메타데이터로만 관리
- preview는 classify와 병렬로 실행 (AI 결과 불필요)
## UI 구조
```
┌──────────────────────────────────────────────────┐
│ [☰ 사이드바] [PKM / 문서] [ 정보] 버튼│ ← 상단 nav
├──────────────────────────────────────────────────┤
│ [검색바] [모드] [] │
│ 문서 목록 (30%) — 드래그 업로드 지원 │ ← 상단 영역
│ █ 문서카드 (domain 색상 바 + 포맷 아이콘) │
├──────────────────────────────────────────────────┤
│ 하단 뷰어/편집 (70%) — 전체 너비 │ ← 하단 영역
│ Markdown: split editor (textarea + preview) │
│ PDF: 브라우저 내장 뷰어 │
│ 오피스: PDF 변환 미리보기 + [편집] 새 탭 버튼 │
│ 이미지: img 태그 │
└──────────────────────────────────────────────────┘
사이드바: 평소 접힘, ☰로 오버레이 (domain 트리 + 스마트 그룹 + Inbox)
정보 패널: 버튼 → 우측 전체 높이 drawer (메모/태그 편집/메타/처리상태/편집 URL)
```
## 데이터 계층
1. **원본 파일** (NAS `/volume4/Document_Server/PKM/`) — 유일한 원본, 위치 변경 없음
2. **가공 데이터** (PostgreSQL) — 텍스트 추출, AI 분류, 검색 인덱스, 메모, 태그
3. **파생물** — 벡터 임베딩 (pgvector), PDF 미리보기 캐시 (`.preview/`)
## 코딩 규칙
- Python 3.11+, asyncio, type hints
- SQLAlchemy 2.0+ async 세션
- Svelte 5 runes mode ($state, $derived, $effect — $: 사용 금지)
- 인증 정보는 credentials.env에서 로딩 (하드코딩 금지)
- 로그는 `logs/`에 저장 (Docker 볼륨)
- AI 호출은 반드시 `app/ai/client.py``AIClient`를 통해 (직접 HTTP 호출 금지)
- 한글 주석 사용
- Migration: `migrations/*.sql`에 작성, `init_db()`가 자동 실행 (schema_migrations 추적)
- SQL에 BEGIN/COMMIT 금지 (외부 트랜잭션 깨짐)
- 기존 DB에서는 schema_migrations에 수동 이력 등록 필요할 수 있음
## 개발/배포 워크플로우
```
MacBook Pro (개발) → Gitea push → 서버에서 pull
MacBook Pro (개발) → Gitea push → GPU 서버에서 pull
개발:
cd ~/Documents/code/hyungi_Document_Server/
# 코드 작성 → git commit & push
Mac mini 배포:
GPU 서버 배포 (메인):
ssh hyungi@100.111.160.84
cd ~/Documents/code/hyungi_Document_Server/
git pull
docker compose up -d
GPU 서버 배포:
cd ~/Documents/code/hyungi_Document_Server/gpu-server/
git pull
docker compose up -d
docker compose up -d --build fastapi frontend
```
## v1 코드 참조
@@ -143,12 +194,14 @@ v1(DEVONthink 기반) 코드는 `v1-final` 태그로 보존:
```bash
git show v1-final:scripts/law_monitor.py
git show v1-final:scripts/pkm_utils.py
git show v1-final:scripts/prompts/classify_document.txt
```
## 주의사항
- credentials.env는 git에 올리지 않음 (.gitignore)
- NAS SMB 마운트 경로: Docker 컨테이너 내 `/documents`
- 법령 API (LAW_OC)는 승인 대기 중 — 스크립트만 만들고 실제 호출은 승인 후
- GPU 서버 Tailscale IP는 credentials.env에서 관리
- NAS NFS 마운트 경로: Docker 컨테이너 내 `/documents`
- FastAPI 시작 시 `/documents/PKM` 존재 확인 (NFS 미마운트 방지)
- 법령 API (LAW_OC)는 승인 대기 중
- Ollama/AI Gateway 포트는 127.0.0.1 바인딩 (외부 접근 차단)
- Caddy는 `auto_https off` + `http://` only (HTTPS는 Mac mini nginx에서 처리)
- Synology Office 편집은 새 탭 열기 방식 (iframe 미사용, edit_url 수동 등록)

View File

@@ -1,9 +1,35 @@
pkm.hyungi.net {
reverse_proxy fastapi:8000
{
auto_https off
}
http://document.hyungi.net {
encode gzip
# API + 문서 → FastAPI
handle /api/* {
reverse_proxy fastapi:8000
}
handle /docs {
reverse_proxy fastapi:8000
}
handle /openapi.json {
reverse_proxy fastapi:8000
}
handle /health {
reverse_proxy fastapi:8000
}
handle /setup {
reverse_proxy fastapi:8000
}
# 프론트엔드
handle {
reverse_proxy frontend:3000
}
}
# Synology Office 프록시
office.hyungi.net {
http://office.hyungi.net {
reverse_proxy https://ds1525.hyungi.net:5001 {
header_up Host {upstream_hostport}
transport http {

View File

@@ -24,7 +24,7 @@ Self-hosted 개인 지식관리(PKM) 웹 애플리케이션
## Quick Start
```bash
git clone https://git.hyungi.net/hyungi/hyungi_document_server.git
git clone https://git.hyungi.net/hyungi/hyungi_document_server.git hyungi_Document_Server
cd hyungi_Document_Server
# 인증 정보 설정

View File

@@ -2,6 +2,14 @@ FROM python:3.11-slim
WORKDIR /app
# LibreOffice headless (PDF 변환용) + 한글/CJK 폰트
RUN apt-get update && \
apt-get install -y --no-install-recommends \
libreoffice-core libreoffice-calc libreoffice-writer libreoffice-impress \
fonts-noto-cjk fonts-noto-cjk-extra fonts-nanum \
fonts-noto-core fonts-noto-extra && \
apt-get clean && rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

View File

@@ -1,11 +1,45 @@
"""AI 추상화 레이어 — 통합 클라이언트. 기본값은 항상 Qwen3.5."""
import json
import re
from pathlib import Path
import httpx
from core.config import settings
def strip_thinking(text: str) -> str:
"""Qwen3.5의 <think>...</think> 블록 및 Thinking Process 텍스트 제거"""
# <think> 태그 제거
text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL)
# "Thinking Process:" 등 사고 과정 텍스트 제거 (첫 번째 { 이전의 모든 텍스트)
json_start = text.find("{")
if json_start > 0:
text = text[json_start:]
return text.strip()
def parse_json_response(raw: str) -> dict | None:
"""AI 응답에서 JSON 객체 추출 (think 태그, 코드블록 등 제거)"""
cleaned = strip_thinking(raw)
# 코드블록 내부 JSON 추출
code_match = re.search(r"```(?:json)?\s*(\{.*?\})\s*```", cleaned, re.DOTALL)
if code_match:
cleaned = code_match.group(1)
# 마지막 유효 JSON 객체 찾기
matches = list(re.finditer(r"\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}", cleaned, re.DOTALL))
for m in reversed(matches):
try:
return json.loads(m.group())
except json.JSONDecodeError:
continue
# 최후 시도: 전체 텍스트를 JSON으로
try:
return json.loads(cleaned)
except json.JSONDecodeError:
return None
# 프롬프트 로딩
PROMPTS_DIR = Path(__file__).parent.parent / "prompts"
@@ -61,19 +95,43 @@ class AIClient:
raise
async def _request(self, model_config, prompt: str) -> str:
"""단일 모델 API 호출"""
response = await self._http.post(
model_config.endpoint,
json={
"model": model_config.model,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": model_config.max_tokens,
},
timeout=model_config.timeout,
)
response.raise_for_status()
data = response.json()
return data["choices"][0]["message"]["content"]
"""단일 모델 API 호출 (OpenAI 호환 + Anthropic Messages API)"""
is_anthropic = "anthropic.com" in model_config.endpoint
if is_anthropic:
import os
headers = {
"x-api-key": os.getenv("CLAUDE_API_KEY", ""),
"anthropic-version": "2023-06-01",
"content-type": "application/json",
}
response = await self._http.post(
model_config.endpoint,
headers=headers,
json={
"model": model_config.model,
"max_tokens": model_config.max_tokens,
"messages": [{"role": "user", "content": prompt}],
},
timeout=model_config.timeout,
)
response.raise_for_status()
data = response.json()
return data["content"][0]["text"]
else:
response = await self._http.post(
model_config.endpoint,
json={
"model": model_config.model,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": model_config.max_tokens,
"chat_template_kwargs": {"enable_thinking": False},
},
timeout=model_config.timeout,
)
response.raise_for_status()
data = response.json()
return data["choices"][0]["message"]["content"]
async def close(self):
await self._http.aclose()

201
app/api/auth.py Normal file
View File

@@ -0,0 +1,201 @@
"""인증 API — 로그인, 토큰 갱신, TOTP 검증
access token: 응답 body (프론트에서 메모리 보관)
refresh token: HttpOnly cookie (XSS 방어)
"""
from datetime import datetime, timezone
from typing import Annotated
from fastapi import APIRouter, Cookie, Depends, HTTPException, Request, Response, status
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import (
REFRESH_TOKEN_EXPIRE_DAYS,
create_access_token,
create_refresh_token,
decode_token,
get_current_user,
hash_password,
verify_password,
verify_totp,
)
from core.database import get_session
from models.user import User
router = APIRouter()
# ─── 요청/응답 스키마 ───
class LoginRequest(BaseModel):
username: str
password: str
totp_code: str | None = None
class AccessTokenResponse(BaseModel):
access_token: str
token_type: str = "bearer"
class ChangePasswordRequest(BaseModel):
current_password: str
new_password: str
class UserResponse(BaseModel):
id: int
username: str
is_active: bool
totp_enabled: bool
last_login_at: datetime | None
class Config:
from_attributes = True
# ─── 헬퍼 ───
def _set_refresh_cookie(response: Response, token: str):
"""refresh token을 HttpOnly cookie로 설정"""
response.set_cookie(
key="refresh_token",
value=token,
httponly=True,
secure=False, # Nginx가 TLS 종료, 내부 트래픽은 HTTP
samesite="lax",
max_age=REFRESH_TOKEN_EXPIRE_DAYS * 86400,
path="/api/auth",
)
# ─── 엔드포인트 ───
@router.post("/login", response_model=AccessTokenResponse)
async def login(
body: LoginRequest,
response: Response,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""로그인 → access token(body) + refresh token(cookie)"""
result = await session.execute(
select(User).where(User.username == body.username)
)
user = result.scalar_one_or_none()
if not user or not verify_password(body.password, user.password_hash):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="아이디 또는 비밀번호가 올바르지 않습니다",
)
if not user.is_active:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="비활성화된 계정입니다",
)
# TOTP 검증 (설정된 경우)
if user.totp_secret:
if not body.totp_code:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="TOTP 코드가 필요합니다",
)
if not verify_totp(body.totp_code, user.totp_secret):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="TOTP 코드가 올바르지 않습니다",
)
# 마지막 로그인 시간 업데이트
user.last_login_at = datetime.now(timezone.utc)
await session.commit()
# refresh token → HttpOnly cookie
_set_refresh_cookie(response, create_refresh_token(user.username))
return AccessTokenResponse(
access_token=create_access_token(user.username),
)
@router.post("/refresh", response_model=AccessTokenResponse)
async def refresh_token(
response: Response,
session: Annotated[AsyncSession, Depends(get_session)],
refresh_token: str | None = Cookie(None),
):
"""cookie의 refresh token으로 새 토큰 쌍 발급"""
if not refresh_token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="리프레시 토큰이 없습니다",
)
payload = decode_token(refresh_token)
if not payload or payload.get("type") != "refresh":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="유효하지 않은 리프레시 토큰",
)
username = payload.get("sub")
result = await session.execute(
select(User).where(User.username == username, User.is_active.is_(True))
)
user = result.scalar_one_or_none()
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="유저를 찾을 수 없음",
)
# 새 refresh token → cookie
_set_refresh_cookie(response, create_refresh_token(user.username))
return AccessTokenResponse(
access_token=create_access_token(user.username),
)
@router.post("/logout")
async def logout(response: Response):
"""로그아웃 — refresh cookie 삭제"""
response.delete_cookie("refresh_token", path="/api/auth")
return {"message": "로그아웃 완료"}
@router.get("/me", response_model=UserResponse)
async def get_me(user: Annotated[User, Depends(get_current_user)]):
"""현재 로그인한 유저 정보"""
return UserResponse(
id=user.id,
username=user.username,
is_active=user.is_active,
totp_enabled=bool(user.totp_secret),
last_login_at=user.last_login_at,
)
@router.post("/change-password")
async def change_password(
body: ChangePasswordRequest,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""비밀번호 변경"""
if not verify_password(body.current_password, user.password_hash):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="현재 비밀번호가 올바르지 않습니다",
)
user.password_hash = hash_password(body.new_password)
await session.commit()
return {"message": "비밀번호가 변경되었습니다"}

138
app/api/dashboard.py Normal file
View File

@@ -0,0 +1,138 @@
"""대시보드 위젯 데이터 API"""
from typing import Annotated
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy import func, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import get_current_user
from core.database import get_session
from models.document import Document
from models.queue import ProcessingQueue
from models.user import User
router = APIRouter()
class DomainCount(BaseModel):
domain: str | None
count: int
class RecentDocument(BaseModel):
id: int
title: str | None
file_format: str
ai_domain: str | None
created_at: str
class PipelineStatus(BaseModel):
stage: str
status: str
count: int
class DashboardResponse(BaseModel):
today_added: int
today_by_domain: list[DomainCount]
inbox_count: int
law_alerts: int
recent_documents: list[RecentDocument]
pipeline_status: list[PipelineStatus]
failed_count: int
total_documents: int
@router.get("/", response_model=DashboardResponse)
async def get_dashboard(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""대시보드 위젯 데이터 집계"""
# 오늘 추가된 문서
today_result = await session.execute(
select(Document.ai_domain, func.count(Document.id))
.where(func.date(Document.created_at) == func.current_date())
.group_by(Document.ai_domain)
)
today_rows = today_result.all()
today_added = sum(row[1] for row in today_rows)
# Inbox 미분류 수 (review_status = pending)
inbox_result = await session.execute(
select(func.count(Document.id))
.where(
Document.review_status == "pending",
Document.deleted_at == None,
)
)
inbox_count = inbox_result.scalar() or 0
# 법령 알림 (오늘)
law_result = await session.execute(
select(func.count(Document.id))
.where(
Document.source_channel == "law_monitor",
func.date(Document.created_at) == func.current_date(),
)
)
law_alerts = law_result.scalar() or 0
# 최근 문서 5건
recent_result = await session.execute(
select(Document)
.order_by(Document.created_at.desc())
.limit(5)
)
recent_docs = recent_result.scalars().all()
# 파이프라인 상태 (24h)
pipeline_result = await session.execute(
text("""
SELECT stage, status, COUNT(*)
FROM processing_queue
WHERE created_at > NOW() - INTERVAL '24 hours'
GROUP BY stage, status
""")
)
# 실패 건수
failed_result = await session.execute(
select(func.count())
.select_from(ProcessingQueue)
.where(ProcessingQueue.status == "failed")
)
failed_count = failed_result.scalar() or 0
# 전체 문서 수
total_result = await session.execute(select(func.count(Document.id)))
total_documents = total_result.scalar() or 0
return DashboardResponse(
today_added=today_added,
today_by_domain=[
DomainCount(domain=row[0], count=row[1]) for row in today_rows
],
inbox_count=inbox_count,
law_alerts=law_alerts,
recent_documents=[
RecentDocument(
id=doc.id,
title=doc.title,
file_format=doc.file_format,
ai_domain=doc.ai_domain,
created_at=doc.created_at.isoformat() if doc.created_at else "",
)
for doc in recent_docs
],
pipeline_status=[
PipelineStatus(stage=row[0], status=row[1], count=row[2])
for row in pipeline_result
],
failed_count=failed_count,
total_documents=total_documents,
)

400
app/api/documents.py Normal file
View File

@@ -0,0 +1,400 @@
"""문서 CRUD API"""
import shutil
from datetime import datetime, timezone
from pathlib import Path
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query, UploadFile, status
from fastapi.responses import FileResponse
from pydantic import BaseModel
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import get_current_user
from core.config import settings
from core.database import get_session
from core.utils import file_hash
from models.document import Document
from models.queue import ProcessingQueue
from models.user import User
router = APIRouter()
# ─── 스키마 ───
class DocumentResponse(BaseModel):
id: int
file_path: str
file_format: str
file_size: int | None
file_type: str
title: str | None
ai_domain: str | None
ai_sub_group: str | None
ai_tags: list | None
ai_summary: str | None
document_type: str | None
importance: str | None
ai_confidence: float | None
user_note: str | None
derived_path: str | None
original_format: str | None
conversion_status: str | None
is_read: bool | None
review_status: str | None
edit_url: str | None
preview_status: str | None
source_channel: str | None
data_origin: str | None
extracted_at: datetime | None
ai_processed_at: datetime | None
embedded_at: datetime | None
created_at: datetime
updated_at: datetime
class Config:
from_attributes = True
class DocumentListResponse(BaseModel):
items: list[DocumentResponse]
total: int
page: int
page_size: int
class DocumentUpdate(BaseModel):
title: str | None = None
ai_domain: str | None = None
ai_sub_group: str | None = None
ai_tags: list | None = None
user_note: str | None = None
edit_url: str | None = None
source_channel: str | None = None
data_origin: str | None = None
# ─── 스키마 (트리) ───
class TreeNode(BaseModel):
name: str
path: str
count: int
children: list["TreeNode"]
# ─── 엔드포인트 ───
@router.get("/tree")
async def get_document_tree(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""도메인 트리 (3단계 경로 파싱, 사이드바용)"""
from sqlalchemy import text as sql_text
result = await session.execute(
sql_text("""
SELECT ai_domain, COUNT(*)
FROM documents
WHERE ai_domain IS NOT NULL AND ai_domain != ''
AND deleted_at IS NULL
GROUP BY ai_domain
ORDER BY ai_domain
""")
)
# 경로를 트리로 파싱
root: dict = {}
for domain_path, count in result:
parts = domain_path.split("/")
node = root
for part in parts:
if part not in node:
node[part] = {"_count": 0, "_children": {}}
node[part]["_count"] += count
node = node[part]["_children"]
def build_tree(d: dict, prefix: str = "") -> list[dict]:
nodes = []
for name, data in sorted(d.items()):
path = f"{prefix}/{name}" if prefix else name
children = build_tree(data["_children"], path)
nodes.append({
"name": name,
"path": path,
"count": data["_count"],
"children": children,
})
return nodes
return build_tree(root)
@router.get("/", response_model=DocumentListResponse)
async def list_documents(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
page: int = Query(1, ge=1),
page_size: int = Query(20, ge=1, le=100),
domain: str | None = None,
sub_group: str | None = None,
source: str | None = None,
format: str | None = None,
):
"""문서 목록 조회 (페이지네이션 + 필터, 뉴스 제외)"""
query = select(Document).where(Document.deleted_at == None, Document.source_channel != "news")
if domain:
# prefix 매칭: Industrial_Safety 클릭 시 하위 전부 포함
query = query.where(Document.ai_domain.startswith(domain))
if source:
query = query.where(Document.source_channel == source)
if format:
query = query.where(Document.file_format == format)
# 전체 건수
count_query = select(func.count()).select_from(query.subquery())
total = (await session.execute(count_query)).scalar()
# 페이지네이션
query = query.order_by(Document.created_at.desc())
query = query.offset((page - 1) * page_size).limit(page_size)
result = await session.execute(query)
items = result.scalars().all()
return DocumentListResponse(
items=[DocumentResponse.model_validate(doc) for doc in items],
total=total,
page=page,
page_size=page_size,
)
@router.get("/{doc_id}", response_model=DocumentResponse)
async def get_document(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""문서 단건 조회"""
doc = await session.get(Document, doc_id)
if not doc or doc.deleted_at is not None:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
return DocumentResponse.model_validate(doc)
@router.get("/{doc_id}/file")
async def get_document_file(
doc_id: int,
session: Annotated[AsyncSession, Depends(get_session)],
token: str | None = Query(None, description="Bearer token (iframe용)"),
user: User | None = Depends(lambda: None),
):
"""문서 원본 파일 서빙 (Bearer 헤더 또는 ?token= 쿼리 파라미터)"""
from core.auth import decode_token
# 쿼리 파라미터 토큰 검증
if token:
payload = decode_token(token)
if not payload or payload.get("type") != "access":
raise HTTPException(status_code=401, detail="유효하지 않은 토큰")
else:
# 일반 Bearer 헤더 인증 시도
raise HTTPException(status_code=401, detail="토큰이 필요합니다")
doc = await session.get(Document, doc_id)
if not doc:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
file_path = Path(settings.nas_mount_path) / doc.file_path
if not file_path.exists():
raise HTTPException(status_code=404, detail="파일을 찾을 수 없습니다")
# 미디어 타입 매핑
media_types = {
".pdf": "application/pdf",
".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".png": "image/png", ".gif": "image/gif",
".bmp": "image/bmp", ".tiff": "image/tiff",
".svg": "image/svg+xml",
".txt": "text/plain", ".md": "text/plain",
".html": "text/html", ".csv": "text/csv",
".json": "application/json", ".xml": "application/xml",
}
suffix = file_path.suffix.lower()
media_type = media_types.get(suffix, "application/octet-stream")
return FileResponse(
path=str(file_path),
media_type=media_type,
headers={"Content-Disposition": "inline"},
)
@router.post("/", response_model=DocumentResponse, status_code=201)
async def upload_document(
file: UploadFile,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""파일 업로드 → Inbox 저장 + DB 등록 + 처리 큐 등록"""
if not file.filename:
raise HTTPException(status_code=400, detail="파일명이 필요합니다")
# 파일명 정규화 (경로 이탈 방지)
safe_name = Path(file.filename).name
if not safe_name or safe_name.startswith("."):
raise HTTPException(status_code=400, detail="유효하지 않은 파일명")
# Inbox에 파일 저장
inbox_dir = Path(settings.nas_mount_path) / "PKM" / "Inbox"
inbox_dir.mkdir(parents=True, exist_ok=True)
target = (inbox_dir / safe_name).resolve()
# Inbox 하위 경로 검증
if not str(target).startswith(str(inbox_dir.resolve())):
raise HTTPException(status_code=400, detail="잘못된 파일 경로")
# 중복 파일명 처리
counter = 1
stem, suffix = target.stem, target.suffix
while target.exists():
target = inbox_dir.resolve() / f"{stem}_{counter}{suffix}"
counter += 1
content = await file.read()
target.write_bytes(content)
# 상대 경로 (NAS 루트 기준)
rel_path = str(target.relative_to(Path(settings.nas_mount_path)))
fhash = file_hash(target)
ext = target.suffix.lstrip(".").lower() or "unknown"
# DB 등록
doc = Document(
file_path=rel_path,
file_hash=fhash,
file_format=ext,
file_size=len(content),
file_type="immutable",
title=target.stem,
source_channel="manual",
)
session.add(doc)
await session.flush()
# 처리 큐 등록
session.add(ProcessingQueue(
document_id=doc.id,
stage="extract",
status="pending",
))
await session.commit()
return DocumentResponse.model_validate(doc)
@router.patch("/{doc_id}", response_model=DocumentResponse)
async def update_document(
doc_id: int,
body: DocumentUpdate,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""문서 메타데이터 수정 (수동 오버라이드)"""
doc = await session.get(Document, doc_id)
if not doc:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
update_data = body.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(doc, field, value)
doc.updated_at = datetime.now(timezone.utc)
await session.commit()
return DocumentResponse.model_validate(doc)
@router.put("/{doc_id}/content")
async def save_document_content(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
body: dict = None,
):
"""Markdown 원본 파일 저장 + extracted_text 갱신"""
doc = await session.get(Document, doc_id)
if not doc:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
if doc.file_format not in ("md", "txt"):
raise HTTPException(status_code=400, detail="편집 가능한 포맷이 아닙니다 (md, txt만 가능)")
content = body.get("content", "") if body else ""
file_path = Path(settings.nas_mount_path) / doc.file_path
file_path.write_text(content, encoding="utf-8")
# 메타 갱신
doc.file_size = len(content.encode("utf-8"))
doc.file_hash = file_hash(file_path)
doc.extracted_text = content[:15000]
doc.updated_at = datetime.now(timezone.utc)
await session.commit()
return DocumentResponse.model_validate(doc)
@router.get("/{doc_id}/preview")
async def get_document_preview(
doc_id: int,
session: Annotated[AsyncSession, Depends(get_session)],
token: str | None = Query(None, description="Bearer token (iframe용)"),
):
"""PDF 미리보기 캐시 서빙"""
from core.auth import decode_token
if token:
payload = decode_token(token)
if not payload or payload.get("type") != "access":
raise HTTPException(status_code=401, detail="유효하지 않은 토큰")
else:
raise HTTPException(status_code=401, detail="토큰이 필요합니다")
doc = await session.get(Document, doc_id)
if not doc:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
preview_path = Path(settings.nas_mount_path) / "PKM" / ".preview" / f"{doc_id}.pdf"
if not preview_path.exists():
raise HTTPException(status_code=404, detail="미리보기가 아직 생성되지 않았습니다")
return FileResponse(
path=str(preview_path),
media_type="application/pdf",
headers={"Content-Disposition": "inline"},
)
@router.delete("/{doc_id}")
async def delete_document(
doc_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
delete_file: bool = Query(False, description="NAS 파일도 함께 삭제"),
):
"""문서 삭제 (기본: DB만 삭제, 파일 유지)"""
doc = await session.get(Document, doc_id)
if not doc:
raise HTTPException(status_code=404, detail="문서를 찾을 수 없습니다")
# soft-delete (물리 파일은 cleanup job에서 나중에 정리)
doc.deleted_at = datetime.now(timezone.utc)
await session.commit()
return {"message": f"문서 {doc_id} soft-delete 완료"}

165
app/api/news.py Normal file
View File

@@ -0,0 +1,165 @@
"""뉴스 소스 관리 API"""
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import get_current_user
from core.database import get_session
from models.news_source import NewsSource
from models.user import User
router = APIRouter()
class NewsSourceResponse(BaseModel):
id: int
name: str
country: str | None
feed_url: str
feed_type: str
category: str | None
language: str | None
enabled: bool
last_fetched_at: str | None
created_at: str
class Config:
from_attributes = True
class NewsSourceCreate(BaseModel):
name: str
country: str | None = None
feed_url: str
feed_type: str = "rss"
category: str | None = None
language: str | None = None
class NewsSourceUpdate(BaseModel):
name: str | None = None
feed_url: str | None = None
category: str | None = None
enabled: bool | None = None
@router.get("/sources")
async def list_sources(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
result = await session.execute(select(NewsSource).order_by(NewsSource.id))
return [NewsSourceResponse.model_validate(s) for s in result.scalars().all()]
@router.post("/sources")
async def create_source(
body: NewsSourceCreate,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
source = NewsSource(**body.model_dump())
session.add(source)
await session.commit()
return NewsSourceResponse.model_validate(source)
@router.patch("/sources/{source_id}")
async def update_source(
source_id: int,
body: NewsSourceUpdate,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
source = await session.get(NewsSource, source_id)
if not source:
raise HTTPException(status_code=404)
for field, value in body.model_dump(exclude_unset=True).items():
setattr(source, field, value)
await session.commit()
return NewsSourceResponse.model_validate(source)
@router.delete("/sources/{source_id}")
async def delete_source(
source_id: int,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
source = await session.get(NewsSource, source_id)
if not source:
raise HTTPException(status_code=404)
await session.delete(source)
await session.commit()
return {"message": f"소스 {source_id} 삭제됨"}
@router.get("/articles")
async def list_articles(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
source: str | None = None,
unread_only: bool = False,
page: int = 1,
page_size: int = 30,
):
"""뉴스 기사 목록"""
from sqlalchemy import func
from models.document import Document
query = select(Document).where(
Document.source_channel == "news",
Document.deleted_at == None,
)
if source:
query = query.where(Document.ai_sub_group == source)
if unread_only:
query = query.where(Document.is_read == False)
count_q = select(func.count()).select_from(query.subquery())
total = (await session.execute(count_q)).scalar()
query = query.order_by(Document.is_read.asc(), Document.created_at.desc())
query = query.offset((page - 1) * page_size).limit(page_size)
result = await session.execute(query)
items = result.scalars().all()
from api.documents import DocumentResponse
return {
"items": [DocumentResponse.model_validate(doc) for doc in items],
"total": total,
"page": page,
}
@router.post("/mark-all-read")
async def mark_all_read(
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""전체 읽음 처리"""
from sqlalchemy import update
from models.document import Document
result = await session.execute(
update(Document)
.where(Document.source_channel == "news", Document.is_read == False)
.values(is_read=True)
)
await session.commit()
return {"marked": result.rowcount}
@router.post("/collect")
async def trigger_collect(
user: Annotated[User, Depends(get_current_user)],
):
"""수동 수집 트리거"""
from workers.news_collector import run
import asyncio
asyncio.create_task(run())
return {"message": "뉴스 수집 시작됨"}

166
app/api/search.py Normal file
View File

@@ -0,0 +1,166 @@
"""하이브리드 검색 API — FTS + ILIKE + 벡터 (필드별 가중치)"""
from typing import Annotated
from fastapi import APIRouter, Depends, Query
from pydantic import BaseModel
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from ai.client import AIClient
from core.auth import get_current_user
from core.database import get_session
from models.user import User
router = APIRouter()
class SearchResult(BaseModel):
id: int
title: str | None
ai_domain: str | None
ai_summary: str | None
file_format: str
score: float
snippet: str | None
match_reason: str | None = None
class SearchResponse(BaseModel):
results: list[SearchResult]
total: int
query: str
mode: str
@router.get("/", response_model=SearchResponse)
async def search(
q: str,
user: Annotated[User, Depends(get_current_user)],
session: Annotated[AsyncSession, Depends(get_session)],
mode: str = Query("hybrid", pattern="^(fts|trgm|vector|hybrid)$"),
limit: int = Query(20, ge=1, le=100),
):
"""문서 검색 — FTS + ILIKE + 벡터 결합"""
if mode == "vector":
results = await _search_vector(session, q, limit)
else:
results = await _search_text(session, q, limit)
# hybrid: 벡터 결과도 합산
if mode == "hybrid":
vector_results = await _search_vector(session, q, limit)
results = _merge_results(results, vector_results, limit)
return SearchResponse(
results=results,
total=len(results),
query=q,
mode=mode,
)
async def _search_text(session: AsyncSession, query: str, limit: int) -> list[SearchResult]:
"""FTS + ILIKE — 필드별 가중치 적용"""
result = await session.execute(
text("""
SELECT id, title, ai_domain, ai_summary, file_format,
left(extracted_text, 200) AS snippet,
(
-- title 매칭 (가중치 최고)
CASE WHEN coalesce(title, '') ILIKE '%%' || :q || '%%' THEN 3.0 ELSE 0 END
-- ai_tags 매칭 (가중치 높음)
+ CASE WHEN coalesce(ai_tags::text, '') ILIKE '%%' || :q || '%%' THEN 2.5 ELSE 0 END
-- user_note 매칭 (가중치 높음)
+ CASE WHEN coalesce(user_note, '') ILIKE '%%' || :q || '%%' THEN 2.0 ELSE 0 END
-- ai_summary 매칭 (가중치 중상)
+ CASE WHEN coalesce(ai_summary, '') ILIKE '%%' || :q || '%%' THEN 1.5 ELSE 0 END
-- extracted_text 매칭 (가중치 중간)
+ CASE WHEN coalesce(extracted_text, '') ILIKE '%%' || :q || '%%' THEN 1.0 ELSE 0 END
-- FTS 점수 (보너스)
+ coalesce(ts_rank(
to_tsvector('simple', coalesce(title, '') || ' ' || coalesce(extracted_text, '')),
plainto_tsquery('simple', :q)
), 0) * 2.0
) AS score,
-- match reason
CASE
WHEN coalesce(title, '') ILIKE '%%' || :q || '%%' THEN 'title'
WHEN coalesce(ai_tags::text, '') ILIKE '%%' || :q || '%%' THEN 'tags'
WHEN coalesce(user_note, '') ILIKE '%%' || :q || '%%' THEN 'note'
WHEN coalesce(ai_summary, '') ILIKE '%%' || :q || '%%' THEN 'summary'
WHEN coalesce(extracted_text, '') ILIKE '%%' || :q || '%%' THEN 'content'
ELSE 'fts'
END AS match_reason
FROM documents
WHERE deleted_at IS NULL
AND (coalesce(title, '') ILIKE '%%' || :q || '%%'
OR coalesce(ai_tags::text, '') ILIKE '%%' || :q || '%%'
OR coalesce(user_note, '') ILIKE '%%' || :q || '%%'
OR coalesce(ai_summary, '') ILIKE '%%' || :q || '%%'
OR coalesce(extracted_text, '') ILIKE '%%' || :q || '%%'
OR to_tsvector('simple', coalesce(title, '') || ' ' || coalesce(extracted_text, ''))
@@ plainto_tsquery('simple', :q))
ORDER BY score DESC
LIMIT :limit
"""),
{"q": query, "limit": limit},
)
return [SearchResult(**row._mapping) for row in result]
async def _search_vector(session: AsyncSession, query: str, limit: int) -> list[SearchResult]:
"""벡터 유사도 검색 (코사인 거리)"""
try:
client = AIClient()
query_embedding = await client.embed(query)
await client.close()
except Exception:
return []
result = await session.execute(
text("""
SELECT id, title, ai_domain, ai_summary, file_format,
(1 - (embedding <=> cast(:embedding AS vector))) AS score,
left(extracted_text, 200) AS snippet,
'vector' AS match_reason
FROM documents
WHERE embedding IS NOT NULL AND deleted_at IS NULL
ORDER BY embedding <=> cast(:embedding AS vector)
LIMIT :limit
"""),
{"embedding": str(query_embedding), "limit": limit},
)
return [SearchResult(**row._mapping) for row in result]
def _merge_results(
text_results: list[SearchResult],
vector_results: list[SearchResult],
limit: int,
) -> list[SearchResult]:
"""텍스트 + 벡터 결과 합산 (중복 제거, 점수 합산)"""
merged: dict[int, SearchResult] = {}
for r in text_results:
merged[r.id] = r
for r in vector_results:
if r.id in merged:
# 이미 텍스트로 잡힌 문서 — 벡터 점수 가산
existing = merged[r.id]
merged[r.id] = SearchResult(
id=existing.id,
title=existing.title,
ai_domain=existing.ai_domain,
ai_summary=existing.ai_summary,
file_format=existing.file_format,
score=existing.score + r.score * 0.5,
snippet=existing.snippet,
match_reason=f"{existing.match_reason}+vector",
)
elif r.score > 0.3: # 벡터 유사도 최소 threshold
merged[r.id] = r
results = sorted(merged.values(), key=lambda x: x.score, reverse=True)
return results[:limit]

234
app/api/setup.py Normal file
View File

@@ -0,0 +1,234 @@
"""첫 접속 셋업 위자드 API
유저가 0명일 때만 동작. 셋업 완료 후 자동 비활성화.
"""
import time
from pathlib import Path
from typing import Annotated
import pyotp
from fastapi import APIRouter, Depends, HTTPException, Request, status
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from pydantic import BaseModel
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from core.auth import create_access_token, create_refresh_token, hash_password
from core.config import settings
from core.database import get_session
from models.user import User
router = APIRouter()
templates = Jinja2Templates(directory=Path(__file__).parent.parent / "templates")
# ─── Rate Limiting (인메모리, 단일 프로세스) ───
_failed_attempts: dict[str, list[float]] = {}
RATE_LIMIT_MAX = 5
RATE_LIMIT_WINDOW = 300 # 5분
def _check_rate_limit(client_ip: str):
"""5분 내 5회 실패 시 차단"""
now = time.time()
attempts = _failed_attempts.get(client_ip, [])
# 윈도우 밖의 기록 제거
attempts = [t for t in attempts if now - t < RATE_LIMIT_WINDOW]
_failed_attempts[client_ip] = attempts
if len(attempts) >= RATE_LIMIT_MAX:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"너무 많은 시도입니다. {RATE_LIMIT_WINDOW // 60}분 후 다시 시도하세요.",
)
def _record_failure(client_ip: str):
_failed_attempts.setdefault(client_ip, []).append(time.time())
# ─── 헬퍼: 셋업 필요 여부 ───
async def _needs_setup(session: AsyncSession) -> bool:
result = await session.execute(select(func.count(User.id)))
return result.scalar() == 0
async def _require_setup(session: AsyncSession):
if not await _needs_setup(session):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="셋업이 이미 완료되었습니다",
)
# ─── 스키마 ───
class SetupStatusResponse(BaseModel):
needs_setup: bool
class CreateAdminRequest(BaseModel):
username: str
password: str
class CreateAdminResponse(BaseModel):
message: str
access_token: str
refresh_token: str
class TOTPInitResponse(BaseModel):
secret: str
otpauth_uri: str
class TOTPVerifyRequest(BaseModel):
secret: str
code: str
class VerifyNASRequest(BaseModel):
path: str
class VerifyNASResponse(BaseModel):
exists: bool
readable: bool
writable: bool
path: str
# ─── 엔드포인트 ───
@router.get("/status", response_model=SetupStatusResponse)
async def setup_status(session: Annotated[AsyncSession, Depends(get_session)]):
"""셋업 필요 여부 확인"""
return SetupStatusResponse(needs_setup=await _needs_setup(session))
@router.post("/admin", response_model=CreateAdminResponse)
async def create_admin(
body: CreateAdminRequest,
request: Request,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""관리자 계정 생성 (유저 0명일 때만)"""
await _require_setup(session)
client_ip = request.client.host if request.client else "unknown"
_check_rate_limit(client_ip)
# 유효성 검사
if len(body.username) < 2:
_record_failure(client_ip)
raise HTTPException(status_code=400, detail="아이디는 2자 이상이어야 합니다")
if len(body.password) < 8:
_record_failure(client_ip)
raise HTTPException(status_code=400, detail="비밀번호는 8자 이상이어야 합니다")
user = User(
username=body.username,
password_hash=hash_password(body.password),
is_active=True,
)
session.add(user)
await session.commit()
return CreateAdminResponse(
message=f"관리자 '{body.username}' 계정이 생성되었습니다",
access_token=create_access_token(body.username),
refresh_token=create_refresh_token(body.username),
)
@router.post("/totp/init", response_model=TOTPInitResponse)
async def totp_init(
request: Request,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""TOTP 시크릿 생성 + otpauth URI 반환 (DB에 저장하지 않음)"""
await _require_setup(session)
secret = pyotp.random_base32()
totp = pyotp.TOTP(secret)
uri = totp.provisioning_uri(
name="admin",
issuer_name="hyungi Document Server",
)
return TOTPInitResponse(secret=secret, otpauth_uri=uri)
@router.post("/totp/verify")
async def totp_verify(
body: TOTPVerifyRequest,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""TOTP 코드 검증 후 DB에 시크릿 저장"""
await _require_setup(session)
totp = pyotp.TOTP(body.secret)
if not totp.verify(body.code):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="TOTP 코드가 올바르지 않습니다. 다시 시도하세요.",
)
# 가장 최근 생성된 유저에 저장 (셋업 직후이므로 유저 1명)
result = await session.execute(
select(User).order_by(User.id.desc()).limit(1)
)
user = result.scalar_one_or_none()
if not user:
raise HTTPException(status_code=404, detail="유저를 찾을 수 없습니다")
user.totp_secret = body.secret
await session.commit()
return {"message": "TOTP 2FA가 활성화되었습니다"}
@router.post("/verify-nas", response_model=VerifyNASResponse)
async def verify_nas(
body: VerifyNASRequest,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""NAS 마운트 경로 읽기/쓰기 테스트"""
await _require_setup(session)
path = Path(body.path)
exists = path.exists()
readable = path.is_dir() and any(True for _ in path.iterdir()) if exists else False
writable = False
if exists:
test_file = path / ".pkm_write_test"
try:
test_file.write_text("test")
test_file.unlink()
writable = True
except OSError:
pass
return VerifyNASResponse(
exists=exists,
readable=readable,
writable=writable,
path=str(path),
)
@router.get("/", response_class=HTMLResponse)
async def setup_page(
request: Request,
session: Annotated[AsyncSession, Depends(get_session)],
):
"""셋업 위자드 HTML 페이지"""
if not await _needs_setup(session):
from fastapi.responses import RedirectResponse
return RedirectResponse(url="/docs")
return templates.TemplateResponse(request, "setup.html")

View File

@@ -1,14 +1,20 @@
"""JWT + TOTP 2FA 인증"""
from datetime import datetime, timedelta, timezone
from typing import Annotated
import bcrypt
import pyotp
from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from jose import JWTError, jwt
from passlib.context import CryptContext
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from core.config import settings
from core.database import get_session
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
security = HTTPBearer()
# JWT 설정
ALGORITHM = "HS256"
@@ -17,11 +23,11 @@ REFRESH_TOKEN_EXPIRE_DAYS = 7
def verify_password(plain: str, hashed: str) -> bool:
return pwd_context.verify(plain, hashed)
return bcrypt.checkpw(plain.encode(), hashed.encode())
def hash_password(password: str) -> str:
return pwd_context.hash(password)
return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
def create_access_token(subject: str) -> str:
@@ -43,9 +49,37 @@ def decode_token(token: str) -> dict | None:
return None
def verify_totp(code: str) -> bool:
"""TOTP 코드 검증"""
if not settings.totp_secret:
def verify_totp(code: str, secret: str | None = None) -> bool:
"""TOTP 코드 검증 (유저별 secret 또는 글로벌 설정)"""
totp_secret = secret or settings.totp_secret
if not totp_secret:
return True # TOTP 미설정 시 스킵
totp = pyotp.TOTP(settings.totp_secret)
totp = pyotp.TOTP(totp_secret)
return totp.verify(code)
async def get_current_user(
credentials: Annotated[HTTPAuthorizationCredentials, Depends(security)],
session: Annotated[AsyncSession, Depends(get_session)],
):
"""Bearer 토큰에서 현재 유저 조회"""
from models.user import User
payload = decode_token(credentials.credentials)
if not payload or payload.get("type") != "access":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="유효하지 않은 토큰",
)
username = payload.get("sub")
result = await session.execute(
select(User).where(User.username == username, User.is_active.is_(True))
)
user = result.scalar_one_or_none()
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="유저를 찾을 수 없음",
)
return user

View File

@@ -44,6 +44,10 @@ class Settings(BaseModel):
# kordoc
kordoc_endpoint: str = "http://kordoc-service:3100"
# 분류 체계
taxonomy: dict = {}
document_types: list[str] = []
def load_settings() -> Settings:
"""config.yaml + 환경변수에서 설정 로딩"""
@@ -53,8 +57,10 @@ def load_settings() -> Settings:
totp_secret = os.getenv("TOTP_SECRET", "")
kordoc_endpoint = os.getenv("KORDOC_ENDPOINT", "http://kordoc-service:3100")
# config.yaml
config_path = Path(__file__).parent.parent.parent / "config.yaml"
# config.yaml — Docker 컨테이너 내부(/app/config.yaml) 또는 프로젝트 루트
config_path = Path("/app/config.yaml")
if not config_path.exists():
config_path = Path(__file__).parent.parent.parent / "config.yaml"
ai_config = None
nas_mount = "/documents"
nas_pkm = "/documents/PKM"
@@ -79,6 +85,9 @@ def load_settings() -> Settings:
nas_mount = raw["nas"].get("mount_path", nas_mount)
nas_pkm = raw["nas"].get("pkm_root", nas_pkm)
taxonomy = raw.get("taxonomy", {}) if config_path.exists() and raw else {}
document_types = raw.get("document_types", []) if config_path.exists() and raw else []
return Settings(
database_url=database_url,
ai=ai_config,
@@ -87,6 +96,8 @@ def load_settings() -> Settings:
jwt_secret=jwt_secret,
totp_secret=totp_secret,
kordoc_endpoint=kordoc_endpoint,
taxonomy=taxonomy,
document_types=document_types,
)

View File

@@ -1,10 +1,17 @@
"""PostgreSQL 연결 — SQLAlchemy async engine + session factory"""
import logging
import re
import time
from pathlib import Path
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.orm import DeclarativeBase
from core.config import settings
logger = logging.getLogger("migration")
engine = create_async_engine(
settings.database_url,
echo=False,
@@ -19,13 +26,116 @@ class Base(DeclarativeBase):
pass
async def init_db():
"""DB 연결 확인 (스키마는 migrations/로 관리)"""
async with engine.begin() as conn:
# 연결 테스트
await conn.execute(
__import__("sqlalchemy").text("SELECT 1")
# NOTE: 모든 pending migration은 단일 트랜잭션으로 실행됨.
# DDL이 많거나 대량 데이터 변경이 포함된 migration은 장시간 lock을 유발할 수 있음.
_MIGRATION_VERSION_RE = re.compile(r"^(\d+)_")
_MIGRATION_LOCK_KEY = 938475
def _parse_migration_files(migrations_dir: Path) -> list[tuple[int, str, Path]]:
"""migration 파일 스캔 → (version, name, path) 리스트, 버전순 정렬"""
files = []
for p in sorted(migrations_dir.glob("*.sql")):
m = _MIGRATION_VERSION_RE.match(p.name)
if not m:
continue
version = int(m.group(1))
files.append((version, p.name, p))
# 중복 버전 검사
seen: dict[int, str] = {}
for version, name, _ in files:
if version in seen:
raise RuntimeError(
f"migration 버전 중복: {seen[version]} vs {name} (version={version})"
)
seen[version] = name
files.sort(key=lambda x: x[0])
return files
def _validate_sql_content(name: str, sql: str) -> None:
"""migration SQL에 BEGIN/COMMIT이 포함되어 있으면 에러 (외부 트랜잭션 깨짐 방지)"""
# 주석(-- ...) 라인 제거 후 검사
lines = [
line for line in sql.splitlines()
if not line.strip().startswith("--")
]
stripped = "\n".join(lines).upper()
for keyword in ("BEGIN", "COMMIT", "ROLLBACK"):
# 단어 경계로 매칭 (예: BEGIN_SOMETHING은 제외)
if re.search(rf"\b{keyword}\b", stripped):
raise RuntimeError(
f"migration {name}{keyword} 포함됨 — "
f"migration SQL에는 트랜잭션 제어문을 넣지 마세요"
)
async def _run_migrations(conn) -> None:
"""미적용 migration 실행 (호출자가 트랜잭션 관리)"""
from sqlalchemy import text
# schema_migrations 테이블 생성
await conn.execute(text("""
CREATE TABLE IF NOT EXISTS schema_migrations (
version INT PRIMARY KEY,
name TEXT NOT NULL,
applied_at TIMESTAMPTZ DEFAULT NOW()
)
"""))
# advisory lock 획득 (트랜잭션 끝나면 자동 해제)
await conn.execute(text(
f"SELECT pg_advisory_xact_lock({_MIGRATION_LOCK_KEY})"
))
# 적용 이력 조회
result = await conn.execute(text("SELECT version FROM schema_migrations"))
applied = {row[0] for row in result}
# migration 파일 스캔
migrations_dir = Path(__file__).resolve().parent.parent.parent / "migrations"
if not migrations_dir.is_dir():
logger.info("[migration] migrations/ 디렉토리 없음, 스킵")
return
files = _parse_migration_files(migrations_dir)
pending = [(v, name, path) for v, name, path in files if v not in applied]
if not pending:
logger.info("[migration] 미적용 migration 없음")
return
start = time.monotonic()
logger.info(f"[migration] {len(pending)}건 적용 시작")
for version, name, path in pending:
sql = path.read_text(encoding="utf-8")
_validate_sql_content(name, sql)
logger.info(f"[migration] {name} 실행 중...")
await conn.execute(text(sql))
await conn.execute(
text("INSERT INTO schema_migrations (version, name) VALUES (:v, :n)"),
{"v": version, "n": name},
)
logger.info(f"[migration] {name} 완료")
elapsed = time.monotonic() - start
logger.info(f"[migration] 전체 {len(pending)}건 완료 ({elapsed:.1f}s)")
async def init_db():
"""DB 연결 확인 + pending migration 실행"""
from sqlalchemy import text
async with engine.begin() as conn:
await conn.execute(text("SELECT 1"))
try:
await _run_migrations(conn)
except Exception as e:
logger.error(f"[migration] 실패: {e} — 전체 트랜잭션 롤백")
raise
async def get_session() -> AsyncSession:

View File

@@ -44,3 +44,95 @@ def count_log_errors(log_path: str) -> int:
return sum(1 for line in f if "[ERROR]" in line)
except FileNotFoundError:
return 0
# ─── CalDAV 헬퍼 ───
def escape_ical_text(text: str | None) -> str:
"""iCalendar TEXT 값 이스케이프 (RFC 5545 §3.3.11).
SUMMARY, DESCRIPTION, LOCATION 등 TEXT 프로퍼티에 사용.
"""
if not text:
return ""
text = text.replace("\r\n", "\n").replace("\r", "\n") # CRLF 정규화
text = text.replace("\\", "\\\\") # 백슬래시 먼저
text = text.replace("\n", "\\n")
text = text.replace(",", "\\,")
text = text.replace(";", "\\;")
return text
def create_caldav_todo(
caldav_url: str,
username: str,
password: str,
title: str,
description: str = "",
due_days: int = 7,
) -> str | None:
"""Synology Calendar에 VTODO 생성, UID 반환"""
import uuid
from datetime import datetime, timedelta, timezone
import caldav
try:
client = caldav.DAVClient(url=caldav_url, username=username, password=password)
principal = client.principal()
calendars = principal.calendars()
if not calendars:
return None
calendar = calendars[0]
uid = str(uuid.uuid4())
due = datetime.now(timezone.utc) + timedelta(days=due_days)
due_str = due.strftime("%Y%m%dT%H%M%SZ")
vtodo = f"""BEGIN:VCALENDAR
VERSION:2.0
BEGIN:VTODO
UID:{uid}
SUMMARY:{escape_ical_text(title)}
DESCRIPTION:{escape_ical_text(description)}
DUE:{due_str}
STATUS:NEEDS-ACTION
PRIORITY:5
END:VTODO
END:VCALENDAR"""
calendar.save_event(vtodo)
return uid
except Exception as e:
logging.getLogger("caldav").error(f"CalDAV VTODO 생성 실패: {e}")
return None
# ─── SMTP 헬퍼 ───
def send_smtp_email(
host: str,
port: int,
username: str,
password: str,
subject: str,
body: str,
to_addr: str | None = None,
):
"""Synology MailPlus SMTP로 이메일 발송"""
import smtplib
from email.mime.text import MIMEText
to_addr = to_addr or username
msg = MIMEText(body, "plain", "utf-8")
msg["Subject"] = subject
msg["From"] = username
msg["To"] = to_addr
try:
with smtplib.SMTP_SSL(host, port, timeout=30) as server:
server.login(username, password)
server.send_message(msg)
except Exception as e:
logging.getLogger("smtp").error(f"SMTP 발송 실패: {e}")

View File

@@ -2,21 +2,63 @@
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi import FastAPI, Request
from fastapi.responses import RedirectResponse
from sqlalchemy import func, select, text
from api.auth import router as auth_router
from api.dashboard import router as dashboard_router
from api.documents import router as documents_router
from api.news import router as news_router
from api.search import router as search_router
from api.setup import router as setup_router
from core.config import settings
from core.database import init_db
from core.database import async_session, engine, init_db
from models.user import User
@asynccontextmanager
async def lifespan(app: FastAPI):
"""앱 시작/종료 시 실행되는 lifespan 핸들러"""
# 시작: DB 연결, 스케줄러 등록
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.cron import CronTrigger
from workers.daily_digest import run as daily_digest_run
from workers.file_watcher import watch_inbox
from workers.law_monitor import run as law_monitor_run
from workers.mailplus_archive import run as mailplus_run
from workers.news_collector import run as news_collector_run
from workers.queue_consumer import consume_queue
# 시작: DB 연결 확인
await init_db()
# TODO: APScheduler 시작 (Phase 3)
# NAS 마운트 확인 (NFS 미마운트 시 로컬 빈 디렉토리에 쓰는 것 방지)
from pathlib import Path
nas_check = Path(settings.nas_mount_path) / "PKM"
if not nas_check.is_dir():
raise RuntimeError(
f"NAS 마운트 확인 실패: {nas_check} 디렉토리 없음. "
f"NFS 마운트 상태를 확인하세요."
)
# APScheduler: 백그라운드 작업
scheduler = AsyncIOScheduler(timezone="Asia/Seoul")
# 상시 실행
scheduler.add_job(consume_queue, "interval", minutes=1, id="queue_consumer")
scheduler.add_job(watch_inbox, "interval", minutes=5, id="file_watcher")
# 일일 스케줄 (KST)
scheduler.add_job(law_monitor_run, CronTrigger(hour=7), id="law_monitor")
scheduler.add_job(mailplus_run, CronTrigger(hour=7), id="mailplus_morning")
scheduler.add_job(mailplus_run, CronTrigger(hour=18), id="mailplus_evening")
scheduler.add_job(daily_digest_run, CronTrigger(hour=20), id="daily_digest")
scheduler.add_job(news_collector_run, "interval", hours=6, id="news_collector")
scheduler.start()
yield
# 종료: 리소스 정리
# TODO: 스케줄러 종료, DB 연결 해제
# 종료: 스케줄러 DB 순서로 정리
scheduler.shutdown(wait=False)
await engine.dispose()
app = FastAPI(
@@ -26,16 +68,70 @@ app = FastAPI(
lifespan=lifespan,
)
# ─── 라우터 등록 ───
app.include_router(setup_router, prefix="/api/setup", tags=["setup"])
app.include_router(auth_router, prefix="/api/auth", tags=["auth"])
app.include_router(documents_router, prefix="/api/documents", tags=["documents"])
app.include_router(search_router, prefix="/api/search", tags=["search"])
app.include_router(dashboard_router, prefix="/api/dashboard", tags=["dashboard"])
app.include_router(news_router, prefix="/api/news", tags=["news"])
# TODO: Phase 5에서 추가
# app.include_router(tasks.router, prefix="/api/tasks", tags=["tasks"])
# app.include_router(export.router, prefix="/api/export", tags=["export"])
# ─── 셋업 미들웨어: 유저 0명이면 /setup으로 리다이렉트 ───
SETUP_BYPASS_PREFIXES = (
"/api/setup", "/setup", "/health", "/docs", "/openapi.json", "/redoc",
)
@app.middleware("http")
async def setup_redirect_middleware(request: Request, call_next):
path = request.url.path
# 바이패스 경로는 항상 통과
if any(path.startswith(p) for p in SETUP_BYPASS_PREFIXES):
return await call_next(request)
# 유저 존재 여부 확인
try:
async with async_session() as session:
result = await session.execute(select(func.count(User.id)))
user_count = result.scalar()
if user_count == 0:
return RedirectResponse(url="/setup")
except Exception:
pass # DB 연결 실패 시 통과 (health에서 확인 가능)
return await call_next(request)
# ─── 셋업 페이지 라우트 (API가 아닌 HTML 페이지) ───
@app.get("/setup")
async def setup_page_redirect(request: Request):
"""셋업 위자드 페이지로 포워딩"""
from api.setup import setup_page
from core.database import get_session
async for session in get_session():
return await setup_page(request, session)
@app.get("/health")
async def health_check():
return {"status": "ok", "version": "2.0.0"}
"""헬스체크 — DB 연결 상태 포함"""
db_ok = False
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
db_ok = True
except Exception:
pass
# TODO: 라우터 등록 (Phase 0~2)
# from api import documents, search, tasks, dashboard, export
# app.include_router(documents.router, prefix="/api/documents", tags=["documents"])
# app.include_router(search.router, prefix="/api/search", tags=["search"])
# app.include_router(tasks.router, prefix="/api/tasks", tags=["tasks"])
# app.include_router(dashboard.router, prefix="/api/dashboard", tags=["dashboard"])
# app.include_router(export.router, prefix="/api/export", tags=["export"])
return {
"status": "ok" if db_ok else "degraded",
"version": "2.0.0",
"database": "connected" if db_ok else "disconnected",
}

20
app/models/automation.py Normal file
View File

@@ -0,0 +1,20 @@
"""automation_state 테이블 ORM — 자동화 워커 증분 동기화 상태"""
from datetime import datetime
from sqlalchemy import BigInteger, DateTime, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from core.database import Base
class AutomationState(Base):
__tablename__ = "automation_state"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
job_name: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
last_check_value: Mapped[str | None] = mapped_column(Text)
last_run_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.now, onupdate=datetime.now
)

View File

@@ -3,7 +3,7 @@
from datetime import datetime
from pgvector.sqlalchemy import Vector
from sqlalchemy import BigInteger, DateTime, Enum, String, Text
from sqlalchemy import BigInteger, Boolean, DateTime, Enum, String, Text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
@@ -38,16 +38,42 @@ class Document(Base):
ai_sub_group: Mapped[str | None] = mapped_column(String(100))
ai_model_version: Mapped[str | None] = mapped_column(String(50))
ai_processed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
document_type: Mapped[str | None] = mapped_column(String(50))
importance: Mapped[str | None] = mapped_column(String(20), default="medium")
ai_confidence: Mapped[float | None] = mapped_column()
# 3계층: 벡터 임베딩
embedding = mapped_column(Vector(768), nullable=True)
embedding = mapped_column(Vector(1024), nullable=True)
embed_model_version: Mapped[str | None] = mapped_column(String(50))
embedded_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# 사용자 메모
user_note: Mapped[str | None] = mapped_column(Text)
# ODF 변환
derived_path: Mapped[str | None] = mapped_column(Text) # 변환본 경로 (.derived/)
original_format: Mapped[str | None] = mapped_column(String(20))
conversion_status: Mapped[str | None] = mapped_column(String(20), default="none")
# 읽음 상태 (뉴스용)
is_read: Mapped[bool | None] = mapped_column(Boolean, default=False)
# 승인/삭제
review_status: Mapped[str | None] = mapped_column(String(20), default="pending")
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# 외부 편집 URL
edit_url: Mapped[str | None] = mapped_column(Text)
# 미리보기
preview_status: Mapped[str | None] = mapped_column(String(20), default="none")
preview_hash: Mapped[str | None] = mapped_column(String(64))
preview_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# 메타데이터
source_channel: Mapped[str | None] = mapped_column(
Enum("law_monitor", "devonagent", "email", "web_clip",
"tksafety", "inbox_route", "manual", "drive_sync",
"tksafety", "inbox_route", "manual", "drive_sync", "news",
name="source_channel")
)
data_origin: Mapped[str | None] = mapped_column(

25
app/models/news_source.py Normal file
View File

@@ -0,0 +1,25 @@
"""news_sources 테이블 ORM"""
from datetime import datetime
from sqlalchemy import Boolean, DateTime, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from core.database import Base
class NewsSource(Base):
__tablename__ = "news_sources"
id: Mapped[int] = mapped_column(primary_key=True)
name: Mapped[str] = mapped_column(String(100), nullable=False)
country: Mapped[str | None] = mapped_column(String(10))
feed_url: Mapped[str] = mapped_column(Text, nullable=False)
feed_type: Mapped[str] = mapped_column(String(20), default="rss")
category: Mapped[str | None] = mapped_column(String(50))
language: Mapped[str | None] = mapped_column(String(10))
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
last_fetched_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.now
)

View File

@@ -14,7 +14,7 @@ class ProcessingQueue(Base):
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
document_id: Mapped[int] = mapped_column(BigInteger, ForeignKey("documents.id"), nullable=False)
stage: Mapped[str] = mapped_column(
Enum("extract", "classify", "embed", name="process_stage"), nullable=False
Enum("extract", "classify", "embed", "preview", name="process_stage"), nullable=False
)
status: Mapped[str] = mapped_column(
Enum("pending", "processing", "completed", "failed", name="process_status"),

22
app/models/user.py Normal file
View File

@@ -0,0 +1,22 @@
"""users 테이블 ORM"""
from datetime import datetime
from sqlalchemy import BigInteger, Boolean, DateTime, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from core.database import Base
class User(Base):
__tablename__ = "users"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
username: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
password_hash: Mapped[str] = mapped_column(Text, nullable=False)
totp_secret: Mapped[str | None] = mapped_column(String(64))
is_active: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=datetime.now
)
last_login_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))

View File

@@ -1,51 +1,93 @@
당신은 문서 분류 AI입니다. 아래 문서를 분석하고 반드시 JSON 형식으로만 응답하세요. 다른 텍스트는 출력하지 마세요.
You are a document classification AI. Analyze the document below and respond ONLY in JSON format. No other text.
## 응답 형식
## Response Format
{
"tags": ["태그1", "태그2", "태그3"],
"domain": "도메인경로",
"sub_group": "하위그룹",
"sourceChannel": "유입경로",
"dataOrigin": "work 또는 external"
"domain": "Level1/Level2/Level3",
"document_type": "one of document_types",
"confidence": 0.85,
"tags": ["tag1", "tag2"],
"importance": "medium",
"sourceChannel": "inbox_route",
"dataOrigin": "work or external"
}
## 도메인 선택지 (NAS 폴더 경로)
- Knowledge/Philosophy — 철학, 사상, 인문학
- Knowledge/Language — 어학, 번역, 언어학
- Knowledge/Engineering — 공학 전반 기술 문서
- Knowledge/Industrial_Safety — 산업안전, 규정, 인증
- Knowledge/Programming — 개발, 코드, IT 기술
- Knowledge/General — 일반 도서, 독서 노트, 메모
- Reference — 도면, 참고자료, 규격표
## Domain Taxonomy (select the most specific leaf node)
## 하위 그룹 예시 (도메인별)
- Knowledge/Industrial_Safety: Legislation, Standards, Cases
- Knowledge/Programming: Language, Framework, DevOps, AI_ML
- Knowledge/Engineering: Mechanical, Electrical, Network
- 잘 모르겠으면: (비워둠)
Philosophy/
Ethics, Metaphysics, Epistemology, Logic, Aesthetics, Eastern_Philosophy, Western_Philosophy
## 태그 체계
태그는 최대 5개, 한글 사용. 아래 계층 구조 중에서 선택:
- @상태/: 처리중, 검토필요, 완료, 아카이브
- #주제/기술/: 서버관리, 네트워크, AI-ML
- #주제/산업안전/: 법령, 위험성평가, 순회점검, 안전교육, 사고사례, 신고보고, 안전관리자, 보건관리자
- #주제/업무/: 프로젝트, 회의, 보고서
- $유형/: 논문, 법령, 기사, 메모, 이메일, 채팅로그, 도면, 체크리스트
- !우선순위/: 긴급, 중요, 참고
Language/
Korean, English, Japanese, Translation, Linguistics
## sourceChannel 값
- tksafety: TKSafety API 업무 실적
- devonagent: 자동 수집 뉴스
- law_monitor: 법령 API 법령 변경
- inbox_route: Inbox AI 분류 (이 프롬프트에 의한 분류)
- email: MailPlus 이메일
- web_clip: Web Clipper 스크랩
- manual: 직접 추가
- drive_sync: Synology Drive 동기화
Engineering/
Mechanical/ Piping, HVAC, Equipment
Electrical/ Power, Instrumentation
Chemical/ Process, Material
Civil
Network/ Server, Security, Infrastructure
## dataOrigin 값
- work: 자사 업무 관련 (TK, 테크니컬코리아, 공장, 생산, 사내)
- external: 외부 참고 자료 (뉴스, 논문, 법령, 일반 정보)
Industrial_Safety/
Legislation/ Act, Decree, Foreign_Law, Korea_Law_Archive, Enforcement_Rule, Public_Notice, SAPA
Theory/ Industrial_Safety_General, Safety_Health_Fundamentals
Academic_Papers/ Safety_General, Risk_Assessment_Research
Cases/ Domestic, International
Practice/ Checklist, Contractor_Management, Safety_Education, Emergency_Plan, Patrol_Inspection, Permit_to_Work, PPE, Safety_Plan
Risk_Assessment/ KRAS, JSA, Checklist_Method
Safety_Manager/ Appointment, Duty_Record, Improvement, Inspection, Meeting
Health_Manager/ Appointment, Duty_Record, Ergonomics, Health_Checkup, Mental_Health, MSDS, Work_Environment
## 분류 대상 문서
Programming/
Programming_Language/ Python, JavaScript, Go, Rust
Framework/ FastAPI, SvelteKit, React
DevOps/ Docker, CI_CD, Linux_Administration
AI_ML/ Large_Language_Model, Computer_Vision, Data_Science
Database
Software_Architecture
General/
Reading_Notes, Self_Development, Business, Science, History
## Classification Rules
- domain MUST be the most specific leaf node (e.g., Industrial_Safety/Practice/Patrol_Inspection, NOT Industrial_Safety/Practice)
- domain MUST be exactly ONE path
- If content spans multiple domains, choose by PRIMARY purpose
- If safety content is >30%, prefer Industrial_Safety
- If code is included, prefer Programming
- 2-level paths allowed ONLY when no leaf exists (e.g., Engineering/Civil)
## Document Types (select exactly ONE)
Reference, Standard, Manual, Drawing, Template, Note, Academic_Paper, Law_Document, Report, Memo, Checklist, Meeting_Minutes, Specification
### Document Type Detection Rules
- Step-by-step instructions → Manual
- Legal clauses/regulations → Law_Document
- Technical requirements → Specification
- Meeting discussion → Meeting_Minutes
- Checklist format → Checklist
- Academic/research format → Academic_Paper
- Technical drawings → Drawing
- If unclear → Note
## Confidence (0.0 ~ 1.0)
- How confident are you in the domain classification?
- 0.85+ = high confidence, 0.6~0.85 = moderate, <0.6 = uncertain
## Tags
- Free-form tags (Korean or English)
- Include: person names, technology names, concepts, project names
- Maximum 5 tags
## Importance
- high: urgent or critical documents
- medium: normal working documents
- low: reference or archive material
## sourceChannel
- inbox_route (this classification)
## dataOrigin
- work: company-related (TK, Technicalkorea, factory, production)
- external: external reference (news, papers, laws, general info)
## Document to classify
{document_text}

View File

@@ -7,10 +7,12 @@ python-dotenv>=1.0.0
pyyaml>=6.0
httpx>=0.27.0
python-jose[cryptography]>=3.3.0
passlib[bcrypt]>=1.7.4
bcrypt>=4.0.0
pyotp>=2.9.0
caldav>=1.3.0
apscheduler>=3.10.0
anthropic>=0.40.0
markdown>=3.5.0
python-multipart>=0.0.9
jinja2>=3.1.0
feedparser>=6.0.0

405
app/templates/setup.html Normal file
View File

@@ -0,0 +1,405 @@
<!DOCTYPE html>
<html lang="ko">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>hyungi Document Server — 초기 설정</title>
<script src="https://cdn.jsdelivr.net/npm/qrcode@1.5.4/build/qrcode.min.js"></script>
<style>
:root {
--bg: #0f1117;
--surface: #1a1d27;
--border: #2a2d3a;
--text: #e4e4e7;
--text-dim: #8b8d98;
--accent: #6c8aff;
--accent-hover: #859dff;
--error: #f5564e;
--success: #4ade80;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
background: var(--bg);
color: var(--text);
min-height: 100vh;
display: flex;
justify-content: center;
align-items: center;
}
.container {
width: 100%;
max-width: 480px;
padding: 2rem;
}
h1 {
font-size: 1.5rem;
margin-bottom: 0.25rem;
}
.subtitle {
color: var(--text-dim);
font-size: 0.9rem;
margin-bottom: 2rem;
}
.steps {
display: flex;
gap: 0.5rem;
margin-bottom: 2rem;
}
.step-dot {
width: 2.5rem;
height: 4px;
border-radius: 2px;
background: var(--border);
transition: background 0.3s;
}
.step-dot.active { background: var(--accent); }
.step-dot.done { background: var(--success); }
.card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 12px;
padding: 1.5rem;
}
.card h2 {
font-size: 1.1rem;
margin-bottom: 1rem;
}
.field {
margin-bottom: 1rem;
}
label {
display: block;
font-size: 0.85rem;
color: var(--text-dim);
margin-bottom: 0.3rem;
}
input[type="text"], input[type="password"] {
width: 100%;
padding: 0.65rem 0.75rem;
background: var(--bg);
border: 1px solid var(--border);
border-radius: 8px;
color: var(--text);
font-size: 0.95rem;
outline: none;
transition: border-color 0.2s;
}
input:focus { border-color: var(--accent); }
.btn {
display: inline-block;
padding: 0.65rem 1.5rem;
background: var(--accent);
color: #fff;
border: none;
border-radius: 8px;
font-size: 0.95rem;
cursor: pointer;
transition: background 0.2s;
width: 100%;
margin-top: 0.5rem;
}
.btn:hover { background: var(--accent-hover); }
.btn:disabled { opacity: 0.5; cursor: not-allowed; }
.btn-skip {
background: transparent;
border: 1px solid var(--border);
color: var(--text-dim);
margin-top: 0.5rem;
}
.btn-skip:hover { border-color: var(--text-dim); }
.error-msg {
color: var(--error);
font-size: 0.85rem;
margin-top: 0.5rem;
display: none;
}
.success-msg {
color: var(--success);
font-size: 0.85rem;
margin-top: 0.5rem;
display: none;
}
.qr-wrap {
display: flex;
justify-content: center;
margin: 1rem 0;
background: #fff;
border-radius: 8px;
padding: 1rem;
width: fit-content;
margin-left: auto;
margin-right: auto;
}
.secret-text {
font-family: monospace;
font-size: 0.8rem;
color: var(--text-dim);
word-break: break-all;
text-align: center;
margin-bottom: 1rem;
}
.nas-result {
font-size: 0.85rem;
margin-top: 0.5rem;
}
.nas-result span { margin-right: 1rem; }
.check { color: var(--success); }
.cross { color: var(--error); }
.hidden { display: none; }
.done-icon {
font-size: 3rem;
text-align: center;
margin-bottom: 1rem;
}
</style>
</head>
<body>
<div class="container">
<h1>hyungi Document Server</h1>
<p class="subtitle">초기 설정 위자드</p>
<div class="steps">
<div class="step-dot active" id="dot-0"></div>
<div class="step-dot" id="dot-1"></div>
<div class="step-dot" id="dot-2"></div>
</div>
<!-- Step 0: 관리자 계정 -->
<div class="card" id="step-0">
<h2>1. 관리자 계정 생성</h2>
<div class="field">
<label for="username">아이디</label>
<input type="text" id="username" placeholder="admin" autocomplete="username">
</div>
<div class="field">
<label for="password">비밀번호 (8자 이상)</label>
<input type="password" id="password" autocomplete="new-password">
</div>
<div class="field">
<label for="password2">비밀번호 확인</label>
<input type="password" id="password2" autocomplete="new-password">
</div>
<div class="error-msg" id="admin-error"></div>
<button class="btn" onclick="createAdmin()">계정 생성</button>
</div>
<!-- Step 1: TOTP 2FA -->
<div class="card hidden" id="step-1">
<h2>2. 2단계 인증 (TOTP)</h2>
<p style="color: var(--text-dim); font-size: 0.85rem; margin-bottom: 1rem;">
Google Authenticator 등 인증 앱으로 QR 코드를 스캔하세요.
</p>
<div class="qr-wrap" id="qr-container"></div>
<p class="secret-text" id="totp-secret-text"></p>
<div class="field">
<label for="totp-code">인증 코드 6자리</label>
<input type="text" id="totp-code" maxlength="6" placeholder="000000" inputmode="numeric" pattern="[0-9]*">
</div>
<div class="error-msg" id="totp-error"></div>
<div class="success-msg" id="totp-success"></div>
<button class="btn" onclick="verifyTOTP()">인증 확인</button>
<button class="btn btn-skip" onclick="skipTOTP()">건너뛰기</button>
</div>
<!-- Step 2: NAS 경로 확인 -->
<div class="card hidden" id="step-2">
<h2>3. NAS 저장소 경로 확인</h2>
<div class="field">
<label for="nas-path">NAS 마운트 경로</label>
<input type="text" id="nas-path" value="/documents">
</div>
<div class="nas-result hidden" id="nas-result"></div>
<div class="error-msg" id="nas-error"></div>
<button class="btn" onclick="verifyNAS()">경로 확인</button>
<button class="btn btn-skip" onclick="finishSetup()">건너뛰기</button>
</div>
<!-- Step 3: 완료 -->
<div class="card hidden" id="step-3">
<div class="done-icon">&#10003;</div>
<h2 style="text-align:center;">설정 완료</h2>
<p style="color: var(--text-dim); text-align: center; margin: 1rem 0;">
관리자 계정이 생성되었습니다. API 문서에서 엔드포인트를 확인하세요.
</p>
<button class="btn" onclick="location.href='/docs'">API 문서 열기</button>
</div>
</div>
<script>
const API = '/api/setup';
let currentStep = 0;
let authToken = '';
let totpSecret = '';
function showStep(n) {
for (let i = 0; i < 4; i++) {
const el = document.getElementById('step-' + i);
if (el) el.classList.toggle('hidden', i !== n);
}
for (let i = 0; i < 3; i++) {
const dot = document.getElementById('dot-' + i);
dot.classList.remove('active', 'done');
if (i < n) dot.classList.add('done');
else if (i === n) dot.classList.add('active');
}
currentStep = n;
}
function showError(id, msg) {
const el = document.getElementById(id);
el.textContent = msg;
el.style.display = 'block';
}
function hideError(id) {
document.getElementById(id).style.display = 'none';
}
async function createAdmin() {
hideError('admin-error');
const username = document.getElementById('username').value.trim() || 'admin';
const password = document.getElementById('password').value;
const password2 = document.getElementById('password2').value;
if (password !== password2) {
showError('admin-error', '비밀번호가 일치하지 않습니다');
return;
}
if (password.length < 8) {
showError('admin-error', '비밀번호는 8자 이상이어야 합니다');
return;
}
try {
const res = await fetch(API + '/admin', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ username, password }),
});
const data = await res.json();
if (!res.ok) {
showError('admin-error', data.detail || '계정 생성 실패');
return;
}
authToken = data.access_token;
await initTOTP();
showStep(1);
} catch (e) {
showError('admin-error', '서버 연결 실패: ' + e.message);
}
}
async function initTOTP() {
try {
const res = await fetch(API + '/totp/init', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + authToken,
},
});
const data = await res.json();
totpSecret = data.secret;
document.getElementById('totp-secret-text').textContent = '수동 입력: ' + data.secret;
const container = document.getElementById('qr-container');
container.innerHTML = '';
QRCode.toCanvas(document.createElement('canvas'), data.otpauth_uri, {
width: 200,
margin: 0,
}, function(err, canvas) {
if (!err) container.appendChild(canvas);
});
} catch (e) {
console.error('TOTP init failed:', e);
}
}
async function verifyTOTP() {
hideError('totp-error');
const code = document.getElementById('totp-code').value.trim();
if (code.length !== 6) {
showError('totp-error', '6자리 코드를 입력하세요');
return;
}
try {
const res = await fetch(API + '/totp/verify', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ secret: totpSecret, code }),
});
const data = await res.json();
if (!res.ok) {
showError('totp-error', data.detail || 'TOTP 검증 실패');
return;
}
const el = document.getElementById('totp-success');
el.textContent = '2단계 인증이 활성화되었습니다';
el.style.display = 'block';
setTimeout(() => showStep(2), 1000);
} catch (e) {
showError('totp-error', '서버 연결 실패');
}
}
function skipTOTP() {
showStep(2);
}
async function verifyNAS() {
hideError('nas-error');
const path = document.getElementById('nas-path').value.trim();
if (!path) {
showError('nas-error', '경로를 입력하세요');
return;
}
try {
const res = await fetch(API + '/verify-nas', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ path }),
});
const data = await res.json();
if (!res.ok) {
showError('nas-error', data.detail || '경로 확인 실패');
return;
}
const result = document.getElementById('nas-result');
result.innerHTML = `
<span class="${data.exists ? 'check' : 'cross'}">${data.exists ? '&#10003;' : '&#10007;'} 존재</span>
<span class="${data.readable ? 'check' : 'cross'}">${data.readable ? '&#10003;' : '&#10007;'} 읽기</span>
<span class="${data.writable ? 'check' : 'cross'}">${data.writable ? '&#10003;' : '&#10007;'} 쓰기</span>
`;
result.classList.remove('hidden');
if (data.exists && data.readable) {
setTimeout(() => finishSetup(), 1500);
}
} catch (e) {
showError('nas-error', '서버 연결 실패');
}
}
function finishSetup() {
showStep(3);
}
// 초기화: 이미 셋업 완료 상태인지 확인
(async () => {
try {
const res = await fetch(API + '/status');
const data = await res.json();
if (!data.needs_setup) {
location.href = '/docs';
}
} catch (e) {
// 서버 연결 실패 시 그냥 위자드 표시
}
})();
</script>
</body>
</html>

View File

@@ -0,0 +1,127 @@
"""AI 분류 워커 — taxonomy 기반 도메인/문서타입/태그/요약 생성"""
from datetime import datetime, timezone
from sqlalchemy.ext.asyncio import AsyncSession
from ai.client import AIClient, parse_json_response, strip_thinking
from core.config import settings
from core.utils import setup_logger
from models.document import Document
logger = setup_logger("classify_worker")
MAX_CLASSIFY_TEXT = 8000
# settings에서 taxonomy/document_types 로딩
DOCUMENT_TYPES = set(settings.document_types)
def _get_taxonomy_leaf_paths(taxonomy: dict, prefix: str = "") -> set[str]:
"""taxonomy dict에서 모든 유효한 경로를 추출"""
paths = set()
for key, value in taxonomy.items():
current = f"{prefix}/{key}" if prefix else key
if isinstance(value, dict):
if not value:
paths.add(current)
else:
paths.update(_get_taxonomy_leaf_paths(value, current))
elif isinstance(value, list):
if not value:
paths.add(current)
else:
for leaf in value:
paths.add(f"{current}/{leaf}")
paths.add(current) # 2단계도 허용 (leaf가 없는 경우용)
else:
paths.add(current)
return paths
VALID_DOMAIN_PATHS = _get_taxonomy_leaf_paths(settings.taxonomy)
def _validate_domain(domain: str) -> str:
"""domain이 taxonomy에 존재하는지 검증, 없으면 최대한 가까운 경로 찾기"""
if domain in VALID_DOMAIN_PATHS:
return domain
# 부분 매칭 시도 (2단계까지)
parts = domain.split("/")
for i in range(len(parts), 0, -1):
partial = "/".join(parts[:i])
if partial in VALID_DOMAIN_PATHS:
logger.warning(f"[분류] domain '{domain}''{partial}' (부분 매칭)")
return partial
logger.warning(f"[분류] domain '{domain}' taxonomy에 없음, General/Reading_Notes로 대체")
return "General/Reading_Notes"
async def process(document_id: int, session: AsyncSession) -> None:
"""문서 AI 분류 + 요약"""
doc = await session.get(Document, document_id)
if not doc:
raise ValueError(f"문서 ID {document_id}를 찾을 수 없음")
if not doc.extracted_text:
raise ValueError(f"문서 ID {document_id}: extracted_text가 비어있음")
client = AIClient()
try:
# ─── 분류 ───
truncated = doc.extracted_text[:MAX_CLASSIFY_TEXT]
raw_response = await client.classify(truncated)
parsed = parse_json_response(raw_response)
if not parsed:
raise ValueError(f"AI 응답에서 JSON 추출 실패: {raw_response[:200]}")
# domain 검증
domain = _validate_domain(parsed.get("domain", ""))
doc.ai_domain = domain
# sub_group은 domain 경로에서 추출 (호환성)
parts = domain.split("/")
doc.ai_sub_group = parts[1] if len(parts) > 1 else ""
# document_type 검증
doc_type = parsed.get("document_type", "")
doc.document_type = doc_type if doc_type in DOCUMENT_TYPES else "Note"
# confidence
confidence = parsed.get("confidence", 0.5)
doc.ai_confidence = max(0.0, min(1.0, float(confidence)))
# importance
importance = parsed.get("importance", "medium")
doc.importance = importance if importance in ("high", "medium", "low") else "medium"
# tags
doc.ai_tags = parsed.get("tags", [])[:5]
# source/origin
if parsed.get("sourceChannel") and not doc.source_channel:
doc.source_channel = parsed["sourceChannel"]
if parsed.get("dataOrigin") and not doc.data_origin:
doc.data_origin = parsed["dataOrigin"]
# ─── 요약 ───
summary = await client.summarize(doc.extracted_text[:15000])
doc.ai_summary = strip_thinking(summary)
# ─── 메타데이터 ───
doc.ai_model_version = "qwen3.5-35b-a3b"
doc.ai_processed_at = datetime.now(timezone.utc)
logger.info(
f"[분류] document_id={document_id}: "
f"domain={domain}, type={doc.document_type}, "
f"confidence={doc.ai_confidence:.2f}, tags={doc.ai_tags}"
)
finally:
await client.close()
# _move_to_knowledge 제거됨 — 파일은 원본 위치 유지, 분류는 DB 메타데이터만

146
app/workers/daily_digest.py Normal file
View File

@@ -0,0 +1,146 @@
"""일일 다이제스트 워커 — PostgreSQL/CalDAV 쿼리 → Markdown + SMTP
v1 scripts/pkm_daily_digest.py에서 포팅.
DEVONthink/OmniFocus → PostgreSQL/CalDAV 쿼리로 전환.
"""
import os
from datetime import datetime, timezone
from pathlib import Path
from sqlalchemy import func, select, text
from core.config import settings
from core.database import async_session
from core.utils import send_smtp_email, setup_logger
from models.document import Document
from models.queue import ProcessingQueue
logger = setup_logger("daily_digest")
async def run():
"""일일 다이제스트 생성 + 저장 + 발송"""
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
sections = []
async with async_session() as session:
# ─── 1. 오늘 추가된 문서 ───
added = await session.execute(
select(Document.ai_domain, func.count(Document.id))
.where(func.date(Document.created_at) == today)
.group_by(Document.ai_domain)
)
added_rows = added.all()
total_added = sum(row[1] for row in added_rows)
section = f"## 오늘 추가된 문서 ({total_added}건)\n"
if added_rows:
for domain, count in added_rows:
section += f"- {domain or '미분류'}: {count}\n"
else:
section += "- 없음\n"
sections.append(section)
# ─── 2. 법령 변경 ───
law_docs = await session.execute(
select(Document.title)
.where(
Document.source_channel == "law_monitor",
func.date(Document.created_at) == today,
)
)
law_rows = law_docs.scalars().all()
section = f"## 법령 변경 ({len(law_rows)}건)\n"
if law_rows:
for title in law_rows:
section += f"- {title}\n"
else:
section += "- 변경 없음\n"
sections.append(section)
# ─── 3. 이메일 수집 ───
email_count = await session.execute(
select(func.count(Document.id))
.where(
Document.source_channel == "email",
func.date(Document.created_at) == today,
)
)
email_total = email_count.scalar() or 0
sections.append(f"## 이메일 수집\n- {email_total}건 아카이브\n")
# ─── 4. 처리 파이프라인 상태 ───
queue_stats = await session.execute(
text("""
SELECT stage, status, COUNT(*)
FROM processing_queue
WHERE created_at > NOW() - INTERVAL '24 hours'
GROUP BY stage, status
ORDER BY stage, status
""")
)
queue_rows = queue_stats.all()
section = "## 파이프라인 상태 (24h)\n"
if queue_rows:
for stage, status, count in queue_rows:
section += f"- {stage}/{status}: {count}\n"
else:
section += "- 처리 항목 없음\n"
# 실패 건수 강조
failed = await session.execute(
select(func.count())
.select_from(ProcessingQueue)
.where(
ProcessingQueue.status == "failed",
ProcessingQueue.created_at > text("NOW() - INTERVAL '24 hours'"),
)
)
failed_count = failed.scalar() or 0
if failed_count > 0:
section += f"\n⚠️ **실패 {failed_count}건** — 수동 확인 필요\n"
sections.append(section)
# ─── 5. Inbox 미분류 ───
inbox_count = await session.execute(
select(func.count(Document.id))
.where(Document.file_path.like("PKM/Inbox/%"))
)
inbox_total = inbox_count.scalar() or 0
if inbox_total > 0:
sections.append(f"## Inbox 미분류\n- {inbox_total}건 대기 중\n")
# ─── Markdown 조합 ───
date_display = datetime.now(timezone.utc).strftime("%Y년 %m월 %d")
markdown = f"# PKM 일일 다이제스트 — {date_display}\n\n"
markdown += "\n".join(sections)
markdown += f"\n---\n*생성: {datetime.now(timezone.utc).isoformat()}*\n"
# ─── NAS 저장 ───
digest_dir = Path(settings.nas_mount_path) / "PKM" / "Archive" / "digests"
digest_dir.mkdir(parents=True, exist_ok=True)
digest_path = digest_dir / f"{today}_digest.md"
digest_path.write_text(markdown, encoding="utf-8")
# ─── 90일 초과 아카이브 ───
archive_dir = digest_dir / "archive"
archive_dir.mkdir(exist_ok=True)
cutoff = datetime.now(timezone.utc).timestamp() - (90 * 86400)
for old in digest_dir.glob("*_digest.md"):
if old.stat().st_mtime < cutoff:
old.rename(archive_dir / old.name)
# ─── SMTP 발송 ───
smtp_host = os.getenv("MAILPLUS_HOST", "")
smtp_port = int(os.getenv("MAILPLUS_SMTP_PORT", "465"))
smtp_user = os.getenv("MAILPLUS_USER", "")
smtp_pass = os.getenv("MAILPLUS_PASS", "")
if smtp_host and smtp_user:
send_smtp_email(
smtp_host, smtp_port, smtp_user, smtp_pass,
f"PKM 다이제스트 — {date_display}",
markdown,
)
logger.info(f"다이제스트 생성 완료: {digest_path}")

View File

@@ -0,0 +1,44 @@
"""벡터 임베딩 워커 — GPU 서버 bge-m3 호출"""
from datetime import datetime, timezone
from sqlalchemy.ext.asyncio import AsyncSession
from ai.client import AIClient
from core.utils import setup_logger
from models.document import Document
logger = setup_logger("embed_worker")
# 임베딩용 텍스트 최대 길이 (bge-m3: 8192 토큰)
MAX_EMBED_TEXT = 6000
EMBED_MODEL_VERSION = "bge-m3"
async def process(document_id: int, session: AsyncSession) -> None:
"""문서 벡터 임베딩 생성"""
doc = await session.get(Document, document_id)
if not doc:
raise ValueError(f"문서 ID {document_id}를 찾을 수 없음")
if not doc.extracted_text:
raise ValueError(f"문서 ID {document_id}: extracted_text가 비어있음")
# title + 본문 앞부분을 결합하여 임베딩 입력 생성
title_part = doc.title or ""
text_part = doc.extracted_text[:MAX_EMBED_TEXT]
embed_input = f"{title_part}\n\n{text_part}".strip()
if not embed_input:
logger.warning(f"[임베딩] document_id={document_id}: 빈 텍스트, 스킵")
return
client = AIClient()
try:
vector = await client.embed(embed_input)
doc.embedding = vector
doc.embed_model_version = EMBED_MODEL_VERSION
doc.embedded_at = datetime.now(timezone.utc)
logger.info(f"[임베딩] document_id={document_id}: {len(vector)}차원 벡터 생성")
finally:
await client.close()

View File

@@ -0,0 +1,167 @@
"""텍스트 추출 워커 — kordoc / LibreOffice / 직접 읽기"""
import subprocess
from datetime import datetime, timezone
from pathlib import Path
import httpx
from sqlalchemy.ext.asyncio import AsyncSession
from core.config import settings
from core.utils import setup_logger
from models.document import Document
logger = setup_logger("extract_worker")
# kordoc으로 파싱 가능한 포맷
KORDOC_FORMATS = {"hwp", "hwpx", "pdf"}
# 직접 읽기 가능한 텍스트 포맷
TEXT_FORMATS = {"md", "txt", "csv", "json", "xml", "html"}
# LibreOffice로 텍스트 추출 가능한 포맷
OFFICE_FORMATS = {"xlsx", "xls", "docx", "doc", "pptx", "ppt", "odt", "ods", "odp", "odoc", "osheet"}
# OCR 필요 이미지 포맷 (Phase 2)
IMAGE_FORMATS = {"jpg", "jpeg", "png", "tiff", "tif", "bmp", "gif"}
EXTRACTOR_VERSION = "kordoc@1.7"
async def process(document_id: int, session: AsyncSession) -> None:
"""문서 텍스트 추출"""
doc = await session.get(Document, document_id)
if not doc:
raise ValueError(f"문서 ID {document_id}를 찾을 수 없음")
fmt = doc.file_format.lower()
full_path = Path(settings.nas_mount_path) / doc.file_path
# 텍스트 파일 — 직접 읽기
if fmt in TEXT_FORMATS:
if not full_path.exists():
raise FileNotFoundError(f"파일 없음: {full_path}")
text = full_path.read_text(encoding="utf-8", errors="replace")
doc.extracted_text = text
doc.extracted_at = datetime.now(timezone.utc)
doc.extractor_version = "direct_read"
logger.info(f"[텍스트] {doc.file_path} ({len(text)}자)")
return
# 이미지 — 스킵 (Phase 2 OCR)
if fmt in IMAGE_FORMATS:
doc.extracted_text = ""
doc.extracted_at = datetime.now(timezone.utc)
doc.extractor_version = "skip_image"
logger.info(f"[이미지] {doc.file_path} — OCR 미구현, 스킵")
return
# kordoc 파싱 (HWP/HWPX/PDF)
if fmt in KORDOC_FORMATS:
# 컨테이너 내부 경로: /documents/{file_path}
container_path = f"/documents/{doc.file_path}"
async with httpx.AsyncClient(timeout=60) as client:
resp = await client.post(
f"{settings.kordoc_endpoint}/parse",
json={"filePath": container_path},
)
if resp.status_code == 404:
raise FileNotFoundError(f"kordoc: 파일 없음 — {container_path}")
if resp.status_code == 422:
raise ValueError(f"kordoc: 파싱 실패 — {resp.json().get('error', 'unknown')}")
resp.raise_for_status()
data = resp.json()
doc.extracted_text = data.get("markdown", "")
doc.extracted_at = datetime.now(timezone.utc)
doc.extractor_version = EXTRACTOR_VERSION
logger.info(f"[kordoc] {doc.file_path} ({len(doc.extracted_text)}자)")
return
# 오피스 포맷 — LibreOffice 텍스트 변환
if fmt in OFFICE_FORMATS:
if not full_path.exists():
raise FileNotFoundError(f"파일 없음: {full_path}")
import shutil
tmp_dir = Path("/tmp/extract_work")
tmp_dir.mkdir(exist_ok=True)
# 한글 파일명 문제 방지 — 영문 임시 파일로 복사
tmp_input = tmp_dir / f"input_{document_id}.{fmt}"
shutil.copy2(str(full_path), str(tmp_input))
# 스프레드시트는 csv, 나머지는 txt
CALC_FORMATS = {"xlsx", "xls", "ods", "osheet"}
if fmt in CALC_FORMATS:
convert_to = "csv:Text - txt - csv (StarCalc):44,34,76,1"
out_ext = "csv"
else:
convert_to = "txt:Text"
out_ext = "txt"
try:
result = subprocess.run(
["libreoffice", "--headless", "--convert-to", convert_to, "--outdir", str(tmp_dir), str(tmp_input)],
capture_output=True, text=True, timeout=60,
)
out_file = tmp_dir / f"input_{document_id}.{out_ext}"
if out_file.exists():
text = out_file.read_text(encoding="utf-8", errors="replace")
doc.extracted_text = text[:15000]
doc.extracted_at = datetime.now(timezone.utc)
doc.extractor_version = "libreoffice"
out_file.unlink()
logger.info(f"[LibreOffice] {doc.file_path} ({len(text)}자)")
else:
raise RuntimeError(f"LibreOffice 변환 실패: {result.stderr[:300]}")
except subprocess.TimeoutExpired:
raise RuntimeError(f"LibreOffice 텍스트 추출 timeout (60s)")
finally:
tmp_input.unlink(missing_ok=True)
# ─── ODF 변환 (편집용) ───
CONVERT_MAP = {
'xlsx': 'ods', 'xls': 'ods',
'docx': 'odt', 'doc': 'odt',
'pptx': 'odp', 'ppt': 'odp',
}
target_fmt = CONVERT_MAP.get(fmt)
if target_fmt:
try:
# .derived 디렉토리에 변환 (file_path는 원본 유지!)
derived_dir = full_path.parent / ".derived"
derived_dir.mkdir(exist_ok=True)
tmp_input2 = tmp_dir / f"convert_{document_id}.{fmt}"
shutil.copy2(str(full_path), str(tmp_input2))
conv_result = subprocess.run(
["libreoffice", "--headless", "--convert-to", target_fmt, "--outdir", str(tmp_dir), str(tmp_input2)],
capture_output=True, text=True, timeout=60,
)
tmp_input2.unlink(missing_ok=True)
conv_file = tmp_dir / f"convert_{document_id}.{target_fmt}"
if conv_file.exists():
final_path = derived_dir / f"{document_id}.{target_fmt}"
shutil.move(str(conv_file), str(final_path))
nas_root = Path(settings.nas_mount_path)
doc.derived_path = str(final_path.relative_to(nas_root))
doc.original_format = doc.file_format
doc.conversion_status = "done"
logger.info(f"[ODF변환] {doc.file_path} → derived: {doc.derived_path}")
else:
doc.conversion_status = "failed"
logger.warning(f"[ODF변환] 실패: {conv_result.stderr[:200]}")
except Exception as e:
doc.conversion_status = "failed"
logger.error(f"[ODF변환] {doc.file_path} 에러: {e}")
else:
doc.conversion_status = "none"
return
# 미지원 포맷
doc.extracted_text = ""
doc.extracted_at = datetime.now(timezone.utc)
doc.extractor_version = f"unsupported_{fmt}"
logger.warning(f"[미지원] {doc.file_path} (format={fmt})")

100
app/workers/file_watcher.py Normal file
View File

@@ -0,0 +1,100 @@
"""파일 감시 워커 — Inbox 디렉토리 스캔, 새 파일/변경 파일 자동 등록"""
from pathlib import Path
from sqlalchemy import select
from core.config import settings
from core.database import async_session
from core.utils import file_hash, setup_logger
from models.document import Document
from models.queue import ProcessingQueue
logger = setup_logger("file_watcher")
# 무시할 파일
SKIP_NAMES = {".DS_Store", "Thumbs.db", "desktop.ini", "Icon\r"}
SKIP_EXTENSIONS = {".tmp", ".part", ".crdownload"}
def should_skip(path: Path) -> bool:
if path.name in SKIP_NAMES or path.name.startswith("._"):
return True
if path.suffix.lower() in SKIP_EXTENSIONS:
return True
# .derived/ 및 .preview/ 디렉토리 내 파일 제외
if ".derived" in path.parts or ".preview" in path.parts:
return True
return False
async def watch_inbox():
"""Inbox 디렉토리를 스캔하여 새/변경 파일을 DB에 등록"""
inbox_path = Path(settings.nas_mount_path) / "PKM" / "Inbox"
if not inbox_path.exists():
return
files = [f for f in inbox_path.rglob("*") if f.is_file() and not should_skip(f)]
if not files:
return
new_count = 0
changed_count = 0
async with async_session() as session:
for file_path in files:
rel_path = str(file_path.relative_to(Path(settings.nas_mount_path)))
fhash = file_hash(file_path)
# DB에서 기존 문서 확인
result = await session.execute(
select(Document).where(Document.file_path == rel_path)
)
existing = result.scalar_one_or_none()
if existing is None:
# 새 파일 → 등록
ext = file_path.suffix.lstrip(".").lower() or "unknown"
doc = Document(
file_path=rel_path,
file_hash=fhash,
file_format=ext,
file_size=file_path.stat().st_size,
file_type="immutable",
title=file_path.stem,
source_channel="drive_sync",
)
session.add(doc)
await session.flush()
session.add(ProcessingQueue(
document_id=doc.id,
stage="extract",
status="pending",
))
new_count += 1
elif existing.file_hash != fhash:
# 해시 변경 → 재가공
existing.file_hash = fhash
existing.file_size = file_path.stat().st_size
# 기존 pending/processing 큐 항목이 없으면 extract부터 재시작
queue_check = await session.execute(
select(ProcessingQueue).where(
ProcessingQueue.document_id == existing.id,
ProcessingQueue.status.in_(["pending", "processing"]),
)
)
if not queue_check.scalar_one_or_none():
session.add(ProcessingQueue(
document_id=existing.id,
stage="extract",
status="pending",
))
changed_count += 1
await session.commit()
if new_count or changed_count:
logger.info(f"[Inbox] 새 파일 {new_count}건, 변경 파일 {changed_count}건 등록")

364
app/workers/law_monitor.py Normal file
View File

@@ -0,0 +1,364 @@
"""법령 모니터 워커 — 국가법령정보센터 API 연동
26개 법령 모니터링, 편/장 단위 분할 저장, 변경 이력 추적.
매일 07:00 실행 (APScheduler).
"""
import os
import re
from datetime import datetime, timezone
from pathlib import Path
from xml.etree import ElementTree as ET
import httpx
from sqlalchemy import select
from core.config import settings
from core.database import async_session
from core.utils import create_caldav_todo, escape_ical_text, file_hash, send_smtp_email, setup_logger
from models.automation import AutomationState
from models.document import Document
from models.queue import ProcessingQueue
logger = setup_logger("law_monitor")
LAW_SEARCH_URL = "https://www.law.go.kr/DRF/lawSearch.do"
LAW_SERVICE_URL = "https://www.law.go.kr/DRF/lawService.do"
# 모니터링 대상 법령 (26개)
MONITORED_LAWS = [
# 산업안전보건 핵심
"산업안전보건법",
"산업안전보건법 시행령",
"산업안전보건법 시행규칙",
"산업안전보건기준에 관한 규칙",
"유해위험작업의 취업 제한에 관한 규칙",
"중대재해 처벌 등에 관한 법률",
"중대재해 처벌 등에 관한 법률 시행령",
# 건설안전
"건설기술 진흥법",
"건설기술 진흥법 시행령",
"건설기술 진흥법 시행규칙",
"시설물의 안전 및 유지관리에 관한 특별법",
# 위험물/화학
"위험물안전관리법",
"위험물안전관리법 시행령",
"위험물안전관리법 시행규칙",
"화학물질관리법",
"화학물질관리법 시행령",
"화학물질의 등록 및 평가 등에 관한 법률",
# 소방/전기/가스
"소방시설 설치 및 관리에 관한 법률",
"소방시설 설치 및 관리에 관한 법률 시행령",
"전기사업법",
"전기안전관리법",
"고압가스 안전관리법",
"고압가스 안전관리법 시행령",
"액화석유가스의 안전관리 및 사업법",
# 근로/환경
"근로기준법",
"환경영향평가법",
]
async def run():
"""법령 변경 모니터링 실행"""
law_oc = os.getenv("LAW_OC", "")
if not law_oc:
logger.warning("LAW_OC 미설정 — 법령 API 승인 대기 중")
return
async with async_session() as session:
state = await session.execute(
select(AutomationState).where(AutomationState.job_name == "law_monitor")
)
state_row = state.scalar_one_or_none()
last_check = state_row.last_check_value if state_row else None
today = datetime.now(timezone.utc).strftime("%Y%m%d")
if last_check == today:
logger.info("오늘 이미 체크 완료")
return
new_count = 0
async with httpx.AsyncClient(timeout=30) as client:
for law_name in MONITORED_LAWS:
try:
count = await _check_law(client, law_oc, law_name, session)
new_count += count
except Exception as e:
logger.error(f"[{law_name}] 체크 실패: {e}")
# 상태 업데이트
if state_row:
state_row.last_check_value = today
state_row.last_run_at = datetime.now(timezone.utc)
else:
session.add(AutomationState(
job_name="law_monitor",
last_check_value=today,
last_run_at=datetime.now(timezone.utc),
))
await session.commit()
logger.info(f"법령 모니터 완료: {new_count}건 신규/변경 감지")
async def _check_law(
client: httpx.AsyncClient,
law_oc: str,
law_name: str,
session,
) -> int:
"""단일 법령 검색 → 변경 감지 → 분할 저장"""
# 법령 검색 (lawSearch.do)
resp = await client.get(
LAW_SEARCH_URL,
params={"OC": law_oc, "target": "law", "type": "XML", "query": law_name},
)
resp.raise_for_status()
root = ET.fromstring(resp.text)
total = root.findtext(".//totalCnt", "0")
if total == "0":
logger.debug(f"[{law_name}] 검색 결과 없음")
return 0
# 정확히 일치하는 법령 찾기
for law_elem in root.findall(".//law"):
found_name = law_elem.findtext("법령명한글", "").strip()
if found_name != law_name:
continue
mst = law_elem.findtext("법령일련번호", "")
proclamation_date = law_elem.findtext("공포일자", "")
revision_type = law_elem.findtext("제개정구분명", "")
if not mst:
continue
# 이미 등록된 법령인지 확인 (같은 법령명 + 공포일자)
existing = await session.execute(
select(Document).where(
Document.title.like(f"{law_name}%"),
Document.source_channel == "law_monitor",
)
)
existing_docs = existing.scalars().all()
# 같은 공포일자 이미 있으면 skip
for doc in existing_docs:
if proclamation_date in (doc.title or ""):
return 0
# 이전 공포일 찾기 (변경 이력용)
prev_date = ""
if existing_docs:
prev_date = max(
(re.search(r'\d{8}', doc.title or "").group() for doc in existing_docs
if re.search(r'\d{8}', doc.title or "")),
default=""
)
# 본문 조회 (lawService.do)
text_resp = await client.get(
LAW_SERVICE_URL,
params={"OC": law_oc, "target": "law", "MST": mst, "type": "XML"},
)
text_resp.raise_for_status()
# 분할 저장
count = await _save_law_split(
session, text_resp.text, law_name, proclamation_date,
revision_type, prev_date,
)
# DB 먼저 커밋 (알림 실패가 저장을 막지 않도록)
await session.commit()
# CalDAV + SMTP 알림 (실패해도 무시)
try:
_send_notifications(law_name, proclamation_date, revision_type)
except Exception as e:
logger.warning(f"[{law_name}] 알림 발송 실패 (무시): {e}")
return count
return 0
async def _save_law_split(
session, xml_text: str, law_name: str, proclamation_date: str,
revision_type: str, prev_date: str,
) -> int:
"""법령 XML → 장(章) 단위 Markdown 분할 저장"""
root = ET.fromstring(xml_text)
# 조문단위에서 장 구분자 찾기 (조문키가 000으로 끝나는 조문)
units = root.findall(".//조문단위")
chapters = [] # [(장제목, [조문들])]
current_chapter = None
current_articles = []
for unit in units:
key = unit.attrib.get("조문키", "")
content = (unit.findtext("조문내용", "") or "").strip()
# 장 구분자: 키가 000으로 끝나고 내용에 "제X장" 포함
if key.endswith("000") and re.search(r"\d+장", content):
# 이전 장/서문 저장
if current_articles:
chapter_name = current_chapter or "서문"
chapters.append((chapter_name, current_articles))
chapter_match = re.search(r"(제\d+장\s*.+)", content)
current_chapter = chapter_match.group(1).strip() if chapter_match else content.strip()
current_articles = []
else:
current_articles.append(unit)
# 마지막 장 저장
if current_articles:
chapter_name = current_chapter or "서문"
chapters.append((chapter_name, current_articles))
# 장 분할 성공
sections = []
if chapters:
for chapter_title, articles in chapters:
md_lines = [f"# {law_name}\n", f"## {chapter_title}\n"]
for article in articles:
title = article.findtext("조문제목", "")
content = article.findtext("조문내용", "")
if title:
md_lines.append(f"\n### {title}\n")
if content:
md_lines.append(content.strip())
section_name = _safe_name(chapter_title)
sections.append((section_name, "\n".join(md_lines)))
else:
# 장 분할 실패 → 전체 1파일
full_md = _law_xml_to_markdown(xml_text, law_name)
sections.append(("전문", full_md))
# 각 섹션 저장
inbox_dir = Path(settings.nas_mount_path) / "PKM" / "Inbox"
inbox_dir.mkdir(parents=True, exist_ok=True)
count = 0
for section_name, content in sections:
filename = f"{law_name}_{proclamation_date}_{section_name}.md"
file_path = inbox_dir / filename
file_path.write_text(content, encoding="utf-8")
rel_path = str(file_path.relative_to(Path(settings.nas_mount_path)))
# 변경 이력 메모
note = ""
if prev_date:
note = (
f"[자동] 법령 개정 감지\n"
f"이전 공포일: {prev_date}\n"
f"현재 공포일: {proclamation_date}\n"
f"개정구분: {revision_type}"
)
doc = Document(
file_path=rel_path,
file_hash=file_hash(file_path),
file_format="md",
file_size=len(content.encode()),
file_type="immutable",
title=f"{law_name} ({proclamation_date}) {section_name}",
source_channel="law_monitor",
data_origin="work",
user_note=note or None,
)
session.add(doc)
await session.flush()
session.add(ProcessingQueue(
document_id=doc.id, stage="extract", status="pending",
))
count += 1
logger.info(f"[법령] {law_name} ({proclamation_date}) → {count}개 섹션 저장")
return count
def _xml_section_to_markdown(elem) -> str:
"""XML 섹션(편/장)을 Markdown으로 변환"""
lines = []
for article in elem.iter():
tag = article.tag
text = (article.text or "").strip()
if not text:
continue
if "" in tag:
lines.append(f"\n### {text}\n")
elif "" in tag:
lines.append(f"\n{text}\n")
elif "" in tag:
lines.append(f"- {text}")
elif "" in tag:
lines.append(f" - {text}")
else:
lines.append(text)
return "\n".join(lines)
def _law_xml_to_markdown(xml_text: str, law_name: str) -> str:
"""법령 XML 전체를 Markdown으로 변환"""
root = ET.fromstring(xml_text)
lines = [f"# {law_name}\n"]
for elem in root.iter():
tag = elem.tag
text = (elem.text or "").strip()
if not text:
continue
if "" in tag and "제목" not in tag:
lines.append(f"\n## {text}\n")
elif "" in tag and "제목" not in tag:
lines.append(f"\n## {text}\n")
elif "" in tag:
lines.append(f"\n### {text}\n")
elif "" in tag:
lines.append(f"\n{text}\n")
elif "" in tag:
lines.append(f"- {text}")
elif "" in tag:
lines.append(f" - {text}")
return "\n".join(lines)
def _safe_name(name: str) -> str:
"""파일명 안전 변환"""
return re.sub(r'[^\w가-힣-]', '_', name).strip("_")
def _send_notifications(law_name: str, proclamation_date: str, revision_type: str):
"""CalDAV + SMTP 알림"""
# CalDAV
caldav_url = os.getenv("CALDAV_URL", "")
caldav_user = os.getenv("CALDAV_USER", "")
caldav_pass = os.getenv("CALDAV_PASS", "")
if caldav_url and caldav_user:
create_caldav_todo(
caldav_url, caldav_user, caldav_pass,
title=f"법령 검토: {law_name}",
description=f"공포일자: {proclamation_date}, 개정구분: {revision_type}",
due_days=7,
)
# SMTP
smtp_host = os.getenv("MAILPLUS_HOST", "")
smtp_port = int(os.getenv("MAILPLUS_SMTP_PORT", "465"))
smtp_user = os.getenv("MAILPLUS_USER", "")
smtp_pass = os.getenv("MAILPLUS_PASS", "")
if smtp_host and smtp_user:
send_smtp_email(
smtp_host, smtp_port, smtp_user, smtp_pass,
subject=f"[법령 변경] {law_name} ({revision_type})",
body=f"법령명: {law_name}\n공포일자: {proclamation_date}\n개정구분: {revision_type}",
)

View File

@@ -0,0 +1,213 @@
"""이메일 수집 워커 — Synology MailPlus IMAP → NAS 저장 + DB 등록
v1 scripts/mailplus_archive.py에서 포팅.
imaplib (동기)를 asyncio.to_thread()로 래핑.
"""
import asyncio
import email
import imaplib
import os
import re
from datetime import datetime, timedelta, timezone
from email.header import decode_header
from pathlib import Path
from sqlalchemy import select
from core.config import settings
from core.database import async_session
from core.utils import file_hash, send_smtp_email, setup_logger
from models.automation import AutomationState
from models.document import Document
from models.queue import ProcessingQueue
logger = setup_logger("mailplus_archive")
# 업무 키워드 (data_origin 자동 감지)
WORK_KEYWORDS = {"테크니컬코리아", "TK", "공장", "생산", "사내", "안전", "점검"}
def _decode_mime_header(raw: str) -> str:
"""MIME 헤더 디코딩"""
parts = decode_header(raw)
decoded = []
for data, charset in parts:
if isinstance(data, bytes):
decoded.append(data.decode(charset or "utf-8", errors="replace"))
else:
decoded.append(data)
return "".join(decoded)
def _sanitize_filename(name: str, max_len: int = 80) -> str:
"""파일명에 사용 불가한 문자 제거"""
clean = re.sub(r'[<>:"/\\|?*\x00-\x1f]', "_", name)
return clean[:max_len].strip()
def _detect_origin(subject: str, body: str) -> str:
"""work/external 자동 감지"""
text = f"{subject} {body[:500]}".lower()
for kw in WORK_KEYWORDS:
if kw.lower() in text:
return "work"
return "external"
def _fetch_emails_sync(host: str, port: int, user: str, password: str, last_uid: int | None):
"""동기 IMAP 메일 가져오기 (asyncio.to_thread에서 실행)"""
results = []
conn = imaplib.IMAP4_SSL(host, port, timeout=30)
try:
conn.login(user, password)
conn.select("INBOX")
if last_uid:
# 증분 동기화: last_uid 이후
_, data = conn.uid("search", None, f"UID {last_uid + 1}:*")
else:
# 최초 실행: 최근 7일
since = (datetime.now() - timedelta(days=7)).strftime("%d-%b-%Y")
_, data = conn.uid("search", None, f"SINCE {since}")
uids = data[0].split()
for uid_bytes in uids:
uid = int(uid_bytes)
_, msg_data = conn.uid("fetch", uid_bytes, "(RFC822)")
if msg_data[0] is None:
continue
raw = msg_data[0][1]
results.append((uid, raw))
finally:
conn.logout()
return results
async def run():
"""이메일 수집 실행"""
host = os.getenv("MAILPLUS_HOST", "")
port = int(os.getenv("MAILPLUS_PORT", "993"))
user = os.getenv("MAILPLUS_USER", "")
password = os.getenv("MAILPLUS_PASS", "")
if not all([host, user, password]):
logger.warning("MailPlus 인증 정보 미설정")
return
async with async_session() as session:
# 마지막 UID 조회
state = await session.execute(
select(AutomationState).where(AutomationState.job_name == "mailplus")
)
state_row = state.scalar_one_or_none()
last_uid = int(state_row.last_check_value) if state_row and state_row.last_check_value else None
# IMAP 동기 호출을 비동기로 래핑
try:
emails = await asyncio.to_thread(
_fetch_emails_sync, host, port, user, password, last_uid,
)
except Exception as e:
logger.error(f"IMAP 연결 실패: {e}")
return
if not emails:
logger.info("새 이메일 없음")
return
# 이메일 저장 디렉토리
email_dir = Path(settings.nas_mount_path) / "PKM" / "Archive" / "emails"
email_dir.mkdir(parents=True, exist_ok=True)
max_uid = last_uid or 0
archived = []
for uid, raw_bytes in emails:
try:
msg = email.message_from_bytes(raw_bytes)
subject = _decode_mime_header(msg.get("Subject", "제목없음"))
date_str = msg.get("Date", "")
date = datetime.now().strftime("%Y%m%d")
# .eml 파일 저장
safe_subject = _sanitize_filename(subject)
filename = f"{date}_{uid}_{safe_subject}.eml"
eml_path = email_dir / filename
eml_path.write_bytes(raw_bytes)
# 본문 추출 (텍스트 파트)
body = ""
charset = None
if msg.is_multipart():
for part in msg.walk():
if part.get_content_type() == "text/plain":
payload = part.get_payload(decode=True)
if payload is not None:
charset = part.get_content_charset() or "utf-8"
body = payload.decode(charset, errors="replace")
break
else:
payload = msg.get_payload(decode=True)
if payload is not None:
charset = msg.get_content_charset() or "utf-8"
body = payload.decode(charset, errors="replace")
if "\ufffd" in body[:1000]:
logger.debug(f"[메일] charset={charset or 'unknown'} 디코딩 중 replacement 발생")
# DB 등록
rel_path = str(eml_path.relative_to(Path(settings.nas_mount_path)))
origin = _detect_origin(subject, body)
doc = Document(
file_path=rel_path,
file_hash=file_hash(eml_path),
file_format="eml",
file_size=len(raw_bytes),
file_type="immutable",
title=subject,
source_channel="email",
data_origin=origin,
)
session.add(doc)
await session.flush()
safe_subj = subject.replace("\n", " ").replace("\r", " ")[:200]
# TODO: extract_worker가 eml 본문/첨부 파싱 지원 시 이 조건 제거
if doc.file_format != "eml":
session.add(ProcessingQueue(
document_id=doc.id, stage="extract", status="pending",
))
else:
logger.debug(f"[메일] {safe_subj} — eml extract 미지원, 큐 스킵")
archived.append(safe_subj)
max_uid = max(max_uid, uid)
except Exception as e:
logger.error(f"UID {uid} 처리 실패: {e}")
# 상태 업데이트
if state_row:
state_row.last_check_value = str(max_uid)
state_row.last_run_at = datetime.now(timezone.utc)
else:
session.add(AutomationState(
job_name="mailplus",
last_check_value=str(max_uid),
last_run_at=datetime.now(timezone.utc),
))
await session.commit()
# SMTP 알림
smtp_host = os.getenv("MAILPLUS_HOST", "")
smtp_port = int(os.getenv("MAILPLUS_SMTP_PORT", "465"))
if archived and smtp_host:
body = f"이메일 {len(archived)}건 수집 완료:\n\n" + "\n".join(f"- {s}" for s in archived)
send_smtp_email(smtp_host, smtp_port, user, password, "PKM 이메일 수집 알림", body)
logger.info(f"이메일 {len(archived)}건 수집 완료 (max_uid={max_uid})")

View File

@@ -0,0 +1,253 @@
"""뉴스 수집 워커 — RSS/API에서 기사 수집, documents에 저장"""
import hashlib
import re
from datetime import datetime, timezone
from html import unescape
from urllib.parse import urlparse, urlunparse
import feedparser
import httpx
from sqlalchemy import select
from core.database import async_session
from core.utils import setup_logger
from models.document import Document
from models.news_source import NewsSource
from models.queue import ProcessingQueue
logger = setup_logger("news_collector")
# 카테고리 표준화 매핑
CATEGORY_MAP = {
# 한국어
"국제": "International", "정치": "Politics", "경제": "Economy",
"사회": "Society", "문화": "Culture", "산업": "Industry",
"환경": "Environment", "기술": "Technology",
# 영어
"World": "International", "International": "International",
"Technology": "Technology", "Tech": "Technology", "Sci-Tech": "Technology",
"Arts": "Culture", "Culture": "Culture",
"Climate": "Environment", "Environment": "Environment",
# 일본어
"国際": "International", "文化": "Culture", "科学": "Technology",
# 독일어
"Kultur": "Culture", "Wissenschaft": "Technology",
# 프랑스어
"Environnement": "Environment",
}
def _normalize_category(raw: str) -> str:
"""카테고리 표준화"""
return CATEGORY_MAP.get(raw, CATEGORY_MAP.get(raw.strip(), "Other"))
def _clean_html(text: str) -> str:
"""HTML 태그 제거 + 정제"""
if not text:
return ""
text = re.sub(r"<[^>]+>", "", text)
text = unescape(text)
return text.strip()[:1000]
def _normalize_url(url: str) -> str:
"""URL 정규화 (tracking params 제거)"""
parsed = urlparse(url)
return urlunparse((parsed.scheme, parsed.netloc, parsed.path, "", "", ""))
def _article_hash(title: str, published: str, source_name: str) -> str:
"""기사 고유 해시 (중복 체크용)"""
key = f"{title}|{published}|{source_name}"
return hashlib.sha256(key.encode()).hexdigest()[:32]
def _normalize_to_utc(dt) -> datetime:
"""다양한 시간 형식을 UTC로 정규화"""
if isinstance(dt, datetime):
if dt.tzinfo is None:
return dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
return datetime.now(timezone.utc)
async def run():
"""뉴스 수집 실행"""
async with async_session() as session:
result = await session.execute(
select(NewsSource).where(NewsSource.enabled == True)
)
sources = result.scalars().all()
if not sources:
logger.info("활성화된 뉴스 소스 없음")
return
total = 0
for source in sources:
try:
if source.feed_type == "api":
count = await _fetch_api(session, source)
else:
count = await _fetch_rss(session, source)
source.last_fetched_at = datetime.now(timezone.utc)
total += count
except Exception as e:
logger.error(f"[{source.name}] 수집 실패: {e}")
source.last_fetched_at = datetime.now(timezone.utc)
await session.commit()
logger.info(f"뉴스 수집 완료: {total}건 신규")
async def _fetch_rss(session, source: NewsSource) -> int:
"""RSS 피드 수집"""
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.get(source.feed_url)
resp.raise_for_status()
feed = feedparser.parse(resp.text)
count = 0
for entry in feed.entries:
title = entry.get("title", "").strip()
if not title:
continue
summary = _clean_html(entry.get("summary", "") or entry.get("description", ""))
if not summary:
summary = title
link = entry.get("link", "")
published = entry.get("published_parsed") or entry.get("updated_parsed")
pub_dt = datetime(*published[:6], tzinfo=timezone.utc) if published else datetime.now(timezone.utc)
# 중복 체크
article_id = _article_hash(title, pub_dt.strftime("%Y%m%d"), source.name)
normalized_url = _normalize_url(link)
existing = await session.execute(
select(Document).where(
(Document.file_hash == article_id) |
(Document.edit_url == normalized_url)
)
)
if existing.scalar_one_or_none():
continue
category = _normalize_category(source.category or "")
source_short = source.name.split(" ")[0] # "경향신문 문화" → "경향신문"
doc = Document(
file_path=f"news/{source.name}/{article_id}",
file_hash=article_id,
file_format="article",
file_size=len(summary.encode()),
file_type="note",
title=title,
extracted_text=f"{title}\n\n{summary}",
extracted_at=datetime.now(timezone.utc),
extractor_version="rss",
source_channel="news",
data_origin="external",
edit_url=link,
review_status="approved",
ai_domain="News",
ai_sub_group=source_short,
ai_tags=[f"News/{source_short}/{category}"],
)
session.add(doc)
await session.flush()
# embed만 등록 (classify 불필요 — 소스/분야 이미 확정)
days_old = (datetime.now(timezone.utc) - pub_dt).days
if days_old <= 30:
session.add(ProcessingQueue(document_id=doc.id, stage="embed", status="pending"))
count += 1
logger.info(f"[{source.name}] RSS → {count}건 수집")
return count
async def _fetch_api(session, source: NewsSource) -> int:
"""NYT API 수집"""
import os
nyt_key = os.getenv("NYT_API_KEY", "")
if not nyt_key:
logger.warning("NYT_API_KEY 미설정")
return 0
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.get(
f"https://api.nytimes.com/svc/topstories/v2/{source.category or 'world'}.json",
params={"api-key": nyt_key},
)
resp.raise_for_status()
data = resp.json()
count = 0
for article in data.get("results", []):
title = article.get("title", "").strip()
if not title:
continue
summary = _clean_html(article.get("abstract", ""))
if not summary:
summary = title
link = article.get("url", "")
pub_str = article.get("published_date", "")
try:
pub_dt = datetime.fromisoformat(pub_str.replace("Z", "+00:00"))
except (ValueError, AttributeError):
pub_dt = datetime.now(timezone.utc)
article_id = _article_hash(title, pub_dt.strftime("%Y%m%d"), source.name)
normalized_url = _normalize_url(link)
existing = await session.execute(
select(Document).where(
(Document.file_hash == article_id) |
(Document.edit_url == normalized_url)
)
)
if existing.scalar_one_or_none():
continue
category = _normalize_category(article.get("section", source.category or ""))
source_short = source.name.split(" ")[0]
doc = Document(
file_path=f"news/{source.name}/{article_id}",
file_hash=article_id,
file_format="article",
file_size=len(summary.encode()),
file_type="note",
title=title,
extracted_text=f"{title}\n\n{summary}",
extracted_at=datetime.now(timezone.utc),
extractor_version="nyt_api",
source_channel="news",
data_origin="external",
edit_url=link,
review_status="approved",
ai_domain="News",
ai_sub_group=source_short,
ai_tags=[f"News/{source_short}/{category}"],
)
session.add(doc)
await session.flush()
days_old = (datetime.now(timezone.utc) - pub_dt).days
if days_old <= 30:
session.add(ProcessingQueue(document_id=doc.id, stage="embed", status="pending"))
count += 1
logger.info(f"[{source.name}] API → {count}건 수집")
return count

View File

@@ -0,0 +1,116 @@
"""PDF 미리보기 생성 워커 — LibreOffice Headless로 문서→PDF 변환"""
import subprocess
import shutil
from datetime import datetime, timezone
from pathlib import Path
from sqlalchemy.ext.asyncio import AsyncSession
from core.config import settings
from core.utils import setup_logger
logger = setup_logger("preview_worker")
# PDF 변환 대상 포맷
CONVERTIBLE_FORMATS = {
"docx", "xlsx", "pptx", "odt", "ods", "odp", # 안정 지원
"odoc", "osheet", "hwp", "hwpx", # 검증 필요
}
# 이미 PDF이거나 변환 불필요한 포맷
NATIVE_PDF = {"pdf"}
NATIVE_IMAGE = {"jpg", "jpeg", "png", "gif", "bmp", "tiff"}
TEXT_FORMATS = {"md", "txt", "csv", "json", "xml", "html"}
PREVIEW_DIR_NAME = "PKM/.preview"
TIMEOUT_SECONDS = 60
async def process(document_id: int, session: AsyncSession) -> None:
"""문서 PDF 미리보기 생성"""
from models.document import Document
doc = await session.get(Document, document_id)
if not doc:
logger.error(f"[preview] document_id={document_id} 없음")
return
fmt = doc.file_format.lower()
# PDF/이미지/텍스트는 변환 불필요
if fmt in NATIVE_PDF or fmt in NATIVE_IMAGE or fmt in TEXT_FORMATS:
doc.preview_status = "ready" if fmt in NATIVE_PDF else "none"
doc.preview_at = datetime.now(timezone.utc)
await session.commit()
return
if fmt not in CONVERTIBLE_FORMATS:
doc.preview_status = "none"
await session.commit()
logger.info(f"[preview] {doc.title} — 변환 불가 포맷: {fmt}")
return
# 원본 파일 경로
source = Path(settings.nas_mount_path) / doc.file_path
if not source.exists():
doc.preview_status = "failed"
await session.commit()
logger.error(f"[preview] 원본 없음: {source}")
return
# 미리보기 디렉토리
preview_dir = Path(settings.nas_mount_path) / PREVIEW_DIR_NAME
preview_dir.mkdir(parents=True, exist_ok=True)
output_path = preview_dir / f"{document_id}.pdf"
doc.preview_status = "processing"
await session.commit()
# LibreOffice 변환
try:
tmp_dir = Path("/tmp/preview_work")
tmp_dir.mkdir(exist_ok=True)
# 한글 파일명 문제 방지 — 영문 임시 파일로 복사
tmp_input = tmp_dir / f"input_{document_id}{source.suffix}"
shutil.copy2(str(source), str(tmp_input))
result = subprocess.run(
[
"libreoffice", "--headless", "--convert-to", "pdf",
"--outdir", str(tmp_dir),
str(tmp_input),
],
capture_output=True,
text=True,
timeout=TIMEOUT_SECONDS,
)
tmp_input.unlink(missing_ok=True)
if result.returncode != 0:
raise RuntimeError(f"LibreOffice 변환 실패: {result.stderr[:200]}")
# 변환 결과 찾기
converted = tmp_dir / f"input_{document_id}.pdf"
if not converted.exists():
raise RuntimeError(f"변환 결과물 없음: {converted}")
# 캐시로 이동
shutil.move(str(converted), str(output_path))
doc.preview_status = "ready"
doc.preview_hash = doc.file_hash
doc.preview_at = datetime.now(timezone.utc)
await session.commit()
logger.info(f"[preview] {doc.title} → PDF 변환 완료")
except subprocess.TimeoutExpired:
doc.preview_status = "failed"
await session.commit()
logger.error(f"[preview] {doc.title} — 변환 timeout ({TIMEOUT_SECONDS}s)")
except Exception as e:
doc.preview_status = "failed"
await session.commit()
logger.error(f"[preview] {doc.title} — 변환 실패: {e}")

View File

@@ -0,0 +1,140 @@
"""처리 큐 소비자 — APScheduler에서 1분 간격으로 호출"""
from datetime import datetime, timedelta, timezone
from sqlalchemy import select, update
from core.database import async_session
from core.utils import setup_logger
from models.queue import ProcessingQueue
logger = setup_logger("queue_consumer")
# stage별 배치 크기
BATCH_SIZE = {"extract": 5, "classify": 3, "embed": 1, "preview": 2}
STALE_THRESHOLD_MINUTES = 10
async def reset_stale_items():
"""processing 상태로 10분 이상 방치된 항목 복구"""
cutoff = datetime.now(timezone.utc) - timedelta(minutes=STALE_THRESHOLD_MINUTES)
async with async_session() as session:
result = await session.execute(
update(ProcessingQueue)
.where(
ProcessingQueue.status == "processing",
ProcessingQueue.started_at < cutoff,
)
.values(status="pending", started_at=None)
)
if result.rowcount > 0:
await session.commit()
logger.warning(f"stale 항목 {result.rowcount}건 복구")
async def enqueue_next_stage(document_id: int, current_stage: str):
"""현재 stage 완료 후 다음 stage를 pending으로 등록"""
next_stages = {"extract": ["classify", "preview"], "classify": ["embed"]}
stages = next_stages.get(current_stage, [])
if not stages:
return
async with async_session() as session:
for next_stage in stages:
existing = await session.execute(
select(ProcessingQueue).where(
ProcessingQueue.document_id == document_id,
ProcessingQueue.stage == next_stage,
ProcessingQueue.status.in_(["pending", "processing"]),
)
)
if existing.scalar_one_or_none():
continue
session.add(ProcessingQueue(
document_id=document_id,
stage=next_stage,
status="pending",
))
await session.commit()
async def consume_queue():
"""큐에서 pending 항목을 가져와 stage별 워커 실행"""
from workers.classify_worker import process as classify_process
from workers.embed_worker import process as embed_process
from workers.extract_worker import process as extract_process
from workers.preview_worker import process as preview_process
workers = {
"extract": extract_process,
"classify": classify_process,
"embed": embed_process,
"preview": preview_process,
}
await reset_stale_items()
for stage, worker_fn in workers.items():
batch_size = BATCH_SIZE.get(stage, 3)
# pending 항목 조회
async with async_session() as session:
result = await session.execute(
select(ProcessingQueue.id, ProcessingQueue.document_id)
.where(
ProcessingQueue.stage == stage,
ProcessingQueue.status == "pending",
)
.order_by(ProcessingQueue.created_at)
.limit(batch_size)
)
pending_items = result.all()
# 각 항목을 독립 세션에서 처리
for queue_id, document_id in pending_items:
# 상태를 processing으로 변경
async with async_session() as session:
item = await session.get(ProcessingQueue, queue_id)
if not item or item.status != "pending":
continue
item.status = "processing"
item.started_at = datetime.now(timezone.utc)
item.attempts += 1
await session.commit()
# 워커 실행 (독립 세션)
try:
async with async_session() as worker_session:
await worker_fn(document_id, worker_session)
await worker_session.commit()
# 완료 처리
async with async_session() as session:
item = await session.get(ProcessingQueue, queue_id)
if not item:
logger.warning(f"[{stage}] queue_id={queue_id} 없음 (삭제됨?), skip")
continue
item.status = "completed"
item.completed_at = datetime.now(timezone.utc)
await session.commit()
await enqueue_next_stage(document_id, stage)
logger.info(f"[{stage}] document_id={document_id} 완료")
except Exception as e:
# 실패 처리
async with async_session() as session:
item = await session.get(ProcessingQueue, queue_id)
if not item:
logger.warning(f"[{stage}] queue_id={queue_id} 없음 (삭제됨?), skip")
continue
item.error_message = str(e)[:500]
if item.attempts >= item.max_attempts:
item.status = "failed"
logger.error(f"[{stage}] document_id={document_id} 영구 실패: {e}")
else:
item.status = "pending"
item.started_at = None
logger.warning(f"[{stage}] document_id={document_id} 재시도 예정 ({item.attempts}/{item.max_attempts}): {e}")
await session.commit()

View File

@@ -2,18 +2,18 @@
ai:
gateway:
endpoint: "http://gpu-server:8080"
endpoint: "http://ai-gateway:8080"
models:
primary:
endpoint: "http://host.docker.internal:8800/v1/chat/completions"
endpoint: "http://100.76.254.116:8800/v1/chat/completions"
model: "mlx-community/Qwen3.5-35B-A3B-4bit"
max_tokens: 4096
timeout: 60
fallback:
endpoint: "http://gpu-server:11434/v1/chat/completions"
model: "qwen3.5:35b-a3b"
endpoint: "http://ollama:11434/v1/chat/completions"
model: "qwen3.5:9b-q8_0"
max_tokens: 4096
timeout: 120
@@ -25,21 +25,81 @@ ai:
require_explicit_trigger: true
embedding:
endpoint: "http://gpu-server:11434/api/embeddings"
model: "nomic-embed-text"
endpoint: "http://ollama:11434/api/embeddings"
model: "bge-m3"
vision:
endpoint: "http://gpu-server:11434/api/generate"
endpoint: "http://ollama:11434/api/generate"
model: "Qwen2.5-VL-7B"
rerank:
endpoint: "http://gpu-server:11434/api/rerank"
endpoint: "http://ollama:11434/api/rerank"
model: "bge-reranker-v2-m3"
nas:
mount_path: "/documents"
pkm_root: "/documents/PKM"
# ─── 문서 분류 체계 ───
taxonomy:
Philosophy:
Ethics: []
Metaphysics: []
Epistemology: []
Logic: []
Aesthetics: []
Eastern_Philosophy: []
Western_Philosophy: []
Language:
Korean: []
English: []
Japanese: []
Translation: []
Linguistics: []
Engineering:
Mechanical: [Piping, HVAC, Equipment]
Electrical: [Power, Instrumentation]
Chemical: [Process, Material]
Civil: []
Network: [Server, Security, Infrastructure]
Industrial_Safety:
Legislation: [Act, Decree, Foreign_Law, Korea_Law_Archive, Enforcement_Rule, Public_Notice, SAPA]
Theory: [Industrial_Safety_General, Safety_Health_Fundamentals]
Academic_Papers: [Safety_General, Risk_Assessment_Research]
Cases: [Domestic, International]
Practice: [Checklist, Contractor_Management, Safety_Education, Emergency_Plan, Patrol_Inspection, Permit_to_Work, PPE, Safety_Plan]
Risk_Assessment: [KRAS, JSA, Checklist_Method]
Safety_Manager: [Appointment, Duty_Record, Improvement, Inspection, Meeting]
Health_Manager: [Appointment, Duty_Record, Ergonomics, Health_Checkup, Mental_Health, MSDS, Work_Environment]
Programming:
Programming_Language: [Python, JavaScript, Go, Rust]
Framework: [FastAPI, SvelteKit, React]
DevOps: [Docker, CI_CD, Linux_Administration]
AI_ML: [Large_Language_Model, Computer_Vision, Data_Science]
Database: []
Software_Architecture: []
General:
Reading_Notes: []
Self_Development: []
Business: []
Science: []
History: []
document_types:
- Reference
- Standard
- Manual
- Drawing
- Template
- Note
- Academic_Paper
- Law_Document
- Report
- Memo
- Checklist
- Meeting_Minutes
- Specification
schedule:
law_monitor: "07:00"
mailplus_archive: ["07:00", "18:00"]

View File

@@ -10,22 +10,18 @@ POSTGRES_DB=pkm
POSTGRES_USER=pkm
POSTGRES_PASSWORD=
# ─── AI: Mac mini MLX (Qwen3.5, 기본 모델) ───
MLX_ENDPOINT=http://localhost:8800/v1/chat/completions
# ─── AI: Mac mini MLX (Tailscale 경유, Qwen3.5 기본 모델) ───
MLX_ENDPOINT=http://100.76.254.116:8800/v1/chat/completions
MLX_MODEL=mlx-community/Qwen3.5-35B-A3B-4bit
# ─── AI: GPU 서버 ───
GPU_SERVER_IP=
GPU_EMBED_PORT=11434
# ─── AI: Claude API (종량제, 복잡한 분석 전용) ───
CLAUDE_API_KEY=
# ─── AI Gateway (GPU 서버) ───
AI_GATEWAY_ENDPOINT=http://gpu-server:8080
# ─── AI Gateway (같은 Docker 네트워크) ───
AI_GATEWAY_ENDPOINT=http://ai-gateway:8080
# ─── Synology NAS ───
NAS_SMB_PATH=/Volumes/Document_Server
# ─── NAS (NFS 마운트) ───
NAS_NFS_PATH=/mnt/nas/Document_Server
NAS_DOMAIN=ds1525.hyungi.net
NAS_TAILSCALE_IP=100.101.79.37
NAS_PORT=15001
@@ -51,7 +47,3 @@ TOTP_SECRET=
# ─── 국가법령정보센터 (법령 모니터링) ───
LAW_OC=
# ─── TKSafety API (나중에 활성화) ───
#TKSAFETY_HOST=tksafety.technicalkorea.net
#TKSAFETY_PORT=

View File

@@ -1,5 +1,3 @@
version: '3.8'
services:
postgres:
image: pgvector/pgvector:pg16
@@ -11,7 +9,7 @@ services:
POSTGRES_USER: pkm
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
ports:
- "5432:5432"
- "15432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U pkm"]
interval: 5s
@@ -24,20 +22,51 @@ services:
ports:
- "3100:3100"
volumes:
- ${NAS_SMB_PATH:-/Volumes/Document_Server}:/documents:ro
- ${NAS_NFS_PATH:-/mnt/nas/Document_Server}:/documents:ro
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://localhost:3100/health"]
test: ["CMD", "node", "-e", "fetch('http://localhost:3100/health').then(r=>{process.exit(r.ok?0:1)}).catch(()=>process.exit(1))"]
interval: 10s
timeout: 5s
retries: 3
restart: unless-stopped
ollama:
image: ollama/ollama
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
ports:
- "127.0.0.1:11434:11434"
restart: unless-stopped
ai-gateway:
build: ./gpu-server/services/ai-gateway
ports:
- "127.0.0.1:8081:8080"
environment:
- PRIMARY_ENDPOINT=http://100.76.254.116:8800/v1/chat/completions
- FALLBACK_ENDPOINT=http://ollama:11434/v1/chat/completions
- CLAUDE_API_KEY=${CLAUDE_API_KEY:-}
- DAILY_BUDGET_USD=${DAILY_BUDGET_USD:-5.00}
depends_on:
- ollama
restart: unless-stopped
fastapi:
build: ./app
ports:
- "8000:8000"
volumes:
- ${NAS_SMB_PATH:-/Volumes/Document_Server}:/documents
- ${NAS_NFS_PATH:-/mnt/nas/Document_Server}:/documents
- ./config.yaml:/app/config.yaml:ro
- ./scripts:/app/scripts:ro
- ./logs:/app/logs
depends_on:
postgres:
condition: service_healthy
@@ -61,8 +90,7 @@ services:
caddy:
image: caddy:2
ports:
- "80:80"
- "443:443"
- "8080:80"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
@@ -74,3 +102,4 @@ services:
volumes:
pgdata:
caddy_data:
ollama_data:

View File

@@ -542,7 +542,7 @@ POST /to-hwpx
### Caddy 설정 예시
```
pkm.hyungi.net {
document.hyungi.net {
reverse_proxy localhost:8000 # FastAPI
}
@@ -931,7 +931,7 @@ pkm-web/
│ └── migrate_from_devonthink.py ← v1 → v2 마이그레이션 스크립트
├── docs/
│ ├── architecture-v2.md ← 이 문서
│ ├── architecture.md ← 이 문서
│ └── deploy.md ← 배포 가이드
└── tests/

View File

@@ -64,11 +64,11 @@ curl http://localhost:3100/health
### 2-6. 외부 접근 (Caddy)
Caddy가 자동으로 HTTPS 인증서를 발급한다.
- `pkm.hyungi.net`FastAPI (:8000)
HTTPS는 앞단 프록시(Mac mini nginx)에서 처리하고, Caddy는 HTTP only로 동작한다.
- `document.hyungi.net`Mac mini nginx (HTTPS 종료) → GPU 서버 Caddy (:8080) → FastAPI/Frontend
- `office.hyungi.net` → Synology Office (NAS 프록시)
DNS 레코드가 Mac mini의 공인 IP를 가리켜야 한다.
DNS 레코드가 Mac mini의 공인 IP를 가리켜야 한다. Caddy는 `auto_https off` 설정.
## 3. GPU 서버 배포

181
docs/devlog.md Normal file
View File

@@ -0,0 +1,181 @@
# 개발 로그
## 2026-04-02 — v2 전환 설계 완료
### 결정 사항
- DEVONthink 탈피 결정. v1의 구조적 한계(AppleScript 취약성, macOS GUI 의존, 13개 DB 복잡성)를 더 이상 감수하지 않기로 함
- 자체 웹앱 방향 확정. 기술 스택: FastAPI + PostgreSQL/pgvector + SvelteKit + Docker
- OmniFocus 탈피 → Synology Calendar (CalDAV VTODO)로 대체
- Synology 서비스 활용 극대화: Office(문서 편집/미리보기), Drive(파일 관리), Calendar(태스크), MailPlus(이메일+알림)
- Document_Server 전체를 Synology Drive가 관리. PKM 하위 폴더로 자동분류 영역 분리
- 문서 "원본" 정의 확정: immutable(PDF, 수신 HWP 등) / editable(Synology Office 포맷) / note(Markdown)
- .docx/.xlsx는 교환 형식으로 취급. 서버에 영구 보관하지 않음
- 데이터 3계층: 원본(NAS) → 가공(PostgreSQL) → 파생(pgvector+캐시)
- kordoc 통합 결정 (HWP/HWPX/PDF → Markdown 파싱, Node.js 마이크로서비스)
- AI 전략: Qwen3.5-35B-A3B(MLX) 우선, Claude API는 종량제로 최후 수단. GPU 서버에 AI Gateway 배치
- Anthropic 약관 확인: 구독 OAuth의 서드파티 사용은 약관상 금지(2026.01~). 자동화에는 API 키만 사용
- NanoClaw는 선택적 확장(대화형 인터페이스)으로 위치, 핵심 파이프라인 비의존
- 장기 로드맵: GPU 서버 확장 → 메인 서버 승격, Mac mini → Roon Core 전용, Synology 장기 유지
### 산출물
- `docs/architecture-v2.md` — 17개 섹션 + 부록 2개 (전체 시스템 설계)
- 마이그레이션 계획서 — Step 1~5 (프로젝트 리네임+정리) + Phase 0~5 (v2 개발)
- 프로젝트 리네임: DEVONThink_my server → hyungi_Document_Server
### 배경 논의 (Cowork 세션)
- v1에서 16개 커밋 중 절반 이상이 AppleScript 버그 수정이었던 점이 전환의 직접적 계기
- Synology Office iframe 임베드로 DEVONthink 미리보기 대체 가능성 논의
- HWP 대응으로 kordoc(광진구청 류승인 주무관 제작, MIT 라이선스) 조사 및 채택
- 편집 가능 문서의 "원본이 뭐냐" 논의 → Synology Office 포맷이 원본, .docx/.xlsx는 교환용
- 가공 데이터 보관 전략 논의 → 파일로 저장하지 않고 PostgreSQL에만 저장, 버전 추적으로 선택적 재가공
## 2026-04-02 — 프로젝트 리네임 + v2 전환 실행
### Step 1: 사전 정리 ✅
- architecture-v2.md 커밋 (`852b7da`)
- v1-archive 브랜치 + v1-final 태그 생성 (v1 상태 완벽 보존)
### Step 2: v1 파일 정리 ✅
- v1 전용 파일 git rm 완료 (`e48b6a2`)
- 삭제: applescript/, launchd/, v1 scripts, v1 docs, tests/test_classify.py, requirements.txt
- 유지: scripts/prompts/classify_document.txt, credentials.env.example (v2 필드로 갱신)
### Step 3: Gitea 리포 리네임 + 로컬 폴더 리네임 ✅
- Gitea: devonthink_home → hyungi_document_server
- 로컬 폴더: DEVONThink_my server → hyungi_Document_Server
- git remote set-url + git ls-remote 검증 + push 완료
### Step 4: 문서 전면 재작성 ✅
- CLAUDE.md — v2 기준으로 전면 재작성
- README.md — 프로젝트명, 기술 스택, 디렉토리 구조 갱신
- docs/deploy.md — Docker Compose 기반 배포 가이드로 교체
- docs/claude-code-commands.md → docs/development-stages.md 변환
- docs/architecture-v2.md → docs/architecture.md 승격
### Step 5: v2 프로젝트 스캐폴딩 ✅
- 전체 디렉토리 구조 생성 (app/, services/kordoc/, gpu-server/, frontend/, migrations/, tests/)
- 동작하는 최소 코드 수준: FastAPI main.py, PostgreSQL 스키마, kordoc server.js, config.yaml 등
- docker-compose.yml, Caddyfile, credentials.env.example 생성
- tests/__init__.py + conftest.py 포함
### Step 1~5 전체 완료.
---
## 2026-04-02 — Phase 0: 기반 구축 시작
### users 테이블 + ORM 모델 추가 ✅
- `migrations/001_initial_schema.sql`에 users 테이블 포함 (username, password_hash, totp_secret, is_active)
- `app/models/user.py` — SQLAlchemy 2.0 Mapped 스타일 ORM 모델
- architecture.md 섹션 6 스키마와 일치
### Auth API 엔드포인트 구현 ✅
- `app/api/auth.py` — 4개 엔드포인트: POST /login (JWT발급+TOTP), POST /refresh, GET /me, POST /change-password
- `app/core/auth.py` — bcrypt 비밀번호 해싱, JWT 발급/검증, TOTP 검증, get_current_user 의존성
- Pydantic 스키마: LoginRequest, TokenResponse, RefreshRequest, ChangePasswordRequest, UserResponse
### main.py 라우터 등록 + health 강화 ✅
- auth 라우터 등록: `/api/auth` prefix
- health 엔드포인트에 DB 연결 상태 포함 (connected/disconnected)
- lifespan 핸들러로 DB 초기화/정리
### Docker 설정 수정 ✅
### 초기 admin 유저 시드 스크립트 ✅
### 셋업 위자드 구현 ✅ (`a601991`)
- `app/api/setup.py` — 6개 엔드포인트: GET /status, POST /admin, POST /totp/init, POST /totp/verify, POST /verify-nas, GET / (HTML)
- `app/templates/setup.html` — Jinja2 단일 HTML, Vanilla JS + qrcode.js CDN, 3단계 위자드
- `app/main.py` — setup 라우터 등록 + 셋업 미들웨어 (유저 0명 시 /setup 리다이렉트, /health /docs 등 바이패스)
- Rate Limiting: IP당 5분 내 5회 실패 시 차단
- TOTP 흐름: init에서 secret 반환(DB 미저장) → verify에서 코드 검증 후 DB 저장
- scripts/seed_admin.py CLI 백업 수단 유지
- requirements.txt에 jinja2 추가
### Phase 0 완료 기준 달성 상태
- ✅ docker compose up → FastAPI 구동
- ✅ DB 스키마 (users, documents, tasks, processing_queue)
- ✅ JWT + TOTP 인증 (로그인, 토큰 갱신, 비밀번호 변경)
- ✅ 셋업 위자드 (관리자 생성 + TOTP + NAS 확인)
- ✅ /health — DB 연결 상태 포함
- ✅ /docs — OpenAPI 문서
- ⬜ NAS SMB 마운트 실제 검증 (Mac mini 배포 시)
- ⬜ config.yaml 로딩 검증 (Mac mini 배포 시)
---
## 2026-04-02 — Phase 1: 데이터 마이그레이션 파이프라인 구현 완료
### Step 1: kordoc /parse 실제 구현 ✅
- `services/kordoc/server.js` — stub → 실제 파싱 구현 (kordoc ^1.7.0 + pdfjs-dist ^4.0.0)
- HWP/HWPX/PDF → Markdown 변환, .md/.txt 직접 읽기, 이미지는 requires_ocr 플래그 반환
- 타임아웃 30초, 파일 미존재 404, 파싱 실패 422
### Step 2: 큐 소비자 인프라 ✅
- `app/workers/queue_consumer.py` — APScheduler AsyncIOScheduler 1분 간격 실행
- 배치 처리: extract=5, classify=3, embed=1
- stage 체이닝: extract → classify → embed 자동 enqueue
- stale 항목 자동 복구 (processing 상태 10분 초과)
- `app/main.py` — lifespan에 APScheduler 연결, yield 후 shutdown 보장
### Step 3: 텍스트 추출 워커 ✅
- `app/workers/extract_worker.py` — 포맷별 분기 처리
- KORDOC_FORMATS (hwp, hwpx, pdf) → kordoc HTTP POST
- TEXT_FORMATS (md, txt, csv, json, xml, html) → 직접 파일 읽기
- IMAGE_FORMATS → Phase 2 OCR로 연기
### Step 4: AI 분류 워커 ✅
- `app/workers/classify_worker.py` — extracted_text 8000자 → AIClient.classify() 호출
- `app/ai/client.py` — strip_thinking(), parse_json_response() 추가 (v1 pkm_utils.py에서 포팅)
- Qwen3.5의 <think> 태그 제거 + 비정형 JSON 파싱 로직
### Step 5: 벡터 임베딩 워커 ✅
- `app/workers/embed_worker.py` — nomic-embed-text-v1.5 (GPU 서버), 6000자 제한
- GPU 서버 불가 시 graceful fail → 재시도
### Step 6: DEVONthink 마이그레이션 스크립트 ✅
- `scripts/migrate_from_devonthink.py` — --dry-run, --source-dir, --target-dir, --database-url 지원
- DEVONthink 내보내기 → NAS PKM 구조 복사 + documents/processing_queue DB 등록
### Phase 1 완료 기준 달성 상태
- ✅ kordoc 파싱 (HWP/HWPX/PDF → Markdown)
- ✅ 큐 소비자 + APScheduler 연동
- ✅ extract → classify → embed 워커 3개
- ✅ AI 클라이언트 think 태그 / JSON 파싱 보강
- ✅ 마이그레이션 스크립트
- ⬜ Step 7: 통합 테스트 + 배치 실행 (Mac mini 배포 후)
---
## 2026-04-02 — Phase 2: 핵심 기능 구현 완료 (`4b69533`)
### 문서 CRUD API ✅
- `app/api/documents.py` — 5개 엔드포인트
- POST /api/documents/ — 파일 업로드 (Inbox 저장 + extract 큐 등록)
- GET /api/documents/ — 목록 조회 (페이징 + domain/source/format 필터)
- GET /api/documents/{id} — 단건 조회
- PATCH /api/documents/{id} — 메타데이터 수동 수정
- DELETE /api/documents/{id} — DB 삭제 (기본), ?delete_file=true로 파일도 삭제
### 하이브리드 검색 API ✅
- `app/api/search.py` — GET /api/search/?q={query}&mode={mode}
- 4가지 모드: fts, trgm, vector, hybrid (기본)
- hybrid 가중치: FTS 0.4 + Trigram 0.2 + Vector 0.4
- 벡터 불가 시 FTS 0.6 + Trigram 0.4 폴백
- 결과에 snippet(200자) 포함
### 파일 워처 ✅
- `app/workers/file_watcher.py` — Inbox 디렉토리 5분 간격 스캔
- 신규 파일: Document 생성 + extract 큐 등록
- 변경 파일: 해시 비교 → 재추출 큐 등록
- .DS_Store, .tmp, .part 등 무시 파일 처리
### 벡터 인덱스 마이그레이션 ✅
- `migrations/002_vector_index.sql` — IVFFlat 인덱스 (cosine, lists=50)
- 문서 수 증가 시 lists 값 조정 필요
### Phase 2 완료 기준 달성 상태
- ✅ 문서 CRUD API (업로드, 목록, 조회, 수정, 삭제)
- ✅ 하이브리드 검색 (FTS + Trigram + Vector)
- ✅ Inbox 파일 워처 (신규/변경 자동 감지 → 파이프라인 등록)
- ✅ 처리 파이프라인 전체 동작 (upload/watch → extract → classify → embed → search)
- ⬜ 문서 뷰어 UI (Phase 4로 이관)
- ⬜ SvelteKit 프론트엔드 (Phase 4로 이관)

174
docs/gpu-migration-plan.md Normal file
View File

@@ -0,0 +1,174 @@
# GPU 서버 이전 + NFS 전환 — Claude Code 작업 지시서
## 배경
Phase 0~4 완료. 현재 Mac mini에서 Docker 전체 구동 중.
GPU 서버(Ubuntu, RTX 4070 Ti Super)로 애플리케이션 이전.
NAS NFS 마운트로 Synology Drive 데드락 해결.
## 완료된 수동 작업
- ✅ Step 1: NAS NFS 서버 설정 (DSM에서 NFS 활성화, NFSv4.1)
- ✅ Step 2: GPU 서버 NFS 마운트 (`/mnt/nas/Document_Server`, fstab 등록 완료)
- ✅ Step 6: Mac mini MLX 서버 외부 접근 확인 (100.76.254.116:8800 응답 확인)
## 확정된 정보
- Mac mini Tailscale IP: `100.76.254.116`
- NAS 로컬 IP: `192.168.1.227`
- GPU 서버 로컬 IP: `192.168.1.186`
- NFS 마운트 경로: `/mnt/nas/Document_Server`
- MLX 모델: `mlx-community/Qwen3.5-35B-A3B-4bit` (Mac mini에서 계속 서빙)
## 목표 구조
```
GPU 서버 (Ubuntu, 메인 서버):
Docker Compose 단일 파일:
- postgres, fastapi, kordoc-service, frontend, caddy
- ollama (NVIDIA GPU), ai-gateway
NFS → NAS /volume4/Document_Server (/mnt/nas/Document_Server)
Mac mini M4 Pro (AI 서버만):
MLX Server: http://100.76.254.116:8800 (Qwen3.5-35B-A3B)
NAS DS1525+ (파일 저장소):
NFS export → GPU 서버
Synology Office/Calendar/MailPlus 유지
```
---
## Claude Code 작업 목록
### 작업 1: docker-compose.yml 통합
현재 루트 `docker-compose.yml` (Mac mini용)에 `gpu-server/docker-compose.yml`의 서비스를 통합.
변경 사항:
- `version: '3.8'` 제거 (Docker Compose V2 기준)
- NAS 볼륨 변수: `NAS_SMB_PATH``NAS_NFS_PATH`, 기본값 `/mnt/nas/Document_Server`
- Ollama 서비스 추가 (NVIDIA GPU runtime, ollama_data 볼륨)
- AI Gateway 서비스 추가 (Ollama depends_on)
- AI Gateway 환경변수: PRIMARY_ENDPOINT=http://100.76.254.116:8800/v1/chat/completions
- Caddy 포트: `127.0.0.1:8080:80` 유지 (HTTPS는 앞단 프록시(UCG-Fiber)에서 처리, Caddy는 HTTP only)
- ollama_data 볼륨 추가
참고 — 현재 파일:
- 루트 docker-compose.yml: postgres, kordoc-service, fastapi, frontend, caddy
- gpu-server/docker-compose.yml: ollama, ai-gateway
### 작업 2: config.yaml AI 엔드포인트 변경
현재 → 변경:
```yaml
ai:
gateway:
endpoint: "http://ai-gateway:8080" # gpu-server → ai-gateway (같은 Docker 네트워크)
models:
primary:
endpoint: "http://100.76.254.116:8800/v1/chat/completions" # host.docker.internal → Mac mini Tailscale IP
# 나머지 동일
fallback:
endpoint: "http://ollama:11434/v1/chat/completions" # gpu-server → ollama (같은 Docker 네트워크)
# 나머지 동일
embedding:
endpoint: "http://ollama:11434/api/embeddings" # gpu-server → ollama
vision:
endpoint: "http://ollama:11434/api/generate" # gpu-server → ollama
rerank:
endpoint: "http://ollama:11434/api/rerank" # gpu-server → ollama
```
핵심: `gpu-server` 호스트명이 전부 `ollama` 또는 `ai-gateway`로 변경 (같은 Docker 네트워크).
primary만 Mac mini Tailscale IP `100.76.254.116`으로 외부 호출.
### 작업 3: credentials.env.example 갱신
변경 사항:
- `NAS_SMB_PATH``NAS_NFS_PATH=/mnt/nas/Document_Server`
- `MLX_ENDPOINT``http://100.76.254.116:8800/v1/chat/completions`
- `GPU_SERVER_IP` 항목 제거 (로컬이 됨)
- `AI_GATEWAY_ENDPOINT``http://ai-gateway:8080` (같은 Docker 네트워크)
- 주석 업데이트: "Mac mini MLX" → "Mac mini MLX (Tailscale 경유)"
### 작업 4: Caddyfile 확인
변경 불필요. 현재 상태 유지:
- `auto_https off` + `http://document.hyungi.net` (HTTPS는 앞단 프록시 UCG-Fiber에서 처리)
- Caddy 포트: `127.0.0.1:8080:80` (localhost 바인딩, 443 불필요)
### 작업 5: 문서 업데이트
#### CLAUDE.md — 네트워크 환경 섹션 갱신
현재:
```
Mac mini M4 Pro (애플리케이션 서버):
- Docker Compose: FastAPI(:8000), PostgreSQL(:5432), kordoc(:3100), Caddy(:80,:443)
- MLX Server: http://localhost:8800/v1/chat/completions (Qwen3.5-35B-A3B)
- 외부 접근: document.hyungi.net (Caddy 프록시)
```
변경:
```
GPU 서버 (RTX 4070 Ti Super, Ubuntu, 메인 서버):
- Docker Compose: FastAPI(:8000), PostgreSQL(:5432), kordoc(:3100), Caddy(:8080, HTTP only), Ollama(:11434), AI Gateway(:8080), frontend(:3000)
- NFS 마운트: /mnt/nas/Document_Server → NAS /volume4/Document_Server
- 외부 접근: document.hyungi.net (Caddy 프록시)
Mac mini M4 Pro (AI 서버):
- MLX Server: http://100.76.254.116:8800/v1/chat/completions (Qwen3.5-35B-A3B)
- Roon Core
```
GPU 서버 Tailscale IP도 추가. AI 모델 구성 섹션에서 primary endpoint 변경 반영.
#### docs/architecture.md — 섹션 3 (인프라 역할 분담) 갱신
Mac mini가 애플리케이션 서버 → GPU 서버가 메인 서버로 변경.
Mac mini는 AI 서버(MLX)만 담당하는 것으로 변경.
아스키 다이어그램 업데이트.
#### docs/deploy.md — GPU 서버 기준 배포 가이드로 변경
- 전제조건: NFS 마운트 (/mnt/nas/Document_Server)
- clone 경로, docker compose 명령 등 GPU 서버 기준으로 변경
- pg_dump/pg_restore 마이그레이션 절차 추가
#### docs/devlog.md — GPU 이전 기록 추가
Phase 1~2는 이미 기록됨. 아래 추가:
1. Phase 3 완료 기록 (자동화 이전: law_monitor, mailplus_archive, daily_digest, automation_state, APScheduler cron) — 기록 안 되어 있으면 추가
2. Phase 4 완료 기록 (SvelteKit UI: 로그인, 대시보드, 문서 탐색/검색, Inbox, 설정, Docker 통합) — 기록 안 되어 있으면 추가
3. GPU 서버 이전 기록 (NFS 전환, docker-compose 통합, AI 엔드포인트 변경, Caddy HTTP only 구조)
### 작업 6: gpu-server/docker-compose.yml 비활성화
- 파일 상단에 주석 추가: "# 이 파일은 더 이상 사용하지 않음. 루트 docker-compose.yml로 통합됨."
- 또는 gpu-server/docker-compose.yml.bak으로 리네임
---
## 작업 순서 (추천)
1. docker-compose.yml 통합 (작업 1)
2. config.yaml 변경 (작업 2)
3. credentials.env.example 갱신 (작업 3)
4. gpu-server/docker-compose.yml 비활성화 (작업 6)
5. 문서 업데이트 (작업 5) — CLAUDE.md, architecture.md, deploy.md, devlog.md
6. Caddyfile 확인 (작업 4)
## 주의사항
- credentials.env 자체는 git에 올리지 않음 (.gitignore). example만 수정.
- Mac mini Tailscale IP `100.76.254.116`은 config.yaml에 직접 기입 (credentials.env에서 변수로 관리해도 됨)
- NAS 경로: Docker 컨테이너 내부에서는 `/documents`로 접근 (기존과 동일)
- GPU 서버 로컬 IP `192.168.1.186`은 NFS 마운트에만 사용, Docker 설정에는 불필요

2245
frontend/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -2,15 +2,23 @@
"name": "hyungi-document-server-frontend",
"version": "0.0.1",
"private": true,
"type": "module",
"scripts": {
"dev": "vite dev",
"build": "vite build",
"preview": "vite preview"
},
"devDependencies": {
"@sveltejs/adapter-node": "^2.0.0",
"@sveltejs/adapter-node": "^5.0.0",
"@sveltejs/kit": "^2.0.0",
"svelte": "^4.0.0",
"vite": "^5.0.0"
"@tailwindcss/vite": "^4.0.0",
"svelte": "^5.0.0",
"tailwindcss": "^4.0.0",
"vite": "^8.0.0"
},
"dependencies": {
"dompurify": "^3.3.3",
"lucide-svelte": "^0.400.0",
"marked": "^15.0.0"
}
}

67
frontend/src/app.css Normal file
View File

@@ -0,0 +1,67 @@
@import "tailwindcss";
:root {
--bg: #0f1117;
--surface: #1a1d27;
--border: #2a2d3a;
--text: #e4e4e7;
--text-dim: #8b8d98;
--accent: #6c8aff;
--accent-hover: #859dff;
--error: #f5564e;
--success: #4ade80;
--warning: #fbbf24;
/* domain 색상 */
--domain-philosophy: #a78bfa;
--domain-language: #f472b6;
--domain-engineering: #38bdf8;
--domain-safety: #fb923c;
--domain-programming: #34d399;
--domain-general: #94a3b8;
--domain-reference: #fbbf24;
/* sidebar */
--sidebar-w: 320px;
--sidebar-bg: #141720;
}
body {
background: var(--bg);
color: var(--text);
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
margin: 0;
}
/* 스크롤바 */
::-webkit-scrollbar { width: 6px; }
::-webkit-scrollbar-track { background: var(--bg); }
::-webkit-scrollbar-thumb { background: var(--border); border-radius: 3px; }
/* Markdown 렌더링 (GitHub Dark 스타일) */
.markdown-body {
color: var(--text);
line-height: 1.7;
font-size: 14px;
}
.markdown-body h1 { font-size: 1.6em; font-weight: 700; margin: 1.5em 0 0.5em; padding-bottom: 0.3em; border-bottom: 1px solid var(--border); }
.markdown-body h2 { font-size: 1.3em; font-weight: 600; margin: 1.3em 0 0.4em; padding-bottom: 0.2em; border-bottom: 1px solid var(--border); }
.markdown-body h3 { font-size: 1.1em; font-weight: 600; margin: 1.2em 0 0.3em; }
.markdown-body h4 { font-size: 1em; font-weight: 600; margin: 1em 0 0.2em; }
.markdown-body p { margin: 0.6em 0; }
.markdown-body ul, .markdown-body ol { padding-left: 1.5em; margin: 0.5em 0; }
.markdown-body li { margin: 0.2em 0; }
.markdown-body li > ul, .markdown-body li > ol { margin: 0.1em 0; }
.markdown-body blockquote { border-left: 3px solid var(--accent); padding: 0.5em 1em; margin: 0.8em 0; color: var(--text-dim); background: var(--surface); border-radius: 0 4px 4px 0; }
.markdown-body code { background: var(--surface); padding: 0.15em 0.4em; border-radius: 3px; font-size: 0.9em; font-family: 'SF Mono', Menlo, monospace; }
.markdown-body pre { background: var(--surface); padding: 1em; border-radius: 6px; overflow-x: auto; margin: 0.8em 0; border: 1px solid var(--border); }
.markdown-body pre code { background: none; padding: 0; }
.markdown-body table { border-collapse: collapse; width: 100%; margin: 0.8em 0; }
.markdown-body th, .markdown-body td { border: 1px solid var(--border); padding: 0.5em 0.8em; text-align: left; font-size: 0.9em; }
.markdown-body th { background: var(--surface); font-weight: 600; }
.markdown-body tr:nth-child(even) { background: rgba(255,255,255,0.02); }
.markdown-body hr { border: none; border-top: 1px solid var(--border); margin: 1.5em 0; }
.markdown-body a { color: var(--accent); text-decoration: none; }
.markdown-body a:hover { text-decoration: underline; }
.markdown-body strong { font-weight: 600; }
.markdown-body img { max-width: 100%; border-radius: 4px; }

135
frontend/src/lib/api.ts Normal file
View File

@@ -0,0 +1,135 @@
/**
* API fetch 래퍼
*
* - access token: 메모리 변수
* - refresh token: HttpOnly cookie (서버가 관리)
* - refresh 중복 방지: isRefreshing 플래그 + 대기 큐
* - 401 retry: 1회만, 실패 시 강제 logout
*/
const API_BASE = '/api';
let accessToken: string | null = null;
// refresh 큐
let isRefreshing = false;
let refreshQueue: Array<{
resolve: (token: string) => void;
reject: (err: Error) => void;
}> = [];
export function setAccessToken(token: string | null) {
accessToken = token;
}
export function getAccessToken(): string | null {
return accessToken;
}
async function refreshAccessToken(): Promise<string> {
const res = await fetch(`${API_BASE}/auth/refresh`, {
method: 'POST',
credentials: 'include', // cookie 전송
});
if (!res.ok) {
throw new Error('refresh failed');
}
const data = await res.json();
accessToken = data.access_token;
return data.access_token;
}
function processRefreshQueue(error: Error | null, token: string | null) {
refreshQueue.forEach(({ resolve, reject }) => {
if (error) reject(error);
else resolve(token!);
});
refreshQueue = [];
}
async function handleTokenRefresh(): Promise<string> {
if (isRefreshing) {
return new Promise((resolve, reject) => {
refreshQueue.push({ resolve, reject });
});
}
isRefreshing = true;
try {
const token = await refreshAccessToken();
processRefreshQueue(null, token);
return token;
} catch (err) {
const error = err instanceof Error ? err : new Error('refresh failed');
processRefreshQueue(error, null);
// 강제 logout
accessToken = null;
if (typeof window !== 'undefined') {
window.location.href = '/login';
}
throw error;
} finally {
isRefreshing = false;
}
}
export type ApiError = {
status: number;
detail: string;
};
export async function api<T = unknown>(
path: string,
options: RequestInit = {},
): Promise<T> {
const headers: Record<string, string> = {
...(options.headers as Record<string, string> || {}),
};
if (accessToken) {
headers['Authorization'] = `Bearer ${accessToken}`;
}
// FormData일 때는 Content-Type 자동 설정
if (options.body && !(options.body instanceof FormData)) {
headers['Content-Type'] = 'application/json';
}
const res = await fetch(`${API_BASE}${path}`, {
...options,
headers,
credentials: 'include',
});
// 401 → refresh 1회 시도 (로그인/리프레시 엔드포인트는 제외)
const isAuthEndpoint = path.startsWith('/auth/login') || path.startsWith('/auth/refresh');
if (res.status === 401 && accessToken && !isAuthEndpoint) {
try {
await handleTokenRefresh();
headers['Authorization'] = `Bearer ${accessToken}`;
const retryRes = await fetch(`${API_BASE}${path}`, {
...options,
headers,
credentials: 'include',
});
if (!retryRes.ok) {
const err = await retryRes.json().catch(() => ({ detail: 'Unknown error' }));
throw { status: retryRes.status, detail: err.detail || retryRes.statusText } as ApiError;
}
return retryRes.json();
} catch (e) {
if ((e as ApiError).detail) throw e;
throw { status: 401, detail: '인증이 만료되었습니다' } as ApiError;
}
}
if (!res.ok) {
const err = await res.json().catch(() => ({ detail: res.statusText }));
throw { status: res.status, detail: err.detail || res.statusText } as ApiError;
}
// 204 No Content
if (res.status === 204) return {} as T;
return res.json();
}

View File

@@ -0,0 +1,116 @@
<script>
import { goto } from '$app/navigation';
import FormatIcon from './FormatIcon.svelte';
import TagPill from './TagPill.svelte';
let { doc, showDomain = true, selected = false, onselect = null } = $props();
function formatDate(dateStr) {
if (!dateStr) return '';
const d = new Date(dateStr);
const now = new Date();
const diff = now - d;
if (diff < 86400000) return '오늘';
if (diff < 172800000) return '어제';
if (diff < 604800000) return `${Math.floor(diff / 86400000)}일 전`;
return d.toLocaleDateString('ko-KR', { month: 'short', day: 'numeric' });
}
function formatSize(bytes) {
if (!bytes) return '';
if (bytes < 1024) return `${bytes}B`;
if (bytes < 1048576) return `${(bytes / 1024).toFixed(0)}KB`;
return `${(bytes / 1048576).toFixed(1)}MB`;
}
const DOMAIN_COLORS = {
'Knowledge/Philosophy': 'var(--domain-philosophy)',
'Knowledge/Language': 'var(--domain-language)',
'Knowledge/Engineering': 'var(--domain-engineering)',
'Knowledge/Industrial_Safety': 'var(--domain-safety)',
'Knowledge/Programming': 'var(--domain-programming)',
'Knowledge/General': 'var(--domain-general)',
'Reference': 'var(--domain-reference)',
};
let domainColor = $derived(DOMAIN_COLORS[doc.ai_domain] || 'var(--border)');
// 반응형: CSS media query matchMedia 사용
let isDesktop = $state(typeof window !== 'undefined' ? window.matchMedia('(min-width: 1024px)').matches : true);
if (typeof window !== 'undefined') {
window.matchMedia('(min-width: 1024px)').addEventListener('change', (e) => isDesktop = e.matches);
}
function handleClick() {
if (!isDesktop) {
goto(`/documents/${doc.id}`);
return;
}
if (onselect) {
onselect(doc);
} else {
goto(`/documents/${doc.id}`);
}
}
</script>
<button
onclick={handleClick}
aria-label={doc.title || '문서 선택'}
class="flex items-stretch bg-[var(--surface)] border rounded-lg hover:border-[var(--accent)] transition-colors group w-full text-left overflow-hidden
{selected ? 'border-[var(--accent)] bg-[var(--accent)]/5' : 'border-[var(--border)]'}"
>
<!-- domain 색상 바 -->
<div class="w-1 shrink-0 rounded-l-lg" style="background: {domainColor}"></div>
<!-- 콘텐츠 -->
<div class="flex items-start gap-3 p-3 flex-1 min-w-0">
<!-- 포맷 아이콘 -->
<div class="shrink-0 mt-0.5 text-[var(--text-dim)] group-hover:text-[var(--accent)]">
<FormatIcon format={doc.file_format} size={18} />
</div>
<!-- 메인 콘텐츠 -->
<div class="flex-1 min-w-0">
<p class="text-sm font-medium truncate group-hover:text-[var(--accent)]">
{doc.title || '제목 없음'}
</p>
{#if doc.ai_summary}
<p class="text-xs text-[var(--text-dim)] truncate mt-0.5">{doc.ai_summary.replace(/[*#_`~]/g, '').slice(0, 100)}</p>
{/if}
<div class="flex items-center gap-2 mt-1.5 flex-wrap">
{#if showDomain && doc.ai_domain}
<span class="text-[10px] text-[var(--text-dim)]">
{doc.ai_domain.replace('Knowledge/', '')}{doc.ai_sub_group ? ` / ${doc.ai_sub_group}` : ''}
</span>
{/if}
{#if doc.ai_tags?.length}
<div class="flex gap-1">
{#each doc.ai_tags.slice(0, 3) as tag}
<TagPill {tag} />
{/each}
</div>
{/if}
</div>
</div>
<!-- 우측 메타 -->
<div class="shrink-0 flex flex-col items-end gap-1 text-[10px]">
{#if doc.source_channel === 'news' && doc.edit_url}
<span class="text-blue-400">📰</span>
{/if}
{#if doc.score !== undefined}
<span class="text-[var(--accent)] font-medium">{(doc.score * 100).toFixed(0)}%</span>
{/if}
{#if doc.data_origin}
<span class="px-1.5 py-0.5 rounded {doc.data_origin === 'work' ? 'bg-blue-900/30 text-blue-400' : 'bg-gray-800 text-gray-400'}">
{doc.data_origin}
</span>
{/if}
<span class="text-[var(--text-dim)]">{formatDate(doc.created_at)}</span>
{#if doc.file_size}
<span class="text-[var(--text-dim)]">{formatSize(doc.file_size)}</span>
{/if}
</div>
</div>
</button>

View File

@@ -0,0 +1,142 @@
<script>
import { goto } from '$app/navigation';
import FormatIcon from './FormatIcon.svelte';
let { items = [], selectedId = null, onselect = null } = $props();
let sortKey = $state('created_at');
let sortOrder = $state('desc');
// localStorage에서 정렬 상태 복원
if (typeof localStorage !== 'undefined') {
const saved = localStorage.getItem('tableSort');
if (saved) {
try {
const { key, order } = JSON.parse(saved);
sortKey = key;
sortOrder = order;
} catch (e) {}
}
}
function toggleSort(key) {
if (sortKey === key) {
sortOrder = sortOrder === 'asc' ? 'desc' : 'asc';
} else {
sortKey = key;
sortOrder = 'asc';
}
if (typeof localStorage !== 'undefined') {
localStorage.setItem('tableSort', JSON.stringify({ key: sortKey, order: sortOrder }));
}
}
// stable sort
let sortedItems = $derived(() => {
const arr = [...items];
arr.sort((a, b) => {
let va = a[sortKey] ?? '';
let vb = b[sortKey] ?? '';
if (typeof va === 'string') va = va.toLowerCase();
if (typeof vb === 'string') vb = vb.toLowerCase();
if (va === vb) return a.id - b.id;
if (sortOrder === 'asc') return va > vb ? 1 : -1;
return va < vb ? 1 : -1;
});
return arr;
});
function formatDate(dateStr) {
if (!dateStr) return '-';
const d = new Date(dateStr);
return d.toLocaleDateString('ko-KR', { month: 'short', day: 'numeric' });
}
function formatSize(bytes) {
if (!bytes) return '-';
if (bytes < 1024) return `${bytes}B`;
if (bytes < 1048576) return `${(bytes / 1024).toFixed(0)}KB`;
return `${(bytes / 1048576).toFixed(1)}MB`;
}
function handleClick(doc) {
if (typeof window !== 'undefined' && window.innerWidth < 1024) {
goto(`/documents/${doc.id}`);
return;
}
if (onselect) onselect(doc);
else goto(`/documents/${doc.id}`);
}
const DOMAIN_COLORS = {
'Philosophy': 'var(--domain-philosophy)',
'Language': 'var(--domain-language)',
'Engineering': 'var(--domain-engineering)',
'Industrial_Safety': 'var(--domain-safety)',
'Programming': 'var(--domain-programming)',
'General': 'var(--domain-general)',
'Reference': 'var(--domain-reference)',
};
function getDomainColor(domain) {
if (!domain) return 'var(--border)';
const top = domain.split('/')[0];
return DOMAIN_COLORS[top] || 'var(--border)';
}
const columns = [
{ key: 'title', label: '이름', flex: 'flex-1' },
{ key: 'ai_domain', label: '분류', width: 'w-48' },
{ key: 'document_type', label: '타입', width: 'w-24' },
{ key: 'file_size', label: '크기', width: 'w-20' },
{ key: 'created_at', label: '등록일', width: 'w-20' },
];
</script>
<div class="w-full">
<!-- 헤더 -->
<div class="flex items-center gap-1 px-2 py-1.5 border-b border-[var(--border)] text-[10px] text-[var(--text-dim)] uppercase tracking-wider">
{#each columns as col}
<button
onclick={() => toggleSort(col.key)}
class="flex items-center gap-1 {col.flex || col.width || ''} px-1 hover:text-[var(--text)] transition-colors text-left"
>
{col.label}
{#if sortKey === col.key}
<span class="text-[var(--accent)]">{sortOrder === 'asc' ? '↑' : '↓'}</span>
{/if}
</button>
{/each}
</div>
<!-- 행 -->
{#each sortedItems() as doc}
<button
onclick={() => handleClick(doc)}
class="flex items-center gap-1 px-2 py-1.5 w-full text-left border-b border-[var(--border)]/30 hover:bg-[var(--surface)] transition-colors group
{selectedId === doc.id ? 'bg-[var(--accent)]/5 border-l-2 border-l-[var(--accent)]' : ''}"
>
<!-- 이름 -->
<div class="flex-1 flex items-center gap-2 min-w-0">
<span class="w-1 h-4 rounded-full shrink-0" style="background: {getDomainColor(doc.ai_domain)}"></span>
<FormatIcon format={doc.file_format} size={14} />
<span class="text-xs truncate group-hover:text-[var(--accent)]">{doc.title || '제목 없음'}</span>
</div>
<!-- 분류 -->
<div class="w-48 text-[10px] text-[var(--text-dim)] truncate">
{doc.ai_domain?.replace('Industrial_Safety/', 'IS/') || '-'}
</div>
<!-- 타입 -->
<div class="w-24 text-[10px] text-[var(--text-dim)]">
{doc.document_type || doc.file_format?.toUpperCase() || '-'}
</div>
<!-- 크기 -->
<div class="w-20 text-[10px] text-[var(--text-dim)] text-right">
{formatSize(doc.file_size)}
</div>
<!-- 등록일 -->
<div class="w-20 text-[10px] text-[var(--text-dim)] text-right">
{formatDate(doc.created_at)}
</div>
</button>
{/each}
</div>

View File

@@ -0,0 +1,270 @@
<script>
import { api, getAccessToken } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { marked } from 'marked';
import DOMPurify from 'dompurify';
import { ExternalLink, Save, RefreshCw } from 'lucide-svelte';
// marked + sanitize
marked.use({ mangle: false, headerIds: false });
function renderMd(text) {
return DOMPurify.sanitize(marked(text), {
USE_PROFILES: { html: true },
FORBID_TAGS: ['style', 'script'],
FORBID_ATTR: ['onerror', 'onclick'],
ALLOW_UNKNOWN_PROTOCOLS: false,
});
}
let { doc } = $props();
let fullDoc = $state(null);
let loading = $state(true);
let viewerType = $state('none');
// Markdown 편집
let editMode = $state(false);
let editContent = $state('');
let editTab = $state('edit');
let saving = $state(false);
let rawMarkdown = $state('');
function getViewerType(format) {
if (['md', 'txt'].includes(format)) return 'markdown';
if (format === 'pdf') return 'pdf';
if (['hwp', 'hwpx'].includes(format)) return 'preview-pdf';
if (['odoc', 'osheet', 'docx', 'xlsx', 'pptx', 'odt', 'ods', 'odp'].includes(format)) return 'preview-pdf';
if (['jpg', 'jpeg', 'png', 'gif', 'bmp', 'tiff'].includes(format)) return 'image';
if (['csv', 'json', 'xml', 'html'].includes(format)) return 'text';
if (['dwg', 'dxf'].includes(format)) return 'cad';
return 'unsupported';
}
const ODF_FORMATS = ['ods', 'odt', 'odp', 'odoc', 'osheet'];
function getEditInfo(doc) {
// DB에 저장된 편집 URL 우선
if (doc.edit_url) return { url: doc.edit_url, label: '편집' };
// ODF 포맷 → Synology Drive
if (ODF_FORMATS.includes(doc.file_format)) return { url: 'https://link.hyungi.net', label: 'Synology Drive에서 열기' };
// CAD
if (['dwg', 'dxf'].includes(doc.file_format)) return { url: 'https://web.autocad.com', label: 'AutoCAD Web' };
return null;
}
$effect(() => {
if (doc?.id) {
loadFullDoc(doc.id);
editMode = false;
}
});
async function loadFullDoc(id) {
loading = true;
try {
fullDoc = await api(`/documents/${id}`);
viewerType = fullDoc.source_channel === 'news' ? 'article' : getViewerType(fullDoc.file_format);
// Markdown: extracted_text 없으면 원본 파일 직접 가져오기
if (viewerType === 'markdown' && !fullDoc.extracted_text) {
try {
const resp = await fetch(`/api/documents/${id}/file?token=${getAccessToken()}`);
if (resp.ok) rawMarkdown = await resp.text();
} catch (e) { rawMarkdown = ''; }
} else {
rawMarkdown = '';
}
} catch (err) {
fullDoc = null;
viewerType = 'none';
} finally {
loading = false;
}
}
function startEdit() {
editContent = fullDoc?.extracted_text || rawMarkdown || '';
editMode = true;
editTab = 'edit';
}
async function saveContent() {
saving = true;
try {
await api(`/documents/${fullDoc.id}/content`, {
method: 'PUT',
body: JSON.stringify({ content: editContent }),
});
fullDoc.extracted_text = editContent;
editMode = false;
addToast('success', '저장됨');
} catch (err) {
addToast('error', '저장 실패');
} finally {
saving = false;
}
}
function handleKeydown(e) {
if ((e.metaKey || e.ctrlKey) && e.key === 's' && editMode) {
e.preventDefault();
saveContent();
}
}
let editInfo = $derived(fullDoc ? getEditInfo(fullDoc) : null);
</script>
<svelte:window on:keydown={handleKeydown} />
<div class="h-full flex flex-col bg-[var(--surface)] border-t border-[var(--border)]">
<!-- 뷰어 툴바 -->
{#if fullDoc && !loading}
<div class="flex items-center justify-between px-3 py-1.5 border-b border-[var(--border)] bg-[var(--sidebar-bg)] shrink-0">
<span class="text-xs text-[var(--text-dim)] truncate">{fullDoc.title || '제목 없음'}</span>
<div class="flex items-center gap-2">
{#if viewerType === 'markdown'}
{#if editMode}
<button
onclick={saveContent}
disabled={saving}
class="flex items-center gap-1 px-2 py-1 text-xs bg-[var(--accent)] text-white rounded hover:bg-[var(--accent-hover)] disabled:opacity-50"
>
<Save size={12} /> {saving ? '저장 중...' : '저장'}
</button>
<button
onclick={() => editMode = false}
class="px-2 py-1 text-xs text-[var(--text-dim)] hover:text-[var(--text)]"
>취소</button>
{:else}
<button
onclick={startEdit}
class="px-2 py-1 text-xs text-[var(--text-dim)] hover:text-[var(--accent)] border border-[var(--border)] rounded"
>편집</button>
{/if}
{/if}
{#if editInfo}
<a
href={editInfo.url}
target="_blank"
rel="noopener"
class="flex items-center gap-1 px-2 py-1 text-xs text-[var(--text-dim)] hover:text-[var(--accent)] border border-[var(--border)] rounded"
>
<ExternalLink size={12} /> {editInfo.label}
</a>
{/if}
<a
href="/documents/{fullDoc.id}"
class="px-2 py-1 text-xs text-[var(--text-dim)] hover:text-[var(--accent)] border border-[var(--border)] rounded"
>전체 보기</a>
</div>
</div>
{/if}
<!-- 뷰어 본문 -->
<div class="flex-1 overflow-auto min-h-0">
{#if loading}
<div class="flex items-center justify-center h-full">
<p class="text-sm text-[var(--text-dim)]">로딩 중...</p>
</div>
{:else if fullDoc}
{#if viewerType === 'markdown'}
{#if editMode}
<!-- Markdown 편집 (탭 전환) -->
<div class="flex flex-col h-full">
<div class="flex gap-1 px-3 py-1 border-b border-[var(--border)] shrink-0">
<button
onclick={() => editTab = 'edit'}
class="px-3 py-1 text-xs rounded-t {editTab === 'edit' ? 'bg-[var(--surface)] text-[var(--text)]' : 'text-[var(--text-dim)]'}"
>편집</button>
<button
onclick={() => editTab = 'preview'}
class="px-3 py-1 text-xs rounded-t {editTab === 'preview' ? 'bg-[var(--surface)] text-[var(--text)]' : 'text-[var(--text-dim)]'}"
>미리보기</button>
</div>
{#if editTab === 'edit'}
<textarea
bind:value={editContent}
class="flex-1 w-full p-4 bg-[var(--bg)] text-[var(--text)] text-sm font-mono resize-none outline-none"
spellcheck="false"
></textarea>
{:else}
<div class="flex-1 overflow-auto p-4 markdown-body">
{@html renderMd(editContent)}
</div>
{/if}
</div>
{:else}
<div class="p-4 markdown-body">
{@html renderMd(fullDoc.extracted_text || rawMarkdown || '*텍스트 추출 대기 중*')}
</div>
{/if}
{:else if viewerType === 'pdf'}
<iframe
src="/api/documents/{fullDoc.id}/file?token={getAccessToken()}"
class="w-full h-full border-0"
title={fullDoc.title}
></iframe>
{:else if viewerType === 'preview-pdf'}
<iframe
src="/api/documents/{fullDoc.id}/preview?token={getAccessToken()}"
class="w-full h-full border-0"
title={fullDoc.title}
onerror={() => {}}
></iframe>
{:else if viewerType === 'image'}
<div class="flex items-center justify-center h-full p-4">
<img
src="/api/documents/{fullDoc.id}/file?token={getAccessToken()}"
alt={fullDoc.title}
class="max-w-full max-h-full object-contain rounded"
/>
</div>
{:else if viewerType === 'text'}
<div class="p-4">
<pre class="text-sm text-[var(--text)] whitespace-pre-wrap font-mono">{fullDoc.extracted_text || '텍스트 없음'}</pre>
</div>
{:else if viewerType === 'cad'}
<div class="flex flex-col items-center justify-center h-full gap-3">
<p class="text-sm text-[var(--text-dim)]">CAD 미리보기 (향후 지원 예정)</p>
<a
href="https://web.autocad.com"
target="_blank"
class="px-3 py-1.5 text-sm bg-[var(--accent)] text-white rounded hover:bg-[var(--accent-hover)]"
>AutoCAD Web에서 열기</a>
</div>
{:else if viewerType === 'article'}
<!-- 뉴스 전용 뷰어 -->
<div class="p-5 max-w-3xl mx-auto">
<h1 class="text-lg font-bold mb-2">{fullDoc.title}</h1>
<div class="flex items-center gap-2 mb-4 text-xs text-[var(--text-dim)]">
{#if fullDoc.ai_tags?.length}
{#each fullDoc.ai_tags.filter(t => t.startsWith('News/')) as tag}
<span class="px-1.5 py-0.5 rounded bg-blue-900/30 text-blue-400">{tag.replace('News/', '')}</span>
{/each}
{/if}
<span>{new Date(fullDoc.created_at).toLocaleDateString('ko-KR', { year: 'numeric', month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' })}</span>
</div>
<div class="markdown-body mb-6">
{@html renderMd(fullDoc.extracted_text || '')}
</div>
<div class="flex items-center gap-3 pt-4 border-t border-[var(--border)]">
{#if fullDoc.edit_url}
<a
href={fullDoc.edit_url}
target="_blank"
rel="noopener noreferrer"
class="flex items-center gap-1 px-3 py-1.5 text-sm bg-[var(--accent)] text-white rounded-lg hover:bg-[var(--accent-hover)]"
>
<ExternalLink size={14} /> 원문 보기
</a>
{/if}
</div>
</div>
{:else}
<div class="flex items-center justify-center h-full">
<p class="text-sm text-[var(--text-dim)]">미리보기를 지원하지 않는 형식입니다 ({fullDoc.file_format})</p>
</div>
{/if}
{/if}
</div>
</div>

View File

@@ -0,0 +1,41 @@
<script>
import { FileText, File, Image, FileSpreadsheet, Presentation, Mail, FileCode, FileQuestion } from 'lucide-svelte';
let { format = '', size = 16 } = $props();
const ICON_MAP = {
pdf: FileText,
hwp: FileText,
hwpx: FileText,
md: FileCode,
txt: File,
csv: FileSpreadsheet,
json: FileCode,
xml: FileCode,
html: FileCode,
jpg: Image,
jpeg: Image,
png: Image,
gif: Image,
bmp: Image,
tiff: Image,
eml: Mail,
odoc: FileText,
osheet: FileSpreadsheet,
docx: FileText,
doc: FileText,
xlsx: FileSpreadsheet,
xls: FileSpreadsheet,
pptx: Presentation,
ppt: Presentation,
odt: FileText,
ods: FileSpreadsheet,
odp: Presentation,
dwg: FileCode,
dxf: FileCode,
};
let Icon = $derived(ICON_MAP[format?.toLowerCase()] || FileQuestion);
</script>
<svelte:component this={Icon} {size} />

View File

@@ -0,0 +1,344 @@
<script>
import { X, ExternalLink, Plus, Save, Trash2 } from 'lucide-svelte';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import FormatIcon from './FormatIcon.svelte';
import TagPill from './TagPill.svelte';
let { doc, onclose, ondelete = () => {} } = $props();
// 메모 상태
let noteText = $state('');
let noteEditing = $state(false);
let noteSaving = $state(false);
// 태그 편집
let newTag = $state('');
let tagEditing = $state(false);
// 삭제
let deleteConfirm = $state(false);
let deleting = $state(false);
async function deleteDoc() {
deleting = true;
try {
await api(`/documents/${doc.id}?delete_file=true`, { method: 'DELETE' });
addToast('success', '문서 삭제됨');
ondelete();
} catch (err) {
addToast('error', '삭제 실패');
} finally {
deleting = false;
deleteConfirm = false;
}
}
// 편집 URL
let editUrlText = $state('');
let editUrlEditing = $state(false);
// doc 변경 시 초기화
$effect(() => {
if (doc) {
noteText = doc.user_note || '';
editUrlText = doc.edit_url || '';
noteEditing = false;
tagEditing = false;
editUrlEditing = false;
newTag = '';
}
});
async function saveNote() {
noteSaving = true;
try {
await api(`/documents/${doc.id}`, {
method: 'PATCH',
body: JSON.stringify({ user_note: noteText }),
});
doc.user_note = noteText;
noteEditing = false;
addToast('success', '메모 저장됨');
} catch (err) {
addToast('error', '메모 저장 실패');
} finally {
noteSaving = false;
}
}
async function saveEditUrl() {
try {
await api(`/documents/${doc.id}`, {
method: 'PATCH',
body: JSON.stringify({ edit_url: editUrlText.trim() || null }),
});
doc.edit_url = editUrlText.trim() || null;
editUrlEditing = false;
addToast('success', '편집 URL 저장됨');
} catch (err) {
addToast('error', '편집 URL 저장 실패');
}
}
async function addTag() {
const tag = newTag.trim();
if (!tag) return;
const updatedTags = [...(doc.ai_tags || []), tag];
try {
await api(`/documents/${doc.id}`, {
method: 'PATCH',
body: JSON.stringify({ ai_tags: updatedTags }),
});
doc.ai_tags = updatedTags;
newTag = '';
addToast('success', '태그 추가됨');
} catch (err) {
addToast('error', '태그 추가 실패');
}
}
async function removeTag(tagToRemove) {
const updatedTags = (doc.ai_tags || []).filter(t => t !== tagToRemove);
try {
await api(`/documents/${doc.id}`, {
method: 'PATCH',
body: JSON.stringify({ ai_tags: updatedTags }),
});
doc.ai_tags = updatedTags;
addToast('success', '태그 삭제됨');
} catch (err) {
addToast('error', '태그 삭제 실패');
}
}
function formatDate(dateStr) {
if (!dateStr) return '-';
return new Date(dateStr).toLocaleDateString('ko-KR', { year: 'numeric', month: 'short', day: 'numeric' });
}
function formatSize(bytes) {
if (!bytes) return '-';
if (bytes < 1024) return `${bytes}B`;
if (bytes < 1048576) return `${(bytes / 1024).toFixed(0)}KB`;
return `${(bytes / 1048576).toFixed(1)}MB`;
}
</script>
<aside class="h-full flex flex-col bg-[var(--sidebar-bg)] border-l border-[var(--border)] overflow-y-auto">
<!-- 헤더 -->
<div class="flex items-center justify-between px-4 py-3 border-b border-[var(--border)] shrink-0">
<div class="flex items-center gap-2 min-w-0">
<FormatIcon format={doc.file_format} size={16} />
<span class="text-sm font-medium truncate">{doc.title || '제목 없음'}</span>
</div>
<div class="flex items-center gap-1">
<a href="/documents/{doc.id}" class="p-1 rounded hover:bg-[var(--surface)] text-[var(--text-dim)]" title="전체 보기">
<ExternalLink size={14} />
</a>
<button onclick={onclose} class="p-1 rounded hover:bg-[var(--surface)] text-[var(--text-dim)]" aria-label="닫기">
<X size={16} />
</button>
</div>
</div>
<div class="flex-1 p-4 space-y-4">
<!-- 메모 -->
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">메모</h4>
{#if noteEditing}
<textarea
bind:value={noteText}
class="w-full h-24 px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-sm text-[var(--text)] resize-none outline-none focus:border-[var(--accent)]"
placeholder="메모 입력..."
></textarea>
<div class="flex gap-2 mt-1.5">
<button
onclick={saveNote}
disabled={noteSaving}
class="flex items-center gap-1 px-2 py-1 text-xs bg-[var(--accent)] text-white rounded hover:bg-[var(--accent-hover)] disabled:opacity-50"
>
<Save size={12} /> 저장
</button>
<button
onclick={() => { noteEditing = false; noteText = doc.user_note || ''; }}
class="px-2 py-1 text-xs text-[var(--text-dim)] hover:text-[var(--text)]"
>취소</button>
</div>
{:else}
<button
onclick={() => noteEditing = true}
class="w-full text-left px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-sm min-h-[40px]
{noteText ? 'text-[var(--text)]' : 'text-[var(--text-dim)]'}"
>
{noteText || '메모 추가...'}
</button>
{/if}
</div>
<!-- 편집 URL -->
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">편집 링크</h4>
{#if editUrlEditing}
<div class="flex gap-1">
<input
bind:value={editUrlText}
placeholder="Synology Drive URL 붙여넣기..."
class="flex-1 px-2 py-1 bg-[var(--bg)] border border-[var(--border)] rounded text-xs text-[var(--text)] outline-none focus:border-[var(--accent)]"
/>
<button onclick={saveEditUrl} class="px-2 py-1 text-xs bg-[var(--accent)] text-white rounded">저장</button>
<button onclick={() => { editUrlEditing = false; editUrlText = doc.edit_url || ''; }} class="px-2 py-1 text-xs text-[var(--text-dim)]">취소</button>
</div>
{:else if doc.edit_url}
<div class="flex items-center gap-1">
<a href={doc.edit_url} target="_blank" class="text-xs text-[var(--accent)] truncate hover:underline">{doc.edit_url}</a>
<button onclick={() => editUrlEditing = true} class="text-[10px] text-[var(--text-dim)] hover:text-[var(--text)]">수정</button>
</div>
{:else}
<button
onclick={() => editUrlEditing = true}
class="text-xs text-[var(--text-dim)] hover:text-[var(--accent)]"
>+ URL 추가</button>
{/if}
</div>
<!-- 태그 -->
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">태그</h4>
<div class="flex flex-wrap gap-1 mb-2">
{#each doc.ai_tags || [] as tag}
<span class="inline-flex items-center gap-0.5">
<TagPill {tag} clickable={false} />
<button
onclick={() => removeTag(tag)}
class="text-[var(--text-dim)] hover:text-[var(--error)] text-[10px]"
title="삭제"
>×</button>
</span>
{/each}
</div>
{#if tagEditing}
<form onsubmit={(e) => { e.preventDefault(); addTag(); }} class="flex gap-1">
<input
bind:value={newTag}
placeholder="태그 입력..."
class="flex-1 px-2 py-1 bg-[var(--bg)] border border-[var(--border)] rounded text-xs text-[var(--text)] outline-none focus:border-[var(--accent)]"
/>
<button type="submit" class="px-2 py-1 text-xs bg-[var(--accent)] text-white rounded">추가</button>
</form>
{:else}
<button
onclick={() => tagEditing = true}
class="flex items-center gap-1 text-xs text-[var(--text-dim)] hover:text-[var(--accent)]"
>
<Plus size={12} /> 태그 추가
</button>
{/if}
</div>
<!-- AI 분류 -->
{#if doc.ai_domain}
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">분류</h4>
<!-- domain breadcrumb -->
<div class="flex flex-wrap gap-1 mb-2">
{#each doc.ai_domain.split('/') as part, i}
{#if i > 0}<span class="text-[10px] text-[var(--text-dim)]"></span>{/if}
<span class="text-xs text-[var(--accent)]">{part}</span>
{/each}
</div>
<!-- document_type + confidence -->
<div class="flex items-center gap-2">
{#if doc.document_type}
<span class="text-[10px] px-1.5 py-0.5 rounded bg-blue-900/30 text-blue-400">{doc.document_type}</span>
{/if}
{#if doc.ai_confidence}
<span class="text-[10px] px-1.5 py-0.5 rounded {doc.ai_confidence >= 0.85 ? 'bg-green-900/30 text-green-400' : doc.ai_confidence >= 0.6 ? 'bg-amber-900/30 text-amber-400' : 'bg-red-900/30 text-red-400'}">
{(doc.ai_confidence * 100).toFixed(0)}%
</span>
{/if}
{#if doc.importance && doc.importance !== 'medium'}
<span class="text-[10px] px-1.5 py-0.5 rounded {doc.importance === 'high' ? 'bg-red-900/30 text-red-400' : 'bg-gray-800 text-gray-400'}">
{doc.importance}
</span>
{/if}
</div>
</div>
{/if}
<!-- 파일 정보 -->
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">정보</h4>
<dl class="space-y-1.5 text-xs">
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">포맷</dt>
<dd class="uppercase">{doc.file_format}{doc.original_format ? ` (원본: ${doc.original_format})` : ''}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">크기</dt>
<dd>{formatSize(doc.file_size)}</dd>
</div>
{#if doc.source_channel}
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">출처</dt>
<dd>{doc.source_channel}</dd>
</div>
{/if}
{#if doc.data_origin}
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">구분</dt>
<dd>{doc.data_origin}</dd>
</div>
{/if}
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">등록일</dt>
<dd>{formatDate(doc.created_at)}</dd>
</div>
</dl>
</div>
<!-- 처리 상태 -->
<div>
<h4 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-1.5">처리</h4>
<dl class="space-y-1 text-xs">
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">추출</dt>
<dd class={doc.extracted_at ? 'text-[var(--success)]' : 'text-[var(--text-dim)]'}>{doc.extracted_at ? '완료' : '대기'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">분류</dt>
<dd class={doc.ai_processed_at ? 'text-[var(--success)]' : 'text-[var(--text-dim)]'}>{doc.ai_processed_at ? '완료' : '대기'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">임베딩</dt>
<dd class={doc.embedded_at ? 'text-[var(--success)]' : 'text-[var(--text-dim)]'}>{doc.embedded_at ? '완료' : '대기'}</dd>
</div>
</dl>
</div>
<!-- 삭제 -->
<div class="pt-2 border-t border-[var(--border)]">
{#if deleteConfirm}
<div class="flex items-center gap-2">
<span class="text-xs text-[var(--error)]">정말 삭제?</span>
<button
onclick={deleteDoc}
disabled={deleting}
class="px-2 py-1 text-xs bg-[var(--error)] text-white rounded disabled:opacity-50"
>{deleting ? '삭제 중...' : '확인'}</button>
<button
onclick={() => deleteConfirm = false}
class="px-2 py-1 text-xs text-[var(--text-dim)]"
>취소</button>
</div>
{:else}
<button
onclick={() => deleteConfirm = true}
class="flex items-center gap-1 text-xs text-[var(--text-dim)] hover:text-[var(--error)]"
>
<Trash2 size={12} /> 문서 삭제
</button>
{/if}
</div>
</div>
</aside>

View File

@@ -0,0 +1,185 @@
<script>
import { page } from '$app/stores';
import { goto } from '$app/navigation';
import { api } from '$lib/api';
import { ChevronRight, ChevronDown, FolderOpen, Inbox, Clock, Mail, Scale } from 'lucide-svelte';
let tree = $state([]);
let loading = $state(true);
let expanded = $state({});
let activeDomain = $derived($page.url.searchParams.get('domain'));
const DOMAIN_COLORS = {
'Philosophy': 'var(--domain-philosophy)',
'Language': 'var(--domain-language)',
'Engineering': 'var(--domain-engineering)',
'Industrial_Safety': 'var(--domain-safety)',
'Programming': 'var(--domain-programming)',
'General': 'var(--domain-general)',
'Reference': 'var(--domain-reference)',
};
async function loadTree() {
loading = true;
try {
tree = await api('/documents/tree');
} catch (err) {
console.error('트리 로딩 실패:', err);
} finally {
loading = false;
}
}
function toggleExpand(path) {
expanded[path] = !expanded[path];
}
function navigate(path) {
const params = new URLSearchParams($page.url.searchParams);
params.delete('page');
if (path) {
params.set('domain', path);
} else {
params.delete('domain');
}
params.delete('sub_group');
for (const [key, val] of [...params.entries()]) {
if (!val) params.delete(key);
}
const qs = params.toString();
goto(`/documents${qs ? '?' + qs : ''}`, { noScroll: true });
}
$effect(() => { loadTree(); });
$effect(() => {
if (activeDomain) {
// 선택된 경로의 부모들 자동 펼치기
const parts = activeDomain.split('/');
let path = '';
for (const part of parts) {
path = path ? `${path}/${part}` : part;
expanded[path] = true;
}
}
});
let totalCount = $derived(tree.reduce((sum, n) => sum + n.count, 0));
</script>
<aside class="h-full flex flex-col bg-[var(--sidebar-bg)] border-r border-[var(--border)] overflow-y-auto">
<div class="px-4 py-3 border-b border-[var(--border)]">
<h2 class="text-sm font-semibold text-[var(--text-dim)] uppercase tracking-wider">분류</h2>
</div>
<!-- 전체 문서 -->
<div class="px-2 pt-2">
<button
onclick={() => navigate(null)}
class="w-full flex items-center justify-between px-3 py-2 rounded-md text-sm transition-colors
{!activeDomain ? 'bg-[var(--accent)]/15 text-[var(--accent)]' : 'text-[var(--text)] hover:bg-[var(--surface)]'}"
>
<span class="flex items-center gap-2">
<FolderOpen size={16} />
전체 문서
</span>
{#if totalCount > 0}
<span class="text-xs text-[var(--text-dim)]">{totalCount}</span>
{/if}
</button>
</div>
<!-- 트리 -->
<nav class="flex-1 px-2 py-2">
{#if loading}
{#each Array(5) as _}
<div class="h-8 bg-[var(--surface)] rounded-md animate-pulse mx-1 mb-1"></div>
{/each}
{:else}
{#each tree as node}
{@const color = DOMAIN_COLORS[node.name] || 'var(--text-dim)'}
{#snippet treeNode(n, depth)}
{@const isActive = activeDomain === n.path}
{@const isParent = activeDomain?.startsWith(n.path + '/')}
{@const hasChildren = n.children.length > 0}
{@const isExpanded = expanded[n.path]}
<div class="flex items-center" style="padding-left: {depth * 16}px">
{#if hasChildren}
<button
onclick={() => toggleExpand(n.path)}
class="p-0.5 rounded hover:bg-[var(--surface)] text-[var(--text-dim)]"
>
{#if isExpanded}
<ChevronDown size={14} />
{:else}
<ChevronRight size={14} />
{/if}
</button>
{:else}
<span class="w-5"></span>
{/if}
<button
onclick={() => navigate(n.path)}
class="flex-1 flex items-center justify-between px-2 py-1.5 rounded-md text-sm transition-colors
{isActive ? 'bg-[var(--accent)]/15 text-[var(--accent)]' : isParent ? 'text-[var(--text)]' : 'text-[var(--text-dim)] hover:bg-[var(--surface)] hover:text-[var(--text)]'}"
>
<span class="flex items-center gap-2">
{#if depth === 0}
<span class="w-2 h-2 rounded-full shrink-0" style="background: {color}"></span>
{/if}
<span class="truncate">{n.name}</span>
</span>
<span class="text-xs text-[var(--text-dim)] shrink-0 ml-2">{n.count}</span>
</button>
</div>
{#if hasChildren && isExpanded}
{#each n.children as child}
{@render treeNode(child, depth + 1)}
{/each}
{/if}
{/snippet}
{@render treeNode(node, 0)}
{/each}
{/if}
</nav>
<!-- 스마트 그룹 -->
<div class="px-2 py-2 border-t border-[var(--border)]">
<h3 class="px-3 py-1 text-[10px] font-semibold text-[var(--text-dim)] uppercase tracking-wider">스마트 그룹</h3>
<button
onclick={() => goto('/documents', { noScroll: true })}
class="w-full flex items-center gap-2 px-3 py-1.5 rounded-md text-sm text-[var(--text-dim)] hover:bg-[var(--surface)] hover:text-[var(--text)]"
>
<Clock size={14} /> 최근 7일
</button>
<button
onclick={() => { const p = new URLSearchParams(); p.set('source', 'law_monitor'); goto(`/documents?${p}`, { noScroll: true }); }}
class="w-full flex items-center gap-2 px-3 py-1.5 rounded-md text-sm text-[var(--text-dim)] hover:bg-[var(--surface)] hover:text-[var(--text)]"
>
<Scale size={14} /> 법령 알림
</button>
<button
onclick={() => { const p = new URLSearchParams(); p.set('source', 'email'); goto(`/documents?${p}`, { noScroll: true }); }}
class="w-full flex items-center gap-2 px-3 py-1.5 rounded-md text-sm text-[var(--text-dim)] hover:bg-[var(--surface)] hover:text-[var(--text)]"
>
<Mail size={14} /> 이메일
</button>
</div>
<!-- Inbox -->
<div class="px-2 py-2 border-t border-[var(--border)]">
<a
href="/inbox"
class="flex items-center justify-between px-3 py-2 rounded-md text-sm text-[var(--text)] hover:bg-[var(--surface)] transition-colors"
>
<span class="flex items-center gap-2">
<Inbox size={16} />
받은편지함
</span>
</a>
</div>
</aside>

View File

@@ -0,0 +1,40 @@
<script>
import { goto } from '$app/navigation';
import { page } from '$app/stores';
let { tag = '', clickable = true } = $props();
// 계층별 색상
function getColor(t) {
if (t.startsWith('@상태/') || t.startsWith('@')) return { bg: 'bg-amber-900/30', text: 'text-amber-400' };
if (t.startsWith('#주제/') || t.startsWith('#')) return { bg: 'bg-blue-900/30', text: 'text-blue-400' };
if (t.startsWith('$유형/') || t.startsWith('$')) return { bg: 'bg-emerald-900/30', text: 'text-emerald-400' };
if (t.startsWith('!우선순위/') || t.startsWith('!')) return { bg: 'bg-red-900/30', text: 'text-red-400' };
return { bg: 'bg-[var(--border)]', text: 'text-[var(--text-dim)]' };
}
function handleClick(e) {
if (!clickable) return;
e.preventDefault();
e.stopPropagation();
const params = new URLSearchParams($page.url.searchParams);
params.set('tag', tag);
params.delete('page');
goto(`/documents?${params}`, { noScroll: true });
}
let color = $derived(getColor(tag));
</script>
{#if clickable}
<button
onclick={handleClick}
class="inline-flex text-[10px] px-1.5 py-0.5 rounded {color.bg} {color.text} hover:opacity-80 transition-opacity"
>
{tag}
</button>
{:else}
<span class="inline-flex text-[10px] px-1.5 py-0.5 rounded {color.bg} {color.text}">
{tag}
</span>
{/if}

View File

@@ -0,0 +1,129 @@
<script>
import { onMount } from 'svelte';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { Upload } from 'lucide-svelte';
let { onupload = () => {} } = $props();
let dragging = $state(false);
let uploading = $state(false);
let uploadFiles = $state([]);
let dragCounter = 0;
onMount(() => {
function onDragEnter(e) {
e.preventDefault();
e.stopPropagation();
dragCounter++;
dragging = true;
}
function onDragOver(e) {
e.preventDefault();
e.stopPropagation();
}
function onDragLeave(e) {
e.preventDefault();
e.stopPropagation();
dragCounter--;
if (dragCounter <= 0) {
dragging = false;
dragCounter = 0;
}
}
function onDrop(e) {
e.preventDefault();
e.stopPropagation();
dragging = false;
dragCounter = 0;
handleFiles(e.dataTransfer?.files);
}
window.addEventListener('dragenter', onDragEnter);
window.addEventListener('dragover', onDragOver);
window.addEventListener('dragleave', onDragLeave);
window.addEventListener('drop', onDrop);
return () => {
window.removeEventListener('dragenter', onDragEnter);
window.removeEventListener('dragover', onDragOver);
window.removeEventListener('dragleave', onDragLeave);
window.removeEventListener('drop', onDrop);
};
});
async function handleFiles(fileList) {
const files = Array.from(fileList || []);
if (files.length === 0) return;
uploading = true;
uploadFiles = files.map(f => ({ name: f.name, status: 'pending' }));
let success = 0;
let failed = 0;
for (let i = 0; i < files.length; i++) {
uploadFiles[i].status = 'uploading';
uploadFiles = [...uploadFiles];
try {
const formData = new FormData();
formData.append('file', files[i]);
await api('/documents/', { method: 'POST', body: formData });
uploadFiles[i].status = 'done';
success++;
} catch (err) {
uploadFiles[i].status = 'failed';
failed++;
}
uploadFiles = [...uploadFiles];
}
if (success > 0) {
addToast('success', `${success}건 업로드 완료${failed > 0 ? `, ${failed}건 실패` : ''}`);
onupload();
} else {
addToast('error', `업로드 실패 (${failed}건)`);
}
setTimeout(() => {
uploading = false;
uploadFiles = [];
}, 3000);
}
</script>
<!-- 전체 페이지 드래그 오버레이 -->
{#if dragging}
<div class="fixed inset-0 z-50 bg-[var(--accent)]/10 border-2 border-dashed border-[var(--accent)] flex items-center justify-center">
<div class="bg-[var(--surface)] rounded-xl px-8 py-6 shadow-xl text-center">
<Upload size={32} class="mx-auto mb-2 text-[var(--accent)]" />
<p class="text-sm font-medium text-[var(--accent)]">여기에 파일을 놓으세요</p>
</div>
</div>
{/if}
<!-- 업로드 진행 상태 -->
{#if uploading && uploadFiles.length > 0}
<div class="mb-3 bg-[var(--surface)] border border-[var(--border)] rounded-lg p-3">
<p class="text-xs text-[var(--text-dim)] mb-2">업로드 중...</p>
<div class="space-y-1 max-h-32 overflow-y-auto">
{#each uploadFiles as f}
<div class="flex items-center justify-between text-xs">
<span class="truncate">{f.name}</span>
<span class={
f.status === 'done' ? 'text-[var(--success)]' :
f.status === 'failed' ? 'text-[var(--error)]' :
f.status === 'uploading' ? 'text-[var(--accent)]' :
'text-[var(--text-dim)]'
}>
{f.status === 'done' ? '✓' : f.status === 'failed' ? '✗' : f.status === 'uploading' ? '↑' : '…'}
</span>
</div>
{/each}
</div>
</div>
{/if}

View File

@@ -0,0 +1,55 @@
import { writable } from 'svelte/store';
import { api, setAccessToken } from '$lib/api';
interface User {
id: number;
username: string;
is_active: boolean;
totp_enabled: boolean;
last_login_at: string | null;
}
export const user = writable<User | null>(null);
export const isAuthenticated = writable(false);
export async function login(username: string, password: string, totp_code?: string) {
const data = await api<{ access_token: string }>('/auth/login', {
method: 'POST',
body: JSON.stringify({ username, password, totp_code: totp_code || undefined }),
});
setAccessToken(data.access_token);
await fetchUser();
}
export async function fetchUser() {
try {
const data = await api<User>('/auth/me');
user.set(data);
isAuthenticated.set(true);
} catch {
user.set(null);
isAuthenticated.set(false);
}
}
export async function logout() {
try {
await api('/auth/logout', { method: 'POST' });
} catch { /* ignore */ }
setAccessToken(null);
user.set(null);
isAuthenticated.set(false);
}
export async function tryRefresh() {
try {
const data = await api<{ access_token: string }>('/auth/refresh', {
method: 'POST',
});
setAccessToken(data.access_token);
await fetchUser();
return true;
} catch {
return false;
}
}

View File

@@ -0,0 +1,27 @@
import { writable } from 'svelte/store';
export const sidebarOpen = writable(true);
export const selectedDocId = writable<number | null>(null);
// Toast 시스템
interface Toast {
id: number;
type: 'success' | 'error' | 'warning' | 'info';
message: string;
}
let toastId = 0;
export const toasts = writable<Toast[]>([]);
export function addToast(type: Toast['type'], message: string, duration = 5000) {
const id = ++toastId;
toasts.update(t => [...t, { id, type, message }]);
if (duration > 0) {
setTimeout(() => removeToast(id), duration);
}
return id;
}
export function removeToast(id: number) {
toasts.update(t => t.filter(toast => toast.id !== id));
}

View File

@@ -0,0 +1,134 @@
<script>
import { onMount } from 'svelte';
import { browser } from '$app/environment';
import { page } from '$app/stores';
import { goto } from '$app/navigation';
import { isAuthenticated, user, tryRefresh, logout } from '$lib/stores/auth';
import { toasts, removeToast } from '$lib/stores/ui';
import Sidebar from '$lib/components/Sidebar.svelte';
import '../app.css';
const PUBLIC_PATHS = ['/login', '/setup'];
const NO_CHROME_PATHS = ['/login', '/setup'];
let authChecked = $state(false);
let sidebarOpen = $state(false);
onMount(async () => {
// localStorage에서 사이드바 상태 복원
const saved = localStorage.getItem('sidebarOpen');
if (saved === 'true') sidebarOpen = true;
if (!$isAuthenticated) {
await tryRefresh();
}
authChecked = true;
});
// 사이드바 상태 저장
$effect(() => {
if (browser) {
localStorage.setItem('sidebarOpen', String(sidebarOpen));
}
});
$effect(() => {
if (browser && authChecked && !$isAuthenticated && !PUBLIC_PATHS.some(p => $page.url.pathname.startsWith(p))) {
goto('/login');
}
});
let showChrome = $derived($isAuthenticated && !NO_CHROME_PATHS.some(p => $page.url.pathname.startsWith(p)));
function handleKeydown(e) {
if (e.key === '/' && !['INPUT', 'TEXTAREA'].includes(document.activeElement?.tagName)) {
e.preventDefault();
document.querySelector('[data-search-input]')?.focus();
}
if (e.key === 'Escape' && sidebarOpen) {
sidebarOpen = false;
}
}
</script>
<svelte:window on:keydown={handleKeydown} />
{#if !authChecked}
<div class="min-h-screen flex items-center justify-center">
<p class="text-[var(--text-dim)]">로딩 중...</p>
</div>
{:else if $isAuthenticated || PUBLIC_PATHS.some(p => $page.url.pathname.startsWith(p))}
{#if showChrome}
<div class="h-screen flex flex-col">
<!-- 상단 nav -->
<nav class="flex items-center justify-between px-4 py-2 border-b border-[var(--border)] bg-[var(--surface)] shrink-0">
<div class="flex items-center gap-3">
<button
onclick={() => sidebarOpen = !sidebarOpen}
class="p-1.5 rounded-md hover:bg-[var(--border)] text-[var(--text-dim)]"
aria-label="사이드바 토글"
>
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M3 12h18M3 6h18M3 18h18"/>
</svg>
</button>
<a href="/" class="text-sm font-semibold hover:text-[var(--accent)]">PKM</a>
<span class="text-[var(--text-dim)] text-xs">/</span>
<a href="/documents" class="text-xs hover:text-[var(--accent)]">문서</a>
</div>
<div class="flex items-center gap-3">
<a href="/news" class="text-xs text-[var(--text-dim)] hover:text-[var(--text)]">뉴스</a>
<a href="/inbox" class="text-xs text-[var(--text-dim)] hover:text-[var(--text)]">Inbox</a>
<a href="/settings" class="text-xs text-[var(--text-dim)] hover:text-[var(--text)]">설정</a>
<button
onclick={() => logout()}
class="text-xs text-[var(--text-dim)] hover:text-[var(--text)]"
>로그아웃</button>
</div>
</nav>
<!-- 메인 -->
<div class="flex-1 min-h-0 relative">
<!-- 사이드바 오버레이 (모든 화면에서 동일) -->
{#if sidebarOpen}
<div class="fixed inset-0 z-40">
<button
onclick={() => sidebarOpen = false}
class="absolute inset-0 bg-black/40"
aria-label="사이드바 닫기"
></button>
<div class="absolute left-0 top-0 bottom-0 z-50" style="width: var(--sidebar-w)">
<Sidebar />
</div>
</div>
{/if}
<!-- 콘텐츠 -->
<main class="h-full overflow-hidden">
<slot />
</main>
</div>
</div>
{:else}
<slot />
{/if}
{/if}
<!-- Toast -->
<div class="fixed top-4 right-4 z-50 flex flex-col gap-2 max-w-sm" role="status" aria-live="polite">
{#each $toasts as toast (toast.id)}
<button
class="px-4 py-3 rounded-lg shadow-lg text-sm flex items-center gap-2 cursor-pointer text-left"
class:bg-green-900={toast.type === 'success'}
class:bg-red-900={toast.type === 'error'}
class:bg-yellow-900={toast.type === 'warning'}
class:bg-blue-900={toast.type === 'info'}
class:text-green-200={toast.type === 'success'}
class:text-red-200={toast.type === 'error'}
class:text-yellow-200={toast.type === 'warning'}
class:text-blue-200={toast.type === 'info'}
onclick={() => removeToast(toast.id)}
>
{toast.message}
</button>
{/each}
</div>

View File

@@ -1,14 +1,89 @@
<script>
// TODO: Phase 4에서 대시보드 위젯 구현
import { onMount } from 'svelte';
import { goto } from '$app/navigation';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
let dashboard = null;
let loading = true;
onMount(async () => {
try {
dashboard = await api('/dashboard/');
} catch (err) {
addToast('error', '대시보드 로딩 실패');
} finally {
loading = false;
}
});
</script>
<h1>hyungi Document Server</h1>
<p>PKM 대시보드 — Phase 4에서 구현 예정</p>
<div class="p-4 lg:p-6">
<div class="max-w-6xl mx-auto">
<h2 class="text-xl font-bold mb-6">대시보드</h2>
<section>
<h2>시스템 상태</h2>
<ul>
<li>FastAPI: <a href="/api/health">헬스체크</a></li>
<li>API 문서: <a href="/docs">OpenAPI</a></li>
</ul>
</section>
{#if loading}
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
{#each Array(4) as _}
<div class="bg-[var(--surface)] rounded-xl p-5 border border-[var(--border)] animate-pulse h-28"></div>
{/each}
</div>
{:else if dashboard}
<!-- 위젯 그리드 -->
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4 mb-8">
<!-- 전체 문서 -->
<div class="bg-[var(--surface)] rounded-xl p-5 border border-[var(--border)]">
<p class="text-sm text-[var(--text-dim)]">전체 문서</p>
<p class="text-3xl font-bold mt-1">{dashboard.total_documents}</p>
<p class="text-xs text-[var(--text-dim)] mt-1">오늘 +{dashboard.today_added}</p>
</div>
<!-- Inbox -->
<div class="bg-[var(--surface)] rounded-xl p-5 border border-[var(--border)]">
<p class="text-sm text-[var(--text-dim)]">Inbox 미분류</p>
<p class="text-3xl font-bold mt-1" class:text-[var(--warning)]={dashboard.inbox_count > 0}>{dashboard.inbox_count}</p>
{#if dashboard.inbox_count > 0}
<a href="/inbox" class="text-xs text-[var(--accent)] hover:underline mt-1 inline-block">분류하기</a>
{/if}
</div>
<!-- 법령 알림 -->
<div class="bg-[var(--surface)] rounded-xl p-5 border border-[var(--border)]">
<p class="text-sm text-[var(--text-dim)]">법령 알림</p>
<p class="text-3xl font-bold mt-1">{dashboard.law_alerts}</p>
<p class="text-xs text-[var(--text-dim)] mt-1">오늘 변경</p>
</div>
<!-- 파이프라인 -->
<div class="bg-[var(--surface)] rounded-xl p-5 border border-[var(--border)]">
<p class="text-sm text-[var(--text-dim)]">파이프라인</p>
{#if dashboard.failed_count > 0}
<p class="text-3xl font-bold mt-1 text-[var(--error)]">{dashboard.failed_count} 실패</p>
{:else}
<p class="text-3xl font-bold mt-1 text-[var(--success)]">정상</p>
{/if}
</div>
</div>
<!-- 최근 문서 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h3 class="text-sm font-semibold text-[var(--text-dim)] mb-3">최근 문서</h3>
{#if dashboard.recent_documents.length > 0}
<div class="space-y-2">
{#each dashboard.recent_documents as doc}
<a href="/documents/{doc.id}" class="flex items-center justify-between py-2 px-3 rounded-lg hover:bg-[var(--bg)] transition-colors">
<div class="flex items-center gap-3">
<span class="text-xs px-2 py-0.5 rounded bg-[var(--border)] text-[var(--text-dim)] uppercase">{doc.file_format}</span>
<span class="text-sm truncate max-w-md">{doc.title || '제목 없음'}</span>
</div>
<span class="text-xs text-[var(--text-dim)]">{doc.ai_domain || ''}</span>
</a>
{/each}
</div>
{:else}
<p class="text-sm text-[var(--text-dim)]">문서가 없습니다</p>
{/if}
</div>
{/if}
</div>
</div>

View File

@@ -0,0 +1,305 @@
<script>
import { page } from '$app/stores';
import { goto } from '$app/navigation';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { Info } from 'lucide-svelte';
import { List, LayoutGrid } from 'lucide-svelte';
import DocumentCard from '$lib/components/DocumentCard.svelte';
import DocumentTable from '$lib/components/DocumentTable.svelte';
import PreviewPanel from '$lib/components/PreviewPanel.svelte';
import DocumentViewer from '$lib/components/DocumentViewer.svelte';
import UploadDropzone from '$lib/components/UploadDropzone.svelte';
// 뷰 모드 (localStorage 기억)
let viewMode = $state(typeof localStorage !== 'undefined' ? (localStorage.getItem('viewMode') || 'card') : 'card');
function toggleViewMode() {
viewMode = viewMode === 'card' ? 'table' : 'card';
if (typeof localStorage !== 'undefined') localStorage.setItem('viewMode', viewMode);
}
let documents = $state([]);
let total = $state(0);
let loading = $state(true);
let searchQuery = $state('');
let searchMode = $state('hybrid');
let searchResults = $state(null);
let selectedDoc = $state(null);
let infoPanelOpen = $state(false);
// URL params → filter
let currentPage = $derived(parseInt($page.url.searchParams.get('page') || '1'));
let filterDomain = $derived($page.url.searchParams.get('domain') || '');
let filterSubGroup = $derived($page.url.searchParams.get('sub_group') || '');
let filterTag = $derived($page.url.searchParams.get('tag') || '');
let filterSource = $derived($page.url.searchParams.get('source') || '');
$effect(() => {
const _p = currentPage;
const _d = filterDomain;
const _s = filterSubGroup;
const _t = filterTag;
const _src = filterSource;
const urlQ = $page.url.searchParams.get('q') || '';
const urlMode = $page.url.searchParams.get('mode') || 'hybrid';
searchQuery = urlQ;
searchMode = urlMode;
selectedDoc = null;
infoPanelOpen = false;
if (urlQ) {
doSearch(urlQ, urlMode);
} else {
loadDocuments();
}
});
async function loadDocuments() {
loading = true;
searchResults = null;
try {
const params = new URLSearchParams();
params.set('page', String(currentPage));
params.set('page_size', '20');
if (filterDomain) params.set('domain', filterDomain);
if (filterSubGroup) params.set('sub_group', filterSubGroup);
if (filterTag) params.set('tag', filterTag);
if (filterSource) params.set('source', filterSource);
const data = await api(`/documents/?${params}`);
documents = data.items;
total = data.total;
} catch (err) {
addToast('error', '문서 목록 로딩 실패');
} finally {
loading = false;
}
}
function submitSearch() {
const params = new URLSearchParams($page.url.searchParams);
params.delete('page');
if (searchQuery.trim()) {
params.set('q', searchQuery.trim());
} else {
params.delete('q');
}
if (searchMode !== 'hybrid') {
params.set('mode', searchMode);
} else {
params.delete('mode');
}
for (const [key, val] of [...params.entries()]) {
if (!val) params.delete(key);
}
const qs = params.toString();
goto(`/documents${qs ? '?' + qs : ''}`, { noScroll: true });
}
function handleSearchKeydown(e) {
if (e.key === 'Enter' && !e.isComposing) {
e.preventDefault();
submitSearch();
}
}
async function doSearch(q, mode) {
loading = true;
try {
const data = await api(`/search/?q=${encodeURIComponent(q)}&mode=${mode}&limit=50`);
searchResults = data.results;
total = data.total;
} catch (err) {
addToast('error', '검색 실패');
searchResults = [];
} finally {
loading = false;
}
}
function changePage(p) {
const params = new URLSearchParams($page.url.searchParams);
if (p > 1) {
params.set('page', String(p));
} else {
params.delete('page');
}
const qs = params.toString();
goto(`/documents${qs ? '?' + qs : ''}`, { noScroll: true });
}
function clearAllFilters() {
goto('/documents', { noScroll: true });
searchQuery = '';
}
function selectDoc(doc) {
selectedDoc = selectedDoc?.id === doc.id ? null : doc;
}
function handleKeydown(e) {
if (e.key === 'Escape' && infoPanelOpen) {
infoPanelOpen = false;
}
}
let totalPages = $derived(Math.ceil(total / 20));
let items = $derived(searchResults || documents);
let hasActiveFilters = $derived(!!filterDomain || !!filterSubGroup || !!filterTag || !!filterSource || !!searchQuery);
</script>
<svelte:window on:keydown={handleKeydown} />
<div class="flex flex-col h-full">
<!-- 상단 영역 -->
<div class="{selectedDoc ? 'h-[30%] shrink-0 border-b border-[var(--border)]' : 'flex-1'} flex flex-col min-h-0">
<!-- 업로드 드롭존 -->
<UploadDropzone onupload={loadDocuments} />
<!-- 검색바 + 정보 버튼 (고정) -->
<div class="flex gap-2 px-4 py-2 shrink-0">
<input
data-search-input
type="text"
bind:value={searchQuery}
onkeydown={handleSearchKeydown}
placeholder="검색어 입력 후 Enter (/ 키로 포커스)"
class="flex-1 px-3 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded-lg text-[var(--text)] text-sm focus:border-[var(--accent)] outline-none"
/>
<select
bind:value={searchMode}
onchange={() => { if (searchQuery) submitSearch(); }}
class="px-2 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded-lg text-[var(--text)] text-xs"
>
<option value="hybrid">하이브리드</option>
<option value="fts">전문검색</option>
<option value="trgm">부분매칭</option>
<option value="vector">의미검색</option>
</select>
<button
onclick={toggleViewMode}
class="p-1.5 rounded-lg border border-[var(--border)] hover:border-[var(--accent)] text-[var(--text-dim)] hover:text-[var(--accent)] transition-colors"
aria-label="뷰 모드 전환"
title={viewMode === 'card' ? '테이블 뷰' : '카드 뷰'}
>
{#if viewMode === 'card'}
<List size={16} />
{:else}
<LayoutGrid size={16} />
{/if}
</button>
{#if selectedDoc}
<button
onclick={() => infoPanelOpen = !infoPanelOpen}
class="p-1.5 rounded-lg border border-[var(--border)] hover:border-[var(--accent)] text-[var(--text-dim)] hover:text-[var(--accent)] transition-colors
{infoPanelOpen ? 'bg-[var(--accent)]/10 border-[var(--accent)] text-[var(--accent)]' : ''}"
aria-label="문서 정보"
title="문서 정보"
>
<Info size={16} />
</button>
{/if}
</div>
<!-- 스크롤 영역 (목록) -->
<div class="flex-1 overflow-y-auto px-4">
<!-- 결과 헤더 -->
{#if !loading}
<div class="flex items-center justify-between mb-2">
<div class="flex items-center gap-2">
<span class="text-[10px] text-[var(--text-dim)]">{total}</span>
{#if filterDomain}
<span class="text-[10px] text-[var(--accent)] bg-[var(--accent)]/10 px-1.5 py-0.5 rounded">
{filterDomain.replace('Knowledge/', '')}{filterSubGroup ? ` / ${filterSubGroup}` : ''}
</span>
{/if}
</div>
{#if hasActiveFilters}
<button
onclick={clearAllFilters}
class="text-[10px] text-[var(--text-dim)] hover:text-[var(--text)] px-1.5 py-0.5 rounded border border-[var(--border)]"
>
초기화
</button>
{/if}
</div>
{/if}
<!-- 결과 -->
{#if loading}
<div class="space-y-1.5">
{#each Array(5) as _}
<div class="bg-[var(--surface)] rounded-lg p-3 border border-[var(--border)] animate-pulse h-14"></div>
{/each}
</div>
{:else if items.length === 0}
<div class="text-center py-12 text-[var(--text-dim)]">
{#if searchQuery}
<p class="text-sm mb-2">'{searchQuery}'에 대한 결과가 없습니다</p>
<button onclick={clearAllFilters} class="text-xs text-[var(--accent)] hover:underline">필터 초기화</button>
{:else if hasActiveFilters}
<p class="text-sm mb-2">이 분류에 문서가 없습니다</p>
<button onclick={clearAllFilters} class="text-xs text-[var(--accent)] hover:underline">필터 초기화</button>
{:else}
<p class="text-sm">등록된 문서가 없습니다</p>
{/if}
</div>
{:else}
{#if viewMode === 'table'}
<DocumentTable
{items}
selectedId={selectedDoc?.id}
onselect={selectDoc}
/>
{:else}
<div class="space-y-1">
{#each items as doc}
<DocumentCard
{doc}
showDomain={!filterDomain}
selected={selectedDoc?.id === doc.id}
onselect={selectDoc}
/>
{/each}
</div>
{/if}
{#if !searchResults && totalPages > 1}
<div class="flex justify-center gap-1 mt-4">
{#each Array(totalPages) as _, i}
<button
onclick={() => changePage(i + 1)}
class="px-2.5 py-0.5 rounded text-xs transition-colors
{currentPage === i + 1 ? 'bg-[var(--accent)] text-white' : 'bg-[var(--surface)] text-[var(--text-dim)] hover:text-[var(--text)]'}"
>
{i + 1}
</button>
{/each}
</div>
{/if}
{/if}
</div>
</div>
<!-- 하단: 뷰어 (70%, 전체 너비) -->
{#if selectedDoc}
<div class="flex-1 min-h-0">
<DocumentViewer doc={selectedDoc} />
</div>
{/if}
</div>
<!-- 정보 패널: 우측 전체 높이 drawer -->
{#if infoPanelOpen && selectedDoc}
<div class="fixed inset-0 z-40">
<button
onclick={() => infoPanelOpen = false}
class="absolute inset-0 bg-black/40"
aria-label="정보 패널 닫기"
></button>
<div class="absolute right-0 top-0 bottom-0 z-50 w-[320px] shadow-xl">
<PreviewPanel doc={selectedDoc} onclose={() => infoPanelOpen = false} ondelete={() => { selectedDoc = null; infoPanelOpen = false; loadDocuments(); }} />
</div>
</div>
{/if}

View File

@@ -0,0 +1,194 @@
<script>
import { onMount } from 'svelte';
import { page } from '$app/stores';
import { api, getAccessToken } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { marked } from 'marked';
import DOMPurify from 'dompurify';
import TagPill from '$lib/components/TagPill.svelte';
marked.use({ mangle: false, headerIds: false });
function renderMd(text) {
return DOMPurify.sanitize(marked(text), {
USE_PROFILES: { html: true },
FORBID_TAGS: ['style', 'script'],
FORBID_ATTR: ['onerror', 'onclick'],
ALLOW_UNKNOWN_PROTOCOLS: false,
});
}
let doc = $state(null);
let loading = $state(true);
let error = $state(null); // 'not_found' | 'network' | null
let docId = $derived($page.params.id);
onMount(async () => {
try {
doc = await api(`/documents/${docId}`);
} catch (err) {
error = err?.status === 404 ? 'not_found' : 'network';
} finally {
loading = false;
}
});
let viewerType = $derived(doc ? (doc.source_channel === 'news' ? 'article' : getViewerType(doc.file_format)) : 'none');
function getViewerType(format) {
if (['md', 'txt', 'csv', 'html'].includes(format)) return 'markdown';
if (format === 'pdf') return 'pdf';
if (['hwp', 'hwpx'].includes(format)) return 'hwp-markdown';
if (['odoc', 'osheet'].includes(format)) return 'synology';
if (['jpg', 'jpeg', 'png', 'gif', 'bmp'].includes(format)) return 'image';
return 'unsupported';
}
</script>
<div class="p-4 lg:p-6">
<!-- breadcrumb -->
<div class="flex items-center gap-2 text-sm mb-4 text-[var(--text-dim)]">
<a href="/documents" class="hover:text-[var(--text)]">문서</a>
<span>/</span>
<span class="truncate max-w-md text-[var(--text)]">{doc?.title || '로딩...'}</span>
</div>
{#if loading}
<div class="max-w-6xl mx-auto p-6">
<div class="bg-[var(--surface)] rounded-xl p-6 border border-[var(--border)] animate-pulse h-96"></div>
</div>
{:else if error === 'not_found'}
<div class="text-center py-20 text-[var(--text-dim)]">
<p class="text-lg mb-2">문서를 찾을 수 없습니다</p>
<a href="/documents" class="text-sm text-[var(--accent)] hover:underline">목록으로 돌아가기</a>
</div>
{:else if error === 'network'}
<div class="text-center py-20 text-[var(--text-dim)]">
<p class="text-lg mb-2">문서를 불러올 수 없습니다</p>
<button onclick={() => location.reload()} class="text-sm text-[var(--accent)] hover:underline">다시 시도</button>
</div>
{:else if doc}
<div class="max-w-6xl mx-auto p-6 grid grid-cols-1 lg:grid-cols-3 gap-6">
<!-- 뷰어 (2/3) -->
<div class="lg:col-span-2 bg-[var(--surface)] rounded-xl border border-[var(--border)] p-6 min-h-[500px]">
{#if viewerType === 'markdown' || viewerType === 'hwp-markdown'}
<div class="prose prose-invert prose-sm max-w-none">
{@html renderMd(doc.extracted_text || '*텍스트 추출 대기 중*')}
</div>
{:else if viewerType === 'pdf'}
<iframe
src="/api/documents/{doc.id}/file?token={getAccessToken()}"
class="w-full h-[80vh] rounded"
title={doc.title}
></iframe>
{:else if viewerType === 'image'}
<img src="/api/documents/{doc.id}/file?token={getAccessToken()}" alt={doc.title} class="max-w-full rounded" />
{:else if viewerType === 'synology'}
<div class="text-center py-10">
<p class="text-[var(--text-dim)] mb-4">Synology Office 문서</p>
<a
href={doc.edit_url || 'https://link.hyungi.net'}
target="_blank"
class="px-4 py-2 bg-[var(--accent)] text-white rounded-lg hover:bg-[var(--accent-hover)]"
>
새 창에서 열기
</a>
</div>
{:else if viewerType === 'article'}
<!-- 뉴스 전용 뷰어 -->
<div>
<h1 class="text-xl font-bold mb-3">{doc.title}</h1>
<div class="flex items-center gap-2 mb-4 text-xs text-[var(--text-dim)]">
<span>출처: {doc.source_channel}</span>
<span>·</span>
<span>{new Date(doc.created_at).toLocaleDateString('ko-KR', { year: 'numeric', month: 'short', day: 'numeric' })}</span>
</div>
{#if doc.extracted_text}
<div class="markdown-body mb-6">
{@html renderMd(doc.extracted_text)}
</div>
{/if}
{#if doc.edit_url}
<a
href={doc.edit_url}
target="_blank"
rel="noopener noreferrer"
class="inline-flex items-center gap-2 px-4 py-2 bg-[var(--accent)] text-white rounded-lg hover:bg-[var(--accent-hover)]"
>원문 보기 →</a>
{/if}
</div>
{:else}
<div class="text-center py-10">
<p class="text-[var(--text-dim)] mb-2">이 문서 형식은 인앱 미리보기를 지원하지 않습니다</p>
<p class="text-xs text-[var(--text-dim)]">포맷: {doc.file_format}</p>
</div>
{/if}
</div>
<!-- 메타데이터 패널 (1/3) -->
<div class="space-y-4">
<!-- 기본 정보 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h3 class="text-sm font-semibold text-[var(--text-dim)] mb-3">문서 정보</h3>
<dl class="space-y-2 text-sm">
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">포맷</dt>
<dd class="uppercase">{doc.file_format}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">크기</dt>
<dd>{doc.file_size ? (doc.file_size / 1024).toFixed(1) + ' KB' : '-'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">도메인</dt>
<dd>{doc.ai_domain || '미분류'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">출처</dt>
<dd>{doc.source_channel || '-'}</dd>
</div>
</dl>
</div>
<!-- AI 요약 -->
{#if doc.ai_summary}
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h3 class="text-sm font-semibold text-[var(--text-dim)] mb-2">AI 요약</h3>
<div class="text-sm leading-relaxed markdown-body">{@html renderMd(doc.ai_summary)}</div>
</div>
{/if}
<!-- 태그 -->
{#if doc.ai_tags?.length > 0}
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h3 class="text-sm font-semibold text-[var(--text-dim)] mb-2">태그</h3>
<div class="flex flex-wrap gap-1.5">
{#each doc.ai_tags as tag}
<TagPill {tag} />
{/each}
</div>
</div>
{/if}
<!-- 가공 이력 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h3 class="text-sm font-semibold text-[var(--text-dim)] mb-3">가공 이력</h3>
<dl class="space-y-2 text-xs">
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">텍스트 추출</dt>
<dd>{doc.extracted_at ? new Date(doc.extracted_at).toLocaleDateString('ko') : '대기'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">AI 분류</dt>
<dd>{doc.ai_processed_at ? new Date(doc.ai_processed_at).toLocaleDateString('ko') : '대기'}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">벡터 임베딩</dt>
<dd>{doc.embedded_at ? new Date(doc.embedded_at).toLocaleDateString('ko') : '대기'}</dd>
</div>
</dl>
</div>
</div>
</div>
{/if}
</div>

View File

@@ -0,0 +1,190 @@
<script>
import { onMount } from 'svelte';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
let documents = [];
let loading = true;
let selected = new Set();
onMount(loadInbox);
async function loadInbox() {
loading = true;
try {
// Inbox 파일만 필터
const data = await api('/documents/?page_size=100');
documents = data.items.filter(d => d.review_status === 'pending');
} catch (err) {
addToast('error', 'Inbox 로딩 실패');
} finally {
loading = false;
}
}
function toggleSelect(id) {
if (selected.has(id)) selected.delete(id);
else selected.add(id);
selected = selected; // 반응성 트리거
}
function toggleAll() {
if (selected.size === documents.length) {
selected = new Set();
} else {
selected = new Set(documents.map(d => d.id));
}
}
let approving = false;
let showConfirm = false;
function startApprove() {
if (selected.size === 0) {
addToast('warning', '선택된 문서가 없습니다');
return;
}
showConfirm = true;
}
async function confirmApprove() {
showConfirm = false;
approving = true;
let success = 0;
const ids = [...selected];
for (const id of ids) {
try {
// AI 분류 결과 그대로 승인 (Inbox에서 이동은 classify_worker가 처리)
const doc = documents.find(d => d.id === id);
if (doc?.ai_domain) {
await api(`/documents/${id}`, {
method: 'PATCH',
body: JSON.stringify({ source_channel: 'inbox_route' }),
});
success++;
}
} catch { /* skip */ }
}
addToast('success', `${success}건 승인 완료`);
selected = new Set();
approving = false;
loadInbox();
}
async function updateDomain(id, domain) {
try {
await api(`/documents/${id}`, {
method: 'PATCH',
body: JSON.stringify({ ai_domain: domain }),
});
documents = documents.map(d => d.id === id ? { ...d, ai_domain: domain } : d);
addToast('success', '도메인 변경됨');
} catch {
addToast('error', '변경 실패');
}
}
const DOMAINS = [
'Knowledge/Philosophy',
'Knowledge/Language',
'Knowledge/Engineering',
'Knowledge/Industrial_Safety',
'Knowledge/Programming',
'Knowledge/General',
'Reference',
];
</script>
<div class="p-4 lg:p-6">
<div class="flex items-center justify-between mb-4">
<div class="flex items-center gap-3">
<h2 class="text-lg font-semibold">Inbox</h2>
<span class="text-xs px-2 py-0.5 rounded-full bg-[var(--warning)] text-black">{documents.length}</span>
</div>
<div class="flex items-center gap-2">
<button onclick={toggleAll} class="px-3 py-1.5 text-xs bg-[var(--surface)] border border-[var(--border)] rounded-lg">
{selected.size === documents.length ? '전체 해제' : '전체 선택'}
</button>
<button
onclick={startApprove}
disabled={approving || selected.size === 0}
class="px-4 py-1.5 text-xs bg-[var(--accent)] text-white rounded-lg disabled:opacity-50"
>
{approving ? '처리 중...' : `선택 승인 (${selected.size})`}
</button>
</div>
</div>
<div class="max-w-6xl mx-auto p-6">
{#if loading}
<div class="space-y-3">
{#each Array(3) as _}
<div class="bg-[var(--surface)] rounded-lg p-4 border border-[var(--border)] animate-pulse h-24"></div>
{/each}
</div>
{:else if documents.length === 0}
<div class="text-center py-20 text-[var(--text-dim)]">
<p class="text-lg">Inbox가 비어 있습니다</p>
<p class="text-sm mt-1">새 파일이 들어오면 자동으로 표시됩니다</p>
</div>
{:else}
<div class="space-y-3">
{#each documents as doc}
<div class="flex items-start gap-3 p-4 bg-[var(--surface)] border border-[var(--border)] rounded-lg" class:border-[var(--accent)]={selected.has(doc.id)}>
<input
type="checkbox"
checked={selected.has(doc.id)}
onchange={() => toggleSelect(doc.id)}
class="mt-1 accent-[var(--accent)]"
/>
<div class="flex-1 min-w-0">
<div class="flex items-center gap-2 mb-1">
<span class="text-xs px-1.5 py-0.5 rounded bg-[var(--border)] text-[var(--text-dim)] uppercase">{doc.file_format}</span>
<a href="/documents/{doc.id}" class="text-sm font-medium hover:text-[var(--accent)] truncate">{doc.title || '제목 없음'}</a>
</div>
{#if doc.ai_summary}
<p class="text-xs text-[var(--text-dim)] truncate">{doc.ai_summary.slice(0, 120)}</p>
{/if}
<div class="flex items-center gap-2 mt-2">
<span class="text-xs text-[var(--text-dim)]">AI 분류:</span>
<select
value={doc.ai_domain || ''}
onchange={(e) => updateDomain(doc.id, e.target.value)}
class="text-xs px-2 py-1 bg-[var(--bg)] border border-[var(--border)] rounded text-[var(--text)]"
>
<option value="">미분류</option>
{#each DOMAINS as d}
<option value={d}>{d.replace('Knowledge/', '')}</option>
{/each}
</select>
{#if doc.ai_tags?.length > 0}
<div class="flex gap-1 ml-2">
{#each doc.ai_tags.slice(0, 3) as tag}
<span class="text-xs px-1.5 py-0.5 bg-[var(--bg)] rounded text-[var(--accent)]">{tag}</span>
{/each}
</div>
{/if}
</div>
</div>
</div>
{/each}
</div>
{/if}
</div>
<!-- 확인 다이얼로그 -->
{#if showConfirm}
<div class="fixed inset-0 bg-black/60 flex items-center justify-center z-50">
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-6 max-w-sm w-full mx-4">
<h3 class="text-lg font-semibold mb-2">{selected.size}건을 승인합니다</h3>
<p class="text-sm text-[var(--text-dim)] mb-4">AI 분류 결과를 확정하고 Inbox에서 이동합니다.</p>
<div class="flex gap-2 justify-end">
<button onclick={() => showConfirm = false} class="px-4 py-2 text-sm bg-[var(--bg)] border border-[var(--border)] rounded-lg">취소</button>
<button onclick={confirmApprove} class="px-4 py-2 text-sm bg-[var(--accent)] text-white rounded-lg">승인</button>
</div>
</div>
</div>
{/if}
</div>

View File

@@ -0,0 +1,89 @@
<script>
import { goto } from '$app/navigation';
import { login } from '$lib/stores/auth';
import { addToast } from '$lib/stores/ui';
let username = '';
let password = '';
let totpCode = '';
let needsTotp = false;
let loading = false;
let error = '';
async function handleLogin() {
error = '';
loading = true;
try {
await login(username, password, needsTotp ? totpCode : undefined);
goto('/');
} catch (err) {
if (err.detail?.includes('TOTP')) {
needsTotp = true;
error = 'TOTP 코드를 입력하세요';
} else {
error = err.detail || '로그인 실패';
}
} finally {
loading = false;
}
}
</script>
<div class="min-h-screen flex items-center justify-center px-4">
<div class="w-full max-w-sm">
<h1 class="text-2xl font-bold mb-1">hyungi Document Server</h1>
<p class="text-[var(--text-dim)] text-sm mb-8">로그인</p>
<form onsubmit={(e) => { e.preventDefault(); handleLogin(); }} class="space-y-4">
<div>
<label for="username" class="block text-sm text-[var(--text-dim)] mb-1">아이디</label>
<input
id="username"
type="text"
bind:value={username}
class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] focus:border-[var(--accent)] outline-none"
autocomplete="username"
/>
</div>
<div>
<label for="password" class="block text-sm text-[var(--text-dim)] mb-1">비밀번호</label>
<input
id="password"
type="password"
bind:value={password}
class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] focus:border-[var(--accent)] outline-none"
autocomplete="current-password"
/>
</div>
{#if needsTotp}
<div>
<label for="totp" class="block text-sm text-[var(--text-dim)] mb-1">TOTP 코드</label>
<input
id="totp"
type="text"
bind:value={totpCode}
maxlength="6"
inputmode="numeric"
pattern="[0-9]*"
class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] focus:border-[var(--accent)] outline-none tracking-widest text-center text-lg"
autocomplete="one-time-code"
/>
</div>
{/if}
{#if error}
<p class="text-[var(--error)] text-sm">{error}</p>
{/if}
<button
type="submit"
disabled={loading}
class="w-full py-2.5 bg-[var(--accent)] hover:bg-[var(--accent-hover)] text-white rounded-lg font-medium disabled:opacity-50 transition-colors"
>
{loading ? '로그인 중...' : '로그인'}
</button>
</form>
</div>
</div>

View File

@@ -0,0 +1,198 @@
<script>
import { onMount } from 'svelte';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { marked } from 'marked';
import DOMPurify from 'dompurify';
function renderMd(text) {
return DOMPurify.sanitize(marked(text), {
USE_PROFILES: { html: true },
FORBID_TAGS: ['style', 'script'],
FORBID_ATTR: ['onerror', 'onclick'],
ALLOW_UNKNOWN_PROTOCOLS: false,
});
}
let articles = $state([]);
let total = $state(0);
let loading = $state(true);
let selectedArticle = $state(null);
let filterSource = $state('');
let showUnreadOnly = $state(false);
let sources = $state([]);
let currentPage = $state(1);
onMount(async () => {
try {
const srcData = await api('/news/sources');
// 신문사별 유니크
const names = new Set();
srcData.forEach(s => names.add(s.name.split(' ')[0]));
sources = [...names];
} catch (e) {}
loadArticles();
});
async function loadArticles() {
loading = true;
try {
const params = new URLSearchParams();
params.set('page', String(currentPage));
params.set('page_size', '30');
if (filterSource) params.set('source', filterSource);
if (showUnreadOnly) params.set('unread_only', 'true');
const data = await api(`/news/articles?${params}`);
articles = data.items;
total = data.total;
} catch (err) {
addToast('error', '뉴스 로딩 실패');
} finally {
loading = false;
}
}
function selectArticle(article) {
selectedArticle = article;
if (!article.is_read) markRead(article);
}
async function markRead(article) {
try {
await api(`/documents/${article.id}`, {
method: 'PATCH',
body: JSON.stringify({ is_read: true }),
});
article.is_read = true;
articles = [...articles];
} catch (e) {}
}
async function markAllRead() {
try {
const result = await api('/news/mark-all-read', { method: 'POST' });
addToast('success', `${result.marked}건 읽음 처리`);
articles = articles.map(a => ({ ...a, is_read: true }));
} catch (e) {
addToast('error', '실패');
}
}
function timeAgo(dateStr) {
const diff = Date.now() - new Date(dateStr).getTime();
const mins = Math.floor(diff / 60000);
if (mins < 60) return `${mins}분 전`;
const hours = Math.floor(mins / 60);
if (hours < 24) return `${hours}시간 전`;
const days = Math.floor(hours / 24);
return `${days}일 전`;
}
$effect(() => {
const _s = filterSource;
const _u = showUnreadOnly;
currentPage = 1;
loadArticles();
});
</script>
<div class="flex h-full">
<!-- 좌측 필터 -->
<div class="w-48 shrink-0 border-r border-[var(--border)] bg-[var(--sidebar-bg)] p-3 overflow-y-auto">
<h2 class="text-xs font-semibold text-[var(--text-dim)] uppercase mb-3">필터</h2>
<button
onclick={() => { filterSource = ''; }}
class="w-full text-left px-2 py-1.5 rounded text-sm mb-1 {filterSource === '' ? 'bg-[var(--accent)]/15 text-[var(--accent)]' : 'text-[var(--text-dim)] hover:bg-[var(--surface)]'}"
>📰 전체</button>
{#each sources as src}
<button
onclick={() => { filterSource = src; }}
class="w-full text-left px-2 py-1.5 rounded text-sm mb-0.5 {filterSource === src ? 'bg-[var(--accent)]/15 text-[var(--accent)]' : 'text-[var(--text-dim)] hover:bg-[var(--surface)]'}"
>{src}</button>
{/each}
<hr class="my-3 border-[var(--border)]">
<label class="flex items-center gap-2 px-2 text-xs text-[var(--text-dim)]">
<input type="checkbox" bind:checked={showUnreadOnly} class="rounded">
읽지 않음만
</label>
</div>
<!-- 메인 -->
<div class="flex-1 flex flex-col min-h-0">
<!-- 상단 바 -->
<div class="flex items-center justify-between px-4 py-2 border-b border-[var(--border)] shrink-0">
<span class="text-xs text-[var(--text-dim)]">{total}</span>
<button
onclick={markAllRead}
class="text-xs text-[var(--text-dim)] hover:text-[var(--accent)] px-2 py-1 rounded border border-[var(--border)]"
>전체 읽음</button>
</div>
<!-- 상단: 기사 리스트 -->
<div class="overflow-y-auto {selectedArticle ? 'h-[40%] shrink-0 border-b border-[var(--border)]' : 'flex-1'}">
{#if loading}
<div class="p-4 space-y-2">
{#each Array(5) as _}
<div class="h-12 bg-[var(--surface)] rounded animate-pulse"></div>
{/each}
</div>
{:else if articles.length === 0}
<div class="text-center py-16 text-[var(--text-dim)]">
<p class="text-sm">뉴스가 없습니다</p>
</div>
{:else}
{#each articles as article}
<button
onclick={() => selectArticle(article)}
class="w-full text-left px-4 py-2.5 border-b border-[var(--border)]/30 hover:bg-[var(--surface)] transition-colors
{selectedArticle?.id === article.id ? 'bg-[var(--accent)]/5 border-l-2 border-l-[var(--accent)]' : ''}"
>
<div class="flex items-start gap-2">
<span class="mt-1 text-[10px] {article.is_read ? 'text-[var(--text-dim)]' : 'text-[var(--accent)]'}">
{article.is_read ? '○' : '●'}
</span>
<div class="min-w-0">
<p class="text-sm truncate {article.is_read ? 'text-[var(--text-dim)]' : 'font-medium'}">{article.title}</p>
<div class="flex items-center gap-2 mt-0.5 text-[10px] text-[var(--text-dim)]">
<span>{article.ai_sub_group || ''}</span>
{#if article.ai_tags?.length}
<span>{article.ai_tags[0]?.split('/').pop()}</span>
{/if}
<span>{timeAgo(article.created_at)}</span>
</div>
</div>
</div>
</button>
{/each}
{/if}
</div>
<!-- 하단: 기사 미리보기 -->
{#if selectedArticle}
<div class="flex-1 overflow-y-auto p-5 bg-[var(--surface)]">
<h2 class="text-lg font-bold mb-2">{selectedArticle.title}</h2>
<div class="flex items-center gap-2 mb-4 text-xs text-[var(--text-dim)]">
<span>{selectedArticle.ai_sub_group}</span>
<span>·</span>
<span>{timeAgo(selectedArticle.created_at)}</span>
</div>
<div class="markdown-body mb-4">
{@html renderMd(selectedArticle.extracted_text || '')}
</div>
{#if selectedArticle.edit_url}
<a
href={selectedArticle.edit_url}
target="_blank"
rel="noopener noreferrer"
class="inline-flex items-center gap-2 px-4 py-2 bg-[var(--accent)] text-white rounded-lg hover:bg-[var(--accent-hover)] text-sm"
>원문 보기 →</a>
{/if}
</div>
{/if}
</div>
</div>

View File

@@ -0,0 +1,213 @@
<script>
import { onMount } from 'svelte';
import { api } from '$lib/api';
import { addToast } from '$lib/stores/ui';
import { user } from '$lib/stores/auth';
let currentPassword = '';
let newPassword = '';
let confirmPassword = '';
let changing = false;
// 뉴스 소스
let sources = $state([]);
let loadingSources = $state(true);
let collecting = $state(false);
let newSource = $state({ name: '', feed_url: '', category: '', language: 'ko', feed_type: 'rss', country: '' });
let showAddForm = $state(false);
onMount(async () => {
try {
sources = await api('/news/sources');
} catch (e) {}
loadingSources = false;
});
async function changePassword() {
if (newPassword !== confirmPassword) {
addToast('error', '새 비밀번호가 일치하지 않습니다');
return;
}
if (newPassword.length < 8) {
addToast('error', '비밀번호는 8자 이상이어야 합니다');
return;
}
changing = true;
try {
await api('/auth/change-password', {
method: 'POST',
body: JSON.stringify({ current_password: currentPassword, new_password: newPassword }),
});
addToast('success', '비밀번호가 변경되었습니다');
currentPassword = ''; newPassword = ''; confirmPassword = '';
} catch (err) {
addToast('error', err.detail || '비밀번호 변경 실패');
} finally { changing = false; }
}
async function toggleSource(source) {
try {
await api(`/news/sources/${source.id}`, {
method: 'PATCH',
body: JSON.stringify({ enabled: !source.enabled }),
});
source.enabled = !source.enabled;
sources = [...sources];
} catch (e) {
addToast('error', '변경 실패');
}
}
async function deleteSource(id) {
try {
await api(`/news/sources/${id}`, { method: 'DELETE' });
sources = sources.filter(s => s.id !== id);
addToast('success', '삭제됨');
} catch (e) {
addToast('error', '삭제 실패');
}
}
async function addSource() {
try {
const created = await api('/news/sources', {
method: 'POST',
body: JSON.stringify(newSource),
});
sources = [...sources, created];
showAddForm = false;
newSource = { name: '', feed_url: '', category: '', language: 'ko', feed_type: 'rss', country: '' };
addToast('success', '소스 추가됨');
} catch (e) {
addToast('error', '추가 실패');
}
}
async function collectNow() {
collecting = true;
try {
await api('/news/collect', { method: 'POST' });
addToast('success', '뉴스 수집 시작됨');
} catch (e) {
addToast('error', '수집 실패');
} finally {
setTimeout(() => collecting = false, 5000);
}
}
</script>
<div class="p-4 lg:p-6">
<div class="max-w-2xl mx-auto space-y-6">
<!-- 계정 정보 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h2 class="text-lg font-semibold mb-3">계정 정보</h2>
<dl class="space-y-2 text-sm">
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">아이디</dt>
<dd>{$user?.username}</dd>
</div>
<div class="flex justify-between">
<dt class="text-[var(--text-dim)]">2FA (TOTP)</dt>
<dd class={$user?.totp_enabled ? 'text-[var(--success)]' : 'text-[var(--text-dim)]'}>
{$user?.totp_enabled ? '활성' : '비활성'}
</dd>
</div>
</dl>
</div>
<!-- 비밀번호 변경 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<h2 class="text-lg font-semibold mb-3">비밀번호 변경</h2>
<form onsubmit={(e) => { e.preventDefault(); changePassword(); }} class="space-y-3">
<div>
<label for="pw-current" class="block text-sm text-[var(--text-dim)] mb-1">현재 비밀번호</label>
<input id="pw-current" type="password" bind:value={currentPassword} class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] outline-none focus:border-[var(--accent)]" />
</div>
<div>
<label for="pw-new" class="block text-sm text-[var(--text-dim)] mb-1">새 비밀번호</label>
<input id="pw-new" type="password" bind:value={newPassword} class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] outline-none focus:border-[var(--accent)]" />
</div>
<div>
<label for="pw-confirm" class="block text-sm text-[var(--text-dim)] mb-1">새 비밀번호 확인</label>
<input id="pw-confirm" type="password" bind:value={confirmPassword} class="w-full px-3 py-2 bg-[var(--bg)] border border-[var(--border)] rounded-lg text-[var(--text)] outline-none focus:border-[var(--accent)]" />
</div>
<button type="submit" disabled={changing} class="w-full py-2.5 bg-[var(--accent)] text-white rounded-lg disabled:opacity-50">
{changing ? '변경 중...' : '비밀번호 변경'}
</button>
</form>
</div>
<!-- 뉴스 소스 관리 -->
<div class="bg-[var(--surface)] rounded-xl border border-[var(--border)] p-5">
<div class="flex items-center justify-between mb-4">
<h2 class="text-lg font-semibold">뉴스 소스</h2>
<div class="flex gap-2">
<button
onclick={collectNow}
disabled={collecting}
class="px-3 py-1.5 text-xs bg-[var(--accent)] text-white rounded-lg disabled:opacity-50"
>{collecting ? '수집 중...' : '지금 수집'}</button>
<button
onclick={() => showAddForm = !showAddForm}
class="px-3 py-1.5 text-xs border border-[var(--border)] rounded-lg text-[var(--text-dim)] hover:text-[var(--text)]"
>{showAddForm ? '취소' : '+ 추가'}</button>
</div>
</div>
<!-- 추가 폼 -->
{#if showAddForm}
<form onsubmit={(e) => { e.preventDefault(); addSource(); }} class="mb-4 p-3 bg-[var(--bg)] rounded-lg space-y-2">
<div class="grid grid-cols-2 gap-2">
<input bind:value={newSource.name} placeholder="이름" class="px-2 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded text-sm text-[var(--text)]" />
<input bind:value={newSource.feed_url} placeholder="RSS URL" class="px-2 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded text-sm text-[var(--text)]" />
<input bind:value={newSource.category} placeholder="분야" class="px-2 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded text-sm text-[var(--text)]" />
<select bind:value={newSource.language} class="px-2 py-1.5 bg-[var(--surface)] border border-[var(--border)] rounded text-sm text-[var(--text)]">
<option value="ko">한국어</option>
<option value="en">English</option>
<option value="ja">日本語</option>
<option value="fr">Français</option>
<option value="zh">中文</option>
<option value="de">Deutsch</option>
</select>
</div>
<button type="submit" class="px-3 py-1.5 text-xs bg-[var(--accent)] text-white rounded">추가</button>
</form>
{/if}
<!-- 소스 목록 -->
{#if loadingSources}
<div class="text-sm text-[var(--text-dim)]">로딩 중...</div>
{:else if sources.length === 0}
<div class="text-sm text-[var(--text-dim)]">등록된 소스가 없습니다</div>
{:else}
<div class="space-y-1">
{#each sources as source}
<div class="flex items-center justify-between p-2 rounded-lg hover:bg-[var(--bg)] text-sm">
<div class="flex items-center gap-2 min-w-0">
<button
onclick={() => toggleSource(source)}
class="w-8 h-4 rounded-full transition-colors {source.enabled ? 'bg-[var(--accent)]' : 'bg-[var(--border)]'}"
>
<div class="w-3 h-3 rounded-full bg-white transition-transform {source.enabled ? 'translate-x-4' : 'translate-x-0.5'}"></div>
</button>
<span class="truncate {source.enabled ? '' : 'text-[var(--text-dim)]'}">{source.name}</span>
<span class="text-[10px] text-[var(--text-dim)]">{source.category}</span>
</div>
<div class="flex items-center gap-2 shrink-0">
{#if source.last_fetched_at}
<span class="text-[10px] text-[var(--text-dim)]">
{new Date(source.last_fetched_at).toLocaleDateString('ko-KR', { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' })}
</span>
{/if}
<button
onclick={() => deleteSource(source.id)}
class="text-[10px] text-[var(--text-dim)] hover:text-[var(--error)]"
>삭제</button>
</div>
</div>
{/each}
</div>
{/if}
</div>
</div>
</div>

7
frontend/vite.config.js Normal file
View File

@@ -0,0 +1,7 @@
import { sveltekit } from '@sveltejs/kit/vite';
import tailwindcss from '@tailwindcss/vite';
import { defineConfig } from 'vite';
export default defineConfig({
plugins: [tailwindcss(), sveltekit()],
});

View File

@@ -1,4 +1,7 @@
version: '3.8'
# ═══════════════════════════════════════════════════
# 이 파일은 더 이상 사용하지 않음.
# 루트 docker-compose.yml로 통합됨 (2026-04-03).
# ═══════════════════════════════════════════════════
services:
ollama:

View File

@@ -4,6 +4,17 @@
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
-- users 테이블 (단일 관리자)
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
totp_secret VARCHAR(64),
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ DEFAULT NOW(),
last_login_at TIMESTAMPTZ
);
-- ENUM 타입
CREATE TYPE doc_type AS ENUM ('immutable', 'editable', 'note');
CREATE TYPE source_channel AS ENUM (

View File

@@ -0,0 +1,11 @@
-- 벡터 유사도 인덱스 (코사인 거리)
-- 주의: lists 값은 문서 수에 따라 조정 필요
-- 문서 수 < 1,000: 인덱스 불필요 (seq scan이 더 빠름)
-- 문서 수 1,000~10,000: lists = 문서수 / 50
-- 문서 수 10,000+: lists = 문서수 / 100
-- 초기 마이그레이션 후 문서 수 확인하여 lists 값 조정할 것
-- 최초 실행 시 lists=50으로 시작 (500~2,500건 최적)
CREATE INDEX IF NOT EXISTS idx_documents_embedding
ON documents USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 50);

View File

@@ -0,0 +1,8 @@
-- 자동화 워커 상태 저장 (증분 동기화용)
CREATE TABLE automation_state (
id BIGSERIAL PRIMARY KEY,
job_name VARCHAR(50) NOT NULL UNIQUE,
last_check_value TEXT,
last_run_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ DEFAULT NOW()
);

View File

@@ -0,0 +1,2 @@
-- 사용자 메모 컬럼 추가
ALTER TABLE documents ADD COLUMN IF NOT EXISTS user_note TEXT;

View File

@@ -0,0 +1,4 @@
-- 문서 미리보기 상태 필드 추가
ALTER TABLE documents ADD COLUMN IF NOT EXISTS preview_status VARCHAR(20) DEFAULT 'none';
ALTER TABLE documents ADD COLUMN IF NOT EXISTS preview_hash VARCHAR(64);
ALTER TABLE documents ADD COLUMN IF NOT EXISTS preview_at TIMESTAMPTZ;

View File

@@ -0,0 +1,2 @@
-- 외부 편집 URL (Synology Drive 공유 링크 등)
ALTER TABLE documents ADD COLUMN IF NOT EXISTS edit_url TEXT;

View File

@@ -0,0 +1,5 @@
-- 원본/변환 분리 필드 추가
ALTER TABLE documents ADD COLUMN IF NOT EXISTS original_path TEXT;
ALTER TABLE documents ADD COLUMN IF NOT EXISTS original_format VARCHAR(20);
ALTER TABLE documents ADD COLUMN IF NOT EXISTS original_hash VARCHAR(64);
ALTER TABLE documents ADD COLUMN IF NOT EXISTS conversion_status VARCHAR(20) DEFAULT 'none';

View File

@@ -0,0 +1,4 @@
-- 분류 체계 확장 필드
ALTER TABLE documents ADD COLUMN IF NOT EXISTS document_type VARCHAR(50);
ALTER TABLE documents ADD COLUMN IF NOT EXISTS importance VARCHAR(20) DEFAULT 'medium';
ALTER TABLE documents ADD COLUMN IF NOT EXISTS ai_confidence FLOAT;

View File

@@ -0,0 +1,8 @@
-- Inbox 승인 상태 분리 + derived_path
ALTER TABLE documents ADD COLUMN IF NOT EXISTS review_status VARCHAR(20) DEFAULT 'pending';
ALTER TABLE documents ADD COLUMN IF NOT EXISTS derived_path TEXT;
-- 기존 문서는 전부 approved (마이그레이션 이후 신규만 pending)
UPDATE documents SET review_status = 'approved';
CREATE INDEX IF NOT EXISTS idx_documents_review_status ON documents(review_status);

View File

@@ -0,0 +1,7 @@
-- Soft-delete 지원
ALTER TABLE documents ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_documents_not_deleted ON documents(deleted_at) WHERE deleted_at IS NULL;
-- active documents 뷰 (raw SQL 누락 방지)
CREATE OR REPLACE VIEW active_documents AS SELECT * FROM documents WHERE deleted_at IS NULL;

View File

@@ -0,0 +1,7 @@
-- 벡터 차원 변경: 768 → 1024 (nomic-embed-text → bge-m3)
-- 기존 임베딩 데이터 초기화 (재생성 필요)
ALTER TABLE documents DROP COLUMN IF EXISTS embedding;
ALTER TABLE documents ADD COLUMN embedding vector(1024);
-- 기존 임베딩 메타 초기화
UPDATE documents SET embedded_at = NULL, embed_model_version = NULL;

View File

@@ -0,0 +1,16 @@
-- 뉴스 소스 관리 테이블
CREATE TABLE IF NOT EXISTS news_sources (
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL,
country VARCHAR(10),
feed_url TEXT NOT NULL,
feed_type VARCHAR(20) DEFAULT 'rss',
category VARCHAR(50),
language VARCHAR(10),
enabled BOOLEAN DEFAULT true,
last_fetched_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- source_channel enum에 'news' 추가
ALTER TYPE source_channel ADD VALUE IF NOT EXISTS 'news';

View File

@@ -0,0 +1,9 @@
-- 읽음 상태
ALTER TABLE documents ADD COLUMN IF NOT EXISTS is_read BOOLEAN DEFAULT false;
-- 뉴스 인덱스
CREATE INDEX IF NOT EXISTS idx_documents_is_read ON documents(is_read) WHERE source_channel = 'news';
CREATE INDEX IF NOT EXISTS idx_news_feed ON documents(source_channel, created_at DESC);
-- 기존 뉴스 재분류 (taxonomy에서 분리)
UPDATE documents SET ai_domain = 'News', ai_sub_group = '' WHERE source_channel = 'news';

View File

@@ -0,0 +1,235 @@
"""DEVONthink → NAS PKM 마이그레이션 스크립트
.dtBase2 번들의 Files.noindex/ 디렉토리에서 직접 파일을 추출하여
NAS PKM 폴더 구조로 복사하고 DB에 등록합니다.
사용법:
# Dry-run (실제 복사/DB 등록 없이 시뮬레이션)
python scripts/migrate_from_devonthink.py --source-dir ~/Documents/Databases --dry-run
# 실제 실행
python scripts/migrate_from_devonthink.py \
--source-dir ~/Documents/Databases \
--target-dir /mnt/nas/Document_Server \
--database-url postgresql+asyncpg://pkm:PASSWORD@localhost:15432/pkm
"""
import argparse
import asyncio
import os
import shutil
import sys
from pathlib import Path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from core.utils import file_hash, setup_logger
logger = setup_logger("migrate")
# ─── DEVONthink DB → NAS PKM 폴더 매핑 ───
FOLDER_MAPPING = {
"00_Inbox DB": "PKM/Inbox",
"Inbox": "PKM/Inbox",
"00_Note_BOX": "PKM/Knowledge",
"01_Philosophie": "PKM/Knowledge/Philosophy",
"02_Language": "PKM/Knowledge/Language",
"03_Engineering": "PKM/Knowledge/Engineering",
"04_Industrial safety": "PKM/Knowledge/Industrial_Safety",
"05_Programming": "PKM/Knowledge/Programming",
"07_General Book": "PKM/Knowledge/General",
"97_Production drawing": "PKM/References",
"99_Reference Data": "PKM/References",
"99_Home File": "PKM/References",
"Archive": "PKM/Archive",
"Projects": "PKM/Knowledge",
"99_Technicalkorea": "Technicalkorea",
# 스킵 대상
"98_명일방주 엔드필드": None,
}
# 무시할 파일
SKIP_NAMES = {".DS_Store", "Thumbs.db", "desktop.ini", "Icon\r", "Icon"}
SKIP_EXTENSIONS = {".dtMeta", ".dtBase2", ".sparseimage", ".dtStore", ".dtCloud"}
def should_skip(path: Path) -> bool:
if path.name in SKIP_NAMES or path.name.startswith("._"):
return True
if path.suffix.lower() in SKIP_EXTENSIONS:
return True
if path.stat().st_size == 0:
return True
return False
async def migrate(
source_dir: str,
target_dir: str,
database_url: str,
dry_run: bool = False,
batch_size: int = 100,
):
"""마이그레이션 실행"""
source = Path(source_dir)
target = Path(target_dir)
if not source.exists():
logger.error(f"소스 디렉토리 없음: {source}")
return
engine = create_async_engine(database_url)
async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
stats = {"total": 0, "copied": 0, "skipped": 0, "duplicates": 0, "errors": 0}
batch = []
# .dtBase2 번들 탐색
for dtbase in sorted(source.glob("*.dtBase2")):
db_name = dtbase.stem # "04_Industrial safety"
target_prefix = FOLDER_MAPPING.get(db_name)
if target_prefix is None:
logger.info(f"[스킵] {db_name} (매핑: None)")
continue
files_dir = dtbase / "Files.noindex"
if not files_dir.exists():
logger.warning(f"[스킵] {db_name}: Files.noindex 없음")
continue
logger.info(f"[DB] {db_name}{target_prefix}")
# Files.noindex 하위의 모든 파일 (format/hash/filename.ext 구조)
files = [f for f in files_dir.rglob("*") if f.is_file() and not should_skip(f)]
for source_file in files:
stats["total"] += 1
# 대상 경로: PKM/{domain}/{파일명}
dest_rel = f"{target_prefix}/{source_file.name}"
dest_path = target / dest_rel
if dry_run:
logger.info(f"[DRY-RUN] {source_file.name}{dest_rel}")
stats["copied"] += 1
continue
try:
# 파일 복사
dest_path.parent.mkdir(parents=True, exist_ok=True)
# 중복 파일명 처리
counter = 1
stem, suffix = dest_path.stem, dest_path.suffix
while dest_path.exists():
dest_path = dest_path.parent / f"{stem}_{counter}{suffix}"
dest_rel = str(dest_path.relative_to(target))
counter += 1
shutil.copy2(source_file, dest_path)
ext = source_file.suffix.lstrip(".").lower() or "unknown"
fhash = file_hash(dest_path)
fsize = dest_path.stat().st_size
batch.append({
"file_path": dest_rel,
"file_hash": fhash,
"file_format": ext,
"file_size": fsize,
"file_type": "immutable",
"import_source": f"devonthink:{db_name}",
"title": source_file.stem,
"source_channel": "manual",
})
stats["copied"] += 1
except Exception as e:
logger.error(f"[오류] {source_file}: {e}")
stats["errors"] += 1
# 배치 커밋
if len(batch) >= batch_size:
dups = await _insert_batch(async_session_factory, batch)
stats["duplicates"] += dups
batch.clear()
logger.info(f" 진행: {stats['copied']}건 처리됨")
# 남은 배치 처리
if batch and not dry_run:
dups = await _insert_batch(async_session_factory, batch)
stats["duplicates"] += dups
await engine.dispose()
logger.info("=" * 50)
logger.info(f"마이그레이션 {'시뮬레이션' if dry_run else '완료'}")
logger.info(f" 전체 파일: {stats['total']}")
logger.info(f" 복사/등록: {stats['copied']}")
logger.info(f" 스킵: {stats['skipped']}")
logger.info(f" 중복: {stats['duplicates']}")
logger.info(f" 오류: {stats['errors']}")
async def _insert_batch(async_session_factory, batch: list[dict]) -> int:
"""배치 단위로 documents + processing_queue 삽입"""
duplicates = 0
async with async_session_factory() as session:
for item in batch:
try:
result = await session.execute(
text("""
INSERT INTO documents (file_path, file_hash, file_format, file_size,
file_type, import_source, title, source_channel)
VALUES (:file_path, :file_hash, :file_format, :file_size,
:file_type, :import_source, :title, :source_channel)
ON CONFLICT (file_path) DO NOTHING
RETURNING id
"""),
item,
)
row = result.fetchone()
if row is None:
duplicates += 1
continue
doc_id = row[0]
await session.execute(
text("""
INSERT INTO processing_queue (document_id, stage, status)
VALUES (:doc_id, 'extract', 'pending')
ON CONFLICT DO NOTHING
"""),
{"doc_id": doc_id},
)
except Exception as e:
logger.error(f"등록 실패: {item['file_path']}: {e}")
await session.commit()
return duplicates
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="DEVONthink → NAS PKM 마이그레이션")
parser.add_argument("--source-dir", required=True, help="DEVONthink Databases 디렉토리")
parser.add_argument("--target-dir", default="/mnt/nas/Document_Server", help="NAS 루트 경로")
parser.add_argument(
"--database-url",
default="postgresql+asyncpg://pkm:pkm@localhost:15432/pkm",
help="PostgreSQL 연결 URL",
)
parser.add_argument("--dry-run", action="store_true", help="시뮬레이션만 실행")
parser.add_argument("--batch-size", type=int, default=100, help="배치 커밋 크기")
args = parser.parse_args()
asyncio.run(migrate(
source_dir=args.source_dir,
target_dir=args.target_dir,
database_url=args.database_url,
dry_run=args.dry_run,
batch_size=args.batch_size,
))

78
scripts/seed_admin.py Normal file
View File

@@ -0,0 +1,78 @@
"""초기 관리자 계정 생성 스크립트
사용법:
# Docker 컨테이너 내부에서 실행
docker compose exec fastapi python /app/scripts/seed_admin.py
# 로컬에서 실행 (DATABASE_URL 환경변수 필요)
python scripts/seed_admin.py
"""
import asyncio
import getpass
import os
import sys
# 프로젝트 루트의 app/ 디렉토리를 경로에 추가
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "app"))
from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from core.auth import hash_password
async def seed_admin():
database_url = os.getenv(
"DATABASE_URL",
"postgresql+asyncpg://pkm:pkm@localhost:5432/pkm",
)
engine = create_async_engine(database_url)
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
print("=== hyungi_Document_Server 관리자 계정 생성 ===\n")
username = input("관리자 아이디 [admin]: ").strip() or "admin"
password = getpass.getpass("비밀번호: ")
if not password:
print("비밀번호는 필수입니다.")
return
password_confirm = getpass.getpass("비밀번호 확인: ")
if password != password_confirm:
print("비밀번호가 일치하지 않습니다.")
return
password_hash = hash_password(password)
async with async_session() as session:
# 이미 존재하는지 확인
result = await session.execute(
text("SELECT id FROM users WHERE username = :username"),
{"username": username},
)
if result.scalar_one_or_none():
print(f"'{username}' 계정이 이미 존재합니다. 비밀번호를 업데이트합니다.")
await session.execute(
text("UPDATE users SET password_hash = :hash WHERE username = :username"),
{"hash": password_hash, "username": username},
)
else:
await session.execute(
text(
"INSERT INTO users (username, password_hash, is_active) "
"VALUES (:username, :hash, TRUE)"
),
{"username": username, "hash": password_hash},
)
print(f"'{username}' 계정이 생성되었습니다.")
await session.commit()
await engine.dispose()
print("\n완료!")
if __name__ == "__main__":
asyncio.run(seed_admin())

View File

@@ -1,6 +1,7 @@
{
"name": "kordoc-service",
"version": "1.0.0",
"type": "module",
"description": "HWP/HWPX/PDF 문서 파싱 마이크로서비스",
"main": "server.js",
"scripts": {
@@ -8,6 +9,7 @@
},
"dependencies": {
"express": "^4.18.0",
"kordoc": "^1.7.0"
"kordoc": "^1.7.0",
"pdfjs-dist": "^4.0.0"
}
}

View File

@@ -1,10 +1,15 @@
/**
* kordoc 마이크로서비스 — HWP/HWPX/PDF → Markdown 변환 API
* kordoc 마이크로서비스 — HWP/HWPX/PDF/텍스트 → Markdown 변환 API
*/
const express = require('express');
import express from 'express';
import fs from 'fs';
import path from 'path';
import { parse, detectFormat } from 'kordoc';
const app = express();
const PORT = 3100;
const PARSE_TIMEOUT_MS = 30000;
app.use(express.json({ limit: '500mb' }));
@@ -13,26 +18,103 @@ app.get('/health', (req, res) => {
res.json({ status: 'ok', service: 'kordoc' });
});
// 문서 파싱
// 지원 포맷 목록
const TEXT_FORMATS = new Set(['.md', '.txt', '.csv', '.json', '.xml', '.html']);
const PARSEABLE_FORMATS = new Set(['.hwp', '.hwpx', '.pdf']);
const IMAGE_FORMATS = new Set(['.jpg', '.jpeg', '.png', '.tiff', '.tif', '.bmp', '.gif']);
/**
* 문서 파싱 — 파일 경로를 받아 Markdown으로 변환
*/
app.post('/parse', async (req, res) => {
const startTime = Date.now();
try {
const { filePath } = req.body;
if (!filePath) {
return res.status(400).json({ error: 'filePath is required' });
}
// TODO: kordoc 라이브러리 연동 (Phase 1에서 구현)
// const kordoc = require('kordoc');
// const result = await kordoc.parse(filePath);
// return res.json(result);
// 파일 존재 확인
if (!fs.existsSync(filePath)) {
return res.status(404).json({ error: `파일을 찾을 수 없습니다: ${filePath}` });
}
const ext = path.extname(filePath).toLowerCase();
const stat = fs.statSync(filePath);
// 100MB 초과 파일 거부
if (stat.size > 100 * 1024 * 1024) {
return res.status(413).json({ error: '파일 크기 100MB 초과' });
}
// 텍스트 파일 — 직접 읽기
if (TEXT_FORMATS.has(ext)) {
const text = fs.readFileSync(filePath, 'utf-8');
return res.json({
success: true,
markdown: text,
metadata: { format: ext.slice(1), fileSize: stat.size },
format: ext.slice(1),
requires_ocr: false,
});
}
// 이미지 파일 — OCR 필요 플래그
if (IMAGE_FORMATS.has(ext)) {
return res.json({
success: true,
markdown: '',
metadata: { format: ext.slice(1), fileSize: stat.size },
format: ext.slice(1),
requires_ocr: true,
});
}
// HWP/HWPX/PDF — kordoc 파싱
if (PARSEABLE_FORMATS.has(ext)) {
const buffer = fs.readFileSync(filePath);
// 타임아웃 처리
const result = await Promise.race([
parse(buffer),
new Promise((_, reject) =>
setTimeout(() => reject(new Error('파싱 타임아웃 (30초)')), PARSE_TIMEOUT_MS)
),
]);
if (!result.success) {
return res.status(422).json({
error: '문서 파싱 실패',
warnings: result.warnings || [],
});
}
return res.json({
success: true,
markdown: result.markdown || '',
metadata: {
...(result.metadata || {}),
format: ext.slice(1),
fileSize: stat.size,
parseTime: Date.now() - startTime,
},
format: ext.slice(1),
requires_ocr: false,
});
}
// 미지원 포맷
return res.json({
success: true,
markdown: '',
metadata: {},
format: 'unknown',
message: 'kordoc 파싱은 Phase 1에서 구현 예정'
metadata: { format: ext.slice(1), fileSize: stat.size },
format: ext.slice(1),
requires_ocr: false,
unsupported: true,
});
} catch (err) {
console.error(`[ERROR] /parse: ${err.message}`);
res.status(500).json({ error: err.message });
}
});
@@ -45,7 +127,7 @@ app.post('/compare', async (req, res) => {
return res.status(400).json({ error: 'filePathA and filePathB are required' });
}
// TODO: kordoc compare 구현 (Phase 2)
// TODO: Phase 2에서 kordoc compare 구현
return res.json({ diffs: [], message: 'compare는 Phase 2에서 구현 예정' });
} catch (err) {
res.status(500).json({ error: err.message });