feat: DEVONthink 제거 + 모닝 브리핑 추가

- DEVONthink 의존성 제거 → kb_writer 전환 (news_digest, inbox_processor, mail pipeline)
- devonthink_bridge.py, plist 삭제
- morning_briefing.py 신규 (매일 07:30, 일정·메일·보고·뉴스 → Synology Chat)
- intent_service.py 분류기 프롬프트 개선 + 키워드 fallback
- migrate-v5.sql (news_digest_log kb_path 컬럼)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Hyungi Ahn
2026-03-19 14:12:38 +09:00
parent fd8925637d
commit 782caf5130
15 changed files with 479 additions and 240 deletions

View File

@@ -56,9 +56,6 @@ IMAP_USER=hyungi
IMAP_PASSWORD=changeme
IMAP_SSL=true
# DEVONthink (devonthink_bridge.py — 지식 저장소)
DEVONTHINK_APP_NAME=DEVONthink
# Karakeep (NAS Docker — 북마크/뉴스 저장)
KARAKEEP_URL=http://192.168.1.227:3000
KARAKEEP_API_KEY=changeme
@@ -70,7 +67,6 @@ API_MONTHLY_LIMIT=10.00
HEIC_CONVERTER_URL=http://host.docker.internal:8090
CHAT_BRIDGE_URL=http://host.docker.internal:8091
CALDAV_BRIDGE_URL=http://host.docker.internal:8092
DEVONTHINK_BRIDGE_URL=http://host.docker.internal:8093
MAIL_BRIDGE_URL=http://host.docker.internal:8094
KB_WRITER_URL=http://host.docker.internal:8095
NOTE_BRIDGE_URL=http://host.docker.internal:8098

View File

@@ -23,7 +23,7 @@ bot-n8n (맥미니 Docker) — 51노드 파이프라인
│ ├─ calendar → CalDAV Bridge → Synology Calendar
│ ├─ reminder → calendar로 통합
│ ├─ mail → 메일 요약 조회
│ ├─ note → DEVONthink 저장
│ ├─ note → KB Writer 저장
│ └─ fallback → 일반 대화 (RAG + 3단계 라우팅)
├─④ [needs_rag=true] 멀티-컬렉션 RAG
@@ -39,7 +39,7 @@ bot-n8n (맥미니 Docker) — 51노드 파이프라인
└── Qdrant (벡터 검색, 3컬렉션)
⑥ 응답 전송 + chat_logs 저장 + API 사용량 UPSERT
⑦ [비동기] 선택적 메모리 (Qwen 판단 → 가치 있으면 벡터화 + DEVONthink 저장)
⑦ [비동기] 선택적 메모리 (Qwen 판단 → 가치 있으면 벡터화 + KB 저장)
별도 워크플로우:
Mail Processing Pipeline (9노드) — mail_bridge 날짜 기반 조회 → dedup → 분류 → mail_logs
@@ -48,19 +48,16 @@ bot-n8n (맥미니 Docker) — 51노드 파이프라인
heic_converter (:8090) — HEIC→JPEG 변환 (macOS sips)
chat_bridge (:8091) — DSM Chat API 브릿지 (사진 폴링/다운로드)
caldav_bridge (:8092) — CalDAV REST 래퍼 (Synology Calendar, VEVENT+VTODO)
devonthink_bridge (:8093) — DEVONthink AppleScript 래퍼
mail_bridge (:8094) — IMAP 날짜 기반 메일 조회 (MailPlus)
kb_writer (:8095) — 마크다운 KB 저장
note_bridge (:8098) — Note Station REST 래퍼 (메모 생성/추가)
intent_service (:8099) — 의도 분류 + 날짜 파싱 + Claude fallback
inbox_processor (5분) — OmniFocus Inbox 폴링 (LaunchAgent)
news_digest (매일 07:00) — 뉴스 번역·요약 (LaunchAgent)
morning_briefing (매일 07:30) — 일정·메일·보고·뉴스 모닝 브리핑 (LaunchAgent)
NAS (192.168.1.227):
Synology Chat / Synology Calendar (CalDAV) / MailPlus (IMAP)
DEVONthink 4 (맥미니):
AppleScript 경유 문서 저장·검색
```
## 인프라
@@ -76,17 +73,16 @@ DEVONthink 4 (맥미니):
| heic_converter | 네이티브 (맥미니) | 8090 | HEIC→JPEG 변환 (macOS sips) |
| chat_bridge | 네이티브 (맥미니) | 8091 | DSM Chat API 브릿지 (사진 폴링/다운로드) |
| caldav_bridge | 네이티브 (맥미니) | 8092 | CalDAV REST 래퍼 (Synology Calendar) |
| devonthink_bridge | 네이티브 (맥미니) | 8093 | DEVONthink AppleScript 래퍼 |
| mail_bridge | 네이티브 (맥미니) | 8094 | IMAP 날짜 기반 메일 조회 (MailPlus) |
| kb_writer | 네이티브 (맥미니) | 8095 | 마크다운 KB 저장 |
| note_bridge | 네이티브 (맥미니) | 8098 | Note Station REST 래퍼 (메모 생성/추가) |
| intent_service | 네이티브 (맥미니) | 8099 | 의도 분류 + 날짜 파싱 + Claude fallback |
| inbox_processor | 네이티브 (맥미니) | — | OmniFocus Inbox 폴링 (LaunchAgent, 5분) |
| news_digest | 네이티브 (맥미니) | — | 뉴스 번역·요약 (LaunchAgent, 매일 07:00) |
| morning_briefing | 네이티브 (맥미니) | — | 모닝 브리핑 (LaunchAgent, 매일 07:30) |
| Synology Chat | NAS (192.168.1.227) | — | 사용자 인터페이스 |
| Synology Calendar | NAS (192.168.1.227) | CalDAV | 캘린더 서비스 |
| MailPlus | NAS (192.168.1.227) | IMAP | 메일 서비스 |
| DEVONthink 4 | 네이티브 (맥미니) | — | 문서 저장·검색 (AppleScript) |
## 3단계 라우팅

View File

@@ -36,7 +36,6 @@ curl -s http://192.168.1.186:11434/api/generate -d '{"model":"id-9b:latest","pro
| chat_bridge | http://localhost:8091 |
| HEIC converter | http://localhost:8090 |
| caldav_bridge | http://localhost:8092 |
| devonthink_bridge | http://localhost:8093 |
## Docker 명령어
@@ -89,7 +88,6 @@ echo "=== 네이티브 서비스 ===" && \
curl -s http://localhost:8090/health && echo && \
curl -s http://localhost:8091/health && echo && \
curl -s http://localhost:8092/health && echo && \
curl -s http://localhost:8093/health && echo && \
./manage_services.sh status && \
echo "=== n8n ===" && \
curl -s -o /dev/null -w ' HTTP %{http_code}' http://localhost:5678 && echo && \
@@ -109,9 +107,9 @@ syn-chat-bot/
├── heic_converter.py ← HEIC→JPEG 변환 API (macOS sips, port 8090)
├── chat_bridge.py ← DSM Chat API 브릿지 (사진 폴링/다운로드, port 8091)
├── caldav_bridge.py ← CalDAV REST 래퍼 (Synology Calendar, port 8092)
├── devonthink_bridge.py ← DEVONthink AppleScript 래퍼 (port 8093)
├── inbox_processor.py ← OmniFocus Inbox 폴링 (LaunchAgent, 5분)
├── news_digest.py ← 뉴스 번역·요약 (LaunchAgent, 매일 07:00)
├── morning_briefing.py ← 모닝 브리핑 (LaunchAgent, 매일 07:30)
├── manage_services.sh ← 네이티브 서비스 관리 (start/stop/status)
├── deploy_workflows.sh ← n8n 워크플로우 자동 배포 (REST API)
├── start-bridge.sh ← 브릿지 서비스 시작 헬퍼
@@ -180,14 +178,14 @@ tail -50 /tmp/chat-bridge.err
# caldav_bridge 상태 확인
curl -s http://localhost:8092/health | python3 -m json.tool
# devonthink_bridge 상태 확인
curl -s http://localhost:8093/health | python3 -m json.tool
# inbox_processor 로그
tail -50 /tmp/inbox-processor.log
# news_digest 로그
tail -50 /tmp/news-digest.log
# morning_briefing 로그
tail -50 /tmp/morning-briefing.log
```
## n8n 접속 정보
@@ -248,11 +246,10 @@ NAS에서 Outgoing Webhook 설정 필요:
- [x] /보고서 월간 보고서 생성 구현
- [x] report_cache 캐시 + --force 재생성
### Phase 6: 캘린더·메일·DEVONthink·OmniFocus·뉴스
### Phase 6: 캘린더·메일·OmniFocus·뉴스
- [x] mail_logs, calendar_events 테이블
- [x] 분류기 v3 (calendar, reminder, mail, note intent 추가)
- [x] caldav_bridge.py — CalDAV REST 래퍼 (Synology Calendar)
- [x] devonthink_bridge.py — DEVONthink AppleScript 래퍼
- [x] inbox_processor.py — OmniFocus Inbox 폴링 (LaunchAgent, 5분)
- [x] news_digest.py — 뉴스 번역·요약 (LaunchAgent, 매일 07:00)
- [x] manage_services.sh — 네이티브 서비스 관리
@@ -264,7 +261,6 @@ NAS에서 Outgoing Webhook 설정 필요:
### 서비스 기동 전제조건
- Synology Calendar (CalDAV) — NAS에서 활성화 필요
- Synology MailPlus — NAS에서 활성화 + 계정 설정 필요
- DEVONthink 4 — 맥미니에 설치 필요 (AppleScript 접근)
- OmniFocus — 맥미니에 설치 필요 (Inbox 폴링)
## 검증 체크리스트
@@ -281,4 +277,4 @@ NAS에서 Outgoing Webhook 설정 필요:
10. 10초 내 6건 → rate limit
11. "내일 회의 잡아줘" → calendar intent → CalDAV 이벤트 생성
12. "최근 메일 확인" → mail intent → 메일 요약 반환
13. "이거 메모해둬" → note intent → DEVONthink 저장
13. "이거 메모해둬" → note intent → KB Writer 저장

View File

@@ -3,23 +3,26 @@
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.syn-chat-bot.devonthink-bridge</string>
<string>com.syn-chat-bot.morning-briefing</string>
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/opt/python@3.14/bin/python3.14</string>
<string>-S</string>
<string>-c</string>
<string>import sys; sys.path.insert(0,'/Users/hyungi/Documents/code/syn-chat-bot/.venv/lib/python3.14/site-packages'); sys.path.insert(0,'/Users/hyungi/Documents/code/syn-chat-bot'); import uvicorn; uvicorn.run('devonthink_bridge:app',host='127.0.0.1',port=8093)</string>
<string>import sys; sys.path.insert(0,'/Users/hyungi/Documents/code/syn-chat-bot/.venv/lib/python3.14/site-packages'); sys.path.insert(0,'/Users/hyungi/Documents/code/syn-chat-bot'); from morning_briefing import main; main()</string>
</array>
<key>WorkingDirectory</key>
<string>/Users/hyungi/Documents/code/syn-chat-bot</string>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StartCalendarInterval</key>
<dict>
<key>Hour</key>
<integer>7</integer>
<key>Minute</key>
<integer>30</integer>
</dict>
<key>StandardOutPath</key>
<string>/tmp/devonthink-bridge.log</string>
<string>/tmp/morning-briefing.log</string>
<key>StandardErrorPath</key>
<string>/tmp/devonthink-bridge.err</string>
<string>/tmp/morning-briefing.err</string>
</dict>
</plist>

View File

@@ -1,125 +0,0 @@
"""DEVONthink Bridge — AppleScript REST API 래퍼 (port 8093)"""
import json
import logging
import os
import subprocess
from dotenv import load_dotenv
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
load_dotenv()
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("devonthink_bridge")
DT_APP = os.getenv("DEVONTHINK_APP_NAME", "DEVONthink")
app = FastAPI()
def _run_applescript(script: str, timeout: int = 15) -> str:
"""AppleScript 실행."""
result = subprocess.run(
["osascript", "-e", script],
capture_output=True, text=True, timeout=timeout,
)
if result.returncode != 0:
raise RuntimeError(f"AppleScript error: {result.stderr.strip()}")
return result.stdout.strip()
def _escape_as(s: str) -> str:
"""AppleScript 문자열 이스케이프."""
return s.replace("\\", "\\\\").replace('"', '\\"')
@app.post("/save")
async def save_record(request: Request):
body = await request.json()
title = body.get("title", "Untitled")
content = body.get("content", "")
record_type = body.get("type", "markdown")
database = body.get("database")
group = body.get("group")
tags = body.get("tags", [])
# Map type to DEVONthink record type
type_map = {"markdown": "markdown", "text": "txt", "html": "html"}
dt_type = type_map.get(record_type, "markdown")
# Build AppleScript
esc_title = _escape_as(title)
esc_content = _escape_as(content)
tags_str = ", ".join(f'"{_escape_as(t)}"' for t in tags) if tags else ""
if database:
db_line = f'set theDB to open database "{_escape_as(database)}"'
else:
db_line = "set theDB to first database"
if group:
group_line = f'set theGroup to get record at "/{_escape_as(group)}" in theDB'
else:
group_line = "set theGroup to incoming group of theDB"
script = f'''tell application "{DT_APP}"
{db_line}
{group_line}
set theRecord to create record with {{name:"{esc_title}", type:{dt_type}, plain text:"{esc_content}"}} in theGroup
{f'set tags of theRecord to {{{tags_str}}}' if tags_str else ''}
set theUUID to uuid of theRecord
set theName to name of theRecord
return theUUID & "|" & theName
end tell'''
try:
result = _run_applescript(script)
parts = result.split("|", 1)
uuid_val = parts[0] if parts else result
name_val = parts[1] if len(parts) > 1 else title
logger.info(f"Record saved: {uuid_val} '{name_val}'")
return JSONResponse({"success": True, "uuid": uuid_val, "name": name_val})
except Exception as e:
logger.error(f"Save failed: {e}")
return JSONResponse({"success": False, "error": str(e)}, status_code=500)
@app.get("/databases")
async def list_databases():
script = f'''tell application "{DT_APP}"
set dbList to {{}}
repeat with theDB in databases
set end of dbList to (name of theDB) & "|" & (uuid of theDB)
end repeat
set AppleScript's text item delimiters to "\\n"
return dbList as text
end tell'''
try:
result = _run_applescript(script)
databases = []
for line in result.strip().split("\n"):
if "|" in line:
parts = line.split("|", 1)
databases.append({"name": parts[0], "uuid": parts[1]})
return JSONResponse({"databases": databases})
except Exception as e:
logger.error(f"List databases failed: {e}")
return JSONResponse({"databases": [], "error": str(e)}, status_code=500)
@app.get("/health")
async def health():
devonthink_running = False
try:
result = _run_applescript(
f'tell application "System Events" to return (name of processes) contains "{DT_APP}"',
timeout=5,
)
devonthink_running = result.lower() == "true"
except Exception:
pass
return {"status": "ok", "devonthink_running": devonthink_running}

View File

@@ -31,7 +31,6 @@ services:
- HEIC_CONVERTER_URL=http://host.docker.internal:8090
- CHAT_BRIDGE_URL=http://host.docker.internal:8091
- CALDAV_BRIDGE_URL=http://host.docker.internal:8092
- DEVONTHINK_BRIDGE_URL=http://host.docker.internal:8093
- MAIL_BRIDGE_URL=http://host.docker.internal:8094
- KB_WRITER_URL=http://host.docker.internal:8095
- NODE_FUNCTION_ALLOW_BUILTIN=crypto,http,https,url

View File

@@ -1,6 +1,6 @@
# Progress — syn-chat-bot
> 최종 업데이트: 2026-03-17
> 최종 업데이트: 2026-03-19
## Phase별 구현 현황
@@ -30,30 +30,37 @@
- `fetch()`/`$http.request()``require('http')`/`require('https')` 패턴 전환
- `NODE_FUNCTION_ALLOW_BUILTIN=crypto,http,https,url` 설정
### Phase 5-6: Calendar/Mail/DEVONthink/OmniFocus/News (커밋 `612933c`, 2026-03-14)
### Phase 5-6: Calendar/Mail/OmniFocus/News (커밋 `612933c`, 2026-03-14)
- 분류기 v3: calendar, reminder, mail, note intent 추가
- caldav_bridge.py (:8092) — CalDAV REST 래퍼 (Synology Calendar CRUD)
- devonthink_bridge.py (:8093) — DEVONthink AppleScript 래퍼
- inbox_processor.py — OmniFocus Inbox 폴링 (LaunchAgent, 5분 간격)
- news_digest.py — RSS 뉴스 번역·요약 (LaunchAgent, 매일 07:00)
- Mail Processing Pipeline (7노드) — IMAP 폴링 → Qwen 분류 → mail_logs
- 메인 파이프라인 51노드로 확장 (calendar/mail/note 핸들러)
### DEVONthink 제거 + 모닝 브리핑 (2026-03-19)
- DEVONthink 의존성 제거 → kb_writer로 전환 (news_digest, inbox_processor, mail pipeline)
- devonthink_bridge.py 삭제 (macOS AppleScript 의존성 제거)
- morning_briefing.py — 모닝 브리핑 (LaunchAgent, 매일 07:30)
- 일정(CalDAV), 메일(mail_logs), 보고(field_reports), 뉴스(Karakeep) → Synology Chat
- migrate-v5.sql — news_digest_log에 kb_path 컬럼 추가
---
## 파일 인벤토리
### Python 서비스 (6개, 1,227줄)
### Python 서비스 (6개)
| 파일 | 줄수 | 포트/실행 | 역할 |
|------|------|----------|------|
| `caldav_bridge.py` | 269 | :8092 | CalDAV REST 래퍼 |
| `chat_bridge.py` | 293 | :8091 | DSM Chat API 브릿지 |
| `devonthink_bridge.py` | 125 | :8093 | DEVONthink AppleScript 래퍼 |
| `heic_converter.py` | 25 | :8090 | HEIC→JPEG 변환 |
| `inbox_processor.py` | 225 | LaunchAgent 5분 | OmniFocus Inbox 폴링 |
| `news_digest.py` | 290 | LaunchAgent 07:00 | 뉴스 번역·요약 |
| 파일 | 포트/실행 | 역할 |
|------|----------|------|
| `caldav_bridge.py` | :8092 | CalDAV REST 래퍼 |
| `chat_bridge.py` | :8091 | DSM Chat API 브릿지 |
| `heic_converter.py` | :8090 | HEIC→JPEG 변환 |
| `inbox_processor.py` | LaunchAgent 5분 | OmniFocus Inbox 폴링 |
| `news_digest.py` | LaunchAgent 07:00 | 뉴스 번역·요약 |
| `morning_briefing.py` | LaunchAgent 07:30 | 모닝 브리핑 |
### LaunchAgent plist (6개)
@@ -61,10 +68,10 @@
|------|--------|
| `com.syn-chat-bot.caldav-bridge.plist` | caldav_bridge |
| `com.syn-chat-bot.chat-bridge.plist` | chat_bridge |
| `com.syn-chat-bot.devonthink-bridge.plist` | devonthink_bridge |
| `com.syn-chat-bot.heic-converter.plist` | heic_converter |
| `com.syn-chat-bot.inbox-processor.plist` | inbox_processor |
| `com.syn-chat-bot.news-digest.plist` | news_digest |
| `com.syn-chat-bot.morning-briefing.plist` | morning_briefing |
### n8n 워크플로우 (2개)
@@ -80,6 +87,7 @@
| `init/init.sql` | 초기 스키마 (5테이블) |
| `init/migrate-v2.sql` | v2 마이그레이션 (7테이블 추가) |
| `init/migrate-v3.sql` | v3 마이그레이션 (calendar_events 확장) |
| `init/migrate-v5.sql` | v5 마이그레이션 (DEVONthink→kb_writer 전환) |
### 기타
@@ -120,7 +128,6 @@ pip install caldav aiohttp
### 3. macOS 앱 설정
- **DEVONthink 4**: 설치 + 데이터베이스 열기 (AppleScript 접근 허용)
- **OmniFocus**: 설치 + Inbox 사용 설정
- **Ollama (맥미니)**: `ollama pull bge-m3 && ollama pull bge-reranker-v2-m3`
@@ -175,7 +182,6 @@ bash manage_services.sh load
curl http://localhost:8090/health # heic_converter
curl http://localhost:8091/health # chat_bridge
curl http://localhost:8092/health # caldav_bridge
curl http://localhost:8093/health # devonthink_bridge
# Docker 상태
docker compose ps
@@ -195,4 +201,4 @@ docker exec bot-postgres psql -U bot chatbot -c '\dt'
- [ ] CalDAV 양방향 동기화
- [ ] 메일 발송 (SMTP via MailPlus)
- [ ] reminder 실구현 (알림 시간에 Synology Chat 푸시)
- [ ] DEVONthink 검색 결과 RAG 연동
- [ ] 모닝 브리핑 고도화 (주간 요약, 커스텀 섹션)

View File

@@ -30,7 +30,7 @@
│ ├─ calendar → CalDAV Bridge → Synology Calendar │
│ ├─ reminder → calendar로 통합 처리 │
│ ├─ mail → 메일 요약 조회 (mail_logs) │
│ ├─ note → DEVONthink Bridge → 문서 저장
│ ├─ note → KB Writer → 문서 저장
│ └─ fallback → 일반 대화 (RAG + 3단계 라우팅) │
│ │
│ ④ [needs_rag=true] 멀티-컬렉션 RAG 검색 │
@@ -43,7 +43,7 @@
│ └─ api_heavy → 예산 체크 → Claude Opus (or 다운그레이드) │
│ │
│ ⑥ 응답 전송 + chat_logs + api_usage_monthly │
│ ⑦ [비동기] Qwen 메모리 판단 → 가치 있으면 벡터화 + DEVONthink
│ ⑦ [비동기] Qwen 메모리 판단 → 가치 있으면 벡터화 + KB 저장
│ └─ classification_logs 기록 │
└──┬──────────┬───────────┬───────────┬──────────────────────┘
│ │ │ │
@@ -77,25 +77,20 @@
│ caldav_bridge.py (:8092) │
│ └─ CalDAV REST 래퍼 (Synology Calendar CRUD) │
│ │
│ devonthink_bridge.py (:8093) │
│ └─ DEVONthink AppleScript 래퍼 (문서 저장·검색) │
│ │
│ inbox_processor.py (LaunchAgent, 5분) │
│ └─ OmniFocus Inbox 폴링 → Qwen 분류 → 자동 정리 │
│ │
│ news_digest.py (LaunchAgent, 매일 07:00) │
│ └─ RSS 뉴스 수집 → Qwen 번역·요약 → Qdrant + Synology Chat│
│ │
│ morning_briefing.py (LaunchAgent, 매일 07:30) │
│ └─ 일정·메일·보고·뉴스 → 요약 → Synology Chat 전송 │
└────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────┐
│ NAS 서비스 (192.168.1.227) │
│ Synology Chat / Calendar (CalDAV) / MailPlus │
└────────────────────────────────────────────────┘
┌────────────────────────────────────────────────┐
│ DEVONthink 4 (맥미니) │
│ AppleScript 경유 문서 저장·검색 │
└────────────────────────────────────────────────┘
```
## 3단계 라우팅 상세
@@ -131,7 +126,7 @@
- `calendar` — CalDAV Bridge로 일정 생성/조회 (Synology Calendar)
- `reminder` — calendar로 통합 (알림 시간 포함 일정 생성)
- `mail` — mail_logs에서 최근 메일 요약 조회
- `note`DEVONthink Bridge로 문서 저장
- `note`KB Writer로 문서 저장
```
### 프리필터 → 분류기 → 모델 라우팅 흐름
@@ -342,7 +337,7 @@ Webhook POST /chat
│ ├─ calendar → [Handle Calendar] → CalDAV Bridge → 확인응답
│ ├─ reminder → [Handle Calendar] (calendar로 통합)
│ ├─ mail → [Handle Mail] → mail_logs 조회 → 요약응답
│ ├─ note → [Handle Note] → DEVONthink Bridge → 확인응답
│ ├─ note → [Handle Note] → KB Writer → 확인응답
│ └─ fallback → [Needs RAG?]
├─ needs_rag=true
@@ -364,7 +359,7 @@ Webhook POST /chat
▼ [비동기]
[Memorization Check] → [Should Memorize?]
├─ true → [Embed & Save Memory] + [DEVONthink 저장]
├─ true → [Embed & Save Memory] + [KB 저장]
└─ false → (끝)
```
@@ -413,10 +408,9 @@ Webhook POST /chat
- API 사용량 추적 (api_usage_monthly UPSERT)
- HEIC→JPEG 변환 (heic_converter.py) + chat_bridge.py (DSM Chat API 브릿지)
### Phase 6: 캘린더·메일·DEVONthink·OmniFocus·뉴스
### Phase 6: 캘린더·메일·OmniFocus·뉴스
- 분류기 v3: calendar, reminder, mail, note intent 추가
- caldav_bridge.py: CalDAV REST 래퍼 (Synology Calendar CRUD)
- devonthink_bridge.py: DEVONthink AppleScript 래퍼
- inbox_processor.py: OmniFocus Inbox 폴링 (LaunchAgent, 5분)
- news_digest.py: 뉴스 번역·요약 (LaunchAgent, 매일 07:00)
- Mail Processing Pipeline (7노드): IMAP 폴링 → 분류 → mail_logs
@@ -428,4 +422,4 @@ Webhook POST /chat
- CalDAV 양방향 동기화 (Synology Calendar → bot-postgres)
- 메일 발송 (SMTP via MailPlus)
- reminder 실구현 (알림 시간에 Synology Chat 푸시)
- DEVONthink 검색 결과 RAG 연동
- 모닝 브리핑 고도화 (주간 요약, 커스텀 섹션)

View File

@@ -16,7 +16,7 @@ logger = logging.getLogger("inbox_processor")
GPU_OLLAMA_URL = os.getenv("GPU_OLLAMA_URL", "http://192.168.1.186:11434")
CALDAV_BRIDGE_URL = os.getenv("CALDAV_BRIDGE_URL", "http://127.0.0.1:8092")
DEVONTHINK_BRIDGE_URL = os.getenv("DEVONTHINK_BRIDGE_URL", "http://127.0.0.1:8093")
KB_WRITER_URL = os.getenv("KB_WRITER_URL", "http://127.0.0.1:8095")
def run_applescript(script: str, timeout: int = 15) -> str:
@@ -166,23 +166,26 @@ def route_calendar(cls: dict, task_id: str) -> None:
def route_note(cls: dict, task_id: str) -> None:
"""DEVONthink 브릿지로 메모 저장."""
"""kb_writer로 메모 저장."""
content = cls.get("content") or cls.get("title", "")
title = cls.get("title", "OmniFocus 메모")
try:
resp = httpx.post(
f"{DEVONTHINK_BRIDGE_URL}/save",
f"{KB_WRITER_URL}/save",
json={
"title": title,
"content": content,
"type": "markdown",
"type": "note",
"tags": ["omnifocus", "inbox"],
"username": "inbox-processor",
"source": "omnifocus",
"topic": "omnifocus",
},
timeout=10,
)
if resp.json().get("success"):
logger.info(f"Note saved to DEVONthink: {title}")
logger.info(f"Note saved to KB: {title}")
mark_processed(task_id)
complete_task(task_id)
except Exception as e:

4
init/migrate-v5.sql Normal file
View File

@@ -0,0 +1,4 @@
-- migrate-v5.sql: DEVONthink → kb_writer 전환
-- 실행: docker exec -i bot-postgres psql -U bot -d chatbot < init/migrate-v5.sql
ALTER TABLE news_digest_log ADD COLUMN IF NOT EXISTS kb_path VARCHAR(200);
-- devonthink_uuid 컬럼은 기존 데이터 유지를 위해 삭제하지 않음

View File

@@ -48,26 +48,51 @@ ID_SYSTEM_PROMPT = """너는 '이드'라는 이름의 AI 비서야. 한국어로
간결하고 실용적으로 답변하되, 친근한 톤을 유지해.
불필요한 인사나 꾸밈말은 생략하고 핵심만 전달해."""
# 의도 분류 프롬프트
CLASSIFY_PROMPT = """사용자 메시지를 분석하여 JSON으로 응답하라.
# 의도 분류 프롬프트 (n8n 파이프라인 호환)
def _build_classify_prompt(user_text: str) -> str:
now = datetime.now(KST)
today = now.strftime("%Y-%m-%d")
current_time = now.strftime("%H:%M:%S")
day_names = ["", "", "", "", "", "", ""]
day_of_week = day_names[now.weekday()]
분류 기준:
- calendar: 일정/약속/회의 등 시간이 정해진 이벤트
- todo: 작업/할일/과제 등 기한이 있는 태스크
- note: 메모/기록/저장 요청
- chat: 일반 대화, 질문, 인사
return f"""현재: {today} {current_time} (KST, {day_of_week}요일). 사용자 메시지를 분석하고 아래 JSON 형식으로만 응답하세요. 다른 텍스트는 출력하지 마세요.
반드시 아래 JSON 형식만 출력:
{"intent": "calendar|todo|note|chat", "confidence": 0.0~1.0, "title": "추출된 제목", "raw_datetime": "원문의 날짜/시간 표현"}
{{
"intent": "greeting|question|log_event|calendar|todo|reminder|mail|note|photo|command|report|other",
"response_tier": "local|api_light|api_heavy",
"needs_rag": true/false,
"rag_target": ["documents", "tk_company", "chat_memory"],
"department_hint": "안전|생산|구매|품질|총무|시설|null",
"report_domain": "안전|시설설비|품질|null",
"query": "검색용 쿼리 (needs_rag=false면 null)",
"title": "추출된 제목 (calendar/todo/note 시)",
"raw_datetime": "원문의 날짜/시간 표현 (calendar/todo 시)"
}}
예시:
- "내일 3시 회의"{"intent": "calendar", "confidence": 0.95, "title": "회의", "raw_datetime": "내일 3시"}
- "이번주까지 보고서 작성"{"intent": "todo", "confidence": 0.9, "title": "보고서 작성", "raw_datetime": "이번주까지"}
- "메모해둬: 부품 발주 필요"{"intent": "note", "confidence": 0.95, "title": "부품 발주 필요", "raw_datetime": ""}
- "안녕"{"intent": "chat", "confidence": 0.99, "title": "", "raw_datetime": ""}
- "내일 자료 정리"{"intent": "todo", "confidence": 0.6, "title": "자료 정리", "raw_datetime": "내일"}
intent 분류:
- log_event: 사실 기록/등록 요청 ("~구입","~완료","~교체","~점검","~수령","~입고","~등록")
- report: 긴급 사고/재해 신고만 ("사고","부상","화재","누수","폭발","붕괴" + 즉각 대응 필요)
- question: 정보 질문/조회
- greeting: 인사/잡담/감사
- calendar: 일정 등록/조회/삭제 ("일정","회의","미팅","약속","~시에 ~등록","오늘 일정","내일 뭐 있어")
- todo: 작업/할일/과제 ("~까지 ~작성","~해야 해","할 일","작업")
- reminder: 알림 설정 ("~시에 알려줘","리마인드") → calendar로 처리
- mail: 메일 관련 조회 ("메일 확인","받은 메일","이메일","메일 왔어?")
- note: 메모/기록 요청 ("기록해","메모해","저장해","적어둬")
※ 애매하면 log_event로 분류 (기록 누락보다 안전)
사용자 메시지: """
response_tier 판단:
- local: 인사, 잡담, log_event, report, calendar, todo, reminder, note, 단순 질문, mail 간단조회
- api_light: 장문 요약(200자+), 다국어 번역, 비교 분석, RAG 결과 종합
- api_heavy: 법률 해석, 복잡한 다단계 추론, 다중 문서 교차 분석
※ 판단이 애매하면 local 우선
needs_rag 판단:
- true: 회사문서/절차 질문, 이전 기록 조회("최근","아까","전에"), 기술질문
- false: 인사, 잡담, 일반상식, log_event, report, calendar, todo, note
사용자 메시지: {user_text}"""
app = FastAPI(title="Intent Service")
@@ -369,18 +394,60 @@ async def _call_claude(prompt: str, system: str | None = None,
# ==================== 엔드포인트 ====================
def _keyword_fallback(text: str) -> dict:
"""AI 실패 시 키워드 기반 분류 (ultimate safety net)."""
t = text
intent = "question"
response_tier = "api_light"
needs_rag = False
rag_target = []
if re.search(r'일정|회의|미팅|약속|스케줄|캘린더', t) and re.search(r'등록|잡아|추가|만들|넣어|수정|삭제|취소', t):
intent, response_tier = "calendar", "local"
elif re.search(r'일정|스케줄|뭐\s*있', t) and re.search(r'오늘|내일|이번|다음', t):
intent, response_tier = "calendar", "local"
elif re.search(r'까지|해야|할\s*일|작업', t) and re.search(r'작성|보고서|정리|준비|제출', t):
intent, response_tier = "todo", "local"
elif re.search(r'기록해|메모해|저장해|적어둬|메모\s*저장|노트', t):
intent, response_tier = "note", "local"
elif re.search(r'메일|이메일|받은\s*편지|mail', t) or (re.search(r'매일', t) and re.search(r'확인|왔|온|요약|읽', t)):
intent, response_tier = "mail", "local"
elif re.search(r'\d+시', t) and re.search(r'알려|리마인드|알림', t):
intent, response_tier = "calendar", "local"
elif re.search(r'구입|완료|교체|점검|수령|입고|발주', t) and not re.search(r'\?|까$|나$', t):
intent, response_tier = "log_event", "local"
else:
if len(text) <= 30 and not re.search(r'요약|번역|분석|비교', t):
response_tier = "local"
needs_rag = bool(re.search(r'회사|절차|문서|안전|품질|규정|아까|전에|기억', t))
if needs_rag:
rag_target = ["documents"]
if re.search(r'회사|절차|안전|품질', t):
rag_target.append("tk_company")
if re.search(r'아까|이전|전에|기억', t):
rag_target.append("chat_memory")
return {
"intent": intent, "response_tier": response_tier,
"needs_rag": needs_rag, "rag_target": rag_target,
"department_hint": None, "report_domain": None,
"query": text, "title": "", "raw_datetime": "",
"fallback": True, "fallback_method": "keyword",
}
@app.post("/classify")
async def classify_intent(request: Request):
"""의도 분류. body: {message: str}
Returns: {intent, confidence, title, raw_datetime, source: "ollama"|"claude"}
n8n 호환 출력: {intent, response_tier, needs_rag, rag_target, ..., title, raw_datetime, source}
"""
body = await request.json()
message = body.get("message", "").strip()
if not message:
return JSONResponse({"success": False, "error": "message required"}, status_code=400)
prompt = CLASSIFY_PROMPT + message
prompt = _build_classify_prompt(message)
# 1차: Ollama
result_text = await _call_ollama(prompt, system="/no_think")
@@ -392,40 +459,50 @@ async def classify_intent(request: Request):
result_text, _, _ = await _call_claude(prompt, system="JSON만 출력하라. 다른 텍스트 없이.")
source = "claude"
# 완전 실패
# 완전 실패 → 키워드 fallback
if not result_text:
return JSONResponse({"success": False,
"error": "AI 서비스 일시 중단. 잠시 후 다시 시도해주세요."})
logger.warning("All AI classification failed, using keyword fallback")
fb = _keyword_fallback(message)
fb["source"] = "keyword"
fb["success"] = True
return fb
# JSON 파싱
try:
# Ollama가 JSON 외 텍스트를 붙일 수 있으므로 추출
json_match = re.search(r'\{[^}]+\}', result_text)
json_match = re.search(r'\{[^}]+\}', result_text, re.DOTALL)
if json_match:
parsed = json.loads(json_match.group())
else:
parsed = json.loads(result_text)
except json.JSONDecodeError:
logger.warning(f"JSON parse failed: {result_text[:200]}")
# 파싱 실패 → chat으로 폴백
parsed = {"intent": "chat", "confidence": 0.5, "title": "", "raw_datetime": ""}
fb = _keyword_fallback(message)
fb["source"] = source
fb["success"] = True
return fb
intent = parsed.get("intent", "chat")
confidence = float(parsed.get("confidence", 0.5))
intent = parsed.get("intent", "question")
response_tier = parsed.get("response_tier", "api_light")
needs_rag = parsed.get("needs_rag", False)
rag_target = parsed.get("rag_target", [])
if not isinstance(rag_target, list):
rag_target = []
title = parsed.get("title", "")
raw_datetime = parsed.get("raw_datetime", "")
# confidence 낮으면 재질문 신호
needs_clarification = confidence < 0.7
return {
"success": True,
"intent": intent,
"confidence": confidence,
"response_tier": response_tier,
"needs_rag": needs_rag,
"rag_target": rag_target,
"department_hint": parsed.get("department_hint"),
"report_domain": parsed.get("report_domain"),
"query": parsed.get("query", message),
"title": title,
"raw_datetime": raw_datetime,
"needs_clarification": needs_clarification,
"source": source,
"fallback": False,
}
@@ -439,25 +516,32 @@ async def parse_date(request: Request):
@app.post("/chat")
async def chat(request: Request):
"""자유 대화. body: {message: str, system?: str}
"""자유 대화. body: {message: str, system?: str, rag_context?: str}
1차 Ollama → 실패 시 Claude API (응답에 source 표시).
1차 Ollama → 실패 시 Claude API (응답에 ☁️ 표시).
"""
body = await request.json()
message = body.get("message", "").strip()
system = body.get("system", ID_SYSTEM_PROMPT)
rag_context = body.get("rag_context", "")
if not message:
return JSONResponse({"success": False, "error": "message required"}, status_code=400)
# RAG 컨텍스트가 있으면 프롬프트에 추가
prompt = ""
if rag_context:
prompt += f"[참고 자료]\n{rag_context}\n\n"
prompt += f"사용자: {message}\n이드:"
# 1차: Ollama (id-9b, 대화 모델)
response = await _call_ollama(message, system=system, model=OLLAMA_CHAT_MODEL, timeout=30)
response = await _call_ollama(prompt, system=system, model=OLLAMA_CHAT_MODEL, timeout=30)
source = "ollama"
# 2차: Claude fallback
if response is None:
logger.info("Chat fallback to Claude API")
response, _, _ = await _call_claude(message, system=system)
response, _, _ = await _call_claude(prompt, system=system)
source = "claude"
# 완전 실패

View File

@@ -6,8 +6,8 @@ SERVICES=(
"com.syn-chat-bot.chat-bridge"
"com.syn-chat-bot.heic-converter"
"com.syn-chat-bot.caldav-bridge"
"com.syn-chat-bot.devonthink-bridge"
"com.syn-chat-bot.kb-writer"
"com.syn-chat-bot.morning-briefing"
"com.syn-chat-bot.mail-bridge"
"com.syn-chat-bot.inbox-processor"
"com.syn-chat-bot.news-digest"

280
morning_briefing.py Normal file
View File

@@ -0,0 +1,280 @@
"""모닝 브리핑 — 일정·메일·보고·뉴스 요약 → Synology Chat (LaunchAgent, 매일 07:30)"""
import json
import logging
import os
from datetime import datetime, timedelta, timezone
import httpx
from dotenv import load_dotenv
load_dotenv()
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("morning_briefing")
CALDAV_BRIDGE_URL = os.getenv("CALDAV_BRIDGE_URL", "http://127.0.0.1:8092")
KARAKEEP_URL = os.getenv("KARAKEEP_URL", "http://localhost:3000")
KARAKEEP_API_KEY = os.getenv("KARAKEEP_API_KEY", "")
GPU_OLLAMA_URL = os.getenv("GPU_OLLAMA_URL", "http://192.168.1.186:11434")
SYNOLOGY_CHAT_WEBHOOK_URL = os.getenv("SYNOLOGY_CHAT_WEBHOOK_URL", "")
PG_HOST = os.getenv("PG_HOST", "127.0.0.1")
PG_PORT = int(os.getenv("PG_PORT", "15478"))
PG_USER = os.getenv("POSTGRES_USER", "bot")
PG_PASS = os.getenv("POSTGRES_PASSWORD", "")
PG_DB = os.getenv("POSTGRES_DB", "chatbot")
KST = timezone(timedelta(hours=9))
WEEKDAYS = ["", "", "", "", "", "", ""]
def get_db_connection():
import psycopg2
return psycopg2.connect(
host=PG_HOST, port=PG_PORT,
user=PG_USER, password=PG_PASS, dbname=PG_DB,
)
def fetch_today_events() -> list[dict]:
"""CalDAV 브릿지에서 오늘 일정 조회."""
now = datetime.now(KST)
start = now.replace(hour=0, minute=0, second=0, microsecond=0).isoformat()
end = now.replace(hour=23, minute=59, second=59, microsecond=0).isoformat()
try:
resp = httpx.post(
f"{CALDAV_BRIDGE_URL}/calendar/query",
json={"start": start, "end": end},
timeout=10,
)
data = resp.json()
if not data.get("success"):
return []
events = data.get("events", [])
events.sort(key=lambda e: e.get("start", ""))
return events
except Exception as e:
logger.error(f"CalDAV fetch failed: {e}")
return []
def fetch_important_mails() -> list[dict]:
"""DB에서 최근 24시간 업무 메일 조회."""
try:
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute(
"SELECT from_address, subject, summary FROM mail_logs "
"WHERE label = '업무' AND mail_date > NOW() - INTERVAL '24 hours' "
"ORDER BY mail_date DESC LIMIT 10"
)
rows = cur.fetchall()
conn.close()
return [{"from": r[0], "subject": r[1], "summary": r[2]} for r in rows]
except Exception as e:
logger.error(f"Mail fetch failed: {e}")
return []
def fetch_open_reports() -> list[dict]:
"""DB에서 미해결 현장보고 조회."""
try:
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute(
"SELECT category, description, created_at FROM field_reports "
"WHERE status = 'open' "
"ORDER BY created_at DESC LIMIT 10"
)
rows = cur.fetchall()
conn.close()
return [{"category": r[0], "description": r[1], "created_at": r[2]} for r in rows]
except Exception as e:
logger.error(f"Reports fetch failed: {e}")
return []
def fetch_news(since: datetime) -> list[dict]:
"""Karakeep에서 최근 24시간 뉴스 조회."""
headers = {"Authorization": f"Bearer {KARAKEEP_API_KEY}"} if KARAKEEP_API_KEY else {}
try:
resp = httpx.get(
f"{KARAKEEP_URL}/api/v1/bookmarks",
params={"limit": 50},
headers=headers,
timeout=15,
)
resp.raise_for_status()
data = resp.json()
bookmarks = data.get("bookmarks", data if isinstance(data, list) else [])
articles = []
for bm in bookmarks:
created = bm.get("createdAt") or bm.get("created_at") or ""
if created:
try:
dt = datetime.fromisoformat(created.replace("Z", "+00:00"))
if dt < since:
continue
except ValueError:
pass
title = bm.get("title") or bm.get("content", {}).get("title", "")
if title:
articles.append({"title": title})
return articles
except Exception as e:
logger.error(f"Karakeep fetch failed: {e}")
return []
def summarize_news(articles: list[dict]) -> list[str]:
"""LLM으로 뉴스 한줄 요약. 실패 시 원본 제목 fallback."""
if not articles:
return []
titles = [a["title"] for a in articles[:5]]
prompt = "다음 뉴스 제목들을 각각 한 줄로 짧게 한국어 요약하세요. 번호 없이 줄바꿈으로 구분.\n\n" + "\n".join(titles)
try:
resp = httpx.post(
f"{GPU_OLLAMA_URL}/api/generate",
json={
"model": "id-9b:latest",
"system": "/no_think",
"prompt": prompt,
"stream": False,
"think": False,
},
timeout=15,
)
lines = [l.strip() for l in resp.json().get("response", "").strip().split("\n") if l.strip()]
if lines:
return lines[:5]
except Exception as e:
logger.error(f"News summarize failed: {e}")
return titles[:5]
def format_briefing(events: list | None, mails: list | None,
reports: list | None, news: list[str] | None) -> str | None:
"""브리핑 텍스트 조립. 전체 데이터 없으면 None."""
now = datetime.now(KST)
weekday = WEEKDAYS[now.weekday()]
header = f"[모닝 브리핑] {now.strftime('%Y-%m-%d')} ({weekday})"
sections = []
# 일정
if events:
limit = 10
lines = []
for e in events[:limit]:
start = e.get("start", "")
time_str = start[11:16] if len(start) >= 16 else ""
title = e.get("summary") or e.get("title", "")
location = e.get("location", "")
entry = f"- {time_str} {title}" if time_str else f"- {title}"
if location:
entry += f" ({location})"
lines.append(entry)
count_str = f"{len(events)}"
if len(events) > limit:
count_str = f"{limit}건, 외 {len(events) - limit}"
sections.append(f"[일정] 오늘 ({count_str})\n" + "\n".join(lines))
# 메일
if mails:
limit = 5
lines = []
for m in mails[:limit]:
sender = m.get("from", "").split("<")[0].strip().strip('"')
lines.append(f"- {sender}: {m.get('subject', '')}")
count_str = f"{len(mails)}"
if len(mails) > limit:
count_str = f"{limit}건, 외 {len(mails) - limit}"
sections.append(f"[메일] 주요 ({count_str})\n" + "\n".join(lines))
# 보고
if reports:
limit = 5
lines = []
for r in reports[:limit]:
cat = r.get("category", "")
desc = (r.get("description") or "")[:50]
created = r.get("created_at")
date_str = ""
if created:
if isinstance(created, datetime):
date_str = created.strftime("%m/%d")
else:
date_str = str(created)[:10]
entry = f"- [{cat}] {desc}"
if date_str:
entry += f" -- {date_str} 접수"
lines.append(entry)
count_str = f"{len(reports)}"
if len(reports) > limit:
count_str = f"{limit}건, 외 {len(reports) - limit}"
sections.append(f"[보고] 미해결 ({count_str})\n" + "\n".join(lines))
# 뉴스
if news:
lines = [f"- {n}" for n in news[:5]]
count_str = f"{len(news)}"
sections.append(f"[뉴스] ({count_str})\n" + "\n".join(lines))
if not sections:
return None
return header + "\n\n" + "\n\n".join(sections)
def send_briefing(text: str) -> None:
"""Synology Chat 웹훅으로 브리핑 전송."""
if not SYNOLOGY_CHAT_WEBHOOK_URL:
logger.warning("SYNOLOGY_CHAT_WEBHOOK_URL not set")
return
try:
httpx.post(
SYNOLOGY_CHAT_WEBHOOK_URL,
data={"payload": json.dumps({"text": text})},
verify=False,
timeout=10,
)
logger.info("Briefing sent to Synology Chat")
except Exception as e:
logger.error(f"Chat send failed: {e}")
def main():
logger.info("Morning briefing started")
since = datetime.now(KST) - timedelta(hours=24)
# 데이터 수집 (각각 독립, 실패해도 계속)
events = fetch_today_events() or None
mails = fetch_important_mails() or None
reports = fetch_open_reports() or None
news_articles = fetch_news(since)
news_lines = summarize_news(news_articles) if news_articles else None
text = format_briefing(events, mails, reports, news_lines)
if not text:
logger.info("No data for briefing — skipping")
return
send_briefing(text)
logger.info("Morning briefing complete")
if __name__ == "__main__":
main()

View File

@@ -109,7 +109,7 @@
},
{
"parameters": {
"jsCode": "function httpPost(url, body, { timeout = 15000, headers = {} } = {}) {\n return new Promise((resolve, reject) => {\n const data = JSON.stringify(body);\n const u = require('url').parse(url);\n const mod = require(u.protocol === 'https:' ? 'https' : 'http');\n const req = mod.request({\n hostname: u.hostname, port: u.port, path: u.path,\n method: 'POST',\n headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data), ...headers }\n }, (res) => {\n let body = '';\n res.on('data', c => body += c);\n res.on('end', () => {\n if (res.statusCode >= 400) return reject(new Error(url + ' \\u2192 ' + res.statusCode + ': ' + body.slice(0, 200)));\n try { resolve(JSON.parse(body)); } catch(e) { reject(new Error('JSON parse error: ' + body.slice(0, 200))); }\n });\n });\n req.on('error', reject);\n req.setTimeout(timeout, () => { req.destroy(); reject(new Error(url + ' \\u2192 timeout after ' + timeout + 'ms')); });\n req.write(data);\n req.end();\n });\n}\n\nfunction httpPut(url, body, { timeout = 10000, headers = {} } = {}) {\n return new Promise((resolve, reject) => {\n const data = JSON.stringify(body);\n const u = require('url').parse(url);\n const mod = require(u.protocol === 'https:' ? 'https' : 'http');\n const req = mod.request({\n hostname: u.hostname, port: u.port, path: u.path,\n method: 'PUT',\n headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data), ...headers }\n }, (res) => {\n let body = '';\n res.on('data', c => body += c);\n res.on('end', () => {\n if (res.statusCode >= 400) return reject(new Error(url + ' \\u2192 ' + res.statusCode + ': ' + body.slice(0, 200)));\n try { resolve(JSON.parse(body)); } catch(e) { reject(new Error('JSON parse error: ' + body.slice(0, 200))); }\n });\n });\n req.on('error', reject);\n req.setTimeout(timeout, () => { req.destroy(); reject(new Error(url + ' \\u2192 timeout after ' + timeout + 'ms')); });\n req.write(data);\n req.end();\n });\n}\n\nconst item = $input.first().json;\nconst embText = `${item.subject} ${item.summary}`;\n\ntry {\n const emb = await httpPost(`${$env.LOCAL_OLLAMA_URL}/api/embeddings`, { model: 'bge-m3', prompt: embText });\n if (emb.embedding) {\n const qdrantUrl = $env.QDRANT_URL || 'http://host.docker.internal:6333';\n await httpPut(`${qdrantUrl}/collections/documents/points`, { points: [{ id: Date.now(), vector: emb.embedding, payload: {\n text: embText, source: 'mail', from_address: item.from,\n mail_date: item.mailDate, label: item.label,\n created_at: new Date().toISOString()\n }}]});\n }\n} catch(e) {}\n\n// DEVONthink 저장 (graceful)\ntry {\n const dtUrl = $env.DEVONTHINK_BRIDGE_URL || 'http://host.docker.internal:8093';\n await httpPost(`${dtUrl}/save`, {\n title: item.subject, content: item.summary,\n type: 'markdown', tags: ['mail', item.label]\n }, { timeout: 5000 });\n} catch(e) {}\n\nreturn [{ json: { ...item, embedded: true } }];"
"jsCode": "function httpPost(url, body, { timeout = 15000, headers = {} } = {}) {\n return new Promise((resolve, reject) => {\n const data = JSON.stringify(body);\n const u = require('url').parse(url);\n const mod = require(u.protocol === 'https:' ? 'https' : 'http');\n const req = mod.request({\n hostname: u.hostname, port: u.port, path: u.path,\n method: 'POST',\n headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data), ...headers }\n }, (res) => {\n let body = '';\n res.on('data', c => body += c);\n res.on('end', () => {\n if (res.statusCode >= 400) return reject(new Error(url + ' \\u2192 ' + res.statusCode + ': ' + body.slice(0, 200)));\n try { resolve(JSON.parse(body)); } catch(e) { reject(new Error('JSON parse error: ' + body.slice(0, 200))); }\n });\n });\n req.on('error', reject);\n req.setTimeout(timeout, () => { req.destroy(); reject(new Error(url + ' \\u2192 timeout after ' + timeout + 'ms')); });\n req.write(data);\n req.end();\n });\n}\n\nfunction httpPut(url, body, { timeout = 10000, headers = {} } = {}) {\n return new Promise((resolve, reject) => {\n const data = JSON.stringify(body);\n const u = require('url').parse(url);\n const mod = require(u.protocol === 'https:' ? 'https' : 'http');\n const req = mod.request({\n hostname: u.hostname, port: u.port, path: u.path,\n method: 'PUT',\n headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data), ...headers }\n }, (res) => {\n let body = '';\n res.on('data', c => body += c);\n res.on('end', () => {\n if (res.statusCode >= 400) return reject(new Error(url + ' \\u2192 ' + res.statusCode + ': ' + body.slice(0, 200)));\n try { resolve(JSON.parse(body)); } catch(e) { reject(new Error('JSON parse error: ' + body.slice(0, 200))); }\n });\n });\n req.on('error', reject);\n req.setTimeout(timeout, () => { req.destroy(); reject(new Error(url + ' \\u2192 timeout after ' + timeout + 'ms')); });\n req.write(data);\n req.end();\n });\n}\n\nconst item = $input.first().json;\nconst embText = `${item.subject} ${item.summary}`;\n\ntry {\n const emb = await httpPost(`${$env.LOCAL_OLLAMA_URL}/api/embeddings`, { model: 'bge-m3', prompt: embText });\n if (emb.embedding) {\n const qdrantUrl = $env.QDRANT_URL || 'http://host.docker.internal:6333';\n await httpPut(`${qdrantUrl}/collections/documents/points`, { points: [{ id: Date.now(), vector: emb.embedding, payload: {\n text: embText, source: 'mail', from_address: item.from,\n mail_date: item.mailDate, label: item.label,\n created_at: new Date().toISOString()\n }}]});\n }\n} catch(e) {}\n\n// KB 저장 (graceful)\ntry {\n const kbUrl = $env.KB_WRITER_URL || 'http://host.docker.internal:8095';\n await httpPost(`${kbUrl}/save`, {\n title: item.subject, content: item.summary,\n type: 'note', tags: ['mail', item.label],\n username: 'mail-pipeline', source: 'mailplus', topic: item.label || 'mail'\n }, { timeout: 5000 });\n} catch(e) {}\n\nreturn [{ json: { ...item, embedded: true } }];"
},
"id": "m1000001-0000-0000-0000-000000000005",
"name": "Embed & Save",

View File

@@ -20,7 +20,7 @@ GPU_OLLAMA_URL = os.getenv("GPU_OLLAMA_URL", "http://192.168.1.186:11434")
LOCAL_OLLAMA_URL = os.getenv("LOCAL_OLLAMA_URL", "http://127.0.0.1:11434")
QDRANT_URL = os.getenv("QDRANT_URL", "http://127.0.0.1:6333")
SYNOLOGY_CHAT_WEBHOOK_URL = os.getenv("SYNOLOGY_CHAT_WEBHOOK_URL", "")
DEVONTHINK_BRIDGE_URL = os.getenv("DEVONTHINK_BRIDGE_URL", "http://127.0.0.1:8093")
KB_WRITER_URL = os.getenv("KB_WRITER_URL", "http://127.0.0.1:8095")
# Postgres 연결 (직접 접속)
PG_HOST = os.getenv("PG_HOST", "127.0.0.1")
@@ -179,21 +179,24 @@ def embed_to_qdrant(text: str) -> str | None:
return None
def save_to_devonthink(title: str, content: str) -> str | None:
"""DEVONthink에 저장."""
def save_to_kb(title: str, content: str) -> str | None:
"""kb_writer에 저장."""
try:
resp = httpx.post(
f"{DEVONTHINK_BRIDGE_URL}/save",
f"{KB_WRITER_URL}/save",
json={
"title": title,
"content": content,
"type": "markdown",
"type": "news",
"tags": ["news", "digest"],
"username": "news-digest",
"source": "karakeep",
"topic": "news",
},
timeout=10,
)
data = resp.json()
return data.get("uuid") if data.get("success") else None
return data.get("path") if data.get("success") else None
except Exception:
return None
@@ -258,7 +261,7 @@ def main():
emb_text = f"{result['title_ko']} {result['summary_ko']}"
qdrant_id = embed_to_qdrant(emb_text)
dt_uuid = save_to_devonthink(
kb_path = save_to_kb(
result["title_ko"],
f"**원문**: {bm['url']}\n**출처**: {bm.get('source', '')}\n\n{result['summary_ko']}",
)
@@ -268,9 +271,9 @@ def main():
try:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO news_digest_log (article_url,source,original_lang,title_ko,summary_ko,qdrant_id,devonthink_uuid) "
"INSERT INTO news_digest_log (article_url,source,original_lang,title_ko,summary_ko,qdrant_id,kb_path) "
"VALUES (%s,%s,%s,%s,%s,%s,%s) ON CONFLICT (article_url) DO NOTHING",
(bm["url"], bm.get("source", ""), lang, result["title_ko"], result["summary_ko"], qdrant_id, dt_uuid),
(bm["url"], bm.get("source", ""), lang, result["title_ko"], result["summary_ko"], qdrant_id, kb_path),
)
conn.commit()
except Exception as e: