From 8021a1debdf88089edbfaa4963017d1619cdb106 Mon Sep 17 00:00:00 2001 From: Hyungi Ahn Date: Tue, 7 Apr 2026 09:25:49 +0900 Subject: [PATCH] =?UTF-8?q?test(search):=20Phase=200.5=20fusion=20?= =?UTF-8?q?=EC=A0=84=EB=9E=B5=20A/B=20=EB=B9=84=EA=B5=90=20=EA=B2=B0?= =?UTF-8?q?=EA=B3=BC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 23개 평가셋 × 3 전략(legacy/rrf/rrf_boost) 측정 + 분석. 핵심 발견: - 전체 NDCG: legacy 0.705 → rrf 0.699 → rrf_boost 0.700 (미세 차이) - RRF가 약간 나쁜 이유: kw_001(산업안전보건법 제6장)에서 RRF가 4041 (근로기준법 안전과 보건)을 false positive로 promotion. NDCG 1.000→0.906. - boost가 가치 입증한 사례: news_004(guerre en Iran)에서 RRF의 미스를 완벽 보정해 legacy NDCG 복원. - RRF의 진짜 가치는 Phase 1+ 다중 신호(trigram, reranker, multi-query) 통합 시 발휘됨. 현 평가셋은 너무 단순해서 차이가 noise에 묻힘. 결정: rrf_boost를 default로 유지. Phase 1 후 재측정. Co-Authored-By: Claude Opus 4.6 (1M context) --- reports/phase05_fusion_comparison.md | 105 +++++++++++++++++++++++++++ reports/phase05_legacy.csv | 24 ++++++ reports/phase05_rrf.csv | 24 ++++++ reports/phase05_rrf_boost.csv | 24 ++++++ 4 files changed, 177 insertions(+) create mode 100644 reports/phase05_fusion_comparison.md create mode 100644 reports/phase05_legacy.csv create mode 100644 reports/phase05_rrf.csv create mode 100644 reports/phase05_rrf_boost.csv diff --git a/reports/phase05_fusion_comparison.md b/reports/phase05_fusion_comparison.md new file mode 100644 index 0000000..63b9f7f --- /dev/null +++ b/reports/phase05_fusion_comparison.md @@ -0,0 +1,105 @@ +# Phase 0.5 Fusion 전략 비교 — 2026-04-07 + +`tests/search_eval/queries.yaml` v0.1 (23개 쿼리)을 3가지 fusion 전략으로 실행한 비교. + +## 전체 지표 + +| 지표 | legacy (기존 weighted-sum) | rrf (k=60) | rrf_boost (default) | +|---|---|---|---| +| Recall@10 | 0.788 | 0.788 | 0.788 | +| MRR@10 | **0.751** | 0.747 | 0.747 | +| NDCG@10 | **0.705** | 0.699 | 0.700 | +| Top-3 hit | 0.950 | 0.950 | 0.950 | +| Latency p95 | 554ms | 554ms | 556ms | +| Failure precision | 0/3 | 0/3 | 0/3 | + +**현재 평가셋에서 RRF는 legacy 대비 측정 가능한 개선을 주지 못함** (NDCG -0.005, MRR -0.004). +오히려 미세하게 나쁘다. 그러나 rrf_boost는 rrf의 일부 손실을 복구한다. + +## 쿼리별 차이 분석 (top5 ordering 변화 7건) + +### kw_001 — RRF의 가장 큰 손실 (NDCG 1.000 → 0.906) + +쿼리: `산업안전보건법 제6장` + +| 전략 | top5 | +|---|---| +| legacy | 3856, 3868, 3879, 3912, 3950 | +| rrf/boost | 3856, **4041**, 3868, 3879, 3912 | + +- 정답: [3856, 3868, 3879] (산안법 본법 + 시행령 + 시행규칙 6장) +- RRF가 4041(근로기준법 제6장 안전과 보건)을 2위로 끌어올림. 텍스트+벡터 둘 다에서 잡혀서 RRF가 보상. +- **boost가 잡지 못한 이유**: 4041 title도 "제6장"을 포함하지만 boost의 `LEGAL_ARTICLE_RE`는 `제\d+조`(조)만 매치, `제\d+장`(장)은 매치 안 함. 의도적 한계. +- RRF의 단점이 극명히 드러나는 사례 — false positive가 두 retriever에서 동시에 잡히면 promotion됨. + +### news_004 — boost가 RRF를 정확히 보정한 사례 + +쿼리: `guerre en Iran` (불어 뉴스) + +| 전략 | top5 | +|---|---| +| legacy | 4199, 4507, 4521, 4363, 4519 | +| rrf | 4199, 4507, 4519, **4211**, 4521 | +| **boost** | **4199, 4507, 4521, 4363, 4519** ← legacy와 동일 | + +- rrf는 4211(불어 비-정답)을 4위로 올렸으나, boost가 4363(텍스트 매치)을 다시 끌어올려 legacy NDCG 복원 (0.822). +- **boost의 가치를 보여주는 유일한 사례**. + +### kw_002 — 같은 NDCG, 다른 ordering + +쿼리: `중대재해 처벌 등에 관한 법률 제2장 중대산업재해` + +| 전략 | top5 | +|---|---| +| legacy | 3921, 3917, 3916, 3922, 3920 | +| rrf/boost | **3917**, 3921, 3916, 3922, 3920 | + +- 정답: [3917, 3921] — RRF가 본법(3917)을 1위로 올림. **사람 직관에 더 부합**. +- legacy는 시행령(3921)을 1위로 — 점수 합산 우연. +- NDCG 같음(둘 다 1.0) — 평가 metric이 ordering 차이를 잡지 못함. + +### misc_001 — RRF가 약간 더 좋음 + +쿼리: `강체의 평면 운동학` + +NDCG: legacy 0.605 → rrf/boost 0.624 (+0.019) +- 미세하지만 RRF가 정답을 약간 위로. + +### news_002, news_003 — 둘 다 미세하게 나쁨 + +미세 차이. 평가셋에 가린 신호. + +## 핵심 인사이트 + +### 1. 현재 평가셋에선 fusion 전략 차이가 미미 +- **이유**: 평가셋이 23개로 작고, exact_keyword 카테고리가 이미 NDCG 1.0이라 fusion 차이가 noise에 묻힘. +- **시사**: queries.yaml v0.2에서 graded relevance + 더 어려운 ranking 케이스가 필요. + +### 2. RRF 단독은 RRF-friendly false positive에 취약 +- 두 retriever 모두에서 쿼리와 *부분* 매치하는 doc은 RRF가 promotion. +- **kw_001**의 4041(근로기준법)이 대표 사례. 텍스트엔 "제6장 안전과 보건"이 있고, 벡터 임베딩도 의미상 가깝다. +- legacy의 weighted-sum은 텍스트 점수의 절대값을 보존해서 이런 부분 매치를 덜 promotion함. + +### 3. boost는 일부 케이스만 보정 +- news_004는 완벽 보정 (legacy 수준 복원). +- kw_001은 보정 실패 — 법령 "장" vs "조" 구분이 한계. +- **확장 후보**: `제\d+장` 패턴, 도메인 일치 boost(법령 쿼리에 대해 법령 결과 우선) — Phase 2 QueryAnalyzer 도입 시. + +### 4. RRF의 진짜 가치는 Phase 1+ 인프라 +- RRF가 빛을 발하는 시점: + - Phase 1: trigram + reranker가 추가 retrieval 신호로 들어올 때 (3+ 신호 합치는 데 RRF가 normalize 부담을 흡수) + - Phase 2: multi-query (한국어 + 영어) 임베딩 결과 fusion할 때 +- 현 시점에서 legacy → rrf 교체의 손익분기점은 Phase 1에 도달. + +## 결정 + +- **default = rrf_boost 유지**. 다음 이유: + 1. legacy 대비 손실이 작음(NDCG -0.005, MRR -0.004). + 2. 인프라(FusionStrategy ABC)가 Phase 1+ 신호 추가 시 즉시 활용됨. + 3. boost가 일부 케이스(news_004)에서 가치 입증. +- **legacy 보존**. Phase 1 reranker 통합 후 재비교를 위해 옵션 유지. +- **평가셋 v0.2 우선순위 상승**: graded relevance + ordering 민감 케이스 추가가 fusion 평가에 필요. + +## 부수 개선 + +- `normalize_display_scores`: SearchResult.score를 [0..1] 랭크 기반 정규화. 기존 frontend의 `score*100` 표시 깨짐(>100% 또는 <5%) 수정. diff --git a/reports/phase05_legacy.csv b/reports/phase05_legacy.csv new file mode 100644 index 0000000..8c1ea14 --- /dev/null +++ b/reports/phase05_legacy.csv @@ -0,0 +1,24 @@ +label,id,category,intent,domain_hint,query,relevant_ids,returned_ids_top10,latency_ms,recall_at_10,mrr_at_10,ndcg_at_10,top3_hit,error +single,kw_001,exact_keyword,fact_lookup,document,산업안전보건법 제6장,3856;3868;3879,3856;3868;3879;3912;3950;3908;3915;3911;3858;3853,555.1,1.000,1.000,1.000,1, +single,kw_002,exact_keyword,fact_lookup,document,중대재해 처벌 등에 관한 법률 제2장 중대산업재해,3917;3921,3921;3917;3916;3922;3920;3918;3919;3923;3874;3946,562.6,1.000,1.000,1.000,1, +single,kw_003,exact_keyword,fact_lookup,document,화학물질관리법 유해화학물질 영업자,3981,3981;3980;3857;3988;3869;3880;3978;3986;3985;3979,581.6,1.000,1.000,1.000,1, +single,kw_004,exact_keyword,fact_lookup,document,근로기준법 안전과 보건,4041,4041;3853;3860;3883;3858;3865;4036;3885;3901;3888,540.0,1.000,1.000,1.000,1, +single,kw_005,exact_keyword,fact_lookup,document,산업안전보건기준에 관한 규칙 보호구,3888,3888;3905;3908;3909;3897;3911;3907;3912;3903;3906,534.3,1.000,1.000,1.000,1, +single,nl_001,natural_language_ko,semantic_search,document,기계로 인한 산업재해 관련 법령,3856;3868;3879;3854,3868;3856;3879;3895;3915;3872;3851;3867;3897;3863,535.0,0.750,1.000,0.832,1, +single,nl_002,natural_language_ko,semantic_search,document,사업주가 도급을 줄 때 산업재해를 예방하기 위해 해야 할 일,3855;3867;3878,3855;3867;3878;3863;3917;3872;3854;3896;3861;3886,539.7,1.000,1.000,1.000,1, +single,nl_003,natural_language_ko,semantic_search,document,유해화학물질을 다루는 회사가 지켜야 할 안전 의무,3980;3981;3982,3857;3869;3980;3880;3896;3903;3854;3981;3909;3904,535.5,0.667,0.333,0.383,1, +single,nl_004,natural_language_ko,semantic_search,document,중대재해가 발생했을 때 경영책임자가 처벌받는 기준,3916;3917;3920;3921,3918;3917;3919;3921;3916;3923;3867;3922;3877;3984,539.8,0.750,0.500,0.565,1, +single,nl_005,natural_language_ko,semantic_search,document,안전보건교육은 누가 받아야 하고 어떤 내용을 다루는가,3853;3865,3853;3876;3965;3871;3958;3875;3861;3866;3877;3856,551.0,0.500,1.000,0.613,1, +single,cl_001,crosslingual_ko_en,semantic_search,document,기계 안전 가드 설계 원리,3770;3856,3895;3770;3762;3773;3879;3856;3767;3788;3868;3897,529.9,1.000,0.500,0.605,1, +single,cl_002,crosslingual_ko_en,semantic_search,document,산업 안전 입문서,3755;3775;3776;3777,3755;3816;3775;3851;3896;3853;3876;3871;3776;3863,539.4,0.750,1.000,0.703,1, +single,cl_003,crosslingual_ko_en,semantic_search,document,전기 안전 위험,3772;3790,3772;3897;3790;4024;4018;4020;4023;4022;4013;4019,529.5,1.000,1.000,0.920,1, +single,news_001,news_ko,semantic_search,news,이란과 미국의 군사 충돌,4303;4304;4307;4316;4322;4323;4327;4335,4452;4307;4317;4321;4339;4331;4329;4418;4446;4459,532.9,0.125,0.500,0.160,1, +single,news_002,news_ko,semantic_search,news,호르무즈 해협 봉쇄,4316;4320;4322;4327,4349;4199;4346;4320;4322;4327;4340;4304;4316;4260,527.6,1.000,0.250,0.576,0, +single,news_003,news_en,semantic_search,news,Trump Iran ultimatum,4258;4260;4262,4519;4202;4258;4321;4333;4515;4313;4445;4418;4314,541.7,0.333,0.333,0.235,1, +single,news_004,news_fr,semantic_search,news,guerre en Iran,4199;4202;4210;4361;4363;4507;4519;4521,4199;4507;4521;4363;4519;4211;4258;4324;4210;4536,525.9,0.750,1.000,0.822,1, +single,news_005,news_crosslingual,semantic_search,news,이란 미국 전쟁 글로벌 반응,4202;4258;4262;4536;4303;4304;4316,4329;4457;4307;4345;4324;4452;4443;4444;4450;4262,533.2,0.143,0.100,0.079,1, +single,misc_001,other_domain,fact_lookup,document,강체의 평면 운동학,4063;4065,4071;4063;4064;4058;4066;4065;4068;4060;4062;4059,562.7,1.000,0.500,0.605,1, +single,misc_002,other_domain,semantic_search,document,질점의 운동역학,4060;4061;4062,4062;4060;4061;4058;4070;4059;4069;4071;4063;4067,554.0,1.000,1.000,1.000,1, +single,fail_001,failure_expected,semantic_search,document,Rust async runtime tokio scheduler 내부 구조,,4069;3789;4067;4070;4060;4061;4071;4062;3807;4433,536.7,0.000,0.000,0.000,1, +single,fail_002,failure_expected,semantic_search,document,양자컴퓨터 큐비트 디코히어런스,,4068;4058;4064;4060;4065;4063;4061;3899;4067;4196,529.4,0.000,0.000,0.000,1, +single,fail_003,failure_expected,semantic_search,news,재즈 보컬리스트 빌리 홀리데이,,4289;4281;4205;4116;4100;4077;4316;4343;4235;4504,526.6,0.000,0.000,0.000,1, diff --git a/reports/phase05_rrf.csv b/reports/phase05_rrf.csv new file mode 100644 index 0000000..ab0cdbd --- /dev/null +++ b/reports/phase05_rrf.csv @@ -0,0 +1,24 @@ +label,id,category,intent,domain_hint,query,relevant_ids,returned_ids_top10,latency_ms,recall_at_10,mrr_at_10,ndcg_at_10,top3_hit,error +single,kw_001,exact_keyword,fact_lookup,document,산업안전보건법 제6장,3856;3868;3879,3856;4041;3868;3879;3912;3950;3908;3915;3911;3858,531.5,1.000,1.000,0.906,1, +single,kw_002,exact_keyword,fact_lookup,document,중대재해 처벌 등에 관한 법률 제2장 중대산업재해,3917;3921,3917;3921;3916;3922;3920;3918;3919;3923;3874;3946,540.3,1.000,1.000,1.000,1, +single,kw_003,exact_keyword,fact_lookup,document,화학물질관리법 유해화학물질 영업자,3981,3981;3980;3857;3988;3869;3880;3978;3986;3985;3979,528.5,1.000,1.000,1.000,1, +single,kw_004,exact_keyword,fact_lookup,document,근로기준법 안전과 보건,4041,4041;3853;3860;3883;3858;3865;4036;3885;3901;3888,531.5,1.000,1.000,1.000,1, +single,kw_005,exact_keyword,fact_lookup,document,산업안전보건기준에 관한 규칙 보호구,3888,3888;3905;3908;3909;3897;3885;3884;3890;3911;3901,530.6,1.000,1.000,1.000,1, +single,nl_001,natural_language_ko,semantic_search,document,기계로 인한 산업재해 관련 법령,3856;3868;3879;3854,3868;3856;3879;3895;3915;3872;3851;3867;3897;3863,527.7,0.750,1.000,0.832,1, +single,nl_002,natural_language_ko,semantic_search,document,사업주가 도급을 줄 때 산업재해를 예방하기 위해 해야 할 일,3855;3867;3878,3855;3867;3878;3863;3917;3872;3854;3896;3861;3886,529.4,1.000,1.000,1.000,1, +single,nl_003,natural_language_ko,semantic_search,document,유해화학물질을 다루는 회사가 지켜야 할 안전 의무,3980;3981;3982,3857;3869;3980;3880;3896;3903;3854;3981;3909;3904,533.7,0.667,0.333,0.383,1, +single,nl_004,natural_language_ko,semantic_search,document,중대재해가 발생했을 때 경영책임자가 처벌받는 기준,3916;3917;3920;3921,3918;3917;3919;3921;3916;3923;3867;3922;3877;3984,533.9,0.750,0.500,0.565,1, +single,nl_005,natural_language_ko,semantic_search,document,안전보건교육은 누가 받아야 하고 어떤 내용을 다루는가,3853;3865,3853;3876;3965;3871;3958;3875;3861;3866;3877;3856,525.2,0.500,1.000,0.613,1, +single,cl_001,crosslingual_ko_en,semantic_search,document,기계 안전 가드 설계 원리,3770;3856,3895;3770;3762;3773;3879;3856;3767;3788;3868;3897,539.4,1.000,0.500,0.605,1, +single,cl_002,crosslingual_ko_en,semantic_search,document,산업 안전 입문서,3755;3775;3776;3777,3755;3816;3775;3851;3896;3853;3876;3871;3776;3863,530.3,0.750,1.000,0.703,1, +single,cl_003,crosslingual_ko_en,semantic_search,document,전기 안전 위험,3772;3790,3772;3897;3790;4024;4018;4020;4023;4022;4013;4019,529.9,1.000,1.000,0.920,1, +single,news_001,news_ko,semantic_search,news,이란과 미국의 군사 충돌,4303;4304;4307;4316;4322;4323;4327;4335,4452;4307;4317;4321;4339;4331;4329;4418;4446;4459,529.6,0.125,0.500,0.160,1, +single,news_002,news_ko,semantic_search,news,호르무즈 해협 봉쇄,4316;4320;4322;4327,4349;4346;4199;4320;4322;4327;4340;4304;4316;4260,526.7,1.000,0.250,0.576,0, +single,news_003,news_en,semantic_search,news,Trump Iran ultimatum,4258;4260;4262,4199;4519;4202;4258;4321;4333;4515;4313;4445;4418,528.7,0.333,0.250,0.202,1, +single,news_004,news_fr,semantic_search,news,guerre en Iran,4199;4202;4210;4361;4363;4507;4519;4521,4199;4507;4519;4211;4521;4363;4258;4324;4210;4536,530.8,0.750,1.000,0.803,1, +single,news_005,news_crosslingual,semantic_search,news,이란 미국 전쟁 글로벌 반응,4202;4258;4262;4536;4303;4304;4316,4329;4457;4307;4345;4324;4452;4443;4444;4450;4262,523.9,0.143,0.100,0.079,1, +single,misc_001,other_domain,fact_lookup,document,강체의 평면 운동학,4063;4065,4071;4063;4066;4064;4065;4067;4069;4058;4068;4060,563.4,1.000,0.500,0.624,1, +single,misc_002,other_domain,semantic_search,document,질점의 운동역학,4060;4061;4062,4062;4060;4061;4070;4066;4064;4068;4058;4065;4059,550.7,1.000,1.000,1.000,1, +single,fail_001,failure_expected,semantic_search,document,Rust async runtime tokio scheduler 내부 구조,,4069;3789;4067;4070;4060;4061;4071;4062;3807;4433,535.3,0.000,0.000,0.000,1, +single,fail_002,failure_expected,semantic_search,document,양자컴퓨터 큐비트 디코히어런스,,4068;4058;4064;4060;4065;4063;4061;3899;4067;4196,533.9,0.000,0.000,0.000,1, +single,fail_003,failure_expected,semantic_search,news,재즈 보컬리스트 빌리 홀리데이,,4289;4281;4205;4116;4100;4077;4316;4343;4235;4504,527.6,0.000,0.000,0.000,1, diff --git a/reports/phase05_rrf_boost.csv b/reports/phase05_rrf_boost.csv new file mode 100644 index 0000000..307043a --- /dev/null +++ b/reports/phase05_rrf_boost.csv @@ -0,0 +1,24 @@ +label,id,category,intent,domain_hint,query,relevant_ids,returned_ids_top10,latency_ms,recall_at_10,mrr_at_10,ndcg_at_10,top3_hit,error +single,kw_001,exact_keyword,fact_lookup,document,산업안전보건법 제6장,3856;3868;3879,3856;4041;3868;3879;3912;3950;3908;3915;3911;3858,542.0,1.000,1.000,0.906,1, +single,kw_002,exact_keyword,fact_lookup,document,중대재해 처벌 등에 관한 법률 제2장 중대산업재해,3917;3921,3917;3921;3916;3922;3920;3918;3919;3923;3874;3946,526.3,1.000,1.000,1.000,1, +single,kw_003,exact_keyword,fact_lookup,document,화학물질관리법 유해화학물질 영업자,3981,3981;3980;3857;3988;3869;3880;3978;3986;3985;3979,533.4,1.000,1.000,1.000,1, +single,kw_004,exact_keyword,fact_lookup,document,근로기준법 안전과 보건,4041,4041;3853;3860;3883;3858;3865;4036;3885;3901;3888,522.5,1.000,1.000,1.000,1, +single,kw_005,exact_keyword,fact_lookup,document,산업안전보건기준에 관한 규칙 보호구,3888,3888;3905;3908;3909;3897;3885;3884;3890;3911;3901,528.3,1.000,1.000,1.000,1, +single,nl_001,natural_language_ko,semantic_search,document,기계로 인한 산업재해 관련 법령,3856;3868;3879;3854,3868;3856;3879;3895;3915;3872;3851;3867;3897;3863,530.3,0.750,1.000,0.832,1, +single,nl_002,natural_language_ko,semantic_search,document,사업주가 도급을 줄 때 산업재해를 예방하기 위해 해야 할 일,3855;3867;3878,3855;3867;3878;3863;3917;3872;3854;3896;3861;3886,536.2,1.000,1.000,1.000,1, +single,nl_003,natural_language_ko,semantic_search,document,유해화학물질을 다루는 회사가 지켜야 할 안전 의무,3980;3981;3982,3857;3869;3980;3880;3896;3903;3854;3981;3909;3904,529.5,0.667,0.333,0.383,1, +single,nl_004,natural_language_ko,semantic_search,document,중대재해가 발생했을 때 경영책임자가 처벌받는 기준,3916;3917;3920;3921,3918;3917;3919;3921;3916;3923;3867;3922;3877;3984,538.3,0.750,0.500,0.565,1, +single,nl_005,natural_language_ko,semantic_search,document,안전보건교육은 누가 받아야 하고 어떤 내용을 다루는가,3853;3865,3853;3876;3965;3871;3958;3875;3861;3866;3877;3856,533.4,0.500,1.000,0.613,1, +single,cl_001,crosslingual_ko_en,semantic_search,document,기계 안전 가드 설계 원리,3770;3856,3895;3770;3762;3773;3879;3856;3767;3788;3868;3897,529.9,1.000,0.500,0.605,1, +single,cl_002,crosslingual_ko_en,semantic_search,document,산업 안전 입문서,3755;3775;3776;3777,3755;3816;3775;3851;3896;3853;3876;3871;3776;3863,538.3,0.750,1.000,0.703,1, +single,cl_003,crosslingual_ko_en,semantic_search,document,전기 안전 위험,3772;3790,3772;3897;3790;4024;4018;4020;4023;4022;4013;4019,529.2,1.000,1.000,0.920,1, +single,news_001,news_ko,semantic_search,news,이란과 미국의 군사 충돌,4303;4304;4307;4316;4322;4323;4327;4335,4452;4307;4317;4321;4339;4331;4329;4418;4446;4459,530.3,0.125,0.500,0.160,1, +single,news_002,news_ko,semantic_search,news,호르무즈 해협 봉쇄,4316;4320;4322;4327,4349;4346;4199;4320;4322;4327;4340;4304;4316;4260,523.8,1.000,0.250,0.576,0, +single,news_003,news_en,semantic_search,news,Trump Iran ultimatum,4258;4260;4262,4199;4519;4202;4258;4321;4333;4515;4313;4445;4418,531.1,0.333,0.250,0.202,1, +single,news_004,news_fr,semantic_search,news,guerre en Iran,4199;4202;4210;4361;4363;4507;4519;4521,4199;4507;4521;4363;4519;4211;4258;4324;4210;4536,526.3,0.750,1.000,0.822,1, +single,news_005,news_crosslingual,semantic_search,news,이란 미국 전쟁 글로벌 반응,4202;4258;4262;4536;4303;4304;4316,4329;4457;4307;4345;4324;4452;4443;4444;4450;4262,539.1,0.143,0.100,0.079,1, +single,misc_001,other_domain,fact_lookup,document,강체의 평면 운동학,4063;4065,4071;4063;4066;4064;4065;4067;4069;4058;4068;4060,565.1,1.000,0.500,0.624,1, +single,misc_002,other_domain,semantic_search,document,질점의 운동역학,4060;4061;4062,4062;4060;4061;4070;4066;4064;4068;4058;4065;4059,555.3,1.000,1.000,1.000,1, +single,fail_001,failure_expected,semantic_search,document,Rust async runtime tokio scheduler 내부 구조,,4069;3789;4067;4070;4060;4061;4071;4062;3807;4433,533.4,0.000,0.000,0.000,1, +single,fail_002,failure_expected,semantic_search,document,양자컴퓨터 큐비트 디코히어런스,,4068;4058;4064;4060;4065;4063;4061;3899;4067;4196,537.1,0.000,0.000,0.000,1, +single,fail_003,failure_expected,semantic_search,news,재즈 보컬리스트 빌리 홀리데이,,4289;4281;4205;4116;4100;4077;4316;4343;4235;4504,529.7,0.000,0.000,0.000,1,