# Document Server 검색 평가셋 v0.2 — graded relevance baseline
#
# Phase 1 산출물 (peppy-hugging-nest.md Phase 1) — Phase 2 모델 점검과 Phase 3
# Markdown 전환의 효과를 객관 측정할 도구 마련. graded relevance 1년 미작성
# (document-server-2026-05-21.md:24) gap 해소.
#
# v0.2 schema 변경:
#   - category         : 사용자 권장 7 카테고리
#                        (standards / korean_only / english_only / mixed /
#                         exam / ocr_derived / failure_expected)
#   - legacy_category  : v0.1 카테고리 보존 (호환)
#   - language         : ko / en / mixed
#   - ocr_derived      : marker/OCR pipeline 통한 chunk 포함 여부
#   - graded_relevance : {doc_id: grade}
#                        (0=irrelevant / 1=marginal / 2=relevant / 3=highly)
#   - failure_expected : 결과 0건 expected (별도 flag)
#
# v0.1 호환:
#   - relevant_ids / top3_ids 는 deprecated 가 아니라 호환용 (--mode v0.1 재현)
#   - run_eval.py --mode v0.1 에서 옛 점수 ±0.001 재현 가능
#
# graded 등급 기준:
#   - 3 : highly relevant (top-3 강제, direct match — 본법/본 chapter/본 article)
#   - 2 : relevant       (정답 후보, 보조 정답, top-10 within OK)
#   - 1 : marginal       (약한 관련, top-10 within = 가점, 밖 = 무감점)
#   - 0 : irrelevant     (검색 결과 잡히면 감점 후보, 현재는 표시만)

version: "0.2"
created_at: "2026-05-23"
corpus_size: 753  # v0.1 기준. PR-Eval-V0_2-Baseline-Analysis 에서 갱신
notes: |
  Phase 1 plan: ~/.claude/plans/phase-1-graded-eval-v0-2.md
  본 PR (PR-Eval-V0_2-Schema-Harness) = v0.1 23 case → v0.2 23 case schema swap
  + run_eval graded 함수 + README. 신규 28 case + baseline 박제는 후속 PR.

queries:
  # ─────────────────────────────────────────────────────────
  # 1. standards (법령/규칙/기준) — v0.1 exact_keyword 흡수
  # ─────────────────────────────────────────────────────────
  - id: kw_001
    query: "산업안전보건법 제6장"
    category: standards
    legacy_category: exact_keyword
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3856: 3
      3868: 2
      3879: 2
    relevant_ids: [3856, 3868, 3879]
    top3_ids: [3856]
    notes: |
      "유해·위험 기계 등에 대한 조치"가 들어있는 6장.
      Act(3856) = grade 3 (본법, top-3 강제).
      Decree(3868)/Rule(3879) = grade 2 (보조 정답, 본법 우선).

  - id: kw_002
    query: "중대재해 처벌 등에 관한 법률 제2장 중대산업재해"
    category: standards
    legacy_category: exact_keyword
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3917: 3
      3921: 2
    relevant_ids: [3917, 3921]
    top3_ids: [3917]
    notes: |
      본법 제2장(3917) = grade 3.
      시행령 동일 장(3921) = grade 2.

  - id: kw_003
    query: "화학물질관리법 유해화학물질 영업자"
    category: standards
    legacy_category: exact_keyword
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3981: 3
    relevant_ids: [3981]
    top3_ids: [3981]
    notes: 화학물질관리법 제4장 = 유해화학물질 영업자 (3981, grade 3).

  - id: kw_004
    query: "근로기준법 안전과 보건"
    category: standards
    legacy_category: exact_keyword
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4041: 3
    relevant_ids: [4041]
    top3_ids: [4041]
    notes: 근로기준법 제6장 = 안전과 보건 (4041, grade 3).

  - id: kw_005
    query: "산업안전보건기준에 관한 규칙 보호구"
    category: standards
    legacy_category: exact_keyword
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3888: 3
    relevant_ids: [3888]
    top3_ids: [3888]
    notes: 산업안전보건기준 규칙 제4장 = 보호구 (3888, grade 3).

  # ─────────────────────────────────────────────────────────
  # 2. korean_only — v0.1 natural_language_ko + news_ko + other_domain 흡수
  # ─────────────────────────────────────────────────────────
  - id: nl_001
    query: "기계로 인한 산업재해 관련 법령"
    category: korean_only
    legacy_category: natural_language_ko
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3856: 3
      3868: 2
      3879: 2
      3854: 1
    relevant_ids: [3856, 3868, 3879, 3854]
    top3_ids: [3856]
    notes: |
      플랜의 대표 예시 쿼리. 기계 안전 = 산안법 6장(3856, grade 3).
      Decree(3868)/Rule(3879) = grade 2.
      4장 유해·위험 방지 조치(3854) = grade 1 (의미상 약한 관련).

  - id: nl_002
    query: "사업주가 도급을 줄 때 산업재해를 예방하기 위해 해야 할 일"
    category: korean_only
    legacy_category: natural_language_ko
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3855: 3
      3867: 2
      3878: 2
    relevant_ids: [3855, 3867, 3878]
    top3_ids: [3855]
    notes: 산안법 제5장 도급 시 산업재해 예방. Act(3855) = grade 3, Decree(3867)/Rule(3878) = grade 2.

  - id: nl_003
    query: "유해화학물질을 다루는 회사가 지켜야 할 안전 의무"
    category: korean_only
    legacy_category: natural_language_ko
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3980: 2
      3981: 2
      3982: 2
    relevant_ids: [3980, 3981, 3982]
    notes: |
      화관법 제3-5장(유해화학물질 관리/영업자/사고 대응). 셋 다 동등하게 정답 후보.
      특정 chapter 강제 없음 → grade 2 셋 (top3_ids 없음).

  - id: nl_004
    query: "중대재해가 발생했을 때 경영책임자가 처벌받는 기준"
    category: korean_only
    legacy_category: natural_language_ko
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3917: 3
      3916: 2
      3920: 2
      3921: 2
    relevant_ids: [3916, 3917, 3920, 3921]
    top3_ids: [3917]
    notes: 중대재해처벌법 본법+시행령 제1-2장. 제2장(3917) = grade 3, 나머지 = grade 2.

  - id: nl_005
    query: "안전보건교육은 누가 받아야 하고 어떤 내용을 다루는가"
    category: korean_only
    legacy_category: natural_language_ko
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3853: 3
      3865: 2
    relevant_ids: [3853, 3865]
    top3_ids: [3853]
    notes: 산안법 제3장 안전보건교육 (3853, grade 3) + 시행령 제3장 (3865, grade 2).

  - id: news_001
    query: "이란과 미국의 군사 충돌"
    category: korean_only
    legacy_category: news_ko
    intent: semantic_search
    domain_hint: news
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4303: 2
      4304: 2
      4307: 2
      4316: 2
      4322: 2
      4323: 2
      4327: 2
      4335: 2
    relevant_ids: [4303, 4304, 4307, 4316, 4322, 4323, 4327, 4335]
    notes: |
      경향신문의 이란-미국 전쟁 보도. recall 위주 평가, 동등 정답 8건 (grade 2).
      diversity 제약 적용 후에도 최소 5건은 top-10에 들어와야 함.

  - id: news_002
    query: "호르무즈 해협 봉쇄"
    category: korean_only
    legacy_category: news_ko
    intent: semantic_search
    domain_hint: news
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4316: 3
      4320: 2
      4322: 2
      4327: 2
    relevant_ids: [4316, 4320, 4322, 4327]
    top3_ids: [4316]
    notes: 호르무즈 해협 직접 언급 기사. 4316 = grade 3 (direct), 나머지 = grade 2.

  - id: misc_001
    query: "강체의 평면 운동학"
    category: korean_only
    legacy_category: other_domain
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4063: 3
      4065: 2
    relevant_ids: [4063, 4065]
    top3_ids: [4063]
    notes: 공업역학 동역학 ch16(4063, grade 3), ch18(4065, grade 2).

  - id: misc_002
    query: "질점의 운동역학"
    category: korean_only
    legacy_category: other_domain
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4060: 2
      4061: 2
      4062: 2
    relevant_ids: [4060, 4061, 4062]
    notes: 공업역학 동역학 ch13~15 (질점 운동역학). 동등 정답 3건 (grade 2).

  # ─────────────────────────────────────────────────────────
  # 3. english_only — v0.1 news_en 흡수
  # ─────────────────────────────────────────────────────────
  - id: news_003
    query: "Trump Iran ultimatum"
    category: english_only
    legacy_category: news_en
    intent: semantic_search
    domain_hint: news
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4258: 2
      4260: 2
      4262: 2
    relevant_ids: [4258, 4260, 4262]
    notes: Der Spiegel 영어판 Iran 관련 기사. 동등 정답 3건 (grade 2).

  # ─────────────────────────────────────────────────────────
  # 4. mixed — v0.1 crosslingual_ko_en + news_fr + news_crosslingual 흡수
  # ─────────────────────────────────────────────────────────
  - id: cl_001
    query: "기계 안전 가드 설계 원리"
    category: mixed
    legacy_category: crosslingual_ko_en
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3770: 3
      3856: 2
    relevant_ids: [3770, 3856]
    top3_ids: [3770]
    notes: |
      Industrial Safety and Health Management(7-ED) Ch15 Machine Guarding(3770, grade 3)이
      한국어 쿼리로 검색되어야 함. 한국 산안법 6장(3856) = grade 2.

  - id: cl_002
    query: "산업 안전 입문서"
    category: mixed
    legacy_category: crosslingual_ko_en
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3755: 2
      3775: 2
      3776: 2
      3777: 2
    relevant_ids: [3755, 3775, 3776, 3777]
    notes: |
      Safety and Health for Engineers / Industrial Safety and Health Management
      영문 교재 입문 챕터들이 한국어 쿼리로 검색되어야 함. 동등 정답 4건 (grade 2).

  - id: cl_003
    query: "전기 안전 위험"
    category: mixed
    legacy_category: crosslingual_ko_en
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3772: 2
      3790: 2
    relevant_ids: [3772, 3790]
    notes: |
      Electrical Hazards(3772), Electrical Safety(3790) 영문 챕터.
      한국어 안전기준 규칙 중 전기 관련 장도 있을 수 있음(보수적으로 영문만 정답).

  - id: news_004
    query: "guerre en Iran"
    category: mixed
    legacy_category: news_fr
    intent: semantic_search
    domain_hint: news
    language: mixed  # 정확히는 fr 이지만 v0.2 enum (ko/en/mixed) 에서 mixed 매핑
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4199: 2
      4202: 2
      4210: 2
      4361: 2
      4363: 2
      4507: 2
      4519: 2
      4521: 2
    relevant_ids: [4199, 4202, 4210, 4361, 4363, 4507, 4519, 4521]
    notes: |
      Le Monde 불어 Iran 전쟁 보도. 동등 정답 8건 (grade 2).
      불어 query 라 v0.2 의 ko/en/mixed enum 에서는 mixed 매핑 (정확히는 fr).
      후속 PR 에서 language enum 확장 검토.

  - id: news_005
    query: "이란 미국 전쟁 글로벌 반응"
    category: mixed
    legacy_category: news_crosslingual
    intent: semantic_search
    domain_hint: news
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4202: 2
      4258: 2
      4262: 2
      4536: 2
      4303: 2
      4304: 2
      4316: 2
    relevant_ids: [4202, 4258, 4262, 4536, 4303, 4304, 4316]
    notes: |
      한국어 쿼리로 한/영/불/독 뉴스가 골고루 검색되어야 함. 동등 정답 7건 (grade 2).
      Phase 1 domain-aware retrieval + multilingual embedding 효과 측정용.
      diversity 제약(국가당 max 2)이 동작하면 최소 4개국 이상 노출.

  # ─────────────────────────────────────────────────────────
  # 5. failure_expected — 결과 0건 expected
  # ─────────────────────────────────────────────────────────
  - id: fail_001
    query: "Rust async runtime tokio scheduler 내부 구조"
    category: failure_expected
    legacy_category: failure_expected
    intent: semantic_search
    domain_hint: document
    language: mixed  # 한국어 + 영문 keyword 혼합
    ocr_derived: false
    failure_expected: true
    graded_relevance: {}
    relevant_ids: []
    notes: |
      코퍼스에 Rust/프로그래밍 문서 없음. 결과 0건 = correct.
      Phase 0.3 search_failure_logs로 자동 수집되어야 함.
      Phase 1+에서 confidence 점수 < 0.5로 분류되는지 확인.

  - id: fail_002
    query: "양자컴퓨터 큐비트 디코히어런스"
    category: failure_expected
    legacy_category: failure_expected
    intent: semantic_search
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: true
    graded_relevance: {}
    relevant_ids: []
    notes: 코퍼스에 양자물리 문서 없음. 결과 0건 = correct.

  - id: fail_003
    query: "재즈 보컬리스트 빌리 홀리데이"
    category: failure_expected
    legacy_category: failure_expected
    intent: semantic_search
    domain_hint: news
    language: ko
    ocr_derived: false
    failure_expected: true
    graded_relevance: {}
    relevant_ids: []
    notes: 코퍼스에 음악/재즈 문서 없음. 결과 0건 = correct.

  # ═════════════════════════════════════════════════════════
  # PR-Eval-V0_2-Baseline-Analysis 신규 28 case (2026-05-23)
  # 분배 (계획 plan 의 +28):
  #   standards +6 / english_only +8 / mixed +5 / exam +7 / failure_expected +2
  #   ocr_derived 0 — TBD-O FAILED (chunks.source = RSS feed 명,
  #   extract_meta / md_extraction_engine 전부 NULL → OCR 식별 불가).
  #   재배분: standards +1 / english_only +2 / mixed +1 → analysis md 명시.
  # ═════════════════════════════════════════════════════════

  # ─────────────────────────────────────────────────────────
  # 1b. standards 신규 6 (한국 KGS code + 산안기준 후반 편 + 고압가스법)
  # ─────────────────────────────────────────────────────────
  - id: kw_006
    query: "산업안전보건기준에 관한 규칙 작업장 통로"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3886: 3
      3887: 2
    relevant_ids: [3886, 3887]
    top3_ids: [3886]
    notes: 산안기준 규칙 1편 제2장 작업장(3886, grade 3) + 제3장 통로(3887, grade 2).

  - id: kw_007
    query: "산업안전보건기준 폭발 화재 위험물 누출 방지"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3896: 3
      3766: 1
    relevant_ids: [3896, 3766]
    top3_ids: [3896]
    notes: |
      산안기준 규칙 2편 제2장 폭발ㆍ화재(3896) = grade 3.
      Industrial Safety and Health Management ch11 Flammable and Explosive(3766)
      = grade 1 (영문 도서, 주제 일치 but 한국 법령 query → cross-lingual marginal).

  - id: kw_008
    query: "고압가스 안전관리법 전문"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4025: 3
      4026: 2
    relevant_ids: [4025, 4026]
    top3_ids: [4025]
    notes: 고압가스법 본법(4025, grade 3) + 시행령(4026, grade 2). "전문" 키워드 매칭.

  - id: kw_009
    query: "KGS FP111 가스설비 배관설비 기준"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      13305: 3
    relevant_ids: [13305]
    top3_ids: [13305]
    notes: KGS FP111 § 2.4~2.5 가스설비·배관설비(13305) = grade 3. KGS code exact lookup.

  - id: kw_010
    query: "KGS FU551 가스설비 압력조정기 가스계량기"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      13652: 3
    relevant_ids: [13652]
    top3_ids: [13652]
    notes: KGS FU551 § 2.4 가스설비 기준(13652) = grade 3. KGS code exact lookup.

  - id: kw_011
    query: "산업안전보건기준 전기로 인한 위험 방지"
    category: standards
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3897: 3
      3772: 1
    relevant_ids: [3897, 3772]
    top3_ids: [3897]
    notes: |
      산안기준 규칙 2편 제3장 전기 위험 방지(3897) = grade 3.
      Industrial Safety and Health Management ch17 Electrical Hazards(3772) = grade 1
      (영문, 주제 일치).

  # ─────────────────────────────────────────────────────────
  # 3b. english_only 신규 8 (Pressure Vessel Design Manual + ASME VIII/IX
  #     + Hydrogen ASME + Industrial Safety 영문 교재 + Structural Analysis)
  # ─────────────────────────────────────────────────────────
  - id: en_001
    query: "pressure vessel flange design"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5144: 3
      5136: 2
    relevant_ids: [5144, 5136]
    top3_ids: [5144]
    notes: |
      Pressure Vessel Design Manual ch03 Flange Design(5144) = grade 3.
      Manual_00 contents(5136) = grade 2 (catalog).

  - id: en_002
    query: "ASME Section VIII introduction"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5204: 3
      5206: 2
    relevant_ids: [5204, 5206]
    top3_ids: [5204]
    notes: ASME VIII Introduction(5204) = grade 3, Design_Basic_R1(5206) = grade 2.

  - id: en_003
    query: "impact test requirements ASME"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5205: 3
      5148: 1
    relevant_ids: [5205, 5148]
    top3_ids: [5205]
    notes: |
      ASME VIII Impact Test(5205) = grade 3. Structural Analysis Materials of
      Construction(5148) = grade 1 (재료 관련 약한 연관).

  - id: en_004
    query: "design of vessel supports"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5149: 3
    relevant_ids: [5149]
    top3_ids: [5149]
    notes: Pressure Vessel Design Manual ch04 Design of Vessel Supports(5149) = grade 3.

  - id: en_005
    query: "hydrogen piping ASME code"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5178: 3
    relevant_ids: [5178]
    top3_ids: [5178]
    notes: Hydrogen_Piping_and_Pipelines_ASME_Code(5178) = grade 3. exact title match.

  - id: en_006
    query: "ASME welding qualification requirements"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5209: 3
      3771: 1
    relevant_ids: [5209, 3771]
    top3_ids: [5209]
    notes: |
      ASME IX Welding(5209) = grade 3.
      Industrial Safety and Health Management ch16 Welding(3771) = grade 1
      (안전 관점 영문 도서, qualification 주제 약한 연관).

  - id: en_007
    query: "pressure vessel fabrication and inspection"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5208: 3
      5145: 2
    relevant_ids: [5208, 5145]
    top3_ids: [5208]
    notes: |
      ASME VIII Fabrication-Inspection_R1(5208) = grade 3.
      Structural Analysis ch02 Selection of Vessel/Specifications(5145) = grade 2.

  - id: en_008
    query: "Industrial Safety and Health Management ergonomics"
    category: english_only
    intent: semantic_search
    domain_hint: document
    language: en
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      3763: 3
      3755: 1
    relevant_ids: [3763, 3755]
    top3_ids: [3763]
    notes: |
      Industrial Safety and Health Management(7-ED) ch08 Ergonomics(3763) = grade 3.
      Contents(3755) = grade 1 (목차 매칭, 본 챕터 우선).

  # ─────────────────────────────────────────────────────────
  # 4b. mixed 신규 5 (양언어 ASME / 한국 KGS ↔ 영문 ASME / 영문 query → 한국 doc)
  # ─────────────────────────────────────────────────────────
  - id: cl_004
    query: "ASME 압력용기 설계 실무"
    category: mixed
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5207: 3
      5210: 3
      5139: 2
      5135: 2
    relevant_ids: [5207, 5210, 5139, 5135]
    top3_ids: [5207]
    notes: |
      한국어 query → 영문/혼합 ASME 도서.
      ASME VIII Design_Practical_R1(5207, en) = grade 3 (실무 정확 일치).
      ASME B-PV CODE SECTION Ⅷ Div.1 압력용기 코드 2007(5210, mixed title) = grade 3.
      압력용기 핸드북_설계와 제작(5139, ko) = grade 2.
      압력용기 핸드북_기하학과 배치도(5135, ko) = grade 2.

  - id: cl_005
    query: "ASME 용접 코드 해설"
    category: mixed
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5224: 3
      5209: 2
    relevant_ids: [5224, 5209]
    top3_ids: [5224]
    notes: |
      asme_code해설[용접](5224, ko 해설) = grade 3 (직접 일치).
      ASME IX Welding(5209, en) = grade 2 (원본 코드).

  - id: cl_006
    query: "pressure vessel Korean industrial safety regulation"
    category: mixed
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      4025: 2
      3856: 2
      5136: 1
    relevant_ids: [4025, 3856, 5136]
    notes: |
      영문 query → 한국 법령. 고압가스 안전관리법(4025, ko) = grade 2,
      산안법 6장(3856, ko) = grade 2, PV Design Manual(5136, en) = grade 1
      (보조 영문 자료). 동등 정답 2건, top3 강제 없음.

  - id: cl_007
    query: "KGS 코드 LPG safety standard"
    category: mixed
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11647: 3
      11689: 2
      11645: 2
      4025: 1
    relevant_ids: [11647, 11689, 11645, 4025]
    top3_ids: [11647]
    notes: |
      혼합 keyword (KGS + LPG + safety standard).
      04_KGS_Code(11647, ko 개요) = grade 3.
      02_KGS_FU_사용(11689) / 02_LPG법(11645) = grade 2.
      고압가스 안전관리법(4025) = grade 1 (관련 법령).

  - id: cl_008
    query: "수소 가스 안전 기준 hydrogen safety"
    category: mixed
    intent: semantic_search
    domain_hint: document
    language: mixed
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      5178: 3
      5169: 2
    relevant_ids: [5178, 5169]
    top3_ids: [5178]
    notes: |
      Hydrogen_Piping_and_Pipelines_ASME_Code(5178, en) = grade 3.
      수소인프라 이슈 및 과제(5169, ko) = grade 2.
      한국 KGS 수소 관련 별도 doc 부재 → 영문 우선 확정.

  # ─────────────────────────────────────────────────────────
  # 6. exam 신규 7 (가스기사 study_questions → library 개념 docs)
  # ─────────────────────────────────────────────────────────
  - id: exam_001
    query: "레이놀즈수 정의 공식"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11504: 3
      11505: 2
    relevant_ids: [11504, 11505]
    top3_ids: [11504]
    notes: |
      study_questions 2019-1회 #2 "레이놀즈수를 옳게 나타낸 것은?" 매핑.
      01_레이놀즈수(11504) = grade 3 (직접 정답 개념).
      02_층류와난류(11505) = grade 2 (연관 개념).

  - id: exam_002
    query: "탱크 바닥 구멍 유체 유속 토리첼리"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11500: 3
      11495: 2
      11496: 2
    relevant_ids: [11500, 11495, 11496]
    top3_ids: [11500]
    notes: |
      study_questions 2019-1회 #1 "수면의 높이가 10m로 일정한 탱크 바닥 5mm 구멍 유속"
      매핑. 01_토리첼리(11500) = grade 3. 베르누이 개념/식(11495/11496) = grade 2.

  - id: exam_003
    query: "이상기체 음속 마하수"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11514: 3
      11515: 2
    relevant_ids: [11514, 11515]
    top3_ids: [11514]
    notes: |
      study_questions 2019-1회 #4 "이상기체 속에서의 음속을 옳게 나타낸 식" 매핑.
      01_음속과마하수(11514) = grade 3. 02_등엔트로피유동(11515) = grade 2.

  - id: exam_004
    query: "고압가스 용기 내압시험 영구증가량"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11591: 3
      11644: 2
      11691: 2
    relevant_ids: [11591, 11644, 11691]
    top3_ids: [11591]
    notes: |
      study_questions 2019-1회 #58 "고압가스 용접용기 내압검사 전증가량 250 mL
      영구증가량" 매핑. 02_고압가스용기(11591) = grade 3.
      01_고압가스안전관리법(11644) / 04_KGS_AC_용기(11691) = grade 2 (법령/용기 코드).

  - id: exam_005
    query: "도시가스 배관 매설 이격거리"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11627: 3
      11625: 2
      11646: 1
    relevant_ids: [11627, 11625, 11646]
    top3_ids: [11627]
    notes: |
      study_questions 2019-1회 #76 "고압가스일반제조 사업소 밖 배관 매몰 설치
      매설물 최소 이격거리" 매핑. 06_도시가스배관매설(11627) = grade 3.
      04_도시가스배관(11625) = grade 2. 03_도시가스사업법(11646) = grade 1.

  - id: exam_006
    query: "LPG 저장탱크 안전거리 분말소화기"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11617: 3
      11669: 2
      11620: 1
    relevant_ids: [11617, 11669, 11620]
    top3_ids: [11617]
    notes: |
      study_questions 2019-1회 #69 (LPG지상저장탱크 거리) + #80 (소형저장탱크
      분말소화기) 매핑. 03_LPG저장설비(11617) = grade 3.
      01_LPG안전(11669) = grade 2. 06_LPG사용설비(11620) = grade 1.

  - id: exam_007
    query: "오리피스 차압식 유량계"
    category: exam
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: false
    graded_relevance:
      11712: 3
      11711: 2
      11503: 2
    relevant_ids: [11712, 11711, 11503]
    top3_ids: [11712]
    notes: |
      가스기사 계측 분야 (study_questions 회차 빈출). 02_오리피스유량계(11712) =
      grade 3. 01_차압식유량계(11711) = grade 2. 04_오리피스(11503, 유체역학)
      = grade 2 (원리 설명).

  # ─────────────────────────────────────────────────────────
  # 5b. failure_expected 신규 2 (의도적 no-result)
  # ─────────────────────────────────────────────────────────
  - id: fail_004
    query: "KGS AC999 임의 가스 코드"
    category: failure_expected
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: true
    graded_relevance: {}
    relevant_ids: []
    notes: |
      KGS AC 시리즈 (용기) 에 999 번호 부재. 의도적 no-result.
      AC101/AC201/AC213 등 실재 번호와 혼동 회피 위해 999 선택.

  - id: fail_005
    query: "초전도 안전 관리법 시행규칙"
    category: failure_expected
    intent: fact_lookup
    domain_hint: document
    language: ko
    ocr_derived: false
    failure_expected: true
    graded_relevance: {}
    relevant_ids: []
    notes: 존재하지 않는 가짜 법령명. 결과 0건 = correct.
