spanscore · enterprise-rag-gold

dataset enterprise-rag-gold · 60 cases · 2026-07-09 18:23 UTC · vs baseline enterprise-rag-gold

Scorecard

recall@1
0.875
95% CI [0.792, 0.942] · n=60
recall@3
1.000
95% CI [1.000, 1.000] · n=60
recall@5
1.000
95% CI [1.000, 1.000] · n=60
mrr
0.956
95% CI [0.917, 0.992] · n=60
context_sufficiency
1.000
95% CI [1.000, 1.000] · n=60
faithfulness
0.983
95% CI [0.950, 1.000] · n=60
answer_correctness
0.988
95% CI [0.971, 1.000] · n=60
citation_coverage
0.931
95% CI [0.881, 0.972] · n=60
budget_compliance
1.000
95% CI [1.000, 1.000] · n=60

Gate vs baseline

MetricVerdictΔ95% CInDetail
recall@1 pass +0.0250 [+0.0000, +0.0667] 60 delta +0.0250
recall@3 pass +0.0667 [+0.0167, +0.1333] 60 delta +0.0667
mrr pass +0.0347 [+0.0083, +0.0681] 60 delta +0.0347
context_sufficiency pass +0.1583 [+0.0750, +0.2583] 60 delta +0.1583
faithfulness pass +0.0639 [+0.0167, +0.1306] 60 delta +0.0639
answer_correctness pass +0.0792 [+0.0208, +0.1500] 60 delta +0.0792
citation_coverage pass +0.0806 [+0.0027, +0.1667] 60 delta +0.0806

Cases

claim-deadline recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately states the requirement to file a claim within 30 days and does not contradict any information from the REFERENCE.
citation_coverage 0.500
budget_compliance 1.000
rag claim-deadline 5258.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claim-requirements recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer includes all the facts present in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag claim-requirements 4687.3 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claim-late recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the conditions under which a claim can still be filed after the 30-day deadline, matching the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag claim-late 4752.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claim-inspection recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag claim-inspection 4597.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claim-assessor recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag claim-assessor 3504.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claim-incomplete recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all the facts presented in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag claim-incomplete 4616.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payout-time recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer matches the REFERENCE exactly without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag payout-time 4541.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payout-late recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all the facts from the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag payout-late 4543.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payout-advance recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 0.500
budget_compliance 1.000
rag payout-advance 1577.6 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payout-currency recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag payout-currency 986.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

flood-standard recall@1 0.50 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.500
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag flood-standard 1016.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

home-deductible-premium recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag home-deductible-premium 1291.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

home-theft recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE by stating that theft after forced entry is covered, with no contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag home-theft 973.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

home-wear recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag home-wear 1052.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

auto-partial-casco recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer includes all the facts present in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag auto-partial-casco 1101.6 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

auto-deductible-full recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately states the deductible for full casco as 300 EUR per claim, matching the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag auto-deductible-full 1203.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

auto-glass recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag auto-glass 1049.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

auto-ridehailing recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE by stating that a business rider is required for ride-hailing, with no contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag auto-ridehailing 1073.3 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

travel-medical-limit recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the coverage amount stated in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag travel-medical-limit 1234.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

travel-cancellation recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the coverage amount and conditions stated in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag travel-cancellation 1628.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

travel-trip-length recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag travel-trip-length 1285.3 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

travel-missed-connection recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag travel-missed-connection 1102.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

flood-rider-deductible recall@1 0.00 recall@3 1.00 recall@5 1.00 mrr 0.50 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.000
recall@3 1.000
recall@5 1.000
mrr 0.500
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately states the deductible amount on the flood rider without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag flood-rider-deductible 952.8 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

flood-rider-waiting recall@1 0.50 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.500
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE by stating the correct waiting period without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag flood-rider-waiting 961.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

valuables-unlisted recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information provided in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag valuables-unlisted 1109.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

natural-hazards-zone recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag natural-hazards-zone 963.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

annual-discount recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag annual-discount 1062.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payment-failed recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 0.75 citation_coverage 0.33 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 0.750 The CANDIDATE correctly states the retry and suspension details but adds an unnecessary fact about policy term extension that is not in the REFERENCE.
citation_coverage 0.333
budget_compliance 1.000
rag payment-failed 1143.3 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

payment-methods recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately lists all the payment methods mentioned in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag payment-methods 1039.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

quarterly-discount recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag quarterly-discount 1105.8 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

cancel-refund recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all facts from the REFERENCE without any contradictions.
citation_coverage 0.500
budget_compliance 1.000
rag cancel-refund 1357.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

overpayment-small recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag overpayment-small 1244.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

no-claims-bonus-max recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately states the maximum no-claims discount and correctly describes how it is earned, matching the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag no-claims-bonus-max 1191.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

no-claims-after-claim recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 0.75 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 0.750 The CANDIDATE correctly states that a single claim reduces the bonus by two steps but includes additional information about earning a discount that is not present in the REFERENCE.
citation_coverage 1.000
budget_compliance 1.000
rag no-claims-after-claim 1465.8 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

data-location recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag data-location 1186.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

claims-retention recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag claims-retention 1102.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

call-recordings recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag call-recordings 1316.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

data-export-time recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE, providing the same fact without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag data-export-time 931.2 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

support-hours-week recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all the information provided in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag support-hours-week 1050.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

support-saturday recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag support-saturday 1004.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

burst-pipe-night recall@1 0.50 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 0.75 citation_coverage 0.50 budget_compliance 1.00
recall@1 0.500
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 0.750 The CANDIDATE correctly states that a burst pipe can be reported via the emergency line 24 hours a day, but it includes additional information about filing a formal claim that is not present in the REFERENCE.
citation_coverage 0.500
budget_compliance 1.000
rag burst-pipe-night 1452.2 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

renewal-notice recall@1 0.50 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.500
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag renewal-notice 3819.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

address-change recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the main fact from the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag address-change 865.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

downgrade-when recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag downgrade-when 995.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

cancel-after-claim recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag cancel-after-claim 1133.9 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

cooling-off recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE answer fully matches the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag cooling-off 1218.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

cancel-by-phone recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag cancel-by-phone 1049.2 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

complaint-response recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all the facts presented in the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag complaint-response 1034.2 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

ombudsman-limit recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information provided in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag ombudsman-limit 1220.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

ombudsman-limitation recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag ombudsman-limitation 5602.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

bundle-discount recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag bundle-discount 1206.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

alarm-discount recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the information in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag alarm-discount 1083.7 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

telematics-saving recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the fact from the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag telematics-saving 1112.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

burst-pipe-first recall@1 0.00 recall@3 1.00 recall@5 1.00 mrr 0.50 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.000
recall@3 1.000
recall@5 1.000
mrr 0.500
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately presents all the facts from the REFERENCE without any contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag burst-pipe-first 1231.0 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

break-in-police recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the REFERENCE by stating that a police report number is required for the theft claim after a break-in, with no contradictions.
citation_coverage 1.000
budget_compliance 1.000
rag break-in-police 1164.4 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

abroad-receipts recall@1 1.00 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 0.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 1.000
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 0.000 The context states that reimbursement without receipts is capped at 250 EUR, but it does not specify that this applies to emergency treatment abroad.
answer_correctness 1.000 The CANDIDATE accurately reflects the main fact from the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag abroad-receipts 1064.8 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

what-is-deductible recall@1 0.00 recall@3 1.00 recall@5 1.00 mrr 0.33 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 0.000
recall@3 1.000
recall@5 1.000
mrr 0.333
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the definition of a deductible as stated in the REFERENCE and does not contain any contradictions.
citation_coverage 0.500
budget_compliance 1.000
rag what-is-deductible 1723.5 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

what-is-casco recall@1 0.00 recall@3 1.00 recall@5 1.00 mrr 0.50 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 1.00 budget_compliance 1.00
recall@1 0.000
recall@3 1.000
recall@5 1.000
mrr 0.500
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects the definition of casco as provided in the REFERENCE without any contradictions or omissions.
citation_coverage 1.000
budget_compliance 1.000
rag what-is-casco 1316.8 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

deductible-home-vs-auto recall@1 0.50 recall@3 1.00 recall@5 1.00 mrr 1.00 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 0.500
recall@3 1.000
recall@5 1.000
mrr 1.000
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately states the deductibles for both the premium home tier and full casco, confirming that the premium home tier has the lower deductible without any contradictions.
citation_coverage 0.500
budget_compliance 1.000
rag deductible-home-vs-auto 1321.2 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error

suspend-vs-cancel recall@1 0.00 recall@3 1.00 recall@5 1.00 mrr 0.50 context_sufficiency 1.00 faithfulness 1.00 answer_correctness 1.00 citation_coverage 0.50 budget_compliance 1.00
recall@1 0.000
recall@3 1.000
recall@5 1.000
mrr 0.500
context_sufficiency 1.000
faithfulness 1.000
answer_correctness 1.000 The CANDIDATE accurately reflects all the facts presented in the REFERENCE without any contradictions.
citation_coverage 0.500
budget_compliance 1.000
rag suspend-vs-cancel 1407.1 ms
retrieval 0.0 ms
chat gpt-4o-mini 0.0 ms

llm retrieval tool error