Question 740
An enterprise knowledge index is updated weekly, but its evaluation results compare answers without tracking document snapshots. What must be versioned?
Explanation
Correct Answer
Document changes can alter expected evidence; reproducible A/B comparisons require immutable retrieval-source versions.
Incorrect Answers
The alternative, Score evidence relevance and version agreement for each retrieved citation, belongs to a different technical condition: A support agent returns accurate facts but often cites passages that discuss a different product release. There the source-specific mechanism or proof is: Answers can be broadly correct yet inappropriate for the requested release when sources refer to other versions. The condition asked about here is: An enterprise knowledge index is updated weekly, but its evaluation results compare answers without tracking document snapshots. A suitable correction is Capture corpus and index snapshot identifiers with each evaluation run; the targeted justification or result is: Document changes can alter expected evidence; reproducible A/B comparisons require immutable retrieval-source versions.
Apply predefined safety guardrails and withhold promotion until risk thresholds are satisfied. That operation would be relevant if: Variant B yields better answer relevance but causes a sharp increase in unsafe responses. Its corresponding reason or verification is: Quality gains do not justify deployment when safety regression violates the experiment's acceptance criteria. However the problem in this question is: An enterprise knowledge index is updated weekly, but its evaluation results compare answers without tracking document snapshots. The requirement instead calls for Capture corpus and index snapshot identifiers with each evaluation run; the question-specific check is: Document changes can alter expected evidence; reproducible A/B comparisons require immutable retrieval-source versions.
Measure citation-to-claim support and source attribution correctness separately from answer fluency. A separate source case supports that approach when: A generated answer quotes a policy correctly but cites an unrelated page. The supporting rationale or observable result is: A fluent correct-looking response is not properly grounded when the cited document does not support its statement. This question describes a different issue: An enterprise knowledge index is updated weekly, but its evaluation results compare answers without tracking document snapshots. Address it through Capture corpus and index snapshot identifiers with each evaluation run, with the required justification or evidence: Document changes can alter expected evidence; reproducible A/B comparisons require immutable retrieval-source versions.