Question 120
Following a proposed change in a cross-team productivity initiative at Yellowstone Education, the regional operations team reports that it addressed the following need under restricted source access: Scanned documents arrive as images but a retrieval pipeline assumes they contain machine-readable text. Which observation provides the STRONGEST evidence that the change worked?
Explanation
Correct Answer
Search tests find facts from formerly image-only pages with provenance back to originals. This is the direct observable test of the intended response: Add appropriate extraction and format validation before embedding the documents. In the context of Yellowstone Education's a cross-team productivity initiative, it checks an actual outcome instead of treating completion of a project ticket or an untested assertion as evidence.
Incorrect Answers
Coverage tests demonstrate that the required policy sections are present and retrievable. This would instead validate the different situation: A knowledge corpus has missing policy sections that users expect the assistant to answer from. It would ordinarily follow the action: Measure completeness and repair missing authoritative records before relying on the corpus. It does not establish that Yellowstone Education solved the need specified in this scenario; the relevant check is: Search tests find facts from formerly image-only pages with provenance back to originals.
Retrieval evaluations show more task-relevant references rather than just a larger index. This would instead validate the different situation: A team adds thousands of old off-topic pages hoping raw corpus size will improve answers. It would ordinarily follow the action: Filter and curate source material for relevance to the approved use cases. It does not establish that Yellowstone Education solved the need specified in this scenario; the relevant check is: Search tests find facts from formerly image-only pages with provenance back to originals.
The dataset inventory flags actual target labels rather than merely descriptive input features. This would instead validate the different situation: An analyst confuses labeled outcome examples with unlabeled transaction logs. It would ordinarily follow the action: Identify which examples contain verified target labels before choosing a supervised approach. It does not establish that Yellowstone Education solved the need specified in this scenario; the relevant check is: Search tests find facts from formerly image-only pages with provenance back to originals.