Question 520
Following a proposed change in a cross-team productivity initiative at Oakridge Utilities, the regional operations team reports that it addressed the following need under restricted source access: A team wants the model to follow a new categorization style and has one carefully labeled example. Which observation provides the STRONGEST evidence that the change worked?
Explanation
Correct Answer
Held-out inputs follow the demonstrated label and formatting rule. This is the direct observable test of the intended response: Use one-shot prompting to demonstrate the desired input-output transformation. In the context of Oakridge Utilities's a cross-team productivity initiative, it checks an actual outcome instead of treating completion of a project ticket or an untested assertion as evidence.
Incorrect Answers
Sample answers maintain the specified tone without making unauthorized promises. This would instead validate the different situation: A drafting task needs responses in the voice of a cautious customer-support agent. It would ordinarily follow the action: Use role prompting with explicit scope, tone, and factual constraints. It does not establish that Oakridge Utilities solved the need specified in this scenario; the relevant check is: Held-out inputs follow the demonstrated label and formatting rule.
Schema validation passes across representative inputs or fails safely. This would instead validate the different situation: A downstream workflow fails when free-form model responses deviate from required fields. It would ordinarily follow the action: Specify a strict output schema and validate generated fields rather than trusting prose. It does not establish that Oakridge Utilities solved the need specified in this scenario; the relevant check is: Held-out inputs follow the demonstrated label and formatting rule.
Tool calls occur only when useful and answers cite returned evidence. This would instead validate the different situation: An assistant must decide whether it needs a search tool and then use the result to answer. It would ordinarily follow the action: Use a reason-and-act style workflow with explicitly permitted tools and observable action steps. It does not establish that Oakridge Utilities solved the need specified in this scenario; the relevant check is: Held-out inputs follow the demonstrated label and formatting rule.