Question 80
Following a proposed change in a cross-team productivity initiative at Eastshore Financial Services, the product operations group reports that it addressed the following need under restricted source access: A chatbot answers questions about events after its training cutoff as though it has verified current information. Which observation provides the STRONGEST evidence that the change worked?
Explanation
Correct Answer
Recent factual answers cite retrieved sources and identify cases when information is unavailable. This is the direct observable test of the intended response: Use an authorized, current grounding source and distinguish live evidence from model prior knowledge. In the context of Eastshore Financial Services's a cross-team productivity initiative, it checks an actual outcome instead of treating completion of a project ticket or an untested assertion as evidence.
Incorrect Answers
A pilot meets the task quality and cost threshold without an unnecessary model-training program. This would instead validate the different situation: A startup wants a capable language assistant quickly but has limited labeled examples and no budget to train a foundation model. It would ordinarily follow the action: Start from an existing suitable foundation model and adapt instructions or grounding before contemplating custom training. It does not establish that Eastshore Financial Services solved the need specified in this scenario; the relevant check is: Recent factual answers cite retrieved sources and identify cases when information is unavailable.
Tests with the longest permitted contracts retain relevant clauses and accurate citations. This would instead validate the different situation: A legal team must summarize contracts longer than the context accepted by its current model. It would ordinarily follow the action: Select an appropriate context window or use retrieval and chunking that preserve document provenance. It does not establish that Eastshore Financial Services solved the need specified in this scenario; the relevant check is: Recent factual answers cite retrieved sources and identify cases when information is unavailable.
Legal and security reviewers approve the intended data flow before any sensitive inputs are sent. This would instead validate the different situation: A consultant selects a foundation model without considering confidentiality of the input documents. It would ordinarily follow the action: Include data-handling terms, residency requirements, and security controls in model selection. It does not establish that Eastshore Financial Services solved the need specified in this scenario; the relevant check is: Recent factual answers cite retrieved sources and identify cases when information is unavailable.