Question 50
At Willowbrook Education, a team reports that it corrected the following problem affecting student collaboration platforms: A retrieval-augmented assistant treats instructions embedded in a webpage as authority to reveal private records. Which observation BEST verifies that the correction actually worked?
Explanation
Correct Answer
Injected document instructions cannot trigger private-data disclosure or unapproved tools. This is the decisive evidence for the original problem: A retrieval-augmented assistant treats instructions embedded in a webpage as authority to reveal private records. It verifies that the intended corrective control—Treat retrieved content as untrusted data and enforce authorization and tool policy outside the model—works in the relevant environment; a completed change ticket alone would not prove the outcome at Willowbrook Education.
Incorrect Answers
Poisoned or superseded records are excluded and answers link to the authorized version. This would be relevant evidence for a different failure: An assistant repeats a poisoned document's false policy because its retrieval index lacks source integrity metadata, usually addressed by Validate source provenance, signed approvals, and current revision before incorporating retrieved text. It does not demonstrate that the current problem at Willowbrook Education was resolved. The specific required evidence is: Injected document instructions cannot trigger private-data disclosure or unapproved tools.
The connector executes only after provenance and dependency checks pass. This would be relevant evidence for a different failure: A model agent installs a third-party model connector from an unreviewed source with unknown dependencies, usually addressed by Require signed or verified dependencies, supply-chain scanning, and restricted runtime permissions. It does not demonstrate that the current problem at Willowbrook Education was resolved. The specific required evidence is: Injected document instructions cannot trigger private-data disclosure or unapproved tools.
Attempted command execution is denied while authorized summaries remain functional. This would be relevant evidence for a different failure: A language-model plugin can invoke arbitrary shell commands although the approved purpose is simple file summarization, usually addressed by Remove general execution privileges and expose only a narrow read-only interface with audited scope. It does not demonstrate that the current problem at Willowbrook Education was resolved. The specific required evidence is: Injected document instructions cannot trigger private-data disclosure or unapproved tools.