Question 500
An operator wants to compare old and new model outputs on live traffic without showing unapproved responses to customers. What pattern should be considered?
Explanation
Correct Answer
Side-by-side shadow outputs provide rollout evidence without transferring the customer-facing decision to an unapproved model.
Incorrect Answers
Review safety configuration and test policy outcomes for the exact new deployment version. It addresses another case: A provider model update changes the default content filters unexpectedly. For that case, the distinguishing justification or acceptance check is: Security behavior can change between versions and must be explicitly validated before the replacement is promoted. The present case instead states: An operator wants to compare old and new model outputs on live traffic without showing unapproved responses to customers. It calls for Run a supported shadow evaluation that does not use candidate outputs for customer responses. Showing success for the other case would not establish: Side-by-side shadow outputs provide rollout evidence without transferring the customer-facing decision to an unapproved model.
The alternative, Record project, deployment resource, model version, and request correlation identifiers, belongs to a different technical condition: The same deployment name is reused across projects, and incident responders cannot tell which endpoint produced a complaint. There the source-specific mechanism or proof is: Deployment provenance must uniquely identify the service involved rather than rely only on a repeated friendly name. The condition asked about here is: An operator wants to compare old and new model outputs on live traffic without showing unapproved responses to customers. A suitable correction is Run a supported shadow evaluation that does not use candidate outputs for customer responses; the targeted justification or result is: Side-by-side shadow outputs provide rollout evidence without transferring the customer-facing decision to an unapproved model.
Compare quality and token-cost metrics on representative workloads before full rollout. That operation would be relevant if: A model upgrade improves answer quality but triples average token use, breaking the monthly budget. Its corresponding reason or verification is: Model improvements must satisfy both quality objectives and financial service constraints for the application. However the problem in this question is: An operator wants to compare old and new model outputs on live traffic without showing unapproved responses to customers. The requirement instead calls for Run a supported shadow evaluation that does not use candidate outputs for customer responses; the question-specific check is: Side-by-side shadow outputs provide rollout evidence without transferring the customer-facing decision to an unapproved model.