Question 490
The reliability team at Stonewell Payments is closing a corrective-action ticket for this ML failure: A batch-scored insurance dataset writes results to ephemeral worker storage, so outputs disappear before the compliance team can audit them. Which independently observable result best verifies that the remediation works?
Explanation
Correct Answer
The batch manifest reconciles input and output row counts, and authorized reviewers can retrieve the results after worker termination. This controlled observation specifically validates the proposed remedy: Persist batch inference results and provenance to a governed durable storage sink with an output manifest. It addresses the documented failure at Stonewell Payments: A batch-scored insurance dataset writes results to ephemeral worker storage, so outputs disappear before the compliance team can audit them.
Incorrect Answers
Equivalent raw samples yield the same numerical feature vector through offline and online paths. This is evidence for a different corrective change: Reuse the exact frozen training preprocessing artifact in the serving preprocessing path. That other change addresses: A model uses standardization at training but its online prediction handler applies a newly calculated per-request mean, changing feature semantics. It does not validate the specific Stonewell Payments remediation for: A batch-scored insurance dataset writes results to ephemeral worker storage, so outputs disappear before the compliance team can audit them. The necessary check instead is: The batch manifest reconciles input and output row counts, and authorized reviewers can retrieve the results after worker termination.
A simulated latency breach restores the approved version within the agreed recovery target. This is evidence for a different corrective change: Keep a previous version available and implement a tested, reversible model deployment rollback procedure. That other change addresses: A new model's latency doubles after rollout, but the release automation has no validated route back to the previously approved build. It does not validate the specific Stonewell Payments remediation for: A batch-scored insurance dataset writes results to ephemeral worker storage, so outputs disappear before the compliance team can audit them. The necessary check instead is: The batch manifest reconciles input and output row counts, and authorized reviewers can retrieve the results after worker termination.
Throughput per accelerator rises while per-request latency stays within the target percentile. This is evidence for a different corrective change: Benchmark bounded inference batching with a latency-aware maximum batch size and safe request mapping. That other change addresses: A text embedding service makes one GPU call per small request and wastes device capacity while queue time remains low. It does not validate the specific Stonewell Payments remediation for: A batch-scored insurance dataset writes results to ephemeral worker storage, so outputs disappear before the compliance team can audit them. The necessary check instead is: The batch manifest reconciles input and output row counts, and authorized reviewers can retrieve the results after worker termination.