Question 740
An audit of the proposed fix for Osprey Enterprise Software requires a specific controlled check. The issue under review is: A newly retrained model is promoted directly into production before any shadow comparison with current predictions. What measured outcome provides the strongest check that this problem is resolved?
Explanation
Correct Answer
Version-tagged shadow outputs demonstrate candidate quality and latency within the approved acceptance envelope. This controlled observation specifically validates the proposed remedy: Run shadow or offline-challenger evaluation against production-representative inputs prior to traffic promotion. It addresses the documented failure at Osprey Enterprise Software: A newly retrained model is promoted directly into production before any shadow comparison with current predictions.
Incorrect Answers
A committed transform update produces a new versioned pipeline run and parity checks cover the changed features. This is evidence for a different corrective change: Include preprocessing code changes in CT triggers and verify the pinned transformation artifact before training. That other change addresses: A feature transformation changes in the Git repository but the CT pipeline retrains using the old preprocessing image without any compatibility test. It does not validate the specific Osprey Enterprise Software remediation for: A newly retrained model is promoted directly into production before any shadow comparison with current predictions. The necessary check instead is: Version-tagged shadow outputs demonstrate candidate quality and latency within the approved acceptance envelope.
A rebuild from the release manifest reproduces the approved digest or a documented deterministic equivalent. This is evidence for a different corrective change: Build and sign immutable containers from recorded commits and dependencies for every promoted version. That other change addresses: A continuous training pipeline uses moving source branches and floating dependencies, making it impossible to rebuild a released artifact. It does not validate the specific Osprey Enterprise Software remediation for: A newly retrained model is promoted directly into production before any shadow comparison with current predictions. The necessary check instead is: Version-tagged shadow outputs demonstrate candidate quality and latency within the approved acceptance envelope.
A label-age audit confirms no pending transaction was silently assigned a negative target. This is evidence for a different corrective change: Restrict retraining labels to matured, verified outcomes and explicitly track the label delay window. That other change addresses: A fraud model is retrained daily with outcomes that mature after 30 days, causing recent unlabeled events to be treated as legitimate. It does not validate the specific Osprey Enterprise Software remediation for: A newly retrained model is promoted directly into production before any shadow comparison with current predictions. The necessary check instead is: Version-tagged shadow outputs demonstrate candidate quality and latency within the approved acceptance envelope.