Question 140
The acceptance reviewer at Riverton Finance needs independent proof of the correction to this reported ML problem: An account fraud model joins labels to activity from after the prediction timestamp, accidentally giving the model future knowledge. What measured outcome provides the strongest check that this problem is resolved?
Explanation
Correct Answer
A temporal audit finds zero post-cutoff feature values in training and online replay samples. This controlled observation specifically validates the proposed remedy: Implement point-in-time feature joins that exclude signals produced after each prediction cutoff. It addresses the documented failure at Riverton Finance: An account fraud model joins labels to activity from after the prediction timestamp, accidentally giving the model future knowledge.
Incorrect Answers
Historical replay retrieves status as it was at each event time, rather than the latest account record. This is evidence for a different corrective change: Use event-time or point-in-time feature retrieval against effective-dated feature values. That other change addresses: A recommendation model joins current customer status to old events, creating implausible historical feature values from updated records. It does not validate the specific Riverton Finance remediation for: An account fraud model joins labels to activity from after the prediction timestamp, accidentally giving the model future knowledge. The necessary check instead is: A temporal audit finds zero post-cutoff feature values in training and online replay samples.
A deliberate category-type change triggers the rejection path while schema-valid data reaches the curated table. This is evidence for a different corrective change: Validate type, required fields, and allowed categories at ingestion and quarantine incompatible batches. That other change addresses: A product classification ingestion job silently coerces the string category column into null after upstream schema changes. It does not validate the specific Riverton Finance remediation for: An account fraud model joins labels to activity from after the prediction timestamp, accidentally giving the model future knowledge. The necessary check instead is: A temporal audit finds zero post-cutoff feature values in training and online replay samples.
A batch loader retrieves images by reference with valid labels and no redundant large inline binary columns. This is evidence for a different corrective change: Keep image objects in Cloud Storage and maintain governed labels and object references in a queryable catalog. That other change addresses: An image classification team stores training images directly in BigQuery rows as base64 strings, making batch loading inefficient and costly. It does not validate the specific Riverton Finance remediation for: An account fraud model joins labels to activity from after the prediction timestamp, accidentally giving the model future knowledge. The necessary check instead is: A temporal audit finds zero post-cutoff feature values in training and online replay samples.