Question 420
A silver pipeline must separate corrupt source records from validated output. Engineers have applied Restrict the predicate to required columns per the data contract. Which result establishes the required behavior: An expectation is misconfigured to reject legal nullable optional fields?
Explanation
Correct Answer
The acceptance criterion for “An expectation is misconfigured to reject legal nullable optional fields” is Valid optional-null records remain while truly invalid required fields are rejected; this is the expected evidence after Restrict the predicate to required columns per the data contract.
Incorrect Answers
The silver output excludes the rejected rows while pipeline proceeds would demonstrate the separate action Use `ON VIOLATION DROP ROW` or an equivalent supported expectation setting, which addresses “A valid silver table should omit rows that fail a noncritical quality predicate.” It does not verify the requirement “An expectation is misconfigured to reject legal nullable optional fields” following Restrict the predicate to required columns per the data contract.
The expectation violation count rises while raw records stay available would demonstrate the separate action Use a warning expectation and retain the raw rows, which addresses “All failing records should be counted but retained in a diagnostic bronze target.” It does not verify the requirement “An expectation is misconfigured to reject legal nullable optional fields” following Restrict the predicate to required columns per the data contract.
The update stops with the named expectation failure pertains to a different control (Use `ON VIOLATION FAIL UPDATE` or equivalent strict expectation) and a different need (A critical primary-key validation must stop the entire pipeline on failure). It cannot establish that Restrict the predicate to required columns per the data contract met the stated goal.