Question 370
For Mercury Travel Tech, the engineer must verify a remediation with measured evidence, not just a deployment claim. The original failure was: A network regressor trains with a ranking loss while evaluation demands absolute numeric error, producing inconsistent optimization. Which independently observable result best verifies that the remediation works?
Explanation
Correct Answer
Documented loss and validation metrics are consistent with the numeric prediction task. This controlled observation specifically validates the proposed remedy: Choose a training objective aligned with the target outcome and evaluate with a separately defined suitable metric. It addresses the documented failure at Mercury Travel Tech: A network regressor trains with a ranking loss while evaluation demands absolute numeric error, producing inconsistent optimization.
Incorrect Answers
Malformed input is isolated with a reason code while valid events arrive without silent count loss. This is evidence for a different corrective change: Apply Dataflow or comparable pipeline validation with explicit dead-letter outputs and schema-drift metrics. That other change addresses: A daily telemetry training pipeline drops schema-invalid JSON records silently before the model team measures input quality. It does not validate the specific Mercury Travel Tech remediation for: A network regressor trains with a ranking loss while evaluation demands absolute numeric error, producing inconsistent optimization. The necessary check instead is: Documented loss and validation metrics are consistent with the numeric prediction task.
The training step completes without OOM at documented effective batch size and comparable validation behavior. This is evidence for a different corrective change: Profile device memory and lower microbatch size or use accumulation and memory-efficient precision where compatible. That other change addresses: A custom image training run fails with device out-of-memory errors immediately after increasing batch size. It does not validate the specific Mercury Travel Tech remediation for: A network regressor trains with a ranking loss while evaluation demands absolute numeric error, producing inconsistent optimization. The necessary check instead is: Documented loss and validation metrics are consistent with the numeric prediction task.
The candidate passes predeclared task and safety floors on independent evaluation prompts. This is evidence for a different corrective change: Evaluate both domain-task gains and broad regression, safety, and refusal suites after fine-tuning. That other change addresses: A fine-tuned foundation model forgets general instructions and begins disclosing sensitive output, but only domain quality is measured. It does not validate the specific Mercury Travel Tech remediation for: A network regressor trains with a ranking loss while evaluation demands absolute numeric error, producing inconsistent optimization. The necessary check instead is: Documented loss and validation metrics are consistent with the numeric prediction task.