Question 250
Two candidates have similar validation accuracy but one requires ten times the inference cost. How should deployment selection account for operations?
Explanation
Correct Answer
A slightly better offline score can be unsuitable when its infrastructure requirements violate production service objectives.
Incorrect Answers
Compute a common approved regression metric on both models' predictions. A separate source case supports that approach when: Two regression models are evaluated on identical examples, but one team reports RMSE and the other reports MAE. The supporting rationale or observable result is: Different loss metrics penalize errors differently; side-by-side ranking needs a consistent criterion agreed before selection. This question describes a different issue: Two candidates have similar validation accuracy but one requires ten times the inference cost. Address it through Compare accuracy along with serving latency, resource cost, and throughput requirements, with the required justification or evidence: A slightly better offline score can be unsuitable when its infrastructure requirements violate production service objectives.
Use the same averaging method and label definitions for every candidate's F1 metric. This is useful for the distinct issue: A model selection pipeline ranks candidates by F1 but the scoring code logs micro-average for one and macro-average for another. The relevant technical rationale or evidence for that issue is: Metrics with the same name can represent different aggregation rules; inconsistent definitions invalidate direct comparisons. Here the reported issue is: Two candidates have similar validation accuracy but one requires ten times the inference cost. The relevant response is: Compare accuracy along with serving latency, resource cost, and throughput requirements. The first result cannot verify this different correction: A slightly better offline score can be unsuitable when its infrastructure requirements violate production service objectives.
Compare operating points under the same permitted false-positive rate and recall objective. It addresses another case: A classification model has improved AUROC but its positive-class threshold was not tuned for the organization's false-positive budget. For that case, the distinguishing justification or acceptance check is: A rank-order metric alone does not choose the threshold required for a business decision with constrained false positives. The present case instead states: Two candidates have similar validation accuracy but one requires ten times the inference cost. It calls for Compare accuracy along with serving latency, resource cost, and throughput requirements. Showing success for the other case would not establish: A slightly better offline score can be unsuitable when its infrastructure requirements violate production service objectives.