Exam Bundle

A threshold sweep documents the target false-positive rate and final locked threshold performs as expected on test data. This would verify a different operation: Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. That separate action addresses: A binary classifier produces ranking scores with useful ROC AUC but a default 0.5 threshold yields too many false positives for an operational queue. It cannot establish the required Westhaven Retail correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

The governance report distinguishes predictive contribution from causation and records the limits of SHAP-style explanations. This would verify a different operation: Treat attributions as model-specific explanations and validate findings with controlled experiments or domain review before causal claims. That separate action addresses: SageMaker Clarify generates feature attribution values but the report is interpreted as proof that those features causally determine the outcome. It cannot establish the required Westhaven Retail correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

The final report identifies the confusing class pairs and tracks their individual error rates after model changes. This would verify a different operation: Compute a class confusion matrix and inspect high-confusion pairs with representative labeled examples. That separate action addresses: An image classifier's evaluation logs only overall accuracy while it frequently misclassifies visually similar protected categories. It cannot establish the required Westhaven Retail correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

Configure rank-aware data sharding or distributed sampling so every worker receives the intended subset for the epoch. It does not directly address the Granite SaaS case: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The relevant acceptance criterion for this case is: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

Register model artifacts with lineage and approval metadata in SageMaker Model Registry or equivalent governed artifact catalog. It does not directly address the Granite SaaS case: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The relevant acceptance criterion for this case is: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

Use a suitably small and tested fine-tuning learning rate or parameter-efficient approach with a retained baseline evaluation. It does not directly address the Granite SaaS case: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The relevant acceptance criterion for this case is: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

A completed job records a retrievable model artifact and a clean inference container loads the expected weights. This would verify a different operation: Write trained artifacts to the supported SageMaker model artifact path and confirm the training job uploads the final tarball to its configured S3 output. That separate action addresses: A SageMaker PyTorch training job saves its learned weights only to ephemeral container memory and cannot produce a deployable model artifact on completion. It cannot establish the required Granite SaaS correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

The release report includes tail latency under load and the candidate meets the agreed high-percentile objective. This would verify a different operation: Measure p95 and p99 latency with representative concurrency rather than relying on average inference time alone. That separate action addresses: A production candidate has excellent mean latency but fails the request SLA during high-percentile load spikes. It cannot establish the required Granite SaaS correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

A threshold sweep documents the target false-positive rate and final locked threshold performs as expected on test data. This would verify a different operation: Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. That separate action addresses: A binary classifier produces ranking scores with useful ROC AUC but a default 0.5 threshold yields too many false positives for an operational queue. It cannot establish the required Granite SaaS correction for: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. The decisive acceptance check is instead: A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout.

Apply an appropriate regularization or early-stopping strategy selected against validation behavior instead of increasing epochs blindly. It does not directly address the Bluehaven Transit case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Apply and benchmark an appropriate dropout or weight-decay configuration while keeping the validation comparison controlled. It does not directly address the Bluehaven Transit case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Use SageMaker's configured channel path or environment variables for training data and remove developer-specific bucket references. It does not directly address the Bluehaven Transit case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

The evaluation reveals positive-class misses and explicitly reports the accepted recall and false-positive tradeoff. This would verify a different operation: Use confusion-matrix precision, recall and F1 metrics with a business-defined threshold on a representative holdout set. That separate action addresses: A high-risk fraud classifier shows 99 percent accuracy by predicting the majority class and is approved without examining missed fraud cases. It cannot establish the required Bluehaven Transit correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Predictions grouped near 0.9 have observed frequencies inside the documented calibration tolerance on held-out data. This would verify a different operation: Evaluate reliability diagrams and use an appropriate validation-fitted probability calibration method if needed. That separate action addresses: A binary classifier's predicted probability of 0.9 corresponds to only 0.6 observed positive frequency, making risk estimates unreliable. It cannot establish the required Bluehaven Transit correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

A clean isolated training job starts without public internet egress and logs the expected dependency versions. This would verify a different operation: Package dependencies in a trusted container image or approved private repository accessible to the training subnet. That separate action addresses: A model training run terminates because it assumes access to external Python package indexes blocked by the organization's isolated VPC. It cannot establish the required Bluehaven Transit correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Evaluate reliability diagrams and use an appropriate validation-fitted probability calibration method if needed. It does not directly address the Brookside Manufacturing case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. It does not directly address the Brookside Manufacturing case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Write trained artifacts to the supported SageMaker model artifact path and confirm the training job uploads the final tarball to its configured S3 output. It does not directly address the Brookside Manufacturing case: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The relevant acceptance criterion for this case is: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

A deployed model version resolves to its exact training job, data manifest and approved model package. This would verify a different operation: Register model artifacts with lineage and approval metadata in SageMaker Model Registry or equivalent governed artifact catalog. That separate action addresses: An organization selects a model solely from the latest notebook file and cannot identify which SageMaker training run produced the released weights. It cannot establish the required Brookside Manufacturing correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

The revised run reduces the train-to-validation gap without collapsing performance on the independent holdout. This would verify a different operation: Apply and benchmark an appropriate dropout or weight-decay configuration while keeping the validation comparison controlled. That separate action addresses: A convolutional model overfits despite an adequate dataset because its configured dropout regularization is disabled in training. It cannot establish the required Brookside Manufacturing correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget. This would verify a different operation: Reduce the effective batch or apply supported gradient accumulation and mixed precision after checking numerical stability. That separate action addresses: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. It cannot establish the required Brookside Manufacturing correction for: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. The decisive acceptance check is instead: The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations.

Configure rank-aware data sharding or distributed sampling so every worker receives the intended subset for the epoch. It does not directly address the Harborline Logistics case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

Enable appropriate SageMaker managed Spot training checkpoints in durable storage and a tested resume workflow. It does not directly address the Harborline Logistics case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

Use confusion-matrix precision, recall and F1 metrics with a business-defined threshold on a representative holdout set. It does not directly address the Harborline Logistics case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

All training timestamps precede each evaluation window and the aggregated backtest reflects the real forecast schedule. This would verify a different operation: Use rolling-origin or other chronological backtesting splits appropriate to the prediction horizon. That separate action addresses: A time-series forecast is assessed using random record-level cross-validation that trains on observations later than the validation period. It cannot establish the required Harborline Logistics correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

The adapted model gains target-domain quality without unacceptable regression on the original reference tasks. This would verify a different operation: Use a suitably small and tested fine-tuning learning rate or parameter-efficient approach with a retained baseline evaluation. That separate action addresses: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. It cannot establish the required Harborline Logistics correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion. This would verify a different operation: Inspect training diagnostics and apply suitable learning-rate reduction or gradient clipping with controlled numerical checks. That separate action addresses: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. It cannot establish the required Harborline Logistics correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

Apply and benchmark an appropriate dropout or weight-decay configuration while keeping the validation comparison controlled. It does not directly address the Northgate Telecom case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

Interpret the train-validation generalization gap as likely overfitting and compare regularization or data improvements first. It does not directly address the Northgate Telecom case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

Define meaningful parameter ranges, metric objectives and SageMaker tuning job resource and stopping limits. It does not directly address the Northgate Telecom case: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The relevant acceptance criterion for this case is: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

The adapted model gains target-domain quality without unacceptable regression on the original reference tasks. This would verify a different operation: Use a suitably small and tested fine-tuning learning rate or parameter-efficient approach with a retained baseline evaluation. That separate action addresses: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. It cannot establish the required Northgate Telecom correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

All training timestamps precede each evaluation window and the aggregated backtest reflects the real forecast schedule. This would verify a different operation: Use rolling-origin or other chronological backtesting splits appropriate to the prediction horizon. That separate action addresses: A time-series forecast is assessed using random record-level cross-validation that trains on observations later than the validation period. It cannot establish the required Northgate Telecom correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.

The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget. This would verify a different operation: Reduce the effective batch or apply supported gradient accumulation and mixed precision after checking numerical stability. That separate action addresses: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. It cannot establish the required Northgate Telecom correction for: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. The decisive acceptance check is instead: The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality.