Exam Bundle

A threshold sweep documents the target false-positive rate and final locked threshold performs as expected on test data. This would verify a different operation: Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. That separate action addresses: A binary classifier produces ranking scores with useful ROC AUC but a default 0.5 threshold yields too many false positives for an operational queue. It cannot establish the required Highland Education correction for: A multi-class classifier trained with a single dominant class consistently ignores minority groups, yet training loss alone appears healthy. The decisive acceptance check is instead: Per-class evaluation improves without using synthetic training records inside validation or test partitions.

A representative image set demonstrates whether the managed API meets accuracy and supported-feature requirements within cost bounds. This would verify a different operation: Compare Amazon Rekognition's supported capabilities against the need for custom features before committing to training. That separate action addresses: An image-processing system needs managed detection of standard objects and text in product photos, but a project proposal requires a custom vision model for each image type. It cannot establish the required Highland Education correction for: A multi-class classifier trained with a single dominant class consistently ignores minority groups, yet training loss alone appears healthy. The decisive acceptance check is instead: Per-class evaluation improves without using synthetic training records inside validation or test partitions.

An approved sample recording yields usable time-aligned text and an operating cost comparison documents the managed-service choice. This would verify a different operation: Evaluate Amazon Transcribe as the managed speech-to-text capability before building and maintaining a custom recognizer. That separate action addresses: A customer support application needs to convert recorded calls into searchable text but developers plan to train speech recognition from scratch for a small dataset. It cannot establish the required Highland Education correction for: A multi-class classifier trained with a single dominant class consistently ignores minority groups, yet training loss alone appears healthy. The decisive acceptance check is instead: Per-class evaluation improves without using synthetic training records inside validation or test partitions.

Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. It does not directly address the Eastshore Energy case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. It does not directly address the Eastshore Energy case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Evaluate an appropriate generative foundation model or managed AI capability while constraining input length, retrieval and output validation. It does not directly address the Eastshore Energy case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

The experiment record shows no selection decisions based on final test labels before the locked evaluation. This would verify a different operation: Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. That separate action addresses: A model validation workflow repeatedly tunes its hyperparameters on the same final test set, inflating the reported out-of-sample quality. It cannot establish the required Eastshore Energy correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

A controlled reference request produces equivalent features in both pipelines and the corrected performance gap is reassessed. This would verify a different operation: Compare sampled training and serving feature transformations and distributions before concluding the model weights are defective. That separate action addresses: A model with good offline precision unexpectedly underperforms after deployment because the serving feature values are generated by a different pipeline. It cannot establish the required Eastshore Energy correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

A threshold sweep documents the target false-positive rate and final locked threshold performs as expected on test data. This would verify a different operation: Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. That separate action addresses: A binary classifier produces ranking scores with useful ROC AUC but a default 0.5 threshold yields too many false positives for an operational queue. It cannot establish the required Eastshore Energy correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. It does not directly address the Lakeside Research case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Add macro-averaged and per-class precision-recall reporting alongside micro metrics for the model acceptance decision. It does not directly address the Lakeside Research case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Compare model and deployment options against data residency, access policy and cost constraints before deciding on Bedrock or a supported hosted alternative. It does not directly address the Lakeside Research case: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The relevant acceptance criterion for this case is: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

An auditor can reproduce the compared configurations and identify the changed input or parameter causing the metric difference. This would verify a different operation: Track training jobs, hyperparameters, dataset manifests and metrics in a reproducible SageMaker experiment catalog. That separate action addresses: A model's validation score degrades between two SageMaker runs but the experiment records lack parameter and dataset lineage. It cannot establish the required Lakeside Research correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

The selected approach meets the required recall and precision tradeoff on the untouched production-like class distribution. This would verify a different operation: Select a classifier and decision threshold based on false-negative cost and class-aware metrics rather than majority-class accuracy. That separate action addresses: A defect prediction project has very few confirmed positive labels and proposes to optimize raw accuracy on an unbalanced evaluation dataset. It cannot establish the required Lakeside Research correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

An evaluation report shows numerical prediction errors in the expected units and a baseline-relative improvement. This would verify a different operation: Use suitable regression metrics such as MAE and RMSE with error units and compare against a simple business baseline. That separate action addresses: A regression forecast is evaluated with classification accuracy even though every prediction is a continuous dollar amount. It cannot establish the required Lakeside Research correction for: A custom model has an unstable optimizer and loss becomes nonfinite after a few updates due to excessive gradient magnitudes. The decisive acceptance check is instead: A repeated training run keeps finite gradients and loss while meeting the declared convergence criterion.

Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. It does not directly address the Cedarfield Hospitality case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Evaluate a sequence-aware recommendation or ranking approach and compare it with a simpler nonsequential baseline. It does not directly address the Cedarfield Hospitality case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Compare sampled training and serving feature transformations and distributions before concluding the model weights are defective. It does not directly address the Cedarfield Hospitality case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

The slice report exposes extreme-input errors and documents whether the corrective model stays within accepted thresholds. This would verify a different operation: Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. That separate action addresses: A regression model produces implausible forecasts for extreme inputs but the team inspects only the overall RMSE. It cannot establish the required Cedarfield Hospitality correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

The feasibility report identifies supported series and validates forecasts without silently fabricating missing chronological observations. This would verify a different operation: Assess temporal data sufficiency, sampling frequency and known-future covariates before comparing time-series models. That separate action addresses: An ML engineer chooses a built-in forecasting method without confirming that multiple product series contain sufficient and regularly spaced history. It cannot establish the required Cedarfield Hospitality correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

The report shows every priority class's performance and a rare-category failure cannot be masked by majority volume. This would verify a different operation: Add macro-averaged and per-class precision-recall reporting alongside micro metrics for the model acceptance decision. That separate action addresses: A product classifier reports a high micro-averaged F1 but hides very poor performance on a critical rare product category. It cannot establish the required Cedarfield Hospitality correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Add macro-averaged and per-class precision-recall reporting alongside micro metrics for the model acceptance decision. It does not directly address the Riverton Utilities case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Assess Amazon Translate for the supported target language pair and compare it with any genuinely required custom terminology handling. It does not directly address the Riverton Utilities case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Choose a multi-label output design with independent label scores and threshold selection instead of single-class competition. It does not directly address the Riverton Utilities case: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The relevant acceptance criterion for this case is: The experiment record displays expected named metrics with values matching the training log for a controlled run.

An audited holdout set shows meaningful semantic grouping and the chosen model's distance assumptions are documented. This would verify a different operation: Compare a suitable text vector representation and model family using relevant similarity measures and unsupervised evaluation criteria. That separate action addresses: A classification dataset contains many thousands of sparse token features, and teams choose k-means with an inappropriate raw Euclidean representation. It cannot establish the required Riverton Utilities correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

The experiment record shows no selection decisions based on final test labels before the locked evaluation. This would verify a different operation: Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. That separate action addresses: A model validation workflow repeatedly tunes its hyperparameters on the same final test set, inflating the reported out-of-sample quality. It cannot establish the required Riverton Utilities correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

The held-out chronological evaluation shows whether the model benefits from the actual interaction order. This would verify a different operation: Evaluate a sequence-aware recommendation or ranking approach and compare it with a simpler nonsequential baseline. That separate action addresses: A recommendation model treats independently ordered shopping events as a flat feature bag even though the prediction depends on recent interaction sequence. It cannot establish the required Riverton Utilities correction for: SageMaker managed training metrics do not appear in experiment reports because the custom container never emits the configured metric pattern. The decisive acceptance check is instead: The experiment record displays expected named metrics with values matching the training log for a controlled run.

Choose a multi-label output design with independent label scores and threshold selection instead of single-class competition. It does not directly address the Juniper Media case: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The relevant acceptance criterion for this case is: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.

Compare sampled training and serving feature transformations and distributions before concluding the model weights are defective. It does not directly address the Juniper Media case: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The relevant acceptance criterion for this case is: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.

Treat attributions as model-specific explanations and validate findings with controlled experiments or domain review before causal claims. It does not directly address the Juniper Media case: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The relevant acceptance criterion for this case is: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.

An audited holdout set shows meaningful semantic grouping and the chosen model's distance assumptions are documented. This would verify a different operation: Compare a suitable text vector representation and model family using relevant similarity measures and unsupervised evaluation criteria. That separate action addresses: A classification dataset contains many thousands of sparse token features, and teams choose k-means with an inappropriate raw Euclidean representation. It cannot establish the required Juniper Media correction for: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The decisive acceptance check is instead: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.

The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality. This would verify a different operation: Compare model and deployment options against data residency, access policy and cost constraints before deciding on Bedrock or a supported hosted alternative. That separate action addresses: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. It cannot establish the required Juniper Media correction for: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The decisive acceptance check is instead: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.

The test matrix isolates the marginal impact of the feature and optimizer changes without conflating their effects. This would verify a different operation: Run controlled ablations or reproducible one-factor comparisons while holding other settings fixed. That separate action addresses: A model accuracy experiment introduces a new feature and several new hyperparameters at once, preventing attribution of the observed gain. It cannot establish the required Juniper Media correction for: A team deploys a SageMaker built-in algorithm with an input record format incompatible with the algorithm's required channel parser. The decisive acceptance check is instead: The built-in training algorithm reads an approved small input channel and no longer reports record-deserialization errors.