Exam Bundle

A threshold sweep documents the target false-positive rate and final locked threshold performs as expected on test data. This would verify a different operation: Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. That separate action addresses: A binary classifier produces ranking scores with useful ROC AUC but a default 0.5 threshold yields too many false positives for an operational queue. It cannot establish the required Meadowbrook Finance correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

Holdout predictions include usable value estimates and documented coverage behavior for uncertainty intervals. This would verify a different operation: Choose a calibrated regression or probabilistic forecasting method compatible with the numeric target and uncertainty reporting requirement. That separate action addresses: A pricing model must output predicted revenue with confidence bounds, but the team selects an algorithm intended solely to rank products. It cannot establish the required Meadowbrook Finance correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

The training trace identifies the instability window and a corrected run converges with finite monitored values. This would verify a different operation: Review optimizer, gradients and intermediate tensors using supported SageMaker Debugger or framework diagnostics. That separate action addresses: Training logs show unstable loss values and stalled convergence but reviewers look only at the final validation score rather than intermediate diagnostics. It cannot establish the required Meadowbrook Finance correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

Track training jobs, hyperparameters, dataset manifests and metrics in a reproducible SageMaker experiment catalog. It does not directly address the Redstone Aviation case: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The relevant acceptance criterion for this case is: The same container trains successfully with a different authorized input channel and no code edits.

Tune the operating threshold against the business cost curve on validation data while keeping the final test set untouched. It does not directly address the Redstone Aviation case: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The relevant acceptance criterion for this case is: The same container trains successfully with a different authorized input channel and no code edits.

Include inference latency, compute cost and model size in a feasibility benchmark against smaller compatible candidates. It does not directly address the Redstone Aviation case: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The relevant acceptance criterion for this case is: The same container trains successfully with a different authorized input channel and no code edits.

Holdout predictions include usable value estimates and documented coverage behavior for uncertainty intervals. This would verify a different operation: Choose a calibrated regression or probabilistic forecasting method compatible with the numeric target and uncertainty reporting requirement. That separate action addresses: A pricing model must output predicted revenue with confidence bounds, but the team selects an algorithm intended solely to rank products. It cannot establish the required Redstone Aviation correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

An auditor can reproduce the compared configurations and identify the changed input or parameter causing the metric difference. This would verify a different operation: Track training jobs, hyperparameters, dataset manifests and metrics in a reproducible SageMaker experiment catalog. That separate action addresses: A model's validation score degrades between two SageMaker runs but the experiment records lack parameter and dataset lineage. It cannot establish the required Redstone Aviation correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

A controlled reference request produces equivalent features in both pipelines and the corrected performance gap is reassessed. This would verify a different operation: Compare sampled training and serving feature transformations and distributions before concluding the model weights are defective. That separate action addresses: A model with good offline precision unexpectedly underperforms after deployment because the serving feature values are generated by a different pipeline. It cannot establish the required Redstone Aviation correction for: A custom SageMaker training script downloads its dataset from a hard-coded developer bucket rather than reading the configured training input channel. The decisive acceptance check is instead: The same container trains successfully with a different authorized input channel and no code edits.

Assess temporal data sufficiency, sampling frequency and known-future covariates before comparing time-series models. It does not directly address the Highland Education case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Compute a class confusion matrix and inspect high-confusion pairs with representative labeled examples. It does not directly address the Highland Education case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Evaluate an unsupervised clustering approach and select a suitable distance representation before proposing any business segment labels. It does not directly address the Highland Education case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

An approved multilingual reference set meets translation acceptance checks without running custom model infrastructure when unnecessary. This would verify a different operation: Assess Amazon Translate for the supported target language pair and compare it with any genuinely required custom terminology handling. That separate action addresses: A machine translation feature plans to retrain a multilingual sequence-to-sequence model despite requiring only a common language pair already supported by a managed AWS service. It cannot establish the required Highland Education correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

The final report identifies the confusing class pairs and tracks their individual error rates after model changes. This would verify a different operation: Compute a class confusion matrix and inspect high-confusion pairs with representative labeled examples. That separate action addresses: An image classifier's evaluation logs only overall accuracy while it frequently misclassifies visually similar protected categories. It cannot establish the required Highland Education correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality. This would verify a different operation: Compare model and deployment options against data residency, access policy and cost constraints before deciding on Bedrock or a supported hosted alternative. That separate action addresses: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. It cannot establish the required Highland Education correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Use suitable regression metrics such as MAE and RMSE with error units and compare against a simple business baseline. It does not directly address the Maplewood Schools case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Evaluate Amazon Transcribe as the managed speech-to-text capability before building and maintaining a custom recognizer. It does not directly address the Maplewood Schools case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. It does not directly address the Maplewood Schools case: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The relevant acceptance criterion for this case is: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

An auditor can reproduce the compared configurations and identify the changed input or parameter causing the metric difference. This would verify a different operation: Track training jobs, hyperparameters, dataset manifests and metrics in a reproducible SageMaker experiment catalog. That separate action addresses: A model's validation score degrades between two SageMaker runs but the experiment records lack parameter and dataset lineage. It cannot establish the required Maplewood Schools correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

A governed reference evaluation meets summarization fidelity and latency targets without claiming that a classifier generates prose. This would verify a different operation: Evaluate an appropriate generative foundation model or managed AI capability while constraining input length, retrieval and output validation. That separate action addresses: A customer chat use case needs to summarize long documents, but the engineering plan uses a model designed only for short text classification. It cannot establish the required Maplewood Schools correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

The candidate evaluation demonstrates compliant request routing and authorized data handling as well as acceptable answer quality. This would verify a different operation: Compare model and deployment options against data residency, access policy and cost constraints before deciding on Bedrock or a supported hosted alternative. That separate action addresses: An inference team chooses a managed language model with the largest context window even though the protected request content cannot leave its designated processing environment. It cannot establish the required Maplewood Schools correction for: A high-memory feature matrix repeatedly exhausts GPU memory because training uses an unnecessarily large batch size. The decisive acceptance check is instead: The model trains without GPU out-of-memory errors and produces comparable validation metrics under the selected memory budget.

Evaluate a sequence-aware recommendation or ranking approach and compare it with a simpler nonsequential baseline. It does not directly address the Lakeside Research case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. It does not directly address the Lakeside Research case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

Evaluate both candidate models on the same versioned untouched holdout dataset with identical metric definitions. It does not directly address the Lakeside Research case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

The experiment record shows no selection decisions based on final test labels before the locked evaluation. This would verify a different operation: Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. That separate action addresses: A model validation workflow repeatedly tunes its hyperparameters on the same final test set, inflating the reported out-of-sample quality. It cannot establish the required Lakeside Research correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

The paired model comparison uses matching sample IDs and reproducible evaluation settings for both candidates. This would verify a different operation: Evaluate both candidate models on the same versioned untouched holdout dataset with identical metric definitions. That separate action addresses: An ML review compares two model variants using different holdout sets and claims the one with higher score is superior. It cannot establish the required Lakeside Research correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

A governed reference evaluation meets summarization fidelity and latency targets without claiming that a classifier generates prose. This would verify a different operation: Evaluate an appropriate generative foundation model or managed AI capability while constraining input length, retrieval and output validation. That separate action addresses: A customer chat use case needs to summarize long documents, but the engineering plan uses a model designed only for short text classification. It cannot establish the required Lakeside Research correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

Assess temporal data sufficiency, sampling frequency and known-future covariates before comparing time-series models. It does not directly address the Eastshore Energy case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. It does not directly address the Eastshore Energy case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

Interpret the train-validation generalization gap as likely overfitting and compare regularization or data improvements first. It does not directly address the Eastshore Energy case: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The relevant acceptance criterion for this case is: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

A holdout run produces numeric unit forecasts and is evaluated with documented regression or forecasting error metrics. This would verify a different operation: Select an appropriate regression or time-series modeling approach aligned to the continuous demand target and its evaluation horizon. That separate action addresses: An inventory forecasting team proposes a binary classification algorithm although the desired output is a continuous quantity of product units. It cannot establish the required Eastshore Energy correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

An audited holdout set shows meaningful semantic grouping and the chosen model's distance assumptions are documented. This would verify a different operation: Compare a suitable text vector representation and model family using relevant similarity measures and unsupervised evaluation criteria. That separate action addresses: A classification dataset contains many thousands of sparse token features, and teams choose k-means with an inappropriate raw Euclidean representation. It cannot establish the required Eastshore Energy correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.

A representative image set demonstrates whether the managed API meets accuracy and supported-feature requirements within cost bounds. This would verify a different operation: Compare Amazon Rekognition's supported capabilities against the need for custom features before committing to training. That separate action addresses: An image-processing system needs managed detection of standard objects and text in product photos, but a project proposal requires a custom vision model for each image type. It cannot establish the required Eastshore Energy correction for: A lengthy training job uses Spot Instances without checkpoints and loses all progress when a capacity interruption stops the instance. The decisive acceptance check is instead: An interruption test restarts from the last saved checkpoint rather than the initial epoch and preserves model state.