Exam Bundle

The experiment log proves whether predictions were merely observed or whether an authorized portion of traffic received candidate outputs. This would verify a different operation: Differentiate shadow validation from controlled live traffic allocation and select the appropriate comparison procedure. That separate action addresses: An ML team labels a production shadow experiment as an A/B test even though the shadow predictions never influence user-facing decisions. It cannot establish the required Timberline Foods correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

A representative image set demonstrates whether the managed API meets accuracy and supported-feature requirements within cost bounds. This would verify a different operation: Compare Amazon Rekognition's supported capabilities against the need for custom features before committing to training. That separate action addresses: An image-processing system needs managed detection of standard objects and text in product photos, but a project proposal requires a custom vision model for each image type. It cannot establish the required Timberline Foods correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

A diagnostic plot shows reduced validation loss without relying on increased training-only accuracy. This would verify a different operation: Interpret the train-validation generalization gap as likely overfitting and compare regularization or data improvements first. That separate action addresses: A model has strong training results but weak validation performance, yet the team calls this underfitting and increases its capacity. It cannot establish the required Timberline Foods correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

Compute a class confusion matrix and inspect high-confusion pairs with representative labeled examples. It does not directly address the Keystone BioLabs case: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The relevant acceptance criterion for this case is: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

Evaluate Amazon Transcribe as the managed speech-to-text capability before building and maintaining a custom recognizer. It does not directly address the Keystone BioLabs case: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The relevant acceptance criterion for this case is: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

Benchmark suitable tabular algorithms such as SageMaker XGBoost against the complex model using the same split and budget. It does not directly address the Keystone BioLabs case: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The relevant acceptance criterion for this case is: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

The training trace identifies the instability window and a corrected run converges with finite monitored values. This would verify a different operation: Review optimizer, gradients and intermediate tensors using supported SageMaker Debugger or framework diagnostics. That separate action addresses: Training logs show unstable loss values and stalled convergence but reviewers look only at the final validation score rather than intermediate diagnostics. It cannot establish the required Keystone BioLabs correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

Holdout predictions include usable value estimates and documented coverage behavior for uncertainty intervals. This would verify a different operation: Choose a calibrated regression or probabilistic forecasting method compatible with the numeric target and uncertainty reporting requirement. That separate action addresses: A pricing model must output predicted revenue with confidence bounds, but the team selects an algorithm intended solely to rank products. It cannot establish the required Keystone BioLabs correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

An audited holdout set shows meaningful semantic grouping and the chosen model's distance assumptions are documented. This would verify a different operation: Compare a suitable text vector representation and model family using relevant similarity measures and unsupervised evaluation criteria. That separate action addresses: A classification dataset contains many thousands of sparse token features, and teams choose k-means with an inappropriate raw Euclidean representation. It cannot establish the required Keystone BioLabs correction for: A team fine-tunes a pre-trained model with a learning rate so high that its previously learned useful capabilities deteriorate rapidly. The decisive acceptance check is instead: The adapted model gains target-domain quality without unacceptable regression on the original reference tasks.

Choose a calibrated regression or probabilistic forecasting method compatible with the numeric target and uncertainty reporting requirement. It does not directly address the Granite SaaS case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Compare model and deployment options against data residency, access policy and cost constraints before deciding on Bedrock or a supported hosted alternative. It does not directly address the Granite SaaS case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Select an appropriate regression or time-series modeling approach aligned to the continuous demand target and its evaluation horizon. It does not directly address the Granite SaaS case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

The final report identifies the confusing class pairs and tracks their individual error rates after model changes. This would verify a different operation: Compute a class confusion matrix and inspect high-confusion pairs with representative labeled examples. That separate action addresses: An image classifier's evaluation logs only overall accuracy while it frequently misclassifies visually similar protected categories. It cannot establish the required Granite SaaS correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

An auditor can reproduce the compared configurations and identify the changed input or parameter causing the metric difference. This would verify a different operation: Track training jobs, hyperparameters, dataset manifests and metrics in a reproducible SageMaker experiment catalog. That separate action addresses: A model's validation score degrades between two SageMaker runs but the experiment records lack parameter and dataset lineage. It cannot establish the required Granite SaaS correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

The experiment log proves whether predictions were merely observed or whether an authorized portion of traffic received candidate outputs. This would verify a different operation: Differentiate shadow validation from controlled live traffic allocation and select the appropriate comparison procedure. That separate action addresses: An ML team labels a production shadow experiment as an A/B test even though the shadow predictions never influence user-facing decisions. It cannot establish the required Granite SaaS correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Assess Amazon Translate for the supported target language pair and compare it with any genuinely required custom terminology handling. It does not directly address the Westhaven Retail case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. It does not directly address the Westhaven Retail case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Use a separate validation set or cross-validation for tuning and reserve the final holdout for one fixed-model assessment. It does not directly address the Westhaven Retail case: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The relevant acceptance criterion for this case is: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

The evaluation reveals positive-class misses and explicitly reports the accepted recall and false-positive tradeoff. This would verify a different operation: Use confusion-matrix precision, recall and F1 metrics with a business-defined threshold on a representative holdout set. That separate action addresses: A high-risk fraud classifier shows 99 percent accuracy by predicting the majority class and is approved without examining missed fraud cases. It cannot establish the required Westhaven Retail correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

A diagnostic plot shows reduced validation loss without relying on increased training-only accuracy. This would verify a different operation: Interpret the train-validation generalization gap as likely overfitting and compare regularization or data improvements first. That separate action addresses: A model has strong training results but weak validation performance, yet the team calls this underfitting and increases its capacity. It cannot establish the required Westhaven Retail correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

A reference document can receive multiple correct labels and per-label quality is assessed on a fixed holdout. This would verify a different operation: Choose a multi-label output design with independent label scores and threshold selection instead of single-class competition. That separate action addresses: A business team asks a multi-label model to assign several valid tags per document but data scientists configure mutually exclusive softmax classes. It cannot establish the required Westhaven Retail correction for: A SageMaker training step is rerun months later and produces a different outcome because the container retrieves unpinned dependency versions. The decisive acceptance check is instead: A controlled repeat uses the same artifact hashes and produces model metrics inside its declared reproducibility tolerance.

Use confusion-matrix precision, recall and F1 metrics with a business-defined threshold on a representative holdout set. It does not directly address the Brookside Manufacturing case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

Interpret the train-validation generalization gap as likely overfitting and compare regularization or data improvements first. It does not directly address the Brookside Manufacturing case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

Analyze residuals across feature ranges and slices, and define domain-specific error thresholds for extreme cases. It does not directly address the Brookside Manufacturing case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

The evaluation reveals positive-class misses and explicitly reports the accepted recall and false-positive tradeoff. This would verify a different operation: Use confusion-matrix precision, recall and F1 metrics with a business-defined threshold on a representative holdout set. That separate action addresses: A high-risk fraud classifier shows 99 percent accuracy by predicting the majority class and is approved without examining missed fraud cases. It cannot establish the required Brookside Manufacturing correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

The experiment log proves whether predictions were merely observed or whether an authorized portion of traffic received candidate outputs. This would verify a different operation: Differentiate shadow validation from controlled live traffic allocation and select the appropriate comparison procedure. That separate action addresses: An ML team labels a production shadow experiment as an A/B test even though the shadow predictions never influence user-facing decisions. It cannot establish the required Brookside Manufacturing correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

The test matrix isolates the marginal impact of the feature and optimizer changes without conflating their effects. This would verify a different operation: Run controlled ablations or reproducible one-factor comparisons while holding other settings fixed. That separate action addresses: A model accuracy experiment introduces a new feature and several new hyperparameters at once, preventing attribution of the observed gain. It cannot establish the required Brookside Manufacturing correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

Treat attributions as model-specific explanations and validate findings with controlled experiments or domain review before causal claims. It does not directly address the Bluehaven Transit case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

Select an appropriate regression or time-series modeling approach aligned to the continuous demand target and its evaluation horizon. It does not directly address the Bluehaven Transit case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

Evaluate an unsupervised clustering approach and select a suitable distance representation before proposing any business segment labels. It does not directly address the Bluehaven Transit case: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The relevant acceptance criterion for this case is: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

The release report includes tail latency under load and the candidate meets the agreed high-percentile objective. This would verify a different operation: Measure p95 and p99 latency with representative concurrency rather than relying on average inference time alone. That separate action addresses: A production candidate has excellent mean latency but fails the request SLA during high-percentile load spikes. It cannot establish the required Bluehaven Transit correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

A governed reference evaluation meets summarization fidelity and latency targets without claiming that a classifier generates prose. This would verify a different operation: Evaluate an appropriate generative foundation model or managed AI capability while constraining input length, retrieval and output validation. That separate action addresses: A customer chat use case needs to summarize long documents, but the engineering plan uses a model designed only for short text classification. It cannot establish the required Bluehaven Transit correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.

An approved sample recording yields usable time-aligned text and an operating cost comparison documents the managed-service choice. This would verify a different operation: Evaluate Amazon Transcribe as the managed speech-to-text capability before building and maintaining a custom recognizer. That separate action addresses: A customer support application needs to convert recorded calls into searchable text but developers plan to train speech recognition from scratch for a small dataset. It cannot establish the required Bluehaven Transit correction for: A training job takes too long on one instance even though distributed training is supported and data partitions can be sharded safely. The decisive acceptance check is instead: The benchmark records reduced wall-clock time while parameter convergence and held-out quality remain comparable.