Question 560
For Silverline Insurance, the engineer must verify a remediation with measured evidence, not just a deployment claim. The original failure was: A compact gradient-boosted tree service spends most of its budget on idle GPUs even though prediction is dominated by CPU operations. What measured outcome provides the strongest check that this problem is resolved?
Explanation
Correct Answer
A representative load satisfies p99 latency and throughput at lower measured serving cost. This controlled observation specifically validates the proposed remedy: Benchmark CPU-backed online serving for the model and right-size instances by throughput per dollar. It addresses the documented failure at Silverline Insurance: A compact gradient-boosted tree service spends most of its budget on idle GPUs even though prediction is dominated by CPU operations.
Incorrect Answers
Per-replica metrics identify the bottleneck and a redesigned test improves throughput per cost. This is evidence for a different corrective change: Profile utilization and choose a better architecture, smaller model, or appropriately sized accelerators based on bottleneck evidence. That other change addresses: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth. It does not validate the specific Silverline Insurance remediation for: A compact gradient-boosted tree service spends most of its budget on idle GPUs even though prediction is dominated by CPU operations. The necessary check instead is: A representative load satisfies p99 latency and throughput at lower measured serving cost.
Two independently launched replicas report the same model digest and deterministic fixture outputs. This is evidence for a different corrective change: Pin immutable artifact digests and use a supported storage/loading strategy for model weights. That other change addresses: A large model fetches unversioned weights from mutable object storage on each start, occasionally loading different bytes under the same release label. It does not validate the specific Silverline Insurance remediation for: A compact gradient-boosted tree service spends most of its budget on idle GPUs even though prediction is dominated by CPU operations. The necessary check instead is: A representative load satisfies p99 latency and throughput at lower measured serving cost.
A forced dependency outage produces bounded latency and safe outcomes rather than cascading unbounded failures. This is evidence for a different corrective change: Implement timeouts, circuit breaking, and a safe explicitly documented fallback or rejection behavior. That other change addresses: An endpoint keeps crashing during a downstream feature service outage because it has no bounded-time fallback or error policy. It does not validate the specific Silverline Insurance remediation for: A compact gradient-boosted tree service spends most of its budget on idle GPUs even though prediction is dominated by CPU operations. The necessary check instead is: A representative load satisfies p99 latency and throughput at lower measured serving cost.