Question 600
The reliability team at Silverline Insurance is closing a corrective-action ticket for this ML failure: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth. What measured outcome provides the strongest check that this problem is resolved?
Explanation
Correct Answer
Per-replica metrics identify the bottleneck and a redesigned test improves throughput per cost. This controlled observation specifically validates the proposed remedy: Profile utilization and choose a better architecture, smaller model, or appropriately sized accelerators based on bottleneck evidence. It addresses the documented failure at Silverline Insurance: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth.
Incorrect Answers
Controlled stale features trigger the intended freshness handling rather than silently entering predictions. This is evidence for a different corrective change: Attach feature freshness metadata and reject, fallback, or refresh feature values according to a documented TTL policy. That other change addresses: A feature lookup service returns values with no feature-event timestamp, so the online model unknowingly uses outdated profile data. It does not validate the specific Silverline Insurance remediation for: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth. The necessary check instead is: Per-replica metrics identify the bottleneck and a redesigned test improves throughput per cost.
Two independently launched replicas report the same model digest and deterministic fixture outputs. This is evidence for a different corrective change: Pin immutable artifact digests and use a supported storage/loading strategy for model weights. That other change addresses: A large model fetches unversioned weights from mutable object storage on each start, occasionally loading different bytes under the same release label. It does not validate the specific Silverline Insurance remediation for: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth. The necessary check instead is: Per-replica metrics identify the bottleneck and a redesigned test improves throughput per cost.
Offline field testing completes classification without cloud access and meets memory and accuracy constraints. This is evidence for a different corrective change: Package an edge-compatible model and update mechanism with on-device latency and storage measurements. That other change addresses: An agricultural classifier must run without reliable connectivity but engineers assume every edge device can call a cloud API for inference. It does not validate the specific Silverline Insurance remediation for: A GPU-hosted model's throughput falls even as replica count increases because individual replicas are stuck at full memory bandwidth. The necessary check instead is: Per-replica metrics identify the bottleneck and a redesigned test improves throughput per cost.