Question 450
A team has corrected this HPE compute solution issue: An architect estimates required GPU memory from model weights alone and ignores key-value cache and concurrent sessions. Which continuing practice would best prevent the same problem?
Explanation
Correct Answer
Include inference runtime overhead in accelerator capacity models. This recurring assurance measure is tied to the diagnosed risk: Serving overhead can substantially exceed static model parameter storage. For the active incident, the corrective action remains: Profile inference KV-cache, batching and runtime overhead for expected concurrency.
Incorrect Answers
Serving overhead can substantially exceed static model parameter storage. Using the proposed safeguard would guard against a separate risk: Workspace-level access boundaries do not reflect the data classification plan. Include inference runtime overhead in accelerator capacity models.
Infrastructure telemetry does not itself provide lineage, access review or compliance evidence. The selected safeguard helps with that exposure, but it leaves the recurrence risk here unaddressed. Serving overhead can substantially exceed static model parameter storage. Include inference runtime overhead in accelerator capacity models.
Serving overhead can substantially exceed static model parameter storage. The proposed safeguard is intended for another failure mechanism: Camera decode, batching and concurrent model execution affect the scaling relationship. Include inference runtime overhead in accelerator capacity models.