Question 670
An agent calls a powerful reasoning model for every routine status request, making costs high. What optimization could help?
Explanation
Correct Answer
Task-aware model routing can reduce spend when a smaller model satisfies quality and safety requirements.
Incorrect Answers
Include high-percentile input size and output demand in cost forecasting. A separate source case supports that approach when: A pricing forecast uses only mean prompt length and misses rare 100,000-token document analyses. The supporting rationale or observable result is: Long-tail workloads can dominate token billing even when most requests are small. This question describes a different issue: An agent calls a powerful reasoning model for every routine status request, making costs high. Address it through Route simple tasks to an adequately evaluated lower-cost model tier, with the required justification or evidence: Task-aware model routing can reduce spend when a smaller model satisfies quality and safety requirements.
Report token and model cost metrics per approved tenant or usage cohort. This is useful for the distinct issue: A deployment budget is under control overall, but one enterprise tenant generates disproportionate expensive requests. The relevant technical rationale or evidence for that issue is: Segmented cost telemetry identifies concentrated demand without confusing it with organization-wide averages. Here the reported issue is: An agent calls a powerful reasoning model for every routine status request, making costs high. The relevant response is: Route simple tasks to an adequately evaluated lower-cost model tier. The first result cannot verify this different correction: Task-aware model routing can reduce spend when a smaller model satisfies quality and safety requirements.
Evaluate safe prompt-prefix caching or reuse where supported by the model service. It addresses another case: A large support product is charged for repeated system instructions on every call. For that case, the distinguishing justification or acceptance check is: Caching repeated eligible prompt content can reduce redundant processing while preserving required instructions. The present case instead states: An agent calls a powerful reasoning model for every routine status request, making costs high. It calls for Route simple tasks to an adequately evaluated lower-cost model tier. Showing success for the other case would not establish: Task-aware model routing can reduce spend when a smaller model satisfies quality and safety requirements.