Question 240
A Spark batch repeatedly spills hash aggregate state to disk and has long garbage-collection pauses. What should the engineer verify before declaring success?
Explanation
Correct Answer
This outcome directly verifies the scenario's required behavior: memory pressure and key cardinality should determine sizing and query choices.
Incorrect Answers
Centralizing state increases OOM risk for large groups.
Output file count does not remedy aggregation memory pressure.
Hidden telemetry cannot establish improved execution.