Question 670
A cluster repeatedly reads the same Delta file ranges during exploratory analytics. Engineers have applied Optimize data layout or query access before increasing disk cache settings. Which result establishes the required behavior: A query shows little benefit from disk caching because every run reads new files?
Explanation
Correct Answer
Measured remote read volume falls for the relevant recurring access pattern is the matching outcome for the implemented control (Optimize data layout or query access before increasing disk cache settings) and the stated requirement “A query shows little benefit from disk caching because every run reads new files.”
Incorrect Answers
Subsequent actions reuse cached partitions while memory consumption is bounded would demonstrate the separate action Use `.cache()` or `.persist()` for the reused DataFrame and unpersist afterward, which addresses “A Spark DataFrame is reused many times in one session and explicit compute caching is justified.” It does not verify the requirement “A query shows little benefit from disk caching because every run reads new files” following Optimize data layout or query access before increasing disk cache settings.
This signal belongs to the adjacent case “A developer assumes disk cache permanently pins data across cluster replacement,” whose remedy is Treat disk cache as an ephemeral executor-local optimization. The observed result for the present requirement is Measured remote read volume falls for the relevant recurring access pattern.
That observation is a valid indicator for “An important dataset must remain queryable even after cached worker nodes terminate” when using Store the authoritative table in governed cloud object storage, not cache. The scenario instead tests “A query shows little benefit from disk caching because every run reads new files”; the relevant result is Measured remote read volume falls for the relevant recurring access pattern.