Question 450
A reusable PySpark library transforms multiple transactional feeds. Engineers have applied Encapsulate them in a pure reusable transform function. Which result establishes the required behavior: Several DataFrame operations are used repeatedly by other jobs?
Explanation
Correct Answer
The acceptance criterion for “Several DataFrame operations are used repeatedly by other jobs” is Different jobs apply the same deterministic transformation to shared fixtures; this is the expected evidence after Encapsulate them in a pure reusable transform function.
Incorrect Answers
One child row appears per array element with expected null behavior would demonstrate the separate action Use `explode` or `explode_outer` according to null retention requirements, which addresses “Nested arrays must become normalized child rows for downstream grouping.” It does not verify the requirement “Several DataFrame operations are used repeatedly by other jobs” following Encapsulate them in a pure reusable transform function.
The physical plan includes a distributed filter pertains to a different control (Use DataFrame `.filter()` or `.where()` with a Spark column expression) and a different need (A business predicate must filter rows without collecting the dataset to the driver). It cannot establish that Encapsulate them in a pure reusable transform function met the stated goal.
The output schema includes the derived field and preserves others would demonstrate the separate action Use `.withColumn()` with a supported column expression, which addresses “A new derived column is needed without replacing other columns.” It does not verify the requirement “Several DataFrame operations are used repeatedly by other jobs” following Encapsulate them in a pure reusable transform function.