Databricks Machine Learning Associate Question 149
Single answerYou are working with a Spark DataFrame named sales_df that contains information about monthly sales, including columns region, product, and sales_amount. You want to compute summary statistics such as mean, standard deviation, minimum, and maximum for the sales_amount column. Which of the following code snippets correctly computes these summary statistics for the specified column?
- A
sales_df.summary('mean', 'stddev', 'min', 'max').filter(col('summary') == 'sales_amount').show() - B
sales_df.select('sales_amount').summary('mean', 'stddev', 'min', 'max').show() - C
sales_df.summary('mean', 'stddev', 'min', 'max').show() - D
sales_df.describe('sales_amount').show()
Show answer and explanation
Correct answer: C
Explanation
The .summary() method in Spark DataFrames computes a variety of summary statistics (e.g., mean, stddev, min, max) for all numerical columns in the DataFrame. It cannot be applied to a subset of columns directly, and it provides more flexibility than .describe(). The correct syntax is to call .summary() on the DataFrame and then display the result using .show().
- A. Incorrect.
This code does not work because the
.summary()method does not allow for filtering directly on the 'summary' column. The column name 'sales_amount' is not part of the summary output. - B. Incorrect.
This code is incorrect because
.summary()can only be called on the DataFrame as a whole, not on a selected column subset likesales_df.select('sales_amount'). - C. Correct.
This is the correct answer because
.summary()computes the specified summary statistics for all numerical columns in the DataFrame, includingsales_amount, and displays the results. - D. Incorrect.
This code is incorrect because
.describe()only computes basic statistics (mean, min, max, stddev, and count) and does not allow for specifying additional statistics like.summary()does.