Databricks Machine Learning Associate Question 152: You are working on a Databricks notebook and need to compute summary... | Databricks Machine Learning Associate Dumps

Databricks Machine Learning Associate practice question 152 of 656

Databricks Certified Machine Learning Associate. Associate level, Databricks. Free question with the correct answer and a full explanation.

Databricks Machine Learning Associate Question 152

Select 2

You are working on a Databricks notebook and need to compute summary statistics (e.g., mean, variance, min, max) for a Spark DataFrame. The DataFrame is stored as a Delta table, and you want to save the summary statistics to a CSV file in the Databricks File System (DBFS). Which of the following steps should you take to achieve this?

A
Use the describe method on the Spark DataFrame to compute summary statistics.
B
Use the dbutils.fs.put method to save the summary statistics directly as a CSV file.
C
Convert the output of the describe method to a Pandas DataFrame and use to_csv to write the file to DBFS.
D
Write the results of the describe method directly to a Delta table using the write method.
E
Use the dbutils.fs.cp method to copy the summary statistics from a notebook cell output to a DBFS location.

Show answer and explanation

Correct answers: A, C

Explanation

To compute and save summary statistics in Databricks, you use the describe method of a Spark DataFrame to generate the statistics. The output can then be converted to a Pandas DataFrame using .toPandas() and written to a CSV file in DBFS using the to_csv method. Other methods like dbutils.fs.put or dbutils.fs.cp are not designed for handling DataFrame objects or statistical outputs.

A. Correct.
Correct. The describe method on a Spark DataFrame is specifically designed to compute summary statistics such as mean, variance, min, and max.
B. Incorrect.
Incorrect. The dbutils.fs.put method is used for writing strings to files in DBFS, but it cannot directly handle DataFrame objects or summary statistics.
C. Correct.
Correct. Converting the output of the describe method to a Pandas DataFrame allows you to use its to_csv method to write the summary statistics to a CSV file on DBFS.
D. Incorrect.
Incorrect. The describe method output is not suitable for direct writing to a Delta table, as it is not in a tabular format compatible with Delta Lake.
E. Incorrect.
Incorrect. The dbutils.fs.cp method is used for copying files between locations in DBFS but cannot be used to save computed statistics from a notebook cell output directly.

Databricks Machine Learning Associate practice question 152 of 656

Databricks Machine Learning Associate Question 152

Take a Databricks Machine Learning Associate practice test under exam conditions

More Databricks Machine Learning Associate practice questions