Databricks Machine Learning Professional Question 8
Select 3You are working on a machine learning project where the training data is stored in a Delta table named 'ml_training_data' in your Databricks workspace. You need to load this table into a Spark DataFrame for model training and then save the resulting predictions back into a Delta table named 'ml_predictions'. Which of the following steps correctly achieves this?
- A
Use
spark.read.format('delta').table('ml_training_data')to load the Delta table into a DataFrame. - B
Use
spark.read.delta('ml_training_data')to load the Delta table into a DataFrame. - C
Use
dataframe.write.format('delta').saveAsTable('ml_predictions')to save the predictions as a Delta table. - D
Use
dataframe.write.mode('overwrite').option('format', 'delta').save('ml_predictions')to save the predictions as a Delta table. - E
Use
spark.table('ml_training_data')to load the Delta table into a DataFrame.
Show answer and explanation
Correct answers: A, C, E
Explanation
To read a Delta table into a Spark DataFrame, you can either use spark.read.format('delta').table() or the shorthand spark.table(). Both are valid for Delta tables in Databricks. To save a DataFrame as a Delta table, you need to use the write.format('delta').saveAsTable() method. Understanding these operations is crucial for managing Delta tables when working with machine learning workflows in Databricks.
- A. Correct.
Correct. Using
spark.read.format('delta').table('ml_training_data')is one of the standard ways to load a Delta table into a DataFrame in Databricks. - B. Incorrect.
Incorrect. The method
spark.read.deltais not a valid function in PySpark for reading Delta tables. - C. Correct.
Correct. Using
dataframe.write.format('delta').saveAsTable('ml_predictions')is the correct way to save a DataFrame as a Delta table in a Databricks environment. - D. Incorrect.
Incorrect. The
.option('format', 'delta')syntax is not valid for saving Delta tables. The correct syntax is using.format('delta'). - E. Correct.
Correct. The function
spark.table('ml_training_data')is a valid and concise way to load a Delta table into a DataFrame.