Databricks Machine Learning Professional Question 11
Single answerYou are working on a machine learning project in Databricks, where your model's predictions need to be stored in a Delta table for downstream analytics workflows. The Delta table already exists. You have a DataFrame predictions_df containing the model's predictions. Which of the following commands will correctly write the predictions to the Delta table while preserving its existing data?
- A
predictions_df.write.format('delta').mode('overwrite').saveAsTable('predictions_table')
- B
predictions_df.write.format('delta').mode('append').saveAsTable('predictions_table')
- C
predictions_df.write.format('parquet').mode('append').saveAsTable('predictions_table')
- D
predictions_df.write.format('delta').mode('overwrite').save('/mnt/delta/predictions_table')
Show answer and explanation
Correct answer: B
Explanation
To preserve the existing data in a Delta table while adding new records, the 'append' mode should be used. The 'delta' format ensures that the data is written in the Delta Lake format, which supports advanced features like ACID transactions, schema enforcement, and time travel. The other options either overwrite the table, use an incorrect format, or do not adhere to Delta table requirements.
- A. Incorrect.
This command will overwrite the entire Delta table, which is not suitable if you want to preserve the existing data. Overwriting should only be used when explicitly replacing all the data.
- B. Correct.
This command correctly appends the new predictions to the existing Delta table without deleting its current data, which is the required behavior in this scenario.
- C. Incorrect.
This command uses the Parquet format instead of Delta, which does not support Delta table features such as ACID transactions and versioning. It is therefore incorrect.
- D. Incorrect.
This command attempts to overwrite data directly to a file path instead of appending to the Delta table, which does not meet the requirement of preserving existing data in the table.