Databricks Machine Learning Professional Question 152
Single answerA machine learning team has built a model for predicting customer churn and is storing the predictions in a large Delta table. Analysts frequently query the table to retrieve predictions for specific customers based on their unique customer IDs. The team notices that query times are increasing as the size of the table grows. Which of the following actions can help reduce the query latency in this scenario?
- A
Enable z-ordering on the Delta table using the customer ID column.
- B
Partition the Delta table by the customer ID column.
- C
Increase the number of worker nodes in the cluster.
- D
Enable Delta caching for the Delta table.
Show answer and explanation
Correct answer: A
Explanation
Z-ordering is a technique in Delta Lake that helps improve query performance by co-locating data based on the values in one or more columns, reducing the amount of data scanned during queries. In this case, z-ordering on the customer ID column allows queries to retrieve predictions more efficiently, addressing the performance issue caused by the growing table size.
- A. Correct.
Correct. Z-ordering on the customer ID column will optimize the storage layout of the Delta table, clustering data with similar values for the customer ID column together. This reduces the number of files and rows that need to be scanned during a query, improving query performance.
- B. Incorrect.
Incorrect. Partitioning by the customer ID column is generally not recommended because it can lead to an excessive number of small partitions due to the high cardinality of customer IDs, which can degrade performance rather than improve it.
- C. Incorrect.
Incorrect. While increasing the number of worker nodes might help with general query performance, it does not address the core issue of inefficient data layout in the Delta table.
- D. Incorrect.
Incorrect. Delta caching can improve performance by storing frequently accessed data in memory, but it does not optimize the underlying data layout, which is the root cause of the query latency in this scenario.