Databricks Data Engineer Professional Question 85
Select 3You are tasked with implementing a stream-static join in Databricks to enrich streaming data from a Kafka source with static data stored in a Delta table. The static Delta table contains customer information, while the streaming data contains transactions with a customer_id field. Which of the following steps are necessary to correctly implement the stream-static join in Databricks?
- A
Load the static Delta table into a DataFrame.
- B
Use the
joinoperation with an appropriate join condition based on thecustomer_idfield. - C
Ensure the static DataFrame is continuously updated to reflect new data before performing the join.
- D
Write the joined result to a Delta table using a streaming write operation.
- E
Cache the static DataFrame to improve join performance.
Show answer and explanation
Correct answers: A, B, D
Explanation
To implement a stream-static join in Databricks, the static Delta table must be loaded into a DataFrame, and a join operation must be performed with a valid condition. The joined result should be written to a sink using a streaming write operation. Continuously updating the static DataFrame or caching it is not a strict requirement for this operation.
- A. Correct.
Correct. To perform a stream-static join, the static Delta table must be loaded into a DataFrame so it can be used in the join operation.
- B. Correct.
Correct. A
joinoperation with a valid join condition (e.g., matchingcustomer_idin the streaming and static datasets) is required to combine the data. - C. Incorrect.
Incorrect. The static DataFrame does not need to be continuously updated. Stream-static joins assume the static dataset remains unchanged during the streaming process.
- D. Correct.
Correct. After performing the join, the combined result needs to be written to a sink, such as a Delta table, using a streaming write operation.
- E. Incorrect.
Incorrect. While caching the static DataFrame might improve performance, it is not a mandatory step for implementing stream-static joins.