Databricks Machine Learning Professional Question 169
Single answerA retail company wants to build an ETL pipeline that processes real-time customer transactions to generate aggregated metrics, such as total sales per hour, which will be used for live dashboards. They decide to use Databricks' Structured Streaming for this task. Which of the following reasons best explains why Structured Streaming is a suitable choice?
- A
Structured Streaming supports incremental, fault-tolerant computation for processing real-time data.
- B
Structured Streaming can only process batch data, making it suitable for ETL pipelines that require periodic updates.
- C
Structured Streaming does not require schema definitions, making it faster to implement ETL pipelines.
- D
Structured Streaming automatically scales to handle both real-time and batch data with minimal configuration.
Show answer and explanation
Correct answer: A
Explanation
Structured Streaming is a powerful tool in Databricks for building ETL pipelines that require real-time data processing, such as those needed for live dashboards. Its support for incremental and fault-tolerant computation ensures data is processed reliably and efficiently, making it a suitable choice for the described scenario.
- A. Correct.
Structured Streaming supports incremental, fault-tolerant computation, making it ideal for processing continuous streams of real-time data efficiently. This is the correct explanation for choosing Structured Streaming for real-time ETL pipelines.
- B. Incorrect.
This statement is incorrect because Structured Streaming is specifically designed for real-time data processing, not just batch data.
- C. Incorrect.
This statement is incorrect because Structured Streaming typically requires schema definitions to ensure proper processing of structured data.
- D. Incorrect.
While Structured Streaming can handle both real-time and batch data, its ability to scale automatically is not the primary reason for its suitability in this scenario.