Databricks Data Engineer Associate Question 377
Select 3A data engineering team is deciding between using a triggered pipeline and a continuous pipeline in Databricks for processing incoming streaming data. Which of the following statements correctly compares the two in terms of cost and latency?
- A
Triggered pipelines incur lower costs compared to continuous pipelines because they process data in batches at scheduled intervals.
- B
Continuous pipelines provide lower latency as they process data in near real-time without scheduled triggers.
- C
Triggered pipelines are ideal for scenarios where consistent low-latency processing is required.
- D
Continuous pipelines may incur higher costs because they require resources to run constantly, even during periods of low data volume.
- E
Triggered pipelines are more efficient for workloads with steady, high-frequency data streams.
Show answer and explanation
Correct answers: A, B, D
Explanation
Triggered pipelines are cost-efficient and process data in scheduled batches, making them suitable for workloads with less stringent latency requirements. Continuous pipelines, on the other hand, provide lower latency by processing data as it arrives, but they can incur higher costs due to the constant allocation of resources. Understanding these trade-offs is essential for choosing the right pipeline type based on workload requirements.
- A. Correct.
Correct: Triggered pipelines process data in discrete intervals, which makes them more cost-efficient as resources are not continuously allocated.
- B. Correct.
Correct: Continuous pipelines process data as soon as it arrives, providing lower latency compared to triggered pipelines.
- C. Incorrect.
Incorrect: Triggered pipelines process data in batches, making them less suitable for low-latency requirements.
- D. Correct.
Correct: Continuous pipelines run constantly, which can lead to higher costs due to continuous resource usage, even during periods of low data throughput.
- E. Incorrect.
Incorrect: Triggered pipelines are not ideal for steady, high-frequency streams, as the batch processing nature may introduce latency.