Databricks Machine Learning Professional Question 177
Select 2A retail company is deploying a real-time recommendation system for its e-commerce platform using a Databricks streaming pipeline. The business logic involves filtering fraudulent transactions, aggregating real-time sales data, and dynamically updating recommendations. Why is it critical to handle this complex business logic directly within the streaming deployment?
- A
Streaming deployments require business logic to ensure low-latency processing of incoming data.
- B
Separating complex business logic into offline systems can lead to stale or inconsistent results in a streaming context.
- C
Business logic in streaming deployments ensures model retraining pipelines are automatically triggered.
- D
Real-time business logic enables seamless scalability of the streaming pipeline to handle spikes in data volume.
- E
Handling business logic in streaming deployments ensures compliance with data governance policies.
Show answer and explanation
Correct answers: A, B
Explanation
Complex business logic must be handled within streaming deployments to maintain the real-time nature of the system. This ensures low-latency processing and prevents stale or inconsistent results that could arise from separating logic into offline systems. Streaming systems are designed to provide actionable insights and decisions in real-time, making it critical to embed the necessary business logic directly into the pipeline.
- A. Correct.
Correct: Low-latency processing is a key requirement of streaming systems, and embedding business logic ensures timely decisions and actions on the incoming data.
- B. Correct.
Correct: Separating business logic into offline systems can result in outdated or inconsistent results, which defeats the purpose of real-time streaming systems.
- C. Incorrect.
Incorrect: While business logic might influence model retraining indirectly, it is not a primary reason for embedding complex logic within streaming deployments.
- D. Incorrect.
Incorrect: Scalability is a function of the streaming architecture and infrastructure, not directly tied to business logic handling.
- E. Incorrect.
Incorrect: Data governance is important but not the primary reason for implementing business logic directly in streaming deployments.