Databricks Machine Learning Professional Question 178
Single answerA retail company is deploying a machine learning model for real-time fraud detection on financial transactions. The deployment involves processing a continuous stream of transaction data. Why is it important to handle complex business logic, such as filtering high-risk transactions based on multiple conditions, directly in the streaming pipeline?
- A
To ensure that the business logic is applied consistently and in near real-time during data processing.
- B
To reduce the computational overhead of the streaming system by postponing business logic to batch processing.
- C
To enable the streaming pipeline to handle larger volumes of data by simplifying its structure.
- D
To minimize latency in decision-making by integrating complex business logic directly into the streaming pipeline.
Show answer and explanation
Correct answer: A
Explanation
In streaming deployments, such as real-time fraud detection, the goal is to process data in near real-time with minimal latency. Handling complex business logic directly in the streaming pipeline ensures that decisions are made consistently and immediately as data flows through the system. This is critical for scenarios where timely actions, such as blocking fraudulent transactions, are required. Postponing such logic to batch processing or oversimplifying the pipeline would undermine the effectiveness of the streaming application.
- A. Correct.
Correct. Applying business logic directly within the streaming pipeline ensures that decisions and transformations occur consistently and in near real-time, which is critical in scenarios like fraud detection.
- B. Incorrect.
Incorrect. Postponing business logic to batch processing increases latency and goes against the purpose of real-time streaming deployments, which aim to provide immediate insights.
- C. Incorrect.
Incorrect. Simplifying a streaming pipeline by removing business logic may improve scalability, but it compromises the ability to perform real-time decision-making and violates the use-case requirements.
- D. Incorrect.
Incorrect. While minimizing latency is important, simply integrating business logic without considering consistency and real-time processing does not guarantee correct or reliable outcomes.