AZ-305 Question 98
Single answerYour company needs to integrate data from both on-premises SQL databases (loaded once a day) and real-time sensor data from IoT devices into Azure Data Lake Storage for subsequent analytics in Azure Synapse Analytics. You want a solution that minimizes management overhead, allows you to orchestrate batch and streaming data flows in a single environment, and uses Azure-native services. Which approach most effectively meets these requirements?
- A
Use Azure Synapse Analytics pipelines to orchestrate batch ingestion from on-premises SQL databases, and Azure Event Hubs to process IoT data directly into Azure Data Lake Storage
- B
Use Azure Data Factory pipelines with a self-hosted integration runtime for batch ingestion and connect Azure Event Hubs for real-time data ingestion into Azure Data Lake Storage
- C
Install an on-premises scheduling tool that writes to Azure Blob Storage, and configure Azure IoT Hub to write sensor data to an on-premises SQL server
- D
Use Azure Batch service for orchestrating data movement from on-premises SQL and configure directly with Azure IoT Hub for real-time sensor data landing in Azure Cosmos DB
Show answer and explanation
Correct answer: B
Explanation
In this scenario, you need a unified environment that can coordinate both batch and real-time ingestion using native Azure services. Azure Data Factory (ADF) pipelines are designed for orchestration of data flows, including scheduled batch ingestion from on-premises sources through a self-hosted integration runtime, and they can also integrate with real-time services like Azure Event Hubs. According to Microsoft documentation (https://learn.microsoft.com/azure/data-factory/introduction), ADF is often used in hybrid data integration scenarios, combining cloud-hosted and on-premises data sources for both streaming and batch processing.
- A. Incorrect.
Option 1 is plausible but incomplete. While Azure Synapse Analytics pipelines can handle orchestration and Azure Event Hubs can process real-time feeds, using two separate configurations (Synapse for batch, Event Hubs directly for real-time) may lead to increased management overhead. The question specifically asks for a single environment or a more unified approach.
- B. Correct.
Option 2 is correct. Azure Data Factory can orchestrate both batch ingestion from on-premises SQL (via the self-hosted integration runtime) and real-time data from Azure Event Hubs. This setup provides a unified data integration environment with minimal overhead, allowing you to schedule batch pipelines and handle streaming ingestion effectively into Azure Data Lake Storage.
- C. Incorrect.
Option 3 is incorrect because it reverses the intended data flow: Azure IoT Hub data should feed directly into a cloud service like Event Hubs or an Azure service, not an on-premises SQL server. Also, relying solely on an on-premises scheduling tool to push to Azure could increase administration complexity.
- D. Incorrect.
Option 4 is incorrect because Azure Batch is not designed specifically for orchestrating continuous or scheduled ingestion workflows. It is more suited for parallel batch processing or compute-intensive tasks. Cosmos DB also is not mandated in this scenario for analytical storage; the requirement is explicitly Azure Data Lake Storage.