MLA-C01 Question 16
Select 3You are building a machine learning workflow in AWS to process sensor data from IoT devices. The data is streamed in real-time in JSON format and contains nested structures. The architecture must ensure that the data is validated, stored in a highly available manner, and can be ingested efficiently for training a machine learning model. Which combination of services and approaches should you use?
- A
Use Amazon Kinesis Data Streams to collect and validate the real-time data.
- B
Use AWS Glue to transform and flatten the nested JSON data for storage.
- C
Store the data in Amazon S3 in its raw JSON format for cost-effective storage.
- D
Use Amazon DynamoDB for storing the sensor data and directly querying it for training purposes.
- E
Validate the data using custom code running on Amazon EC2 instances before storing it.
Show answer and explanation
Correct answers: A, B, C
Explanation
The combination of Amazon Kinesis Data Streams, AWS Glue, and Amazon S3 provides a scalable, cost-effective, and managed solution for ingesting, validating, transforming, and storing sensor data in a machine learning workflow. Kinesis handles real-time data ingestion and validation, Glue transforms the data for downstream processing, and S3 provides a central repository for storing raw and processed data. Other options, like DynamoDB or EC2, are less optimal for this scenario due to scalability and cost considerations.
- A. Correct.
Amazon Kinesis Data Streams is designed for real-time data ingestion and can be used to validate the data through AWS Lambda or other processing mechanisms.
- B. Correct.
AWS Glue can transform and flatten nested JSON data, making it suitable for downstream processing and storage in a structured format.
- C. Correct.
Amazon S3 is a cost-effective and highly available storage solution that is well-suited for storing raw data, especially for machine learning workflows.
- D. Incorrect.
Amazon DynamoDB is a NoSQL database service, but it is not optimized for large-scale storage of raw, nested JSON data or direct training workflows.
- E. Incorrect.
While custom code on Amazon EC2 could be used for data validation, this approach is less efficient and scalable compared to managed services like Kinesis and AWS Lambda.