MLA-C01 Question 17
Select 3You are tasked with building a machine learning pipeline on AWS to process a large dataset containing customer transaction logs. The data is stored in JSON format and needs to be ingested into Amazon SageMaker for training. The pipeline must validate the data for schema consistency and handle missing fields. Which combination of services and mechanisms should you use to meet these requirements?
- A
Use AWS Glue to perform data schema validation and transformation before storing the data in Amazon S3.
- B
Use Amazon Kinesis Data Firehose to stream the data directly into Amazon SageMaker for training.
- C
Store the JSON data in Amazon S3 and use AWS Glue DataBrew to identify and handle missing fields.
- D
Use Amazon S3 Event Notifications to trigger an AWS Lambda function for schema validation and preprocessing.
- E
Directly load the JSON data into Amazon SageMaker and use built-in algorithms to handle schema validation.
Show answer and explanation
Correct answers: A, C, D
Explanation
To meet the requirements of validating schema consistency and handling missing fields in the JSON data, you can use AWS Glue for schema validation and transformation, AWS Glue DataBrew for identifying and handling missing fields, and Amazon S3 Event Notifications with AWS Lambda for automated preprocessing. These tools together ensure the data is clean and consistent before being ingested into Amazon SageMaker for training.
- A. Correct.
AWS Glue is designed for data transformation and validation. It integrates well with Amazon S3, making it suitable for preparing data before ML training.
- B. Incorrect.
Amazon Kinesis Data Firehose cannot directly stream data into SageMaker for training. It is more suitable for real-time data delivery to other destinations such as Amazon S3 or Redshift.
- C. Correct.
AWS Glue DataBrew provides a visual interface to clean and validate data, making it an excellent tool for identifying and handling missing fields in the dataset.
- D. Correct.
Amazon S3 Event Notifications can trigger AWS Lambda functions to perform schema validation and preprocessing, ensuring data consistency before it is used for ML training.
- E. Incorrect.
Amazon SageMaker is not designed to natively handle schema validation or preprocessing. These tasks should be handled before the data is fed into the training process.