MLS-C01 Question 33
Single answerYou are a machine learning engineer tasked with creating a real-time data pipeline for processing user activity logs generated by a mobile application. The logs need to be delivered into an S3 bucket in Parquet format for further analysis using Amazon Athena. The incoming data has a high volume and velocity, and you need the solution to handle scale automatically, transform the data into the desired format, and ensure low latency. Which AWS service configuration would best suit this use case?
- A
Use Amazon Kinesis Data Firehose to ingest the data, configure it to transform the data into Parquet format using an AWS Lambda function, and deliver it to Amazon S3.
- B
Use Amazon Kinesis Data Streams to ingest the data, process it with an Amazon EMR cluster, and save the transformed data to Amazon S3.
- C
Use Amazon S3 to directly accept the data, and transform it using AWS Glue to convert it into Parquet format.
- D
Use Amazon Kinesis Data Firehose to ingest the data and deliver it directly to Amazon Redshift for transformation.
Show answer and explanation
Correct answer: A
Explanation
Amazon Kinesis Data Firehose is the most suitable service for this use case because it is designed for ingesting and processing high-velocity data streams, supports data transformation using AWS Lambda, and can deliver data in Parquet format directly to Amazon S3. This solution is highly scalable, minimizes latency, and requires less operational overhead compared to other options.
- A. Correct.
This is the correct answer. Amazon Kinesis Data Firehose supports transformations using AWS Lambda and can directly deliver data in Parquet format to Amazon S3. It is designed for handling high data volumes with minimal latency and scaling automatically.
- B. Incorrect.
While Amazon Kinesis Data Streams can be used for ingestion, it requires additional components like Amazon EMR for processing and transformation, which increases complexity compared to Kinesis Data Firehose.
- C. Incorrect.
Amazon S3 is primarily a storage service and does not natively support real-time ingestion or transformation. AWS Glue can perform transformation, but it is not as efficient for real-time, high-velocity data streams.
- D. Incorrect.
Amazon Kinesis Data Firehose does not natively transform data when delivering to Amazon Redshift. Redshift is primarily a data warehouse and not suited for real-time ingestion and transformation tasks.