DEA-C01 Question 48
Select 2You are working as a data engineer for a company that processes large amounts of log data. The raw log data is ingested into an Amazon S3 bucket in real-time. The data must be transformed, enriched, and analyzed using a serverless approach before being loaded into an Amazon Redshift table for reporting. Which combination of services should you use to build this pipeline efficiently?
- A
Amazon S3, AWS Lambda, Amazon Kinesis Data Firehose, Amazon Redshift
- B
Amazon S3, Amazon EMR, Amazon Redshift, AWS Glue
- C
Amazon S3, AWS Lambda, AWS Glue, Amazon Redshift
- D
Amazon S3, Amazon Kinesis Data Streams, AWS Glue, Amazon Redshift
- E
Amazon S3, Amazon Athena, Amazon Redshift, AWS Batch
Show answer and explanation
Correct answers: A, C
Explanation
For a serverless data processing pipeline, Amazon S3 can be used for raw data storage, AWS Lambda for real-time transformations, AWS Glue for enrichment and cataloging, and Amazon Redshift for reporting. Alternatively, Amazon Kinesis Data Firehose can also be used for real-time delivery directly into Amazon Redshift. Both approaches meet the serverless requirement efficiently.
- A. Correct.
Correct: This combination uses Amazon S3 for storage, AWS Lambda for serverless transformations, Amazon Kinesis Data Firehose for real-time data delivery, and Amazon Redshift for reporting. It is an efficient serverless pipeline.
- B. Incorrect.
Incorrect: Amazon EMR is not serverless and would require cluster management. While it can process large-scale data, it does not align with the serverless requirement.
- C. Correct.
Correct: This combination uses Amazon S3 for storage, AWS Lambda for serverless transformations, AWS Glue for data enrichment and cataloging, and Amazon Redshift for reporting. It's a valid serverless approach.
- D. Incorrect.
Incorrect: While Amazon Kinesis Data Streams could be used for ingestion, it is not as straightforward as Kinesis Data Firehose for real-time delivery. This option also misses AWS Lambda for transformations.
- E. Incorrect.
Incorrect: Amazon Athena is used for querying data in-place on S3 and is not suitable for transforming or enriching data before loading into Amazon Redshift. AWS Batch is also not serverless.