DEA-C01 Question 132
Select 3You are designing a distributed data processing pipeline for an e-commerce platform that processes terabytes of clickstream data daily. The pipeline must scale to handle varying traffic patterns and ensure fault tolerance. Which AWS services should you use to implement a distributed computing solution for this use case?
- A
Amazon EMR for processing the clickstream data using Apache Spark
- B
Amazon S3 for storing raw and processed clickstream data
- C
AWS Lambda for storing the clickstream data in real-time
- D
Amazon EC2 for running a custom-built distributed data processing framework
- E
Amazon Kinesis Data Streams for ingesting clickstream data in real-time
- F
Amazon RDS for processing and analyzing the clickstream data
Show answer and explanation
Correct answers: A, B, E
Explanation
To implement a distributed computing solution for processing terabytes of clickstream data, you need services that can handle distributed processing, storage, and real-time ingestion. Amazon EMR provides managed distributed computing with frameworks like Apache Spark, Amazon S3 offers scalable storage for the data, and Amazon Kinesis Data Streams allows for real-time data ingestion. AWS Lambda and Amazon EC2 are not optimal for this scenario due to their limitations in this distributed use case, while Amazon RDS is not designed for processing large-scale datasets.
- A. Correct.
Amazon EMR is a fully managed service that supports distributed data processing frameworks like Apache Spark and Hadoop, making it ideal for processing large-scale clickstream data.
- B. Correct.
Amazon S3 is a scalable and durable storage service often used for storing raw and processed data in distributed computing pipelines.
- C. Incorrect.
AWS Lambda is a serverless compute service and is not designed for storing data; it is used for executing code in response to events.
- D. Incorrect.
While Amazon EC2 can be used for custom distributed frameworks, it requires significant overhead for configuration, management, and scaling, making it less ideal compared to managed solutions like EMR for this use case.
- E. Correct.
Amazon Kinesis Data Streams is designed for real-time ingestion of streaming data, such as clickstream logs, and integrates well with downstream distributed processing systems.
- F. Incorrect.
Amazon RDS is a relational database service suited for transactional workloads, not for processing and analyzing large-scale distributed datasets like clickstream data.