DOP-C02 Question 226
Select 2Your organization is building a real-time log ingestion pipeline to process application logs from multiple sources and store them for analytics. The logs need to be processed in real-time for monitoring anomalies, and the architecture must be highly scalable and fault-tolerant. Which combination of AWS services should you use to implement this solution?
- A
Amazon Kinesis Data Streams for ingesting logs, AWS Lambda for processing logs, and Amazon S3 for storing processed logs
- B
Amazon SQS for ingesting logs, AWS Lambda for processing logs, and Amazon RDS for storing logs
- C
Amazon Kinesis Firehose for ingesting logs, AWS Glue for transforming logs, and Amazon Redshift for storing logs
- D
Amazon Kinesis Data Streams for ingesting logs, Amazon EMR for processing logs, and Amazon OpenSearch Service for storing and analyzing logs
- E
Amazon EventBridge for ingesting logs, Amazon DynamoDB Streams for processing logs, and Amazon S3 for storing logs
Show answer and explanation
Correct answers: A, D
Explanation
When designing a real-time log ingestion pipeline, you need to select services that support high-throughput, low-latency data streaming, processing, and scalable storage. Amazon Kinesis Data Streams is well-suited for real-time log ingestion, and AWS Lambda or Amazon EMR can be used to process the logs. For storage and analysis, Amazon S3 provides cost-effective storage, while Amazon OpenSearch Service supports log analytics. Options 1 and 4 cover these requirements effectively.
- A. Correct.
This is a valid option. Amazon Kinesis Data Streams is designed for real-time log ingestion, AWS Lambda provides serverless processing capabilities, and Amazon S3 offers scalable and cost-effective storage for processed logs.
- B. Incorrect.
This is incorrect because Amazon SQS is not ideal for real-time log ingestion scenarios. It is better suited for message queuing rather than streaming data. Additionally, Amazon RDS is not optimized for storing logs in a scalable and cost-effective manner.
- C. Incorrect.
This is partially correct. Kinesis Firehose can be used for log ingestion, and Amazon Redshift can store logs for analytics. However, AWS Glue is not suitable for real-time log processing as it is primarily used for ETL jobs.
- D. Correct.
This is a valid option. Amazon Kinesis Data Streams supports real-time log ingestion, Amazon EMR is suitable for scalable log processing, and Amazon OpenSearch Service is designed for log storage and analysis.
- E. Incorrect.
This is incorrect because EventBridge is better suited for event-driven architectures rather than real-time log ingestion. Additionally, DynamoDB Streams is not an appropriate choice for log processing in this scenario.