DOP-C02 Question 224
Select 2Your company is running a highly distributed microservices application on Amazon ECS. The application generates large volumes of logs, which need to be ingested in real-time for analysis and monitoring. The solution must be highly scalable, cost-efficient, and capable of processing millions of events per second. Which combination of services would best meet this requirement?
- A
Use Amazon Kinesis Data Streams to collect and process the logs in real time, and use Amazon Kinesis Data Firehose to route the logs to Amazon S3 for storage.
- B
Use Amazon CloudWatch Logs to directly ingest and analyze the logs in real time.
- C
Set up an Amazon SQS queue to collect the logs, then use an AWS Lambda function to process and store them in Amazon S3.
- D
Use Amazon Kinesis Data Streams to ingest the logs in real time and Amazon Elasticsearch (OpenSearch Service) for real-time analysis.
- E
Configure an Amazon SNS topic to receive log events and replicate them to multiple AWS services for processing.
Show answer and explanation
Correct answers: A, D
Explanation
For real-time log ingestion and processing in a highly scalable and cost-efficient manner, Amazon Kinesis Data Streams is the most suitable service. When paired with Kinesis Data Firehose, it can route logs to Amazon S3 for storage or other destinations for further analysis. Additionally, coupling Kinesis Data Streams with Amazon Elasticsearch (OpenSearch Service) enables real-time analysis of logs, which is critical for monitoring and troubleshooting distributed systems. Other options like CloudWatch Logs, SQS, and SNS are either not designed for high-throughput real-time ingestion or do not provide effective mechanisms for log analysis at scale.
- A. Correct.
Amazon Kinesis Data Streams is designed for real-time ingestion and processing of log data. Combined with Kinesis Data Firehose, it offers a scalable and cost-efficient way to route logs to Amazon S3 for storage and further analysis. This setup is well-suited for real-time logging use cases.
- B. Incorrect.
Amazon CloudWatch Logs is a powerful tool for log management but is not optimized for high-throughput real-time ingestion and processing of millions of events per second. It is better suited for operational monitoring and troubleshooting.
- C. Incorrect.
Amazon SQS is a message queuing service and is not designed for real-time log ingestion or high-throughput processing. Using SQS and Lambda may introduce latency and is less scalable for extremely high log volumes.
- D. Correct.
Amazon Kinesis Data Streams is ideal for real-time log ingestion, and Amazon Elasticsearch (OpenSearch Service) is designed for real-time log analysis. This combination is effective for scenarios requiring both ingestion and immediate search or visualization capabilities.
- E. Incorrect.
Amazon SNS is a pub/sub messaging service and is not designed for real-time log ingestion or analysis. It is more suited for event notification and simple fan-out scenarios.