DEA-C01 Question 160
Select 2Your organization needs to process a large volume of semi-structured data stored in Amazon S3, and the processing involves both batch and interactive query use cases. The data must be transformed and stored in a query-optimized format to reduce storage costs and improve query performance. Which combination of AWS services should you use to meet these requirements?
- A
Amazon Glue for data transformation and Amazon Athena for interactive querying
- B
Amazon Redshift for data transformation and storage
- C
Amazon EMR for data transformation and Amazon S3 with Parquet format for storage
- D
Amazon Kinesis Data Analytics for real-time streaming and Amazon S3 for storage
- E
AWS Glue DataBrew for data transformation and Amazon DynamoDB for storage
Show answer and explanation
Correct answers: A, C
Explanation
To process large volumes of semi-structured data stored in S3, the data must be transformed into a query-optimized format like Parquet or ORC to improve query performance and reduce storage costs. Amazon Glue is an ETL service suitable for this transformation, and Amazon Athena provides serverless, pay-per-query interactive analytics on data stored in S3. Alternatively, Amazon EMR can handle large-scale data transformations using frameworks like Apache Spark, and the transformed data can be stored in S3 with a query-optimized format. Both combinations fulfill the requirements of the scenario.
- A. Correct.
Correct: AWS Glue can perform ETL (Extract, Transform, Load) operations to transform data, and Amazon Athena is a serverless query service that allows interactive querying of data stored in S3. Using Athena with a query-optimized format like Parquet or ORC can improve performance and reduce costs.
- B. Incorrect.
Incorrect: Amazon Redshift is a data warehouse solution that is suitable for analytical queries, but it is not ideal for processing semi-structured data stored directly in S3 or for interactive querying of S3 data.
- C. Correct.
Correct: Amazon EMR can process large volumes of data and transform it into a query-optimized format like Parquet, which can then be queried interactively using services like Athena. Storing data in S3 with a format like Parquet reduces storage costs and improves query efficiency.
- D. Incorrect.
Incorrect: Amazon Kinesis Data Analytics is designed for real-time streaming use cases, not batch processing of large volumes of data stored in S3. It does not address the requirement for batch and interactive use cases described in the scenario.
- E. Incorrect.
Incorrect: AWS Glue DataBrew is primarily a visual data preparation tool for cleaning and normalizing data, but it is not designed to handle large-scale ETL transformations required in this scenario. DynamoDB is a NoSQL database and not optimized for analytic queries or storing transformed semi-structured data.