DEA-C01 Question 274
Select 2You are designing a data pipeline in AWS where large volumes of data are ingested daily into an Amazon S3 data lake. This data is queried frequently using Amazon Athena. To optimize both query performance and cost, which of the following strategies should you implement? (Choose TWO)
- A
Partition the data in Amazon S3 based on commonly queried attributes such as date or region.
- B
Compress the data using a columnar format like Parquet or ORC.
- C
Store the data in a single large file in Amazon S3 to reduce metadata overhead.
- D
Avoid using compression to ensure faster read performance in Amazon Athena.
- E
Use AWS Glue to catalog and index the data for easy discovery and querying.
Show answer and explanation
Correct answers: A, B
Explanation
To optimize Amazon Athena queries on an S3 data lake, partitioning the data helps reduce the amount of data scanned by aligning storage with commonly queried attributes. Additionally, compressing data using formats like Parquet or ORC is a best practice, as columnar storage formats improve query efficiency and reduce costs. These techniques are fundamental for optimizing large-scale data pipelines in AWS.
- A. Correct.
Partitioning the data in Amazon S3 based on commonly queried attributes like date or region helps Amazon Athena reduce the amount of data scanned during queries, improving both performance and cost-efficiency.
- B. Correct.
Compressing data using columnar formats like Parquet or ORC reduces storage costs and improves query performance in Athena because these formats allow for selective reading of data.
- C. Incorrect.
Storing data in a single large file in Amazon S3 is not recommended because it can lead to inefficient parallel processing by Amazon Athena and increase query latency.
- D. Incorrect.
Avoiding compression is not a best practice. Compression reduces data size, which lowers storage costs and improves query performance by reducing the amount of data read during queries.
- E. Incorrect.
While AWS Glue enables data cataloging and discovery, it does not directly optimize query performance or reduce costs in Athena.