DEA-C01 Question 171
Single answerYou are designing a data pipeline to process large volumes of semi-structured data in AWS. The data will be processed using Amazon Athena for ad-hoc querying and stored in Amazon S3. Which storage format should you choose to optimize query performance and reduce storage costs?
- A
CSV
- B
JSON
- C
Parquet
- D
TXT
Show answer and explanation
Correct answer: C
Explanation
Parquet is the best choice for this scenario because it is a columnar storage format. This allows Amazon Athena to scan only the required columns during queries, significantly reducing the amount of data read and improving performance. Additionally, Parquet supports efficient compression, which reduces storage costs in Amazon S3 compared to row-oriented formats like CSV or JSON.
- A. Incorrect.
CSV is a plain-text format that is not optimized for query performance. It lacks features like columnar storage and compression, leading to higher storage costs and slower query performance.
- B. Incorrect.
JSON is semi-structured and flexible but is not optimized for analytics workloads. It does not support columnar storage and often results in increased storage and query costs compared to optimized formats like Parquet.
- C. Correct.
Parquet is a columnar storage format that is highly optimized for analytics workloads. It supports efficient compression and query performance, making it ideal for use with tools like Amazon Athena.
- D. Incorrect.
TXT is a simple text format that is not well-suited for large-scale analytics or optimized querying. It does not provide features like compression or columnar storage.