DEA-C01 Question 175
Single answerA data engineering team is designing a data pipeline for a large-scale analytics workload. The pipeline processes terabytes of log data daily and stores it in Amazon S3. The data will be analyzed using Amazon Athena, and the team needs to optimize storage costs and query performance. Which data storage format should they choose?
- A
CSV
- B
JSON
- C
Parquet
- D
TXT
Show answer and explanation
Correct answer: C
Explanation
Parquet is the best choice for this scenario because it is a columnar storage format optimized for analytics. It allows for efficient compression and faster query performance, making it ideal for processing large-scale data in Amazon Athena. CSV, JSON, and TXT formats either lack columnar storage or compression capabilities, leading to higher storage costs and slower queries.
- A. Incorrect.
CSV is a plain text format that is easy to use but does not support columnar storage or compression, which would result in higher storage costs and slower query performance.
- B. Incorrect.
JSON is a semi-structured format that is suitable for unstructured data but is not optimized for analytics, as it lacks columnar storage and efficient compression.
- C. Correct.
Parquet is a columnar storage format designed for analytics. It supports efficient compression and query performance, making it ideal for large-scale analytics workloads.
- D. Incorrect.
TXT is a plain text format that does not support columnar storage or compression, so it is not suitable for optimizing storage costs or query performance in this scenario.