DEA-C01 Question 172
Single answerYou are designing a data pipeline in AWS that processes large volumes of semi-structured data. The pipeline ingests the data into Amazon S3 and later processes it using AWS Glue for analytics. Which data storage format should you choose to optimize storage costs and query performance?
- A
CSV
- B
JSON
- C
Parquet
- D
TXT
Show answer and explanation
Correct answer: C
Explanation
Parquet is the best choice for this scenario because it is a columnar storage format that compresses data efficiently and allows for faster queries by reading only the required columns. This leads to reduced storage costs and improved analytics performance, making it ideal for use cases involving tools like AWS Glue or Amazon Athena. In contrast, formats like CSV, JSON, and TXT lack these optimizations and are less suitable for large-scale analytics workloads.
- A. Incorrect.
CSV is a plain text format that is not optimized for storage or query performance, especially for large datasets. It does not support columnar storage, which can lead to inefficiencies in analytics.
- B. Incorrect.
JSON is a flexible format for semi-structured data but is not optimized for storage or analytics performance. It has high storage overhead and lacks columnar storage capabilities.
- C. Correct.
Parquet is a columnar storage format that is highly optimized for both storage efficiency and analytics performance. It is particularly well-suited for large datasets processed in tools like AWS Glue and Amazon Athena.
- D. Incorrect.
TXT is a plain text format similar to CSV. It is not designed for efficient storage or query performance, making it unsuitable for large-scale analytics.