DEA-C01 Question 173
Single answerYou are designing a data pipeline in AWS to process large volumes of data stored in Amazon S3. The data will be queried frequently by AWS Glue and Amazon Athena for analytical purposes. Which data storage format should you choose to optimize performance and reduce costs, and why?
- A
CSV
- B
JSON
- C
Parquet
- D
TXT
Show answer and explanation
Correct answer: C
Explanation
Parquet is the optimal choice for this scenario because it is a columnar storage format that is explicitly designed for analytical workloads. When querying large datasets with services like AWS Glue and Amazon Athena, Parquet significantly reduces the amount of data scanned by allowing access to only the relevant columns. Additionally, it supports efficient compression, which minimizes storage costs and accelerates query performance. In contrast, formats like CSV, JSON, and TXT are less efficient for this use case.
- A. Incorrect.
CSV is a plain text format and lacks columnar storage, making it less efficient for read-heavy analytical workloads. It also results in higher storage costs and slower query performance.
- B. Incorrect.
JSON is a semi-structured format and not optimized for analytical workloads. It lacks columnar storage and can be verbose, leading to higher storage costs and slower query performance.
- C. Correct.
Parquet is a columnar storage format, making it highly efficient for analytical workloads where only specific columns are queried. It also supports compression, reducing storage costs and improving query performance in services like AWS Glue and Amazon Athena.
- D. Incorrect.
TXT is a plain text format and is not designed for analytical workloads. It lacks structure, compression, and columnar capabilities, resulting in higher costs and lower performance.