DEA-C01 Question 225
Single answerYou are working as a data engineer for a retail company. The company stores customer transaction data in an Amazon S3 bucket and wants to analyze this data using Amazon Redshift. They have millions of small files in the S3 bucket, and you need to ensure that the data is ingested efficiently into Amazon Redshift. What is the best approach to achieve this?
- A
Use the COPY command in Amazon Redshift to load the files directly from the S3 bucket without any preprocessing.
- B
Use AWS Glue to merge the small files into larger files before loading them into Amazon Redshift using the COPY command.
- C
Use Amazon Redshift Spectrum to directly query the small files in the S3 bucket without loading them into Amazon Redshift.
- D
Upload the small files to an Amazon RDS database and then use Amazon Redshift's federated query feature to analyze the data.
Show answer and explanation
Correct answer: B
Explanation
When dealing with millions of small files in Amazon S3, it is best to merge them into larger files before loading them into Amazon Redshift. AWS Glue can help preprocess and consolidate the files, improving the efficiency of the COPY command in Amazon Redshift. This approach ensures faster data ingestion and better query performance within Amazon Redshift.
- A. Incorrect.
Loading millions of small files directly into Amazon Redshift using the COPY command is inefficient and can lead to poor performance due to the overhead of managing many small files.
- B. Correct.
Using AWS Glue to merge small files into larger files before loading them into Amazon Redshift is the recommended approach. It reduces the number of files and improves the performance of the COPY command.
- C. Incorrect.
Amazon Redshift Spectrum can query data in S3 directly, but it is not the most efficient solution for long-term analytics. For performance and optimization, data should be ingested into Amazon Redshift.
- D. Incorrect.
Uploading the small files to an Amazon RDS database and then using federated queries is not a recommended approach for this use case and introduces unnecessary complexity.