DEA-C01 Question 385
Select 2You are a data engineer tasked with designing a data lake architecture for a retail company on AWS. The company wants to store raw clickstream data from its website in an Amazon S3 bucket for analysis. The data will later be refined and transformed for downstream analytics. To ensure efficient querying and cost-effective storage, what steps should you take while designing the data lake?
- A
Enable S3 Object Lock on the bucket to prevent accidental deletions of raw data.
- B
Partition the data in Amazon S3 based on frequently queried attributes, such as date or region.
- C
Enable S3 Transfer Acceleration to speed up data uploads to the bucket.
- D
Use AWS Glue to create a metadata catalog for the raw data stored in S3.
- E
Store the transformed data in the same bucket without any folder structure to minimize complexity.
Show answer and explanation
Correct answers: B, D
Explanation
To optimize querying and ensure cost-effective storage in an Amazon S3-based data lake, it is essential to partition the data based on commonly queried attributes and use a metadata catalog, such as AWS Glue, for schema and data discovery. These steps improve performance and enable efficient use of downstream analytics tools like Athena or Redshift Spectrum.
- A. Incorrect.
Enabling S3 Object Lock is not directly relevant to the requirements in this scenario. It is used for compliance and data protection but does not enhance query efficiency or cost-effectiveness.
- B. Correct.
Partitioning data based on frequently queried attributes, such as date or region, allows for more efficient querying and reduces the cost of scanning unnecessary data during analytics.
- C. Incorrect.
S3 Transfer Acceleration is helpful for speeding up uploads but is not relevant to improving query efficiency or cost-effective storage in this scenario.
- D. Correct.
AWS Glue can be used to create a metadata catalog, making the data in S3 discoverable and easier to query using tools like Amazon Athena.
- E. Incorrect.
Storing transformed data without any folder structure would make it harder to organize, query, and analyze, leading to inefficiencies.