SAA-C03 Question 360
Single answerYour company is building a data lake on AWS to analyze customer transaction data. The data arrives daily in raw CSV files and needs to be transformed into a Parquet format for efficient querying. Additionally, you must clean the data by removing null values and standardizing the date format before storing it in Amazon S3 for querying with Amazon Athena. Which AWS service should you use to automate this data transformation process?
- A
AWS Glue
- B
Amazon EMR
- C
AWS Lambda
- D
Amazon Redshift
Show answer and explanation
Correct answer: A
Explanation
AWS Glue is the best choice for this scenario because it provides a fully managed ETL service with built-in support for transforming data (e.g., converting CSV to Parquet, cleaning data) and is integrated with Amazon S3 and Athena. This makes it efficient and cost-effective for automating the data transformation process in a data lake architecture.
- A. Correct.
AWS Glue is a fully managed ETL (Extract, Transform, Load) service that is ideal for automating data transformation tasks like converting formats, cleaning data, and preparing it for querying or analysis. It supports integration with Amazon S3 and works seamlessly with Athena.
- B. Incorrect.
Amazon EMR can process large-scale data transformations, but it requires significant management effort and is better suited for complex big data workloads like running Apache Spark or Hadoop jobs. It is not as straightforward or automated as AWS Glue for this use case.
- C. Incorrect.
AWS Lambda can trigger and execute small-scale transformations but is not suitable for large-scale ETL processes like converting large datasets from CSV to Parquet or performing complex data cleaning. It also lacks built-in support for data cataloging and schema inference.
- D. Incorrect.
Amazon Redshift is a data warehousing service and is not designed for ETL-specific tasks. While it can store and query data, it does not handle transformation workflows like AWS Glue does.