DEA-C01 Question 116
Select 3You are managing a data engineering team that builds and deploys ETL pipelines on AWS. Your team has adopted a CI/CD approach to ensure quick and reliable updates to the pipelines. Which combination of AWS services would you use to implement a pipeline that automates the building, testing, and deployment of these ETL pipelines?
- A
AWS CodePipeline
- B
AWS Glue
- C
AWS CodeBuild
- D
AWS CodeDeploy
- E
Amazon EMR
Show answer and explanation
Correct answers: A, C, D
Explanation
To implement CI/CD for ETL pipelines, you need services that automate the build, test, and deployment steps. AWS CodePipeline orchestrates the entire CI/CD workflow, AWS CodeBuild handles building and testing, and AWS CodeDeploy automates deployment. AWS Glue and Amazon EMR are data processing services and are not directly involved in the CI/CD process.
- A. Correct.
AWS CodePipeline is a service designed to orchestrate CI/CD workflows. It can manage the sequence of steps required to build, test, and deploy ETL pipelines in an automated manner.
- B. Incorrect.
AWS Glue is primarily a managed ETL service, not a CI/CD tool. While it can execute ETL jobs, it is not directly involved in CI/CD operations for deploying or testing pipelines.
- C. Correct.
AWS CodeBuild is a fully managed build service that can compile source code, run tests, and produce build artifacts, making it essential for the CI/CD process.
- D. Correct.
AWS CodeDeploy automates the deployment of applications, including ETL pipelines, to various compute services such as Amazon EC2, AWS Lambda, or on-premises servers.
- E. Incorrect.
Amazon EMR is a big data platform for processing vast amounts of data using frameworks like Apache Spark or Hadoop. While it is powerful for data processing, it is not a CI/CD service.