DEA-C01 Question 118
Select 3You are tasked with implementing a CI/CD pipeline for a data engineering team that processes large datasets using AWS Glue. The pipeline must automatically test and deploy any updates to the data pipeline code. Which combination of AWS services should you use to automate the CI/CD process effectively?
- A
AWS CodePipeline for orchestrating the CI/CD process
- B
AWS Glue for running unit tests during the CI phase
- C
AWS CodeCommit for storing the data pipeline's source code
- D
AWS CodeBuild for executing test cases and building deployment artifacts
- E
Amazon S3 for deploying the updated pipeline code
Show answer and explanation
Correct answers: A, C, D
Explanation
To implement a CI/CD pipeline for data pipelines, you need services that handle source control (AWS CodeCommit), orchestration (AWS CodePipeline), and testing/building (AWS CodeBuild). AWS Glue is used for running ETL jobs, not for CI/CD tasks, and Amazon S3 is used for storage, not for automating deployments.
- A. Correct.
Correct: AWS CodePipeline is designed to orchestrate the CI/CD process by defining the stages and steps for building, testing, and deploying code.
- B. Incorrect.
Incorrect: AWS Glue is primarily a data processing service and is not designed for running unit tests. Unit tests should be executed in a testing framework, such as in AWS CodeBuild.
- C. Correct.
Correct: AWS CodeCommit is a fully managed source control service that integrates well with other AWS CI/CD services for storing pipeline code.
- D. Correct.
Correct: AWS CodeBuild is ideal for running test cases, compiling code, and packaging deployment artifacts as part of the CI/CD pipeline.
- E. Incorrect.
Incorrect: Amazon S3 is used for object storage and is not a deployment tool. It does not automate deployment in CI/CD pipelines.