Google Professional Data Engineer Question 117
Select 3Google Cloud PlatformYour team is building a data pipeline on Google Cloud that processes streaming data in real-time. The pipeline includes multiple stages such as ingestion, transformation, and storage in BigQuery. The team wants to implement a CI/CD pipeline to automatically test and deploy updates to the data pipeline code. Which of the following practices should you adopt to effectively implement CI/CD for this use case?
- A
Use Cloud Build to create a pipeline that automates testing, containerization, and deployment of the data pipeline.
- B
Store the data pipeline code and configuration in a version control system, such as Cloud Source Repositories or GitHub.
- C
Manually test the data pipeline after each change to ensure it processes data correctly before deploying.
- D
Implement unit and integration tests for the data pipeline stages and run them automatically as part of the CI/CD pipeline.
- E
Deploy changes directly to production without testing to minimize downtime.
Show answer and explanation
Correct answers: A, B, D
Explanation
Implementing CI/CD for a data pipeline requires using automation tools like Cloud Build, maintaining version control for code and configurations, and automating testing with unit and integration tests. These practices ensure that changes are tested and deployed efficiently and reliably while reducing manual intervention and risks.
- A. Correct.
Correct: Cloud Build is a serverless CI/CD platform on Google Cloud that can automate testing, containerization, and deployment, which are essential for CI/CD implementation.
- B. Correct.
Correct: Storing the pipeline code in a version control system enables tracking changes, collaboration, and integration with CI/CD tools.
- C. Incorrect.
Incorrect: CI/CD pipelines are designed to automate testing, eliminating the need for manual testing after every change. Manual testing is time-consuming and prone to human error.
- D. Correct.
Correct: Automated unit and integration tests ensure the data pipeline stages function as expected and help identify issues early in the CI/CD process.
- E. Incorrect.
Incorrect: Deploying changes directly to production without testing is risky and violates the principles of CI/CD, which emphasize testing to ensure reliability and stability.