Google Professional Machine Learning Engineer Question 394
Single answerGoogle Cloud PlatformYou are a machine learning engineer managing a Google Cloud Composer workflow to automate the retraining of your ML model. The workflow retrieves new training data daily, preprocesses it, trains the model, and then deploys it to Vertex AI. Recently, you noticed that the workflow fails intermittently during the preprocessing step, citing insufficient memory. What is the most effective way to resolve this issue in your Composer environment?
- A
Increase the memory allocated to the Composer worker nodes by customizing the worker machine type.
- B
Increase the number of workers in the Composer environment to distribute the workload.
- C
Use Cloud Dataflow to preprocess the data instead of relying on the Composer worker nodes.
- D
Switch to a different orchestrator, such as Apache Beam, to handle the preprocessing step.
Show answer and explanation
Correct answer: A
Explanation
The intermittent failures during preprocessing are caused by insufficient memory, which can be resolved by customizing the worker machine type in the Composer environment. This allows you to allocate more memory to the tasks without overhauling the existing workflow or migrating to a different tool.
- A. Correct.
This is the correct option because Composer allows you to customize the worker machine types to provide more memory for tasks. This directly addresses the memory issue during preprocessing.
- B. Incorrect.
Increasing the number of workers might help with parallel tasks but does not address the root cause of the memory insufficiency during preprocessing.
- C. Incorrect.
Using Cloud Dataflow could be a valid option for large-scale data processing, but it requires significant changes to your workflow and is unnecessary if increasing worker memory resolves the issue.
- D. Incorrect.
Switching to a different orchestrator is excessive and does not specifically address the memory issue. Composer is designed to handle such workflows effectively when configured correctly.