Databricks Machine Learning Professional Question 29
Select 3You are training multiple machine learning models for a regression problem on Databricks. To ensure reproducibility and track the performance of each experiment, you decide to log key metrics and parameters using MLflow. Which of the following steps are essential to correctly log and track your experiments in MLflow on Databricks?
- A
Use the MLflow tracking API to start an experiment by specifying a unique experiment name or ID.
- B
Log all model parameters, metrics, and artifacts manually to a local directory on the Databricks cluster.
- C
Set the experiment using
mlflow.set_experiment()to associate runs with the correct experiment. - D
Ensure that the Databricks workspace is integrated with an external database for storing MLflow logs.
- E
Use MLflow's autologging feature to automatically capture model parameters, metrics, and artifacts.
Show answer and explanation
Correct answers: A, C, E
Explanation
To track experiments effectively in MLflow on Databricks, you must start an experiment and associate runs using the MLflow tracking API and mlflow.set_experiment(). Additionally, leveraging autologging simplifies the process of logging parameters, metrics, and artifacts. These steps ensure reproducibility and organization across machine learning experiments. Manually logging to local directories or requiring external database integration are not necessary in Databricks' managed environment.
- A. Correct.
Correct. Starting an experiment using the MLflow tracking API by specifying a unique experiment name or ID is essential to track runs in an organized manner.
- B. Incorrect.
Incorrect. MLflow automatically manages the logging of artifacts, metrics, and parameters to its tracking server. Manually logging to a local directory is not required and would not be integrated with the MLflow tracking system.
- C. Correct.
Correct. Setting the experiment context using
mlflow.set_experiment()ensures that the runs are properly associated with the specified experiment in MLflow. - D. Incorrect.
Incorrect. While external databases can be used with MLflow, Databricks provides its own managed MLflow tracking server, so external database integration is not required for standard use cases.
- E. Correct.
Correct. MLflow's autologging feature simplifies the process by automatically capturing model parameters, metrics, and artifacts for supported machine learning frameworks.