Databricks Data Engineer Professional Question 260
Select 3You are tasked with refactoring a Databricks notebook that currently uses a notebook dependency pattern. The notebook imports functions from other notebooks using %run. To improve maintainability and modularity, you decide to adapt this pattern to use Python file dependencies. Which of the following steps should you take to achieve this?
- A
Convert the dependent notebooks into Python files and store them in a Databricks Repos folder.
- B
Use the
dbutils.library.installcommand to install the Python files as libraries. - C
Add the folder containing the Python files to the Python path using
sys.path.append. - D
Replace
%runstatements with standard Pythonimportstatements to reference functions in the Python files. - E
Enable the Python file dependencies by configuring the cluster to recognize the
.pyfiles.
Show answer and explanation
Correct answers: A, C, D
Explanation
To adapt a notebook dependency pattern to use Python file dependencies in Databricks, you must first convert the dependent notebooks into Python files, manage their paths, and replace %run with Python's import statements. These steps ensure better modularity, maintainability, and compatibility with standard Python practices.
- A. Correct.
Correct. Converting dependent notebooks into Python files ensures the dependencies are modular and reusable, which is the goal of this refactor.
- B. Incorrect.
Incorrect.
dbutils.library.installis not a valid command for managing Python file dependencies in Databricks. Libraries are typically managed through init scripts or package managers like pip. - C. Correct.
Correct. Adding the folder containing the Python files to the Python path ensures that Python can locate and import the files.
- D. Correct.
Correct. Replacing
%runstatements withimportstatements is necessary to adapt the notebook dependency pattern to Python file dependencies. - E. Incorrect.
Incorrect. There is no need to configure the cluster specifically to recognize
.pyfiles. Databricks automatically supports Python files.