Databricks Data Engineer Professional Question 264
Select 3You are tasked with refactoring a Databricks notebook that contains multiple utility functions used across several other notebooks. To improve code modularity and reusability, you decide to move these utility functions into a Python file dependency. Which of the following steps are required to correctly use the Python file in place of the notebook dependency?
- A
Save the utility functions into a Python file (e.g.,
utils.py) and upload it to the Databricks Workspace or a DBFS location. - B
Use the
dbutils.library.installcommand to install the Python file as a library in your cluster. - C
Import the Python file in your target notebooks using the
importstatement. - D
Add the Python file path to the Python system path using
sys.path.append()if it is not in the default import path. - E
Use
%runto call the Python file and execute its code in the target notebook.
Show answer and explanation
Correct answers: A, C, D
Explanation
To refactor a notebook dependency pattern into Python file dependencies, you need to save the utility functions in a Python file, make it accessible to the notebooks (e.g., by uploading it to the workspace or DBFS), and import it using the import statement. If the file is not in a default import path, modifying sys.path is also required. %run is not applicable for Python files, and dbutils.library.install is for installing libraries, not importing custom Python files.
- A. Correct.
Correct: Saving the utility functions into a Python file is the first step to modularizing code and enabling reuse.
- B. Incorrect.
Incorrect:
dbutils.library.installis used for installing libraries but is not applicable for loading custom Python files in Databricks. - C. Correct.
Correct: Importing the Python file using the
importstatement is necessary to access the utility functions in other notebooks. - D. Correct.
Correct: Adding the Python file path to
sys.pathis sometimes required if the file is stored in a custom location not included in the default import paths. - E. Incorrect.
Incorrect:
%runis specific to running other notebooks but cannot be used to load Python files.