Databricks Data Engineer Professional Question 262
Select 3You are tasked with refactoring a Databricks notebook that currently uses notebook-to-notebook imports for dependencies. You want to adapt the dependency pattern to use Python file dependencies for better modularity and reusability. Which steps should you take to implement this change?
- A
Convert the dependent notebook code into functions and save them in a Python (.py) file.
- B
Upload the Python file to the Databricks workspace and import it in your notebook using
%run. - C
Store the Python file in a shared location like DBFS and use
sys.path.append()to include it in the Python path. - D
Package the Python file as a library, upload it to the cluster as a wheel file, and import it using standard Python
importstatements. - E
Use the
import_notebookutility to convert the notebook dependencies into Python files at runtime.
Show answer and explanation
Correct answers: A, C, D
Explanation
To adapt a notebook dependency pattern to Python file dependencies, you need to refactor the notebook code into reusable Python functions stored in .py files. These Python files can then be made accessible via shared locations like DBFS or by packaging them as libraries for deployment. While %run and import_notebook are related to notebook workflows, they are not applicable for Python file-based dependency management.
- A. Correct.
Correct: Converting the dependent notebook code into functions and saving them in a Python file is necessary to adapt notebook dependencies into Python file dependencies.
- B. Incorrect.
Incorrect:
%runis specific to notebook-to-notebook imports and does not apply when working with Python files. - C. Correct.
Correct: Storing the Python file in a shared location like DBFS and adding it to the Python path using
sys.path.append()allows the file to be imported as a module. - D. Correct.
Correct: Packaging the Python file as a library and uploading it to the cluster as a wheel file enables standard Python imports, which is a best practice for modularity in production environments.
- E. Incorrect.
Incorrect: The
import_notebookutility is not a valid method for converting notebook dependencies into Python file dependencies.