Databricks Data Engineer Professional Question 267
Select 3A data engineering team maintains reusable Python functions as a custom library packaged in a Wheel file. They want to switch from using the Wheel file to direct imports using relative paths to streamline development and testing in a Databricks notebook. Which steps should they take to successfully implement this change?
- A
Place the Python files in a Databricks repository and ensure the repository is linked to the notebook.
- B
Use the
%pip installcommand in the notebook to install the Wheel file. - C
Modify the imports in the Python files to use relative paths, such as
from .module_name import function_name. - D
Ensure all dependent Python files are in the same directory or properly structured with
__init__.pyfiles for relative imports to work. - E
Add the Wheel file to the Databricks cluster libraries.
Show answer and explanation
Correct answers: A, C, D
Explanation
To move from using Wheel files to direct imports, the Python files must be placed in a location accessible by the notebook, such as a Databricks repository. The import statements in the code must be updated to use relative paths, and the files must be structured correctly to support relative imports. This approach eliminates the need to package the code as a Wheel file, streamlining development and testing.
- A. Correct.
Placing the Python files in a Databricks repository allows the team to directly reference and import the files without needing to package them as a Wheel.
- B. Incorrect.
%pip installis used to install dependencies from PyPI or other Wheel files, which is not applicable if the team is switching to direct imports. - C. Correct.
Modifying imports to use relative paths is required to properly reference modules within the same package or directory structure.
- D. Correct.
For relative imports to work, the files must be in the same directory or organized into a package with
__init__.pyfiles. - E. Incorrect.
Adding the Wheel file to the cluster libraries contradicts the goal of switching to direct imports, as it keeps the dependency on a packaged Wheel.