Databricks Data Engineer Professional Question 270
Single answerA data engineering team is migrating their Databricks workflows from using Python Wheels to direct imports with relative paths. They encounter an issue where the relative imports fail to resolve correctly when running in a Databricks notebook. What is the most likely reason for this issue, and how can they fix it?
- A
The Python module paths are not added to the
sys.path, and they need to manually append the module's directory to thesys.path. - B
Relative imports are not supported in Databricks notebooks, and they need to switch back to using Python Wheels.
- C
The Databricks cluster is not using a compatible Python version, and they need to upgrade the cluster's Python version.
- D
The working directory in Databricks notebooks is different, and they need to adjust the relative import paths accordingly.
Show answer and explanation
Correct answer: A
Explanation
When migrating from Python Wheels to direct imports with relative paths, the key step in Databricks is ensuring the module paths are added to the sys.path. Databricks notebooks do not automatically include the module's directory in the sys.path, so you must do this manually. This is a common issue that arises during such migrations, and the solution involves appending the directory containing the modules to sys.path at runtime.
- A. Correct.
Correct. In Databricks notebooks, the module paths for relative imports are not automatically added to the
sys.path. To enable relative imports, you must append the directory containing the module to thesys.pathat runtime. - B. Incorrect.
Incorrect. Databricks notebooks do support relative imports, but you need to ensure the module paths are correctly configured in the
sys.path. Switching back to Python Wheels is unnecessary. - C. Incorrect.
Incorrect. While Python version compatibility can sometimes cause issues, relative imports are a standard feature and are not dependent on specific Python versions in this case.
- D. Incorrect.
Incorrect. The working directory in a Databricks notebook does not affect relative imports if the module paths are added to
sys.pathcorrectly.