Databricks Data Engineer Associate Question 107
Select 3You are tasked with creating a Delta table in Databricks using data from two different sources: a relational database accessible via JDBC and an external CSV file stored in a cloud storage bucket. Which of the following steps are correct to load the data and create the table?
- A
Use the
spark.read.format('jdbc')method to load data from the database, then write the data to a Delta table usingwrite.format('delta'). - B
Use
CREATE TABLE ... USING CSVto directly create a table from the external CSV file location. - C
Use
spark.read.format('csv')to load the CSV file, transform the data if needed, and then write it to a Delta table usingwrite.format('delta'). - D
Use the
CREATE DELTA TABLESQL syntax to directly load data from a JDBC source into a Delta table. - E
Configure the JDBC connection URL and credentials in a secret scope, then use
spark.read.jdbcto load the data and write it to a Delta table.
Show answer and explanation
Correct answers: A, C, E
Explanation
To create a Delta table in Databricks from a JDBC source or an external CSV file, you should use the appropriate Spark DataFrame API methods (spark.read.format('jdbc') or spark.read.format('csv')) to load the data into a DataFrame. Once loaded, you can write the data to a Delta table using write.format('delta'). Additionally, securely configuring JDBC connection details in a secret scope is considered a best practice.
- A. Correct.
Correct: This is the correct way to load data from a JDBC source into a Delta table. The
spark.read.format('jdbc')method allows you to read the data, andwrite.format('delta')ensures it is written to a Delta table. - B. Incorrect.
Incorrect: While
CREATE TABLE ... USING CSVcan create a table from an external CSV, it does not automatically create a Delta table and would require additional steps to convert the data to Delta format. - C. Correct.
Correct: Using
spark.read.format('csv')is the proper way to load data from a CSV file programmatically. After transformations, writing it withwrite.format('delta')creates a Delta table. - D. Incorrect.
Incorrect: There is no
CREATE DELTA TABLESQL syntax that directly loads data from a JDBC source. You need to use Spark DataFrame APIs for this. - E. Correct.
Correct: Configuring the JDBC connection URL and credentials in a secret scope is a best practice for managing sensitive information securely. Using
spark.read.jdbcto load the data is a valid approach, followed by writing it to a Delta table.