Databricks Data Engineer Associate Question 109
Select 3You are tasked with creating tables in Databricks for a data engineering project. The first table needs to be created from a CSV file stored in an external cloud storage (e.g., S3), and the second table needs to be created from a relational database using a JDBC connection. Which of the following steps are required to accomplish this?
- A
Use the Spark
readAPI to load the CSV file into a DataFrame and then write it as a table using thesaveAsTablemethod. - B
Configure the JDBC connection with a valid URL, driver, and credentials, and then use the Spark
readAPI with the JDBC format to load the data into a DataFrame. - C
Use the Databricks File System (DBFS) commands to directly query the CSV file and create a table from it.
- D
Use the Spark SQL
CREATE TABLEcommand to directly create a table from the JDBC source without loading the data into a DataFrame. - E
Use the Spark
writeAPI to save the DataFrame loaded from the JDBC source as a Delta table.
Show answer and explanation
Correct answers: A, B, E
Explanation
The correct process for creating a table from an external CSV file involves loading the file into a DataFrame using the Spark read API and saving it as a table using the saveAsTable method. For a JDBC source, you must configure the connection details, load the data into a DataFrame using the Spark read API, and then save it as a table using the Spark write API. DBFS commands and direct SQL CREATE TABLE statements are not sufficient for these tasks.
- A. Correct.
Correct: The Spark
readAPI is used to load data from a CSV file into a DataFrame, and thesaveAsTablemethod saves it as a table in Databricks. - B. Correct.
Correct: To create a table from a JDBC connection, you need to provide the valid connection details and use the Spark
readAPI with the JDBC format to load the data into a DataFrame. - C. Incorrect.
Incorrect: DBFS commands can be used to manipulate files, but you cannot directly query a CSV file and create a table without using Spark APIs.
- D. Incorrect.
Incorrect: The Spark SQL
CREATE TABLEcommand cannot directly create a table from a JDBC source; the data must first be loaded into a DataFrame. - E. Correct.
Correct: After loading data from a JDBC source into a DataFrame, the Spark
writeAPI can be used to save it as a Delta table for optimized storage and querying.