Databricks Data Engineer Associate Question 168
Single answerYou are working with a DataFrame in Databricks that contains nested JSON data about customer orders. The schema is as follows:
root |-- order_id: string |-- customer: struct | |-- name: string | |-- contact: struct | |-- email: string | |-- phone: string
How would you extract only the customer's email address using dot syntax while working with this DataFrame?
- A
df.select('customer.contact.email')
- B
df.select('customer.email')
- C
df.select('contact.email')
- D
df.selectExpr('customer.contact.email')
Show answer and explanation
Correct answer: A
Explanation
To extract nested fields in a DataFrame, you use dot syntax to traverse the hierarchy. In this case, 'customer.contact.email' correctly navigates through the nested structure to access the email field. The other options either reference invalid paths or do not adhere to the requirements of the question.
- A. Correct.
Correct. This is the proper way to access a nested field (email) using dot syntax in the DataFrame API.
- B. Incorrect.
Incorrect. There is no field named 'email' directly under the 'customer' struct. It is nested under 'contact'.
- C. Incorrect.
Incorrect. There is no field named 'contact' at the top level of the DataFrame. It is nested under 'customer'.
- D. Incorrect.
Incorrect. While selectExpr can be used for SQL-style expressions, the question specifically asks for dot syntax which is used in the DataFrame API.