Databricks Data Engineer Associate Question 163
Single answerYou are working with a Delta table in Databricks that contains customer data. One of the columns, email_address, contains email addresses in the format username@domain.com. You need to extract only the domain name (e.g., 'domain.com') from the email_address column and store it in a new column called email_domain. Which of the following approaches would correctly achieve this in PySpark?
- A
Use the
regexp_extractfunction with the pattern@(.+)$to extract the domain name. - B
Use the
splitfunction to split the string by the '@' character and select the second element. - C
Use the
substringfunction to extract the portion of the string after the '@' character. - D
Use the
format_stringfunction to reformat theemail_addresscolumn and extract the domain name.
Show answer and explanation
Correct answer: A
Explanation
The regexp_extract function is designed specifically for extracting substrings based on a regular expression pattern. In this case, the regex @(.+)$ captures everything after the '@' character in the email_address column, which is the domain name. This approach is efficient and concise compared to alternatives like split or substring. The format_string function is not relevant for this task.
- A. Correct.
Correct. The
regexp_extractfunction with the regex pattern@(.+)$is the appropriate method to extract the domain name. The pattern captures the portion of the string after the '@' character, which is the domain name. - B. Incorrect.
Incorrect. While the
splitfunction can be used to split the string, it is less efficient and more complex to work with compared toregexp_extractfor this specific task. - C. Incorrect.
Incorrect. The
substringfunction requires you to know the position of the '@' character, which is not guaranteed to be at the same index in every email address. - D. Incorrect.
Incorrect. The
format_stringfunction is used for formatting strings rather than extracting patterns, making it unsuitable for this use case.