Databricks Data Engineer Associate Question 153
Single answerYou are working with a Delta table named events, which contains a column event_time stored as a string in the format 'yyyy-MM-dd HH:mm:ss'. You need to cast the event_time column to a timestamp type for further analysis. Which of the following code snippets correctly performs this operation?
- A
events.withColumn('event_time_ts', events['event_time'].cast('timestamp'))
- B
events.withColumn('event_time_ts', to_timestamp(events['event_time'], 'yyyy-MM-dd HH:mm:ss'))
- C
events.withColumn('event_time_ts', events['event_time'].astype('timestamp'))
- D
events.withColumn('event_time_ts', from_unixtime(events['event_time']))
Show answer and explanation
Correct answer: A
Explanation
The cast('timestamp') method is the appropriate way to convert a string column to a timestamp type in PySpark when the string is already in a valid timestamp format. The other options either use incorrect methods or are inapplicable for this scenario.
- A. Correct.
This is the correct way to cast a column to a timestamp type in PySpark. The
cast('timestamp')method works with columns in a DataFrame. - B. Incorrect.
This is incorrect because
to_timestampis used to convert strings to timestamps, but it is not required here when the string is already in a compatible timestamp format. - C. Incorrect.
This is incorrect because
astypeis not a valid method for casting a column in PySpark. - D. Incorrect.
This is incorrect because
from_unixtimeis used to convert Unix epoch time to a timestamp, which does not apply to the given string format.