Databricks Data Engineer Associate Question 389
Single answerYou are using Auto Loader to ingest JSON files into a Delta table. After inspecting the schema inferred by Auto Loader, you notice that all fields have been inferred as STRING types, even though the data contains different data types such as integers and booleans. What could be the reason for this behavior?
- A
The schema inference is disabled in Auto Loader.
- B
The JSON files contain inconsistent data types for certain fields across different records.
- C
Auto Loader does not support schema inference for JSON files.
- D
The JSON files are improperly formatted, causing Auto Loader to treat all fields as STRING.
Show answer and explanation
Correct answer: B
Explanation
Auto Loader infers the schema of JSON files based on the data in the files. If a field has inconsistent data types across different records (e.g., some records provide an integer and others provide a string), Auto Loader will default to inferring the field as STRING to handle all variations without errors. This behavior ensures data can be ingested without type conflicts but may require additional steps to cast fields to their appropriate data types.
- A. Incorrect.
Schema inference is enabled by default in Auto Loader, so this is not the correct explanation.
- B. Correct.
If the JSON files have inconsistent data types for the same field across records, Auto Loader defaults to inferring these fields as STRING to avoid data type conflicts.
- C. Incorrect.
Auto Loader supports schema inference for JSON files, so this is not the correct explanation.
- D. Incorrect.
Improper formatting of JSON files typically leads to schema inference errors or ingestion failures, not defaulting all fields to STRING.