Databricks Data Engineer Professional exam dumps

Databricks Data Engineer Professional practice question 59 of 313

Databricks Certified Data Engineer Professional. Professional level, Databricks. Free question with the correct answer and a full explanation.

Databricks Data Engineer Professional Question 59

Single answer

You are designing a data pipeline in Databricks to process a large dataset of customer transactions. The dataset contains the following columns: transaction_id, customer_id, transaction_date, store_id, and transaction_amount. The data will be queried frequently by analysts for transactions within specific date ranges and sometimes filtered by store ID. Which strategy should you use to partition the data to optimize query performance and avoid small file issues?

  1. A

    Partition the data by transaction_date and store_id.

  2. B

    Partition the data by customer_id and transaction_id.

  3. C

    Partition the data by transaction_date only.

  4. D

    Do not partition the data to avoid creating too many partitions.

Show answer and explanation

Correct answer: A

Explanation

Partitioning the data by transaction_date and store_id balances query performance and partition management. Analysts frequently query by date ranges and occasionally filter by store ID, so including both columns as partitioning keys ensures efficient data retrieval while reducing the likelihood of small file issues. Partitioning by transaction_date alone is suboptimal since it does not account for store-level filtering, and partitioning by high-cardinality columns like customer_id and transaction_id creates too many partitions, leading to poor performance.

  • A. Correct.

    Partitioning by transaction_date and store_id will optimize the queries that filter by date ranges and store ID, while balancing the number of partitions to avoid small file issues.

  • B. Incorrect.

    Partitioning by customer_id and transaction_id is inefficient because these columns are highly cardinal, leading to excessive partitions and poor query performance.

  • C. Incorrect.

    Partitioning by transaction_date only could improve query performance for date range filters, but it would not optimize queries that also filter by store_id.

  • D. Incorrect.

    Not partitioning the data is generally not recommended for large datasets, as it can lead to inefficiency in querying and processing.

Timed practice exam

Take a Databricks Data Engineer Professional practice test under exam conditions

60 questions in 120 minutes, drawn from this bank, with a score report and a per-question review when you finish.

Start timed exam

More Databricks Data Engineer Professional practice questions

  1. 60.You are working on a Delta Lake table containing a large volume of e-commerce transaction data....
  2. 61.You are working with a PySpark DataFrame in Databricks and need to write the data to disk in...
  3. 62.You are working on a PySpark job where you need to write a DataFrame to disk in Parquet format. To...
  4. 63.You are working on a PySpark job in Databricks, and you need to write a DataFrame to disk in...
  5. 64.You are working on a PySpark job that processes a large dataset and writes the output to disk in...
Back to the Databricks Data Engineer Professional exam dumps