Databricks Machine Learning Associate Question 6
Select 1A data scientist is tasked with training a machine learning model on a small dataset using a single machine learning library, and the training process does not require distributed computation. However, later in the project, they plan to scale the training to a larger dataset requiring distributed processing. Which of the following best describes the cluster choices they should make for these two scenarios?
- A
Use a single-node cluster for the small dataset and a standard cluster for the larger dataset.
- B
Use a standard cluster for both the small and larger datasets.
- C
Use a single-node cluster for both the small and larger datasets.
- D
Use a standard cluster for the small dataset and a single-node cluster for the larger dataset.
Show answer and explanation
Correct answer: A
Explanation
Single-node clusters are best suited for lightweight, non-distributed workloads, such as working with small datasets or testing code. Standard clusters are designed for distributed computations and are necessary when scaling to larger datasets or workloads that require parallel processing across multiple nodes.
- A. Correct.
Correct: A single-node cluster is ideal for small, non-distributed workloads, while a standard cluster is suitable for larger datasets that require distributed computation.
- B. Incorrect.
Incorrect: A standard cluster for the small dataset is unnecessary and incurs additional cost and resources for a non-distributed workload.
- C. Incorrect.
Incorrect: A single-node cluster cannot scale to handle distributed computation for the larger dataset.
- D. Incorrect.
Incorrect: This is the opposite of the correct approach. A single-node cluster is suitable for small tasks, not large distributed ones.