Databricks Machine Learning Professional Question 258
Select 4You are monitoring numerical feature drift in a machine learning model's input data. You are considering using either the Jensen-Shannon (JS) divergence or the Kolmogorov-Smirnov (KS) test. Which of the following statements accurately describe the differences between these two methods?
- A
The JS divergence measures the similarity between two probability distributions, while the KS test measures the maximum difference between their cumulative distributions.
- B
The KS test requires continuous numerical data, while the JS divergence can operate on both continuous and discrete probability distributions.
- C
The JS divergence outputs a probabilistic p-value to indicate drift, while the KS test outputs a divergence score between 0 and 1.
- D
The KS test is non-parametric and does not assume any specific distribution for the data, while the JS divergence assumes the data follows a probability distribution.
- E
The JS divergence is sensitive to changes in the shape of the distribution, while the KS test focuses on detecting large deviations in the cumulative distribution.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
The Jensen-Shannon divergence and Kolmogorov-Smirnov test are both methods for detecting numerical drift, but they differ in their focus and assumptions. The JS divergence measures the similarity between two probability distributions and is better suited for detecting subtle changes in distribution shape. It can handle both discrete and continuous data. The KS test, however, compares the cumulative distributions of two datasets, is non-parametric, and is more sensitive to large deviations. Understanding these differences is crucial when selecting the appropriate metric for drift detection in machine learning workflows.
- A. Correct.
Correct. The JS divergence quantifies the similarity between two probability distributions, whereas the KS test identifies the maximum vertical distance between their cumulative distribution functions.
- B. Correct.
Correct. The KS test specifically requires numerical data and compares their cumulative distributions, whereas the JS divergence works on probability distributions, which can represent both discrete and continuous data.
- C. Incorrect.
Incorrect. The JS divergence does not output a p-value; it outputs a divergence score between 0 and 1 to measure the difference between distributions. The KS test, on the other hand, provides a p-value for hypothesis testing.
- D. Correct.
Correct. The KS test is non-parametric and makes no assumptions about the data's underlying distribution. The JS divergence, however, assumes the data can be represented as a probability distribution.
- E. Correct.
Correct. The JS divergence is effective in detecting subtle changes in distribution shapes, while the KS test is more sensitive to large deviations between cumulative distribution functions.