Databricks Machine Learning Professional Question 259
Select 3You are tasked with monitoring numerical feature drift in a production machine learning pipeline. After evaluating different methods, you decide to compare the use of Jensen-Shannon divergence and the Kolmogorov-Smirnov (KS) test. Which of the following statements accurately explain key differences between the two methods?
- A
Jensen-Shannon divergence is a symmetric measure of distributional difference, while the Kolmogorov-Smirnov test is asymmetric.
- B
The Kolmogorov-Smirnov test provides a p-value to assess statistical significance of drift, while Jensen-Shannon divergence does not.
- C
Jensen-Shannon divergence is specifically designed for categorical distributions, while the Kolmogorov-Smirnov test is designed for numerical data.
- D
The Kolmogorov-Smirnov test evaluates the maximum difference between two empirical cumulative distribution functions (ECDFs), while Jensen-Shannon divergence measures the average divergence between two probability distributions.
- E
Jensen-Shannon divergence requires binning or density estimation for numerical features, while the Kolmogorov-Smirnov test works directly with raw numerical data.
Show answer and explanation
Correct answers: B, D, E
Explanation
Jensen-Shannon divergence and the Kolmogorov-Smirnov test are commonly used for drift detection, but they differ significantly in their methodologies and applications. The Kolmogorov-Smirnov test is a statistical test that operates directly on numerical data and provides a p-value for statistical significance. In contrast, Jensen-Shannon divergence is a symmetric measure of average divergence between probability distributions and requires preprocessing for numerical data. Understanding these differences is crucial for selecting an appropriate method for drift detection in machine learning pipelines.
- A. Incorrect.
This is incorrect. Jensen-Shannon divergence is symmetric, but the Kolmogorov-Smirnov test is also symmetric in comparing two distributions. It does not favor one distribution over the other.
- B. Correct.
This is correct. The Kolmogorov-Smirnov test provides a p-value to determine whether two distributions differ significantly, which is not a feature of Jensen-Shannon divergence.
- C. Incorrect.
This is incorrect. Jensen-Shannon divergence is not specific to categorical distributions; it can also be applied to numerical data with appropriate preprocessing (e.g., binning).
- D. Correct.
This is correct. The Kolmogorov-Smirnov test measures the maximum vertical distance between two ECDFs, while Jensen-Shannon divergence quantifies the average divergence between two probability distributions based on their Kullback-Leibler divergences.
- E. Correct.
This is correct. Jensen-Shannon divergence requires preprocessing such as binning or density estimation to handle numerical data, whereas the Kolmogorov-Smirnov test directly compares raw numerical data without requiring such transformations.