MLS-C01 Question 90
Select 3You are building a supervised machine learning model to classify customer product reviews as 'positive', 'neutral', or 'negative'. Your dataset consists of 10,000 unlabeled text samples. To label this data efficiently, you decide to use Amazon Mechanical Turk. Which of the following steps should you take to ensure high-quality labeled data for your model?
- A
Create clear and detailed instructions for the Mechanical Turk workers, including examples of positive, neutral, and negative reviews.
- B
Set up multiple workers to label each review and implement an aggregation mechanism to resolve conflicts between labels.
- C
Allow workers to label data without any quality checks to speed up the process.
- D
Use qualification tests to ensure workers understand the labeling task before they are allowed to participate.
- E
Accept the first label provided by a worker for each data point to minimize costs.
Show answer and explanation
Correct answers: A, B, D
Explanation
To ensure high-quality labeled data when using Amazon Mechanical Turk, it is essential to provide clear instructions, implement quality control mechanisms such as worker qualification tests and aggregation of multiple labels, and avoid shortcuts that compromise label accuracy. These steps help build a reliable dataset for training your machine learning model.
- A. Correct.
Clear and detailed instructions with examples help ensure workers understand the labeling task and produce consistent results. This is critical for obtaining high-quality labels.
- B. Correct.
Using multiple workers to label the same data point and aggregating their responses (e.g., majority voting) reduces the risk of errors and improves label accuracy.
- C. Incorrect.
Allowing workers to label data without quality checks can lead to inconsistent or incorrect labels, reducing the quality of your dataset.
- D. Correct.
Qualification tests help identify workers who understand the task, ensuring only capable workers participate and contribute high-quality labels.
- E. Incorrect.
Accepting the first label without verification can lead to poor-quality data, as there is no mechanism to confirm the accuracy of the labels.