Just as sourcing ingredients is essential to the quality of a great dish, the quality of labels determines the accuracy of a supervised or semi-supervised machine learning (ML) solution. What is a label?
Data labeling involves taking unclassified data and augmenting each piece of that data with some sort of meaningful ‘tag’, ‘label’, or ‘class’ that is somehow informative or desirable to know. Assigning a label is a judgment task to be performed by an analyst depending upon the kind of variables he/she wants to work with and is an essential part of the ML process.
How to ask Questions?
All tasks involving labels involves, at some point, asking a question to a human being for collecting the data. The questions should be relevant, clear and precise in addition to being plain and understandable and one for which answering does not involve much effort for the individual.
How to debug tasks?
The collected data may sometimes show discrepancies due to:
• Data: Certain factors which can cause bias in the workers have to be eliminated.
• Workers: We have to detect the expected errors arising due to human involvements, say, an error arising due to spammers rather than people making genuine mistakes.
• Tasks: If the problem still persists then there might be a problem with our initial assumptions. Time to rethink from the start.
How to assess Work Quality?
Specific domain based algorithms exist for every point during the project that determine the ongoing quality of work.
Labels are essential and cutting on effort in this regards is most likely to lead you to erroneous results.

For more information visit:
http://blogs.technet.com/b/machinelearning/archive/2015/06/23/label-quality-for-machine-learning.aspx