Crowdsourcing is an efficient approach for knowledge acquisition and data annotation that enables building impressive human-computer systems. In this tutorial, we will discuss the relations between Crowdsourcing and Natural Language Processing, focusing on its practical use for Language Resource construction and evaluation. We will describe the established genres of crowdsourcing, show their strengths and weaknesses on real-world examples and case studies, and provide recommendations for ensuring the high quality of the crowdsourced annotation.

The tutorail will consist of two parts: a lecture and an exercise session. The lecture has no prerequisites and open to everybody. To participate in the exercise part a basic knowledge of Python is required. If you want to participate in the exercise session please bring with you a laptop with preinstalled Python 3.5 or higher and packages from this list: requirements.txt

To participate in the exercise part a basic knowledge of Python is required. If you want to participate in the exercise session please bring with you a laptop with preinstalled Python 3.5 or higher and packages from this list: requirements.txt