Ways to think about crowdsourcing and their limitations


, , , , ,

Crowdsourcing is a huge trend at the moment and is at a tipping point in its development. It is the basis of many paper and research, with big commercial and cultural implications, sometimes with opposite ideals. It can be defined as the use of large number of people, experts and neophytes, to achieve complex tasks that a computer couldn’t do alone. I will resume, analyze and compare three article on the subject, with 2 different views on crowdsourcing. All are based on its application in text analysis.

The first article, Crowdsourcing the Text: Contemporary Approaches to Participatory Resource Creation [1], is a general overview of the concept of crowdsourcing using the Zooniverse (leading crowdsourcing organisation) as an example. It highlights a few problems or challenges that the the branch is facing. Although transcription is one of the most basic task in all text analysis, it requires the help of the crowd. There’s a stratification of the people in the crowd : beginners (who knows nothing but are many) and experts (who knows more but are few). A good system needs to take in account both groups and provide them ways to produce data as efficiently as possible. A solution, proposed by the article, is to have a test that determine if the user is beginner or expert and then send him to two different ‘facilities’ : a training and basic place for the first category and a more powerful one for the latter. Another point touched on by the article is the case of interpretation. It is a higher cognitive task and much more subjective than transcription. We need to determine how we want to tackle this challenge, because a crowd can have as many interpretations as the number of people in it. The article suggest two solutions. Either selection of specific persons in the crowd (but there’s a loss of contact with it) or use cross-interpretation that link all interpretation and compare them (but very hard computationally speaking).

The second article, KinDigi Social: A Mobile-centered Social Annotation Platform for the Kindai Digital Library [2], is a precise application of a mobile crowdsourcing platform based on the KDL, a japanese book archive. The developers have seen that many people actually comments and discuss on these books, and they wanted to create a platform that aggregates all that knowledge into one place for ease of research. The project is very early in development, but they have chosen a free interpretation system (twitter-like). Everybody can annotate and comment, which then generate a discussion to contextualize the passage of the text or book.

The third article, From Crowdsourcing to Knowledge Communities: Creating Meaningful Scholarship Through Digital Collaboration [3], is a research and analysis on crowdsourcing based on three project given to three type of crowd (naïve, expert and paid). All of these had a broadly defined goal and none was about transcription. During the study, they interacted with what they call ‘knowledge community’ which are groups of enthusiasts or neighbours with a very specific knowledge. They are much more involved and are not just extracting data. One interesting thing is none of the projects actually reached its goal. The article then explain that we should not just look at utilitarian questions (is it cheaper, better, quicker, etc) but also look at the opportunities that collaborative works offer for museums and institutes. We can “have original research and engagement with the community” which is important for creating a bond with the people and to keep in touch with isolated people.

The order of the article is not random. At first, we have the theoretical knowledge on crowdsourcing, containing questions on what we should think about when doing these kind of project. It suggest solution to problems and reminder on how we should prepare such study. The second is a ongoing project, that hasn’t encountered problem such as ‘How to crosslink the interpretation’ and that maybe is a bit naïve on the crowd or its goals. The third and last article is a conclusion based on a research already done, showing what the crowd is and that maybe the efficiency is not the real aim of complex crowdsourcing.

When comparing all three, we can see that the first one is a cold utilitarian analysis on how should we use and improve the crowd results whereas the two others are more in touch with the human and the concept of bonding with each other. The Kindigi project wants people to interact with each other and the chief leader, in a egalitarian way. They will be forced to make a choice, choose the first article style and decide that the results is more important or choose the second style and decide that the interactions are more important.

These two visions of crowdsourcing are representative of the general vision of it. Because both are hardly compatible, we must choose : should we use the crowd as an efficient worker and provide it tools to be even more efficient or should we see the crowd as a composition of communities and people, providing it tools for discussion and binding with institute, museum and the world. Both have advantages and drawback. By being utilitarian, we can hypothetically produce more data faster but we may lose the interest of the crowd, transforming them into nothing more than unpaid worker. By being humanist, we can find ways to interest people into the humanities, creating more vocations and bring back people to the culture (museum and such), but we might lose some efficiency in the process.

The future development of crowdsourcing will tell us if the utilitarian or humanist way of thinking will more prominent. We will then see what effect the choice will have.



[1]  D. Powell,  V.Van Hyning, H. Wolfe, J. Tonra, and N. Fraistat, Crowdsourcing the Text: Contemporary Approaches to Participatory Resource Creation

[2] Y. Hashimoto and Y.i Araki KinDigi Social: A Mobile-centered Social Annotation Platform for the Kindai Digital Library

[3]  J Voss, G. Wolfenstein, Z. Frank, R. Heuser, K. Young and N. Stanhope From Crowdsourcing to Knowledge Communities: Creating Meaningful Scholarship Through Digital Collaboration


Get every new post delivered to your Inbox.

Join 141 other followers