Sergey (Nolar) Vasilyev (nolar) wrote,
Sergey (Nolar) Vasilyev

MTurk Quality Experiment #1

As I’ve previously mentioned in my Twitter (nolar), I’ve been experimenting with Amazon MTurk, and here are the preliminary results of the quality of the results for one specific task of translation of the texts from English to Russian.

Initially I wanted to name this article as “Zoo of Mechanical Turkeys”, but changed my mind for better SEO of this article (I mean relevancy, not the traffic and popularity of the Dark SEO). I don’t know why MTurk is so associated with turkeys in my mind, when it has nothing to do with these birds or nationality.

I guess, the results will show why it happens so. But before I will give you the results, I want to describe what the task is, and why is it so unusual.

Read the rest of this entry »

Originally published at Sergey Nolar Vasilyev. You can comment here or there.

Tags: amazon, api, artificial intelligence, aws, boto, experiment, hit, human intelligence task, intelligence, mturk, python, quality, statistics

  • Ideal Employer

    So, I’ve finally decided on what my ideal employer is. It is good when you do anything you want to do and is being paid, for sure. But this…

  • Python Fun #2: generators and exceptions

    Возмём простую задачу, но заюзаем прикольную фичу Python'а -- генераторы, которые, согласно документации, позволяют "замораживать выполнение функции…

  • Python's super() and how to use it

    Сегодня у меня случилось прозрение. Я понял что делает super() в питоне. В руководстве и интернетах часто приводится пример с super() в качестве…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.