Of the 3,000 jobs (1,000 ads each done 3 times) no changes whatsoever where made to 2,068 of them. This could mean that the Google transcription was just that good however of the ads where no changes were made most Turks clicked submit within 10 seconds or less which is not enough time to even review the image and transcription. To me this says that the Turks here were just hoping to slip through any quality control and get paid for doing nothing. Some spot checks of these ads confirms this. Below is a histogram of the time taken on each job when no changes were made.
Of ads where changes were made the Turks spent much more time (an average of over 2 minutes). I will manually review all ads where less than 30 seconds was spent on the task. I will spot check a collection of all ads where changes were made. Below is a histogram of seconds spent where changes were made to the transcripts.
We will be refunded for all the unacceptable results.
I’m going to make a few changes and would like to resubmit the batch for another test.
- Limit the jobs to only highly rated Turks.
- Add a bold warning near the submit button that all transcription errors must be fixed or the task will be rejected.
- Increase the allotted time to 20min per job.
The batch is ~23% complete with an estimated completion time of tomorrow at midnight.
My spot checks on this batch show a much higher level of quality, very acceptable work. The average Turk is now spending 6 minutes reviewing and correcting each ad:
Here is one example of the quality of work.
Google API Transcript:
“ROKE JAIL-On the night of the 3d inst. a Negro Fellow by the name of TOM, about 6 feet high, and about 2 yearn of dge, very slout made, complexion not very blnck, Aud a very likely Negro; recently from Augunta, Geo.. Ala, a mulatio Fellow by the name of JESSE, about 5 feet 10 inches high, and about $3 yearn of age, a barber-hy trndr, and a stout well builu fetlow, large whiskers, and is vers intelligent; he isalao from Auguste, Geo. A liberal reward will be paid for both, or either of thém, by lodging tliem in the Charlenton Jail. B" J. TOBIAS, Jailur, C. D. Jan 5”
Turk Transcript Revision:
“BROKE JAIL-On the night of the 3d inst. a Negro Fellow by the name of TOM, about 6 feet high, and about 32 years of age, very stout made, complexion not very black, And a very likely Negro; recently from Augusta, Geo. [illegible], a mulatto Fellow by the name of JESSE, about 5 feet 10 inches high, and about 33 years of age, a barber by trade, and a stout well built fellow, large whiskers, and is very intelligent; he is also from Augusta, Geo. A liberal reward will be paid for both, or either of them, by lodging them in the Charleston jail. J. TOBIAS, Jailor, C.D. Jan 5 3”