86/100 text annotation batches were of acceptable quality, the other 14 were not. It looked to me like the 14 had pretty basic mistakes and I suspect that those Turks did not bother watching my instruction video so I changed the Turk survey to say that watching the video is required prior to attempting and included a list of common mistakes that will result in rejection. I kicked off the 1,000 ad batch last night which is 93% complete right now. I’ve made the compensation $0.30/job because I figured this annotation is easier than transcribing for which we paid $0.50, I did get one complaint email from a Turk who did a lot of good transcription work for us before and I can see that the average job is taking over 5 min so I might raise the compensation in future batches.
Since the app is connected directly to my SQL server I can query the results in real time as they come in as opposed to the transcription jobs where I had to download them so im going to start building the AI model and just connect its training input directly to the Turk results, that way if I want more training data I just kick off another batch and the data automatically flows into the AI model as the Turks work.
Eric Anderson, CFA
Quantitative Analyst
Skylar Capital Energy Global Master Fund LP
(713) 341-7985 work
(281) 606-9371 cell