Hi All!
Thank you @gcalkins & @caprarom for all the useful tips!
We are all at a conference right now, but @ieva will continue to work on the Edge browser issue depending on her availability these coming days, and we hope to resolve it soon. Hopefully, for the moment catchers will switch to alternative browsers - I included a note and a link to the instructions that @caprarom posted above on Stall Catchers as well (thank you for those!!).
On a topic of a new dataset - we are not receiving new data for a little while still, and the current dataset is not “finished”, at least not exactly. We now have a different system of managing the movies within Stall Catchers, so that each catcher is first shown the movies that don’t have enough annotations, and ones that they haven’t seen before. So some of you who are annotating at a greater speed will have annotated most or all movies of the current dataset at least once, but some of you might not (and most of the users who are less active will not have seen them all). This also ensures that the experience in the game doesn’t change even if you have seen all movies.
So in a sense, @caprarom, you might have “finished” the dataset, because you have seen all movies. In that case, you will start seeing the same movies a second time. It is important to mention though, that those annotations are also very useful to us, but in a different way. First of all, we are using them to improve our crowd methods and understand how individual user answers might change when they have already annotated the same movies, and whether there’s value in allowing them to do it to improve crowd answers in general. In this way, we will also use this data in the new validation research we are doing that we are sending back to the biomedical research lab. So it is definitely useful to us, but we won’t get offended if you have to drop off and come back when we have more data available! We’re going to update you all when we do.
And even more importantly, this should be temporary! I.e. even if some of you (who annotate the most!) are repeating movies in this dataset, it shouldn’t happen in the future. We are completing a process that will enable a steady flow of new data from the lab!
In addition, @caprarom , you really should not see the “same” movies over and over again - we’re looking into this! One possibility might be that a small subset of movies had less annotations than others, and you kept getting movies from that subset to help “complete” it sooner, in a sense.
Anyway - how this clarifies things, and thanks again !!!