Hi,
I’ve participated in another research and found it useful to have images at hand that would show me how a positive and a false positive look like. The instruction you’re giving right now is fairly basic and will surely overlook some more difficult cases. Overall this should provide better training so the results you get are better.
Cheers,
Mahonka
Thanks for the great suggestion, Mahonka.
Posting sample images might help, but it could also be misleading because in a static image, a stall could look the same as a flowing vessel. In many cases, it is only when the movie is running that you can tell the difference. That said, we could post images representing a series of 4 or 5 frames that might capture some of the more difficult examples of stalls.
We intend to add a link to more detailed tutorial content for those seeking more advanced annotation guidance. Stay tuned…
I think part of being able to distinguish between true positives (actual stalls) and false positives (a flowing vessel that looks like a stall) is understanding the 3D nature of the movie. In some sense it is a movie that plays back over time. But in another sense, each subsequent frame of the movie represents a deeper layer of tissue.
Take the following difficult case as an example. If the vessel is oriented vertically through the stack of images, then you would be viewing a cross-section of the vessel. So instead of looking like a winding snake, it would simply appear as a dot (and the green outline would be a small circle).
In this case, if blood were flowing, the dot would appear relatively consistent throughout the movie. But if the dot disappeared completely for a few frames, and then became relatively solid thereafter, that could be a sign of a stall.
I may also be worth mentioning that although we provide participants with many opportunities to annotate stalled vessels, the natural occurrence rate is much lower. The natural frequency of stalls in wild type (normal) animals is about 0.5% and in animals with Alzheimer’s disease it is about 2%. This means that it is very important to not miss a possible stall. So even if you aren’t 100% sure, it is better to catch a stall than to miss one. Each and every stall discovered by the crowd will be individually examined by experts in the lab to verify that it is a true stall.
Thanks again for your thoughtful question and for joining Stall Catchers!
All the best,
Pietro
That’s a particularly important point to highlight here 
However, as @pietro mentions, we will certainly provide some “tricky” movie examples in near future. I hope @vpm22 can help us with that too! 
Examples and explanations of each expert “error” message would be welcome. For example, when the expert sees a stall despite luminescence that makes it invisible to us ordinary mortals. Also, when the absence of black spots is interpreted as a stall, which is counterintuitive and to these eyes appears to be inconsistently applied.
The scoring system may work against marking black spots as stalls when it’s not clear whether they are black or gray or whether they are moving. Especially after one has been hit with the draconian penalty for a mistake - I had my “Maybe” score drop from 81 to 63 points in one turn. Given that most Maybe movies are stall-free as pietro says (stalls in the calibration movies are of course much more frequent), it’s tempting to play the averages and click Flowing when at all in doubt. Guilty as charged.
Ha! You are not alone. One of our developers has also confessed to using this strategy. Rest assured, though, it will not improve your score If you don’t believe me, try answering flowing every single time, and you will see your sensitivity bar gradually drop back down to zero. Your truly best bet for increasing your sensitivity (and hence your score) is to always give your best honest answer based on how the vessel looks to you. That said, feel free to experiment with strategies - the worst you could do is slow the system down (you won’t hurt the ultimate scientific results).
Best,
Pietro
Of course I always give the best answer I can - this isn’t about points, it’s about data - but sometimes I can’t be sure of the answer or even find the vessel, and Flowing is probably a better way to move on than clicking for a Stall I don’t actually see.