Purpose of Levels

Why are there levels for participants? Are incentives given for achieving higher levels? Is there a cap on levels? What is the highest level? If anyone could answer any or all of these questions, it will be greatly appreciated. Thank You :slight_smile:

Hi Stephencpcd -
I think I can answer most of these.

First a little bit of background. You can read more about this in some of the science and training forums and help section.

The goal of Stall Catchers is to identify stalled vessels among the thousands of potential vessels we see in the movies for a given dataset. Ultimately, all candidate stalls identified by the community are verified by the experts. But since this is like searching for a needle in a haystack and can take the experts many, many hours to search through all of the vessels, the community can assist the researchers by identifying highly likely stalled vessels. This way the experts only need to look at a few hundred candidate vessels instead of thousands. Remember that only about 2% of real vessels may be stalled. There is no need for the experts to look at the 98% that are flowing.

The levels are an indicator of a player’s experience. The new dynamic stopping algorithm gives more weight to players with higher experience levels. If 5 or 6 experienced players all indicate that a stall may be present in a given vessel, we probably don’t need to have 14 or 15 more annotations done by other players to suggest this is a likely candidate that should be verified by an expert. Last spring we needed 20 players to annotate every vessel before we could statistically have an answer that would likely be as good as the experts opinion. Based on the research done by Pietro and the Human Computation Institute we can use the experience levels to reduce the total number of annotations needed to have an equally valid community recommendation. We can therefore annotate a new dataset in much less time.

In addition to the scientific and statistical reason to know the experience of a given player, the experience level can also be used during competitions and challenges to allow players of similar experience levels to compete against each other. This has been done for some events and is available depending on the nature of future events.

Not at this time. It has been suggested.

Not to my knowledge.

No limit at this point. However, it takes considerably more points to achieve each additional level.

Hope this helps,

Thank you very much gcalkins for a very informative answer to my questions. I hope these answers you’ve given me helps other new participants in this “game”, to continue in this vital research. Let’s catch some stalls:smile:

Thanks for the question @stephencpcd, and thanks @gcalkins for the excellent answer, as always!

I just have a couple of things to add…

[quote=“stephencpcd, post:1, topic:214”]
Why are there levels for participants? [/quote]

When you achieve higher levels (i.e. are more experienced in the game), you also get much more points for each annotation! (including the “calibration” movies, and the “real” movies, only for those you redeem your points later)

There will be badges you can unlock & similar things in the future :wink: open to any suggestions you may have!

And just to comment on this:

That is not exactly correct - we measure your “expertise” by measuring your sensitivity at the time you give an answer. This is indicated by the blue bar with the eye on the right of the movie. This might fluctuate, so if you’re feeling under the weather, you may have lower sensitivity & we will assign lower weight to your answer, but if you’re doing great then your sensitivity will be higher & we will assign more weight to the answers you give! The rest is spot on, @gcalkins, excellent explanation :wink:

Thanks & keep the good Qs coming!

Thank you seplute for the additional info to my question. Changing the topic for a moment, I love your icon, very cool :), thank you again for helping me. I am spreading the word about this website and the contribution it is making to curing this dreadful ailment.

1 Like

Thank you so much for your support @stephencpcd!! :slight_smile: