Baseball Analogy for Stall Catchers
Here is a slightly whimsical (and imperfect) analogy between Stall Catchers and (one aspect of) the game of baseball. Hopefully this will provide a useful way of thinking about Stall Catchers to a fairly wide audience without resorting to terminology such as ROC curves or detection thresholds.
-
Pitch: movie
-
Swinging at pitch: calling a stall
-
Looking at pitch: calling flowing
-
Swing and a miss: calling stall when it’s actually flowing
-
Taking called strike: calling flowing when it’s actually stalled
You don’t want to chase bad pitches, but you don’t want to look at called strikes either.
-
Swinging for the fences: lower sensitivity (more risk taking)
-
Contact hitting: higher sensitivity (less risk taking)
In baseball, the strike zone is defined explicitly in terms of two dimensions (width of the plate horizontally, roughly shoulder to knee range vertically). In Stall Catchers, the “strike zone” is more complicated (higher dimensional) and defined implicitly by
- A prescribed list of movies where experts have tagged each movie as “stalled” or “flowing”; and
- The judgement of The Crowd
First a standard must be established for what constitutes a ball and a strike, or flowing and stalled.
- MLB Competition Committee: panel of experts labeling movies of gold standard set as “stalled” or “flowing”
In baseball, umpires are trained to recognize balls and strikes (in accordance with the rules promulgated by the Competition Committee), but in reality individualized judgement enters into their decisions. In Stall Catchers, The Crowd plays the umpire role (as well as the Committee role), learning from the examples of the gold standard set of movies. However, the gold standard set provides an imperfect learning experience for The Crowd, and The Crowd itself is not capable of perfect judgement.
In baseball, runs batted in by a player depends roughly on batting average and number of at-bats. For an individual Stall Catcher, points scored depends on sensitivity and number of movies judged. In turn, number of movies judged depends on the product of two factors: rate of viewing movies (movies per hour) and total amount of time viewing movies. So, just as a player who is regularly in the lineup will score more runs than a pinch hitter, a Stall Catcher who makes good, fast judgments and plays for many hours will score more points than someone who makes slower judgments and/or poorer quality judgments, and plays fewer hours.