Issues at Stall Catchers (CHECK HERE before posting)

Hi All!

Thank you @gcalkins & @caprarom for all the useful tips!

We are all at a conference right now, but @ieva will continue to work on the Edge browser issue depending on her availability these coming days, and we hope to resolve it soon. Hopefully, for the moment catchers will switch to alternative browsers - I included a note and a link to the instructions that @caprarom posted above on Stall Catchers as well (thank you for those!!).

On a topic of a new dataset - we are not receiving new data for a little while still, and the current dataset is not “finished”, at least not exactly. :slight_smile: We now have a different system of managing the movies within Stall Catchers, so that each catcher is first shown the movies that don’t have enough annotations, and ones that they haven’t seen before. So some of you who are annotating at a greater speed will have annotated most or all movies of the current dataset at least once, but some of you might not (and most of the users who are less active will not have seen them all). This also ensures that the experience in the game doesn’t change even if you have seen all movies.

So in a sense, @caprarom, you might have “finished” the dataset, because you have seen all movies. In that case, you will start seeing the same movies a second time. It is important to mention though, that those annotations are also very useful to us, but in a different way. First of all, we are using them to improve our crowd methods and understand how individual user answers might change when they have already annotated the same movies, and whether there’s value in allowing them to do it to improve crowd answers in general. In this way, we will also use this data in the new validation research we are doing that we are sending back to the biomedical research lab. So it is definitely useful to us, but we won’t get offended if you have to drop off and come back when we have more data available! :blush: We’re going to update you all when we do.

And even more importantly, this should be temporary! :sweat_smile: I.e. even if some of you (who annotate the most!) are repeating movies in this dataset, it shouldn’t happen in the future. We are completing a process that will enable a steady flow of new data from the lab!

In addition, @caprarom , you really should not see the “same” movies over and over again - we’re looking into this! One possibility might be that a small subset of movies had less annotations than others, and you kept getting movies from that subset to help “complete” it sooner, in a sense.

Anyway - how this clarifies things, and thanks again !!!

1 Like

OK, thanks, Egle. I got back on and my experience is that, aside from calibrations, I see the same two (and only those two) non-calibration movies over and over again. One has been annotated 99 times (10 stalls, 89 flows) and the other 60 times (6 stalls, 54 flows). Those numbers do not change as I add more annotations; therefore, it appears I’ve nothing to contribute for now. Don’t know if that information is of any use to Ieva.
The new process enabling a steady stream of new data sounds great, as does the data management upgrade. Thanks to you & the whole team for your constant efforts to improve the stall-catching experience and efficacy.
Mike C.

Thank you for the extra info @caprarom - I forwarded it to Ieva, it should help resolve it.

Regarding the numbers… [quote=“caprarom, post:61, topic:144”]
One has been annotated 99 times (10 stalls, 89 flows) and the other 60 times (6 stalls, 54 flows). Those numbers do not change as I add more annotations; therefore, it appears I’ve nothing to contribute for now.
[/quote]

They should be going up - that’s actually something we tinkered with, and shouldn’t have, so we are now reversing it :smiley: as I mentioned before, each annotation is useful to us - even if it’s on the same movie, just in a different way.

Also, when I mentioned that you’re seeing the same movies over and over again to Ieva yesterday, she said some of them look very similar - I don’t think that’s the problem though, cause you would be able to tell I guess :slight_smile:

Thanks again!! :blush:

Dear @CarolakaMema, @gcalkins, @caprarom - Ieva tells us the issue with the Edge browser has been resolved. I don’t have Edge and don’t normally use it, so I can’t check - would any of you mind doing that and reporting back here? :slight_smile:

Hope it works :crossed_fingers:

I just tried it on Edge, Egle. Still getting the error messages on the calibration movies (and still getting the same two regular movies).
Mike

1 Like

Same here. Edge still not working with calibration movies.

  • Guy -

Guess I will join the crowd!
Every 3rd or 4th movie was an error tonight. I think I understand about how this all works but I too have been seeing some of the same movies over and over. I just figured they needed more annotations to make their numbers go up.
Last night (6/2/18) the program just froze up and no amount of refreshing the page would help. I logged off, waited 5 minutes, then logged back on and it was still frozen. I couldn’t get anything to move, open, report on, flag the movie, just nothing…so I logged off.
Anyway, I will just plod along, doing as many as my patience will allow.
Miss Egle and her team have the hard job of fixing the bugs and making it easy for us catchers. We all really appreciate the hard work you all do.

Carol

1 Like

Hooboy!!

We received several emails with more error reports as well. Seems like different catchers are experiencing different bugs, however, and most of them we cannot repeat ourselves :confused: So please give us as much info as you can if you experience errors! I posted a general template to help you do so here: http://forumhcinst.org/t/please-use-this-general-template-to-report-all-bugs/405/3

@CarolakaMema can you give us more information on the recent bugs you experienced? See below…

Do you mean the “You have already annotated this movie” message? Are you using Microsoft Edge or a different browser?

Are you sure they are the same movies? That’s strange - even though you may see some of the same movies if you already annotated the whole dataset, it sounds like you’re seeing a small subset of them over and over again (same as @caprarom) and it really shouldn’t happen… Looking into it, among other things.

Extra annotations are certainly useful, but we don’t need to gather more than a few from each user, so it’s strange if that’s what is happening.

Others have reported similar and we’re really concerned about this. What browser and device are you on?

Thank you so much for your patience and your dedication :purple_heart:

:sweat: thank you for your support - we need it! :sweat_smile: not sure what’s happening, but we seem to be in the middle of a bug storm! :sob:

Hi All!

We’ve been holding our breath for more bug reports… but it looks like we might have caught most of them for now. Is everyone able to play? @CarolakaMema @caprarom @gcalkins?

In any case, still looking if we could have overlooked something in the recent software updates - please bear with us if anything else comes up! :sweat: :purple_heart:

Best,
Egle & crew

Hi, Egle. As of 07:40 EDT, the program still is not running properly on the Edge browser. It freezes up every two or three movies, and requires a screen refresh to proceed. I went back to Firefox and it seems to run OK there. Two minor items I’m not sure were mentioned previously, but you probably are aware of:

  1. The bullet indicators to show who is active have not been working on either browser.
  2. The “Played Last” columns on the team screen displays have not updated since May 30th.
    Mike C.

:sob:

OK, thank you @caprarom - that is very helpful. Back to the bug hunt !!

Hi Egle -
I am getting the same freezes as Mike. Below is the image that shows how much is loaded while it is frozen. Still missing the Flowing and Stalled Buttons. Nothing responds.

This is only happening in Microsoft Edge. Sometimes a refresh will work, but other times I need to log out and log back in to get it to load properly.

I noticed that when it hangs, and you do an “Inspect Element” to view the source code, the Preloading Next Video section does not have a new video name loaded. This first copy of code is what it looks like when it hangs.

The next section is what it looks like after I have logged out and back in. This is also what you see during normal play with the other browsers.

I hope this helps,
Guy

1 Like

Hi Egle and catchers,
I don’t know why I didn’t try Chrome last week when it was suggested. I logged on to my regular page and could only do about 15-20 movies before getting totally bummed. Then I looked up where they suggest Chrome or Firefox (yes, I am a little slow tonight!), decided to give it a try and BAM! It was like old times! Nothing froze, stuck, didn’t show up or anything.
I breezed thru my 200 movies and feel great. Although, I did notice that some of the movies were from 2016 and early 2017. I thought we were done with all of those?!?
Anyway, I will be using Chrome until the bugs are worked out. My tip for you is to give it a try, you can always go back to what works for you.

Happy catching,
Carol

2 Likes

Dear All!

I am happy to announce that we finally caught and sorted out the Edge/IE browser bug. You are free to go back to your favorite browsers! :sweat_smile:

This was an out-of-the-ordinary bug situation, so thank you all so much for your patience and sticking through it with us! :purple_heart: We are boosting our internal quality assurance processes to avoid similar situations after big changes in the future. We had a new crew member join us for that too, Vilma (CapriCorn on Stall Catchers) :slight_smile:

AND in case you missed it, here’s a more thorough explanation from @pietro about what has been happening with this bug storm:

Hi Egle -
I am still experiencing the situation I reported above. The “Flowing” and “Stalled” buttons are randomly not getting loaded. It does eventually load after a few refreshes. (Sometimes takes 3 or 4 refresh attempts.) As indicated above the Preloading Video “src” is blank when the buttons are not loaded. Not sure if there is a connection between these two observations.

Thanks,
Guy

Hi Guy,

Oh dear - only seeing this now. It works for us, however - have you tried clearing cache on your browser? I am not sure it’ll help, but until @ieva responds I have nothing better to offer :sweat:

How about others? @caprarom @CarolakaMema? Does Edge work for you?

I tried Edge last evening, Egle. It seemed to function normally for me. I’ll check again later and pay closer attention, but Edge seemed a slight improvement over Firefox. I’ll let you know if I notice anything amiss.

I’m not sure, but now that they are working again the bullet indicators showing the active players, seem to take much longer to clear than previously once a player is inactive. Don’t know if that was by design.

Mike C.

1 Like

Thanks @caprarom! Looking into it, and added the bullet indicator issue to the queue :slight_smile:

Well, here I go again.
I did my 200 on Chrome and flew through them so I thought I would give my old browser (internet explorer) a shot. The 11th movie in got stuck for a few seconds, wouldn’t let me move the bar, then finished like normal. The 19th movie totally froze up and had to refresh to get next movie. The 25th movie did it again and this time refresh would not work. I logged off and then back on to be able to write this.
I was wondering about the bullets as well. Even when someone is not playing (but has) their bullet is still lit up, even if the leaderboard says they are logged off. Again just wondering.
For now, I will stick with Chrome. There is no sticking, freezing and the movies seen clearer to me.

Hope this helps,
Carol

1 Like

Thank you all - forensics are continuing … :spy: