Is anyone else having this issue?
My blue bar/score per film has been decreasing the past few hours after answering calibration vessels correctly. For example…232…232…232…calibration vessel…correct! 4640…231…??? (the 232’s are because we are in a double points period)
I captured it on film and sent it to Pietro but was wondering if anyone else is seeing it?
Not I, Bad. Sorry my crazy idea (which you were crazy enough to try) did not work out. 
When things get this stupid, anything is worth a shot! 
No problem with the blue bar here
Hey, Bad! 9:50 PM EDT - it’s started happening to me also! Get a correct calibration and lose a point or so from the blue bar, etc. Wanna see who can get to zero points first? (I know - you’ve got a head start.)
Any news from the “Devs in training” now that we have a 2nd confirmed case?
Thank you for the reports – we have a couple of hypotheses (could be related to other issues, including the “already annotated this movie” error) – we’re diving into it.
Please understand that we do not have a full time developer now, and Pietro is juggling between that and other things that he has on his plate, but we are doing the best we can.
Confirmed:
Bad’s blue bar reversed at 11:14pmET on 3/20
Caprarom’s blue bar reversed at 12:18pm on 3/21
Carol aka Mema’s blue bar is working properly
sachambers’ blue bar is working properly
christiane’s blue bar is working properly
Conclusion - this is the most bizarre bug I’ve encountered.
More soon…
Pietro, there was another incident as I’ve stated in the box. It’s when you had asked me about logging into my account and we made ha-ha about what you might do to my bar as you check things out with your lack of catching…uh…prowess. If you have logs of the box, it would have occurred the day after we spoke about that I believe. I mention it in the box asking if you were on my account since my bar was down. I assumed you were on and that’s what took it down. No one asked, email or otherwise, for credentials so there was no breach as far as that is concerned.
I hit a few calibration vessels and in no time was back up to 116. That was the 1st time I noticed it. The thing is, I don’t really look at the blue bar since I might make an error in every so many thousand…that’s why I joked at one point about an easter egg for so many vessels viewed without error, so I don’t know if it had happened the day before and I didn’t notice it as it was only about 10 points down and I keep “show answer” off unless I see something that’s tricky and I want to see other’s opinions on it. I just noticed it those times as a “corner of the eye” sort of thing…there’s color, then there’s not color…let’s look over there…and then WT???
!!!!!!*! Just found a screenshot—3/14/19 was the first instance. File says 8:52p EST, but I keep my clock 15 mins. or so ahead. I wrote, “Somebody tried something I see 
Poor blue bar”. I have another screenshot, same day but 4:18a EST that shows a full blue bar and again, I stay logged in unless power goes out or the modem goes down which isn’t often so IP’s should be the same and there should be no other login from my username anywhere else.
Thanks, Bad. I did forget about those other instances of unexplained but temporarily dropped blue bar. I’ll look back - I should be able to find those.
I have a working hypothesis related to how we store events. Each record in our event table represents a single annotation. Over time, we accumulated so many event records it was bogging down our system everytime we had to search to see if a user had already seen a movie. So Ieva came up with the idea to create an “events_history” table and move old events to that table. That way, we could still have access to the information but query only the most recent events for the purpose of selecting a new movie.
In examining the recent data, I noticed that there are overlapping ID numbers for the records in the event and events_history table such that when we join the two, depending on how the information is retrieved, it is conceivable that we would pull the wrong record. I don’t have solid evidence of this yet because I need to find the relevant queries in the code and then run them separately to see how they behave (which is what I’m doing now). In the meantime, I thought i would show you this:
Your user_id is 691. as you can see, for the same record id, there are two records, one is your recent answer to a calibration video, and the older one is a random annotation from 2018 that just happens to have the same row id. Again, my working hypothesis is that these two records are somehow getting conflated. But for some reason, only for you and MikeC, and only recently.
The record with ID 2197226 is the one where it all started. The system thinks you got that one wrong, but the status=5 means you didn’t, yet your FPR changed right after that.
So can you (or anyone) see anything in this table that would explain what is so special about the two records with ID=2197226?
(I’ve been looking at the same for MikeC and there is no pattern apparent to me)
In any case, I am hoping that testing out these queries on those particular records will shed light on the issue.
Best,
Pietro
If I read this table correctly, Bad correctly flagged the latter instance of record 7226 as stalled (indicated by the “1” in the “is_stall…” column). However, last November, a different movie was assigned the same record number, and was not flagged by Bad as a stall (0 in the “is_stall…” column). You are suggesting that when Bad correctly flagged movie 4512 as a stall, the system read the old November data for movie 155234 by mistake? Certainly sounds plausible, but why would the score keep decrementing only slightly instead of dropping the usual amount for a miss? The records look otherwise unremarkable. Still a mystery to me, for sure.
Exactly, @caprarom.
You actually just helped me there by pointing out how slowly it’s changing. I was thinking the system was interpreting them as misses, but it’s actually false positives (FPR), which is why the changes are slower. In other words, the system things you are saying there is a stall when there isn’t. So it’s the opposite of what I thought. The system thinks the correct answer is flowing on these, when it isn’t.
Oh - OK. Glad to be of help, but still a mystery. I feel, however, I must point out that when Bad or I mark one of these as Flowing, we get hammered the usual amount. So, the system either gives us a “miss” for flowing, or a “false positive” for stalled. Either way, we sink. 
my blue bar is working fine
I M getting a lot of black sections decreasing from left to right. Many black areas start over the circled areas, which actually are present over the black area.
Am well aware, @caprarom - the old “damned if you do, damned if you don’t”, which is why I’m spending the weekend on this.
Thanks, @chairgaf - those movies you describe have been taken near the bottom or top of the image stack which is why a few frames at the beginning or end will be black.
Question to @Badstallsbadbad and @caprarom…
I might be on to something.
Have you noticed whether all your training (calibration) movies are stalls? Has the ratio of stalled to flowing in the training movies changed for you recently?
Also, it looks like we might have a new casualty - @christiane - your blue bar is dropping now too, isn’t it?
Pietro
Not 100% sure but from what I remember all the calib. vessels that have preceded the issue have been stalls. Also, sachambers just reported in the chatbox that the bug has hit him.
Yes, I reported to Egle days ago that all the calibrations have been stalls, which she later confirmed. That has not changed.
Thanks, @Badstallsbadbad.
Getting closer…
The last time anyone with a sensitivity > 1 saw a flowing training movie was at 22:11GMT on 3/18.
We use a decay function for our sensitivity calculation. A correct measurement of sensitivity requires a steady influx of flowing training movies. Beginning at 22:11GMT on 3/18, a hidden process began for all users that would cause the blue bar reversal. Due to the decay function being driven by annotation activity, the most active users would experience this first.
Now I am trying to understand exactly what happened at 22:11GMT on 3/18 to precipitate this. Was it a code change? A database change? An infrastructure change?
