I signed up for Mark to cure, but after looking at the instructions, I still have absolutely no idea how to do it. They really need a better instructional video. If anyone can summarize just exactly what we are supposed to be doing, it would be a big help. I can’t even figure out how to do the tutorial. Our game is so much more straightforward. They need a tutorial video that explains step-by-step what we are supposed to be doing, and not just some lecture the guy gave at a symposium that explains the concept!
1 Like
Hi Mike!
Thanks for the suggestion about the video–we’ll definitely take that under consideration! One reason why we don’t have currently don’t have a video tutorial is because of the nature of the task. Unlike a lot of online citizen science projects, Mark2Cure does not have beautiful pictures or amazing videos to look at, and we wanted to try to set the expectation appropriately as early on as possible. Since the tasks in Mark2Cure highly depend on reading comprehension, the tutorials were designed to teach about the task while attempting to set the expectations–it’s not a task that people can do without reading.
Barring any bugs (and the bugs DO happen, so let feel free to reach out to me if you think that’s why you can’t get through it), the tutorial makes use of the following format and available actions:
- Information section: expected actions: read the text / available actions: hit the ‘next’ or ‘submit’ buttons
- Interactive guided response: expected action: read the text / available actions: click on whatever the purple box tells you to click on.
- Interactive comprehension check: expected actions: apply what you read/learned previously / available actions: select and submit the best answer.
best wishes,
ginger (Mark2Cure)
1 Like
I abandon …read 3 times, and still don’t understand.
1 Like
I’m sorry to hear that our tutorial has caused you frustration. Feel free to send a screenshot of where you got stuck to contact@mark2cure.org and I’d be happy to guide you through it.
best wishes,
ginger
Thanks for your reply! I guess my biggest problem with it is that when I get a question wrong, it just says that there’s a better answer. Obviously there’s a better answer! The question is how am I supposed to know what it is? I suppose with the tutorial, I can just keep guessing until I hit the right one, but that won’t help me very much on the actual task. It just seems like every question is different which makes it a hellishly difficult task. In a task where you have only two choices, it’s much easier, but in this kind of task, once you get past 2 choices, a multiple-choice task is much more difficult. One last thing I wanted to say is that in your intro video, the guy says, the first group of people were paid money, but you’re going to do it for free. Certainly doesn’t seem like a good way to motivate people. The first thought that came to my mind is that if they were paid, why can’t I be paid too! Ha ha ha!
Mike
Hi, Mike. Well, I stumbled through what I took to be a tutorial, and I do mean stumbled. Got through that, then wound up at what might have been a “Dashboard?” When you select an item there, you have the opportunity to comment on the selected relationship. Once you do that, you are presented with how your answer compares with others before you. If you see your answer was selected by 9% of people, while another was selected by 43% of people - that’s an educational opportunity (at least, it is for me).
It took several cases before I finally hit on the consensus or predominant response a couple times. Meanwhile, I was scoring points somehow. It’s not clear to me if this is still part of the training, or I’m actually doing something ‘real.’ Regardless, clicking on the little circle shows you the distribution of responses for that selection - and perhaps the “better answer” you were asking about.
I don’t know if that helps at all. I’m still muddled - could not even see how to log out of the site! I’ve not watched any video as yet - not sure I want to based on your input. Perhaps I will watch it next visit/attempt. It might make more sense now that I’m sort-of acquainted with the site. This site promises to be mind-stretching, but it’s going to take some persistence (time) to climb the learning curve, at least for me.
Best of luck,
Mike C.
PS. I have since learned how to log out - there is hope for me.
Hi Mike! Thank you for starting such an interesting conversation!
Agreed it is a hellishly difficult task, and probably THE MOST difficult task of the CitSciMedBlitz challenge! What you’re doing is the very essence of textbook learning! When you read text, your brain identifies the concepts in the text, organizes how the different concepts are related in the text, and then integrates it into your existing framework on the subject. You can think of learning as the addition of new experiences, ideas, etc. into a network of information you’ve gathered based on your experiences. Now imagine just how much knowledge is in the biomedical literature if we think of every article as a knowledge dump of scientific experience! I think we could solve a lot of questions more quickly if there was a better way of harnessing that knowledge!
Text mining tools can help us figure out what genes, diseases, and proteins are in an abstract (to a certain extent), but figuring out how these different things are related? Not easy at all! In fact, there was so much doubt as to whether or not this could be done by non-specialists, that we actually had to have an experiment to test it before we spent the effort developing Mark2Cure (hence the paid amazon mechanical turkers). Agreed that mentioning this isn’t a great motivational tool 
Given how difficult it is to read an abstract, we chose to allow deeper relationship options other than the binary ‘relates to’, ‘has no relationship to’ because the most important information is in the specific relationship between the concepts and it would be a waste of users’ efforts if they read and understood the text, but weren’t able to submit more than ‘relates to’ vs ‘doesn’t relate to’
best wishes,
ginger
1 Like
Hi Caprarom! If you’re getting the user consensus feedback–you’ve reached the actual task! Congratulations on getting through the tutorials and figuring out how to use the system’s feedback mechanisms to learn. The points and feedback were put in place for learning purposes. Given the difficulty level of the task and the variability of what users will encounter in the biomedical text, the tutorials can only do so much. The bulk of learning tends to happen from doing the task itself, and I am always impressed by how knowledgeable users become on different biomedical topics.
best wishes,
ginger
PS- with regards to the log out-- I was 100% against the decision to bury the logout button under the profile!!!
1 Like
Once again, thank you for your reply. I saw on your website that your slogan is if you can read, you can help, or something to that effect. However, I just wanted to point out that that’s a little misleading. These abstracts have to be at a college reading level, or I would even say, even a graduate reading level. If you say it works, then I guess I’ll have to believe you, but it amazes me that someone with no prior knowledge would be able to do the task, but evidently that’s the case. I guess what I’m trying to say is that a lot of these inferences that you’re asking readers to make are usually based on prior knowledge that I would assume most of the readers doing this task do not possess. It absolutely amazes me that people with no prior knowledge of the reading material could do the task at all. I have some experience with studying reading comprehension, and I do know that prior knowledge is one of the main variables that affects reading comprehension, so much so that subjects have to be tested on prior knowledge before reading a text passage in order to determine that they are actually learning something from the text, and not just using their prior knowledge to answer the test questions.
It would seem to me that without prior knowledge, it would be very difficult for someone to do your reading task in this game, but I guess somehow people are just able to examine the grammatical relations in the text, and the operator words, and come to some conclusion about the relationships between the concepts just from that. I guess that’s certainly possible but that’s not the way that most people read a text. I always thought of reading comprehension as more of a top-down process, than a bottom-up process, but I guess information flows in both directions. It may even be that in the absence of prior knowledge it is easier to focus on the basic relationships between concepts because one is less distracted by trying to grapple with the deeper questions that arise from a more thorough understanding of the text.
Thanks for the information. I think I have figured it out enough now to be able to play the game, I am earning points now, so if I got past the tutorial section, I should be all set, but we won’t know until tomorrow if I’m actually playing the game, or still doing the tutorial. I guess if I’m earning points that must mean that the game has started. We will see what happens tomorrow.
It amazes me too! One advantage of NOT having prior knowledge is that the relationships between the concepts are extracted from the abstract and NOT from somewhere else. It’s 100% possible for two concepts to have a different relationship from what someone with prior knowledge may have learned in school. Consider that knowledge in textbooks can be 1-2 years old before publishing and that these 1.3 million of these articles are coming out per year!!!
Thanks, Ginger. Glad the logout thing was not your fault. 
My main source of confusion now is illustrated by the following example. The software misinterpreted the word “lead” (present tense of led) as the word for the element “lead.” For the first question, I selected the response “Lead is not a drug concept.” So far so good - that was the predominant response. On the follow-up question, I tried to select the same response, but it would not allow me to do so. I selected my next best response, and was chagrined to see that the preferred response was again “Lead is not a drug concept.” I’ve several times run into something similar, where my first selection was blocked, even though it is sometimes the most “correct” response. So, I’m still muddled.
Mike C. (Capybara on the Mark2Cure site)
Hi Caprarom! Unfortunately, there is a bug in the system that we haven’t been able to quite pin down. I know it’s annoying and I’m sorry for the frustration it causes our users when they’re trying to help science! It doesn’t appear consistently, though it seems to appear more for firefox users than chrome users. Sadly, this isn’t enough for us to get to the bottom of it. Fortunately, citizen scientists are a creative bunch and there have been two workarounds that users have told us about
- Going back to the dashboard, re-entering that relationship and marking the concept as broken. (preferred method)
- Selecting the ‘cannot be determined’ response and submitting that. (makes the data a little messier to analyze, but not impossible to work with)
Please be assured we’ll take this into account when analyzing the results of the challenge when determining the contributor rankings.
1 Like
Thanks again, Ginger! That clears it up for me. Option one then. I had one where all seven answers were something like “VAL is not a drug concept.” Got the first no problem; the other six - very frustrating. Now I know how to handle it.
Thank YOU for giving us a chance to publicly post the workaround. Super frustrating that users have to resort to that, but not an easy problem to fix.
Hi Ginger -
Struggling to define a drug or not. If a compound can be produced outside the body and then provided in some form can that be considered a drug? The community seems to be calling things like hormones or other compounds normally produced by cells as “not drugs”. I need a better working definition of what is a drug. Thanks.
gcalkins
I wish I could give you a hard and fast definition, gcalkins, but there is ambiguity inherent in language, science, and the language of science. The drugs category is probably one of the more ambiguous ones which is why text mining algorithms for drugs can produce hugely variable results. Since there are differences between the way different databases/resources define drugs and because there are plenty of instances where biological substances produced by cells can go either way, a lot of our veteran users check for the suspected drug’s inclusion in https://www.drugbank.ca or for statements about its use for medical purposes on Wikipedia articles and go from there.
If English is not your native language, it is practically impossible to bring these challenges to a successful conclusion. Sorry, but I can make no positive contribution here.
Ginger, if one encounters a statement that says something like, ‘uridine was used to suppress x-toxicity that leads to y-disease … ,’ is it correct to select a relationship that says uridine can treat y-disease, regardless of whether or not uridine is a drug? (That’s what I did, but it was a minority response as most respondents said uridine was not a drug concept.) It seems like something that shows treatment efficacy should be considered a drug concept, whether or not it’s normally considered a drug, but I might be wrong.
Sorry for the late reply–was trying to get the results of the challenge analyzed so ppl can see how they did. With regards to your question, treatment efficacy is a great heuristic for determining whether or not something should be counted as a drug. Heuristic–not hard fast rule, because this is all biology and biology loves to break rules.
