0

Penalty for Mass Selection on Multiple Answer Questions.

Kerry Pretty

October 07, 2011 00:21

Completed

Follow

Is it possible to set a penalty if students use a mass selection strategy or modified mass selection strategy (pick everything except those they know are incorrect)?

For mulitple answer questions, can you add a mass selection strategy deterrent of some sort, possibly within each question itself, like a loss of 1 point for each answer selected that goes above the number of correct answers for the question. For example, if a student selects 4 answers when there are only 3 correct, they lose 1 mark and if they select 5 answers when only 3 are correct, they lose 2 marks. Then, for questions with say 6 possible answers, if only 2 answers were correct, a mass selection strategy would result in +2 for the correct and -4 for the incorrect or a total of -2 overall for the question.

If something along those lines could be selected as an option like "partial credit, would be appreciated. Perhaps even the number of penalty points for each answer over the number correct could be selected.

• Facebook
• Twitter
• LinkedIn

8 comments

• 0
  
  Adam Sentz April 17, 2014 21:07

  @Kerry - While it's not exactly what you've described here, we did modify the multiple answer grading method in a way that I think addresses the heart of your request. 

  The way it works now is that each option is equally weighted and is either right or wrong. So checking a box that you shouldn't or not checking a box that you should count against you equally. So, say you have a 10 point partial-credit question with 10 options, 5 of which you should check. If you check all of them you would get +5 for the ones you should and -5 for the ones you shouldn't have checked resulting in 0 points for the question.

  If you were to check all the ones you shouldn't and none of the ones you should you'd theoretically earn a -10 for the question, but we've limited questions it to 0 as the lowest possible value.

  So while it doesn't impose an additional penalty for mass selection it's no longer biased towards it. Do you feel like that sufficiently addresses your concerns?

• 0
  
  Patrick Friesen December 20, 2018 17:02

  So how does it work if you don't have an equal number of correct and non-correct answers. Say you have students needing to select 4 of the 10 possible answers on a partial-credit question worth 10 points. Is the formula that is used one that equalizes the correct answers to add up to 10 and all the incorrect answers to also equal -10?

• 0
  
  Adam Sentz December 20, 2018 17:34

  @Patrick - No. Each of the 10 options is either correct and worth 1 point or incorrect and worth -1 point. If a student checked  every answer in this example they'd get 0 points for the question since questions can't be worth negative points (the calculation being 4 - 6).

• 0
  
  Patrick Friesen December 20, 2018 17:54

  I don't think I asked the question clear enough. I'm just trying to figure out the formula that is being applied here. From what you have said my assumption is that the total number of possible correct answers = the total possible points awarded to the question. Similarly the total number of possible wrong answers = the negative total points awarded to the question. Here is another example that I've experienced. 

  • Question is worth 8 points
  • Students are told to select the 8 correct answers out of a possible 10 answers.
  • According to what I'm reading here each correct answer is worth 1 point and each incorrect answer is worth -4 points.
  • If a student were to select 7 of the correct answers and select only 1 of the incorrect the points awarded would be 3. 
  • If a student were to select 6 of the correct answers and select both incorrect answers the points awarded would be 0.

  Is this the way it gets calculated? It is important to know the formula so that I'm not setting up students with bad questions.

• 0
  
  Adam Sentz December 20, 2018 18:31

  "From what you have said my assumption is that the total number of possible correct answers = the total possible points awarded to the question. Similarly the total number of possible wrong answers = the negative total points awarded to the question."

  This is not correct. I think we are using correct/incorrect in different ways. Think of each option as a True/False question where checked means True and unchecked means False. Each option is either correct or incorrect based on whether or not the student answer True when it is supposed to be True or False when it's supposed to be False. This is opposed to checked meaning "correct' and unchecked meaning "incorrect". 

  Let's take a 10 point, partial-credit question with 10 options. Each option is either supposed to be checked or not. For each one you check that you are supposed to check, or that you don't check when you aren't supposed to check it, you will get 1 point, and for each option that you check when you shouldn't or don't check when you should you will get -1 point. 

  In your new example of an 8 point question worth 10 points, each correct answer (checked being checked, unchecked being unchecked) would be worth .8 points (8 points / 10 options). To get all 8 points you must check all 8 that are supposed to be checked and not check the other two.

  In this example "If a student were to select 7 of the correct answers and select only 1 of the incorrect" they would receive 4.8 points (8 correct responses x .8 points) - (2 incorrect responses x .8 points).

  In this example "If a student were to select 6 of the correct answers and select both incorrect answers" they would receive 1.6 points (6 correct responses x .8 points) - (4 incorrect responses x .8 points).

   

  Edited by Adam Sentz December 20, 2018 18:39
• 0
  
  Patrick Friesen December 20, 2018 18:36

  Got it. That helps me explain this to students who are wondering how the calculations are made.

• 0
  
  Adam Hamill September 25, 2024 01:09

  I strongly disagree with this position. 

  Simple example: a 10 point question has 10 choices, 8 of which should be selected. 

  You say that each of these choices is like a True/False sub-question worth one point each, but it's not. A student who correctly selects 6 and correctly de-selects 1 has correctly answered 70% of the individual True/False sub-questions right. If these were 10 one-point True/False questions, the student would and should get seven out of ten points.  

  Populi's partial credit does not do this. Rather than giving credit to the correctly choices, it causes the incorrect choices to cancel out points for the correct choices. In this scenario, correctly selecting/deselecting 7 out of 10 gives a grade of 40% (7 correct minus 3 incorrect).

  I understand the goal of disincentivizing selecting all options, but this goes well beyond this. Rather, let teachers disincentivize selecting all options by coming up with more false choices, rather than this method. And if they want to further disincentivize, they already have the chance to take away partial credit. But teachers cannot give partial credit the way they would want using your system. 

  I am grateful for this conversation. It has highlighted that I cannot use Multiple Answer questions unless and until Partial Answer grading changes. I will now have to comb through my questions and remove all Multiple Answers I can, and re-grade remaining Multiple Answers that students have gotten wrong.

• 0
  
  Adam Sentz September 25, 2024 17:43

  Adam Hamill - We understand that the multiple answer partial credit calculation might not be what every teacher would do in every circumstance. There probably isn't one way that even one teacher would agree with 100% of the time. Right now the only way to deal with that is to adjust the points as desired (which you are always able to do), but we could consider adding the option not to penalize incorrect responses. I'm not finding an existing feature request for that functionality - in case you'd like to add one.

  For the record, I never wrote that "each of these choices is like a True/False sub-question worth one point each." 

  I think you are referring to this that I wrote in 2018:
  
  

  Think of each option as a True/False question where checked means True and unchecked means False. Each option is either correct or incorrect based on whether or not the student answer True when it is supposed to be True or False when it's supposed to be False. This is opposed to checked meaning "correct' and unchecked meaning "incorrect". 

  Let's take a 10 point, partial-credit question with 10 options. Each option is either supposed to be checked or not. For each one you check that you are supposed to check, or that you don't check when you aren't supposed to check it, you will get 1 point, and for each option that you check when you shouldn't or don't check when you should you will get -1 point.