Show
One of my favorite (but not used all that much) test item types is the “matching exercise.” One class I teach has quite a bit of vocabulary that my students just flat-out need to memorize. Matching seems like a good, concise way of testing them with a minimum amount of pain on their part (writing the answers) and my part (creating the test). The sources all agree on the definition:
I was pleased to see this same source describing the types of material that can be used:
This other source, http://teaching.uncc.edu/learning-resources/articles-books/best-practice/assessment-grading/designing-test-questions, lists
As you can see, this test item format is well-suited for testing the Knowledge Level of Bloom’s Taxonomy, however several sources hint that it can apply to the Comprehension Level “if appropriately constructed.” Only one source discusses in detail how to “aim for higher order thinking skills” by describing variations that address, for example, Analysis and Synthesis. (http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf) One variation is to give a Keylist or Masterlist, that is information about several objects, and have the student interpret the meaning of the information, do comparisons (least/greatest, highest/lowest, etc.), and translate symbols. The example gives three elements from the periodic table with the properties listed below them but no title on the properties. The questions ask “Which of the above elements has the largest atomic weight?” and “Which has the lowest melting point?” and other similar inquiries. Another variation is a ranking example:
These directions are followed by a list of events. While I see these variations more as the “fill-in-the-blank” types, their connections to matching properties to objects or events to a time line make it reasonable to treat them as matching types. What are the advantages and disadvantages of matching exercises? (Source: http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/)
(Source: http://www.iub.edu/~best/pdf_docs/better_tests.pdf)
(Source: http://teaching.uncc.edu/learning-resources/articles-books/best-practice/assessment-grading/designing-test-questions)
But
There are design strategies that can reduce the amount of time it takes for students to work through the exercise, and others that don’t put so much emphasis on reading skills. We’ll look at those in the next post. In the previous post we talked about the pros and cons of the alternative-response (e.g., true-false) types of questions as well as their application to Bloom’s Taxonomy. Next we discuss aspects to consider when writing the questions. I found this “Simple Guidelines” list helpful and informative.
All of this makes sense to me. At first I objected to “Make half the statements true and half false” but when I thought about it, I wouldn’t do exactly half necessarily but maybe close to half. In fact this source, http://teaching.uncc.edu/learning-resources/articles-books/best-practice/assessment-grading/designing-test-questions, suggests making the ratio more like 60% false to 40% true since students are more likely to guess the answer is true. I found other points to add to the guidelines list. (Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf)
This same source gives you a nice tip for writing true-false items:
Most of this discussion has been about True-False questions but the category is really Alternative-Response. Let’s look at the variations available to us. (Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf)
In summary, the sources all tend to agree that the best type of Alternative-Response items are those that are unambiguous (“true or false with respect to what?”), concisely written, covering one idea per question, and aimed at more than rote memorization. We should avoid trick questions or questions that test on trivia. And the best tests with A-R items have a lot of questions with a True-to-False ratio of 40:60. Next test item type: Matching! My original intent for the title here was just “True or False Test Items” but one source pointed out that the best name is Alternative-Response. That source is https://www.msu.edu/dept/soweb/writitem.html and defines alternative-response as
It goes on to point out the advantages of this item type:
But there is a major disadvantage: “students have fifty-fifty probability of answering the item correctly by chance alone.” It suggests making up for this by offering “a larger number of alternative-choice items than of other types of items in order to achieve a given level of test reliability.” There are other advantages and disadvantages, such as these offered by http://www.iub.edu/~best/pdf_docs/better_tests.pdf, which only addresses true-false items.
To me this seems like a contradiction: How can they be easy to write but difficult to make unambiguous? This same source offers tips on writing good true-false items which we will address in the next post. This source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf brings up some other points on true-false items.
Bloom’s Taxonomy is of concern for us here, too. Most sources I read indicate these types of questions items address only the first level of Bloom’s, Knowledge. We saw that above with the source listing “Good for” bullet points. Other sources say it bluntly:
*Graphic from http://tips.uark.edu/using-blooms-taxonomy/ It is often thought that multiple choice questions will only test on the first two levels of Bloom’s Taxonomy: remembering and understanding. However, the resources point out that multiple choice questions can be written for the higher levels: applying, analyzing, evaluating, and creating. First, we can recognize the different types of multiple choice questions. While I have used all of these myself, it never occurred to me to classify them. Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf
In fact, this source states:
So what can we do to make multiple choice questions work for higher levels of Bloom’s? Source: http://www.uleth.ca/edu/runte/tests/
Another part of this source brings up the idea of using the “inquiry process” to present a family of problems that ask the student to analyze a quote or situation.
This source gives some good ideas, too. Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf
In short, multiple choice questions, when designed with good structure and strategies, can provide an in-depth evaluation of a student’s knowledge and understanding. It can be challenging to write those good questions but the benefits are worthwhile. I thought about writing a summary of what we have learned about multiple choice questions but found this funny little quiz to be better than anything I could come up with: Can you answer these 6 questions about multiple-choice questions? One type of objective question is multiple choice. We all know what it is but let’s look in detail at its description anyway. Source: https://www.msu.edu/dept/soweb/writitem.html
So the structure of a multiple choice question is a stem followed by options. The options contain one correct answer and a set of distractors. The Stem Some advice for constructing a good stem is Source: http://www.iub.edu/~best/pdf_docs/better_tests.pdf
Here are some examples of good and bad stem design: Source: http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/#stem
The best thought about the stem I have seen on the Internet: Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf
The Options Sometimes known as “the alternatives”, they are composed of one right answer and a group of “foils” or distractors. One point that is emphasized regularly in the resources is that the distractors should all be plausible and attractive answers. Source: http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/
[Ed. Note: I have some issues with this particular example but I get the point of their suggestion.]
Other suggestions from this source:
There is more to think about for multiple choice questions, which we will examine in the next post. Now that we have studied general test writing strategies, ideas, and tips, it is time to pull our focus inward to the details of the questions themselves. In general, question types fall into two categories:
I needed specific definitions for these, which I found here. Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf
This source also suggests guidelines for choosing between them:
And it continues with this bit of advice:
I wanted to see what different sources would say, so I also found this one. Source: http://www.helpteaching.com/about/how_to_write_good_test_questions/
I am not sure that “multiple choice” should be the primary choice but I understand they are suggesting to avoid open-ended questions if you want to measure reasoning or analytic skills or general comprehension. This bothers me a little. It seems to me, from reviewing the previous posts in this blog, that an open-ended question could measure those skills. The example that comes to mind is the question I had in botany about describing the cell types a pin might encounter when passing through a plant stem. That was an essay question measuring general comprehension of plant tissues. The following source brings up good points about analyzing the results. It also notes that objective tests, when “constructed imaginatively,” can test at higher levels of Bloom’s Taxonomy. Source: http://www.calm.hw.ac.uk/GeneralAuthoring/031112-goodpracticeguide-hw.pdf
I like their point about how objective tests cannot test competence to communicate, construct arguments, or offer original answers. Training our students to take only multiple choice tests (or simply answer “true” or “false”) does not help them to learn how to explain their thoughts or even ensure that they can write coherent sentences. This is addressed by the second source and in previous posts. The suggestion is to use a variety of test item types. This can give you a better picture of what your students know, whereas using one single type can be biased against students who are not strong respondents to that type. Which item type is best used when the goal is to measure recall of basic facts?True and false questions are best used when you are looking to test a student's recall ability of specific facts or knowledge.
What type of assessment is most useful for planning for differentiated instruction?What type of assessment is most useful for planning for differentiated instruction? Formative assessment.
What guides students in adopting study strategies when preparing for essay or objective tests?What guides students most in adopting study strategies when preparing for essay or objective tests? Students are likely to study specific details when preparing for an essay test if they know the teacher will require knowing them.
What is the effect when you use experiment interpretation items to assess students recall of a correct interpretation of experimental results?What is the effect when you use experiment-interpretation items to assess students' recall of a correct interpretation of experimental results? It discourages them from learning skills required in interpreting experimental results.
|