Scoring Multiple Choice Tests Overview | Scoring Multiple Choice Tests Overview | Scoring Multiple Choice Tests | TSQS | Academic Technologies

Scoring Multiple Choice Tests Overview

Our main answer sheet for use in general-purpose classroom testing is the University of Alberta General Purpose Answer Sheet which accommodates 200 five-choice questions, 50 twenty-choice questions and 2 fields for entry of instructor-scored sections of a test used, for example, to enter marks from essay questions.

A significant feature of our services is built on an Item Analysis program that is still being enhanced after 40 years of use involving migration through a series of hardware and operating-system platforms. The description, Classical Test Theory, is clearly appropriate for this program. Inspection of item analysis reports can be very useful for identifying problems with a test or its scoring including something as simple as miscoding the answer sheet containing the keyed responses. The wealth of detail included in the reports from this program becomes more informative as the size of the class increases (as artifacts of sampling decrease). Following the inspection of this report, if desired, re-scoring the test can be performed without re-submitting the answer sheets to be processed a second time.

The records that are kept of the students' responses which enable the above also contain sufficient detail to usually determine if the responses on an answer sheet have been modified after processing has taken place. Comments have been made from time to time in the past that the machine made a mistake. This almost always arises after the original answer sheet was returned to the student who then changed one or more responses and brought the problem to the attention of the instructor. Unfortunately, there have been situations where the instructor accepted the student's claim. In well over 99% of the time, the machine does not make a mistake. If a concern arises about the accuracy of the processing, we want to hear about it. If there has been an opportunity for someone to modify the responses on the answer sheet, we want to examine the answer sheet and evaluate it in relation to the record we have about its contents when we first processed it. In any case, it is important for us to hear and deal with concerns regarding the accuracy of our services.

A more recent addition to our services involves the generation of output files intended for uploading to eClass. eClass is frequently used for management and distribution of class assessments. Our basic service matches records obtained from OMR processing with class-registration information so that the multiple-choice test scores may be conveniently entered into the eClass gradebook. New options to this service include two additional text fields:

    • a list of the numbers of the items that the student got wrong (the wrongs report), and
    • a list of the responses that the student gave on the test with correct answers indicated in upper case and incorrect answers shown in lower case (the R/w report).

Two generalized programs are available for scoring instructor-developed multiple choice tests. Each of these programs may be used to score tests when the students' responses have been recorded on one of the three types of General Purpose Answer Sheets that are stocked on campus.
GPSCOR (General Purpose Scoring Program) is used for scoring tests when students are expected to respond with only one answer per question. This is the preferred method for which the OMR machines were designed and, thus, for which the reading of responses is most accurate. In order to score a test, you must supply a key sheet which is simply one of the above General Purpose Answer Sheets on which is marked the correct answer for each question that is to be scored. It is helpful if you write 'KEY' on this sheet so that the operators can easily identify which sheet should be used as the key sheet.

GPSCOR provides a number of scoring options:

    • Up to 8 key sheets may be used to generate scores, or sub-scale scores, on various sets of items in the test. Scores may be computed as the sum of the number of correct answers or the sum of the number of incorrect answers. Incorrect answers are only counted for those items having a correct answer indicated on the key. The absence of a response to an item is not counted as an incorrect answer.
    • The scores from the above keys may be combined into as many as 6 different composite totals. Each score may be weighted by an integer between 0 and 9 before accumulating it to the composite total.
    1. This allows one, for example, to obtain a total score that is the sum of the score on one key plus double the score on a second key.
    2. It also provides a way of allowing several possible correct answers on a particular item (provided that the student only gives one answer). In this situation, the first key sheet could contain the first correct answer to all the items while a second key sheet could contain only the 'second-choice' correct answers to those items that require that provision.
    • Scores may be computed from as many as 4 key sheets using formula scoring (often called correction for guessing). The formula used is R - W/d where

R is the sum of the correct responses
W is the sum of the incorrect responses (omits are not counted)
d is the number of response alternatives minus

Please refer to the instructions for completing the Optical Mark Reader Request for Service Form for further information about GPSCOR.


MRSCOR (Multiple Response Scoring Program) is provided for situations where students are allowed or expected to respond with more than one answer per question.

MRSCOR also provides a variety of scoring options (in all cases, only a single key sheet is used):

    • Scores may be computed by only focusing on the responses that the student made. In this case, the number of correct responses is counted, the number of incorrect answers is counted and a final score is reported which would normally be R - W but the option is provided to apply a weight other than one to either R or W. The total, unweighted, score possible in this case is equal to the number of answers marked on the key sheet. Note that this means that the possible score for a question depends on the number of responses marked for that question on the key.
    • Scores may also be computed by summing the number of correct behaviors performed by the student, i.e., summing both the number of times that a correct answer is marked and the number of times a choice is correctly left blank. The total, unweighted, score possible in this case is equal to the number of questions on the test multiplied by the question length (the number of response choices) of each question.
    • A third alternative is a two step process. After using the second option (above), a program can be run that converts the output file into a GPSCOR file. In the conversion process, complete questions are marked right or wrong depending on whether or not the complete pattern of responses and omits matches the key that has been provided. In this case, the total score possible is equal to the number of questions on the test.

Please refer to the instructions for completing the Optical Mark Reader Request for Service Form for further information about MRSCOR

Academic Technologies Sitemap

Classrooms

Book a Classroom
Classroom Listing

Computer Labs

Book a Lab
Lab Listing

Software

N/A

Test Scoring & Questionnaire Services
General Data Collection
Instructor Designed Questionnaire (IDQ) System
Scoring Multiple Choice Tests
Universal Student Ratings of Instruction (USRI)
Web Surveys
High Performance Computing

Shared Computing
Visualization