MRSCOR | MRSCOR (multiple responses per question) | Scoring Multiple Choice Tests | TSQS | Academic Technologies

MRSCOR

The Lowest Question No. Used may be coded to request that data be picked up from the answer sheets beginning at a point other than question number one. On earlier versions of the Request for Service Sheet, these 3 columns have the heading Batch Number rather than Lowest Question No. Used. Regardless of the column heading, the number coded in this field is used to specify the first question of interest on the answer sheet.

The Highest Question No. Used refers to the last question number (row) used for responses on the answer sheet. For MRSCOR, this number should correspond to the number of items on the test if scoring is to be performed, i.e., unlike GPSCOR, leaving an item blank on the key does not necessarily exclude that item from scoring. Since MRSCOR allows any number of right answers to a question, having no right answers for a question is also allowed. For an alternate method of excluding an item from scoring in MRSCOR, refer to the section describing the Item Weighting Sheet.

The Question Length is the maximum number of choices (response positions) used for any of the questions. For MRSCOR, in particular, it is important to note that it is assumed that all questions have the same Question Length. Note that, for the 10-choice answer sheet, the responses are numbered 0 through 9 but the Question Length is actually 1 through 10.

For MRSCOR, the No. of Keys may only be 0 or 1 to indicate the absence (no scoring) or presence (scoring required) of a key sheet. A key sheet is a General Purpose Answer sheet that has the correct answers marked on it. The nature of scoring using MRSCOR, particularly when option 0 is chosen for Main Parameter 'E', is such that it is assumed that a Right Minus Wrong scoring formula will be appropriate. Thus, the Key Sheet must also include weights that are to be applied to the number of rights (R) and the number of wrongs (W) to derive the final score (T). These weights are communicated by using columns K and L in the Special Codes field of the Key Sheet.

In the first Special Codes column (labelled J in the illustration to the left), indicate the weight to be applied to (R) and, in the second column (K in the example), indicate the weight to be applied to (W). When Main Parameter 'E' is set to 0, the usual weights are 1 and 1 which yields the score T=R-W; the standard scoring formula for a True-False test. When Main Parameter 'E' is set to 1, weights of 1 and 0 (yielding T=R) are appropriate. In both cases, other values may be chosen if you so desire.


The Main Parameters are used to communicate the following information to the MRSCOR scanner program (please write the number in the box provided and fill in the corresponding bubble to request each of the desired actions):

A. Name, (sex, grade and birthdate) processing:

0 = do not include in output file
1 = transfer to data file
2 = reject sheet when the NAME has blank columns between the first column and the last character in the field
3 = reject sheet when the NAME is totally blank

B. Identification number:

0 = do not include in output file
1 = transfer to data file

C. Special codes field (including SA1 and SA2 on the UofA GPA sheet):

0 = do not include in output file
1 = transfer to data file
2 = reject sheet if marks in the special code field are quite light

D. Item responses:

0 = do not include in output file
1 = transfer numeric (0,1) values of responses to data file
2 = transfer alphabetic (T,F) values of responses to data file

E. Scoring mode:

0 = score only the presence of marks, i.e., compare the marks made by the student to the marks on the key
1 = score both the presence and absence of marks, i.e., compare the marks made and omitted by the student to the marks present and omitted on the key and give credit for each kind of correct occurrence

F. Item weighting:

0 = do not apply item weighting
1 = do apply item weighting (see description of Item Weighting Sheet below)

 

The Secondary Parameters are used to communicate the following information to the MRSCOR scanner program (please write the number in the box provided and fill in the corresponding bubble to request each of the desired actions):

K. Check for left-justified Identification Number:

0 - 9 = demand this number of digits in the Identification Number counting from the left-most column

L. Check for right-justified Identification Number:

0 - 9 = demand this number of digits in the Identification Number counting back from the last column in the grid

M. Tolerance value for identifying poor erasures:

Optical mark reading works best when the task of the reading process is to identify the one response position among several where the darkest mark has been made. In the case of MRSCOR, the task is to identify the presence or absence of a mark to each response position. In order to increase the accuracy of this process, the average intensity of all marks appearing on the answer sheet is determined and a cut-off score several levels below this average is used to identify 'light marks' which might be the result of poor erasures or smudges on the answer sheet. The number of levels to step down is the value that should be coded in column 'M'. A value of 4 is suggested. Smaller values will result in a larger number of responses being declared light while larger values will result in a smaller number of responses being declared light. Thus, a value larger than 4 will be more likely to cause a poor erasure to be accepted as a valid mark. During scanning, sheets having marks identified as 'uncertain erasures' are pocketed for visual inspection by the operator (if the 'Pocket Sheets' option has been selected). Attempts will be made to clean up these sheets if necessary (this can significantly increase the operator charge for the scanning job).

If 'E'=0, when a sheet is accepted for processing that has 'uncertain erasures' on it, the questionable response is represented by a blank (in the case of numeric response values) and by a ? in the case of alphabetic response values.

If 'E'=1, the 'uncertain erasures' are coded as though they were omits, i.e., '0' for a numeric response value or 'F' for an alphabetic response value.

NOP. Minimum number of marks that should appear on each answer sheet:

This 3-digit value gives a lower limit for the number of marks that should appear on an answer sheet. Sheets having less than this number are rejected for visual inspection (if the 'Pocket Sheets' option has been selected). This option is used to guard against the possibility that too many marks are treated as erasures as a result of the option selected for 'M'.

 


One of these two options must be chosen. If you request "Pocket Questionable Sheets for MANUAL Verification", sheets that are rejected because they appear to be improperly completed (for reasons such as those requested in the Secondary Parameters above) will be diverted to a special hopper where the operator can review the sheet to see if it can be repaired. After any possible repairs are made, the rejected sheet is scanned again and forced to be accepted even if repairs cannot be successfully performed.

If you request "FLAG Questionable Fields and Process as if Normal", sheets that appear to be improperly completed will not be selected for operator attention. Instead, a '?' will be placed at the beginning of the field that contains questionable results. Choosing this option means that the sheets will be processed without operator assistance, e.g., a sheet which has been completed using pen instead of pencil will be accepted as though it was completely blank.

 


Only the Itemanal program from this list may be used to process the data file produced by MRSCOR.

A special scoring option exists in which the requirement is made that the complete pattern of responses and omits to each question must match the key yielding a possible score of 0 or 1 for each item. To obtain this scoring option, please indicate 'MRTOGP' on the 'Other' line and, also, specify 1 for Option 'E' of the Main Parameters. The MRTOGP program converts the data file from an MRSCOR data file to a GPSCOR data file by scoring each item T or F depending on whether or not the pattern of responses to the item matches the pattern given on the key. If this scoring option is used, it is then possible to ask for Data Processing using Examtab (or even Queseval) rather than Itemanal.


This part of the menu allows you to request generation of a variety of 'class lists' which contain the Name, Identification Number, Special Codes and Scores for each answer sheet. Only the fields that you have requested to have included in the output file (using the Main Parameters options) will appear in these lists, e.g., if you specify '0' under option 'A' of the Main Parameters, the Name field will not be included in the output file and therefore will not be available to include in the class lists. You may ask for lists sorted according to a variety of fields. If you ask for a list sorted by Identification Number, you may also ask that the Name field be dropped from that list. This yields a listing that is suitable for public posting.

The numbers following the 'Sorted by Score' option refer to the sequential number of the score in the output file as follows:

  1. number correct score
  2. number incorrect score
  3. total (R-W) score

When these lists are generated, you may ask that the scores be scaled so that they are 'out of' a total other than the total possible raw score, e.g., you may ask that the scores be converted to percentages or scaled so that the maximum possible score is, say, double the number of questions on the test.

In addition to the printed output described above, you may request that the output be placed in any combination of 3 types of files:

  1. the sorted data may be placed in a plain text file. This is a copy of the file that we send to the printer to generate your printed output when you mark the option Hard Copy. This file will normally have the suffix .srt. If this file is sent to you, it may be opened with a text editor such as WordPad on a PC or TextEdit on a Mac.
  2. marking WebCT requests a file suitable for uploading to eClass which contains each student's last name, id number, user ID, and score. In order to create this file, you will need to provide us with the Registrar's catalog number (Class ID) for the class. This file will normally have the suffix _w.csv
  3. marking Excel requests that the sorted data be placed in a comma-delimited (*.csv) file which may be used to input the data into programs such as Excel. The file generated by this request will contain the combination of name, id number, special codes and scores that have been transferred from the answer sheets. It will be sorted according to the first sort request that is specified, e.g., if you ask for a sort by both name and id the file will contain data sorted by name. This file will normally have the suffix .csv

The options to include the Wrongs and/or Scored Responses columns in the eClass or Excel file are not available with the MRSCOR program.


Item Weighting Sheet: If option 1 is specified for Main Parameter 'F' above, the Key Sheet must be preceded by a Weight Sheet. This is a General Purpose Answer sheet that has a value coded for each item in the test which indicates the weight to be given to the score on that particular item. Although this feature was implemented for more ambitious applications, a practical application of this feature is to use weights of 1 and 0 to indicate inclusion or exclusion of each item in the test. To communicate values of 1 and 0, use a 10-choice General Purpose Answer sheet which includes the value 0 as a possible response. (You may do this even if you have used a 5-choice answer sheet for the key and students' response sheets.) Note that using the 10-choice GPA sheet to communicate the weights will restrict the Highest Question No. to 120 even if the remainder of the answer sheets are 5-choice sheets which allow larger values for the Highest Question No.

The MRTOGP program cannot be used when the Item Weighting Sheet option is used.

Academic Technologies Sitemap

Classrooms

Book a Classroom
Classroom Listing

Computer Labs

Book a Lab
Lab Listing

Software

N/A

Test Scoring & Questionnaire Services
General Data Collection
Instructor Designed Questionnaire (IDQ) System
Scoring Multiple Choice Tests
Universal Student Ratings of Instruction (USRI)
Web Surveys
High Performance Computing

Shared Computing
Visualization