Grading for Developers

home | email

Background

Content

Design

Courses

CLASS Courses

Online Assessment and Grading for Course Developers

The CLASS™ Project faced a number of issues when creating online assessments, most of which involved technology problems. Course development teams also had to devise ways to prepare online students for their evaluations through assignments, self-checks, and other interactive devices.

Test Generators
The CLASS™ Project tried several different software packages and processes to create online assessments, but none of them proved to be user friendly at the time. As a result, all changes in evaluations were entered manually because proprietary software was created and was never quite completed with user interfaces for generation and maintenance. Management systems now offer more developer-friendly packages to accomplish online testing and assessments.
Randomly Generated Questions
Some of the teachers from learning centers – schools, districts, and consortiums - requested randomly generated questions on computer-graded evaluations that allowed students one or more retake opportunities. Some distance teachers, however, objected because this made it nearly impossible to use a key to grade the exams. When a student wanted an explanation as to why he/she had missed a specific question, it was impossible for the teacher to replicate the exact exam question to which the student was referring.
Restrictions on Types of Questions Used
The course development team was severely limited in the types of evaluations that they could use. In general, fill-in-the-blank computer-graded objective evaluations didn't work because every possible correct answer had to be entered into the key due to the fact that the computer is sensitive to upper and lower case letters, punctuation, symbols, and spaces. For example, if the correct answer was x + y + z = 25, every possible combination of: letters (upper and lower case), order of letters, letters and spaces, and reverse of number and letters before and after the equal sign had to be entered as possible correct answers. Clearly, short answer assessments that called for the students to explain concepts or actions were impossible to computer grade because no one could create ALL of the potential correct responses. Vendors of management systems are now offering more teacher-friendly packages to accomplish grading.
Directions
The course development team found that they had to be extremely exacting in their directions for the evaluations because there was very little flexibility if students misinterpreted what they were supposed to do on a particular assessment. For example, some students lost their work on subjective assessments because they didn't use the correct save technique when they left the course. Vendors of management systems are now offering more student-friendly packages to accomplish online testing and assessments.
Internet Connections
One of the most common technical issues involved problems with dial-up Internet connections. In most of the courses, if a student lost his/her Internet connection while taking a computer-graded evaluation, he/she would not be allowed back into the exam because each student was limited to one attempt at taking the exam, and the computer recorded that the student had accessed that exam previously. If this occurred, the student could contact the teacher, who had the capability of resetting the controls and allowing the student to retake his/her evaluation.
Security
The potential for problems with Internet connections also created another potential academic security risk because a student could access the test, copy the questions, find the correct responses, and then tell the teacher that he/she had lost the Internet connection. Since the teacher had no way of knowing if a student's connection had been interrupted, the teacher had little choice but to reset the controls. Vendors of management systems now offer more secure packages to accomplish online testing and assessments, including timers and tracking to last point of access so a teacher can ‘see’ where the student left off.
Feedback
Feedback problems also plagued the development teams. Feedback on assignments and self-checks needed to be explanatory so that students would get the information that they would need for their evaluations. In a classroom the teacher provides this type of input, but distance teachers do not have the time to give every student the individual attention that students may need. This was especially difficult on self-checks and assignments that had open-ended responses. The course designers tried to foresee what kinds of questions the students would have and focus their feedback to alleviate these problems. Multiple-choice self-checks were time consuming also because the reason why each of the wrong answers was incorrect as well as explaining why the right answer was correct is necessary for each of the questions and answers. These are efficient tests however, once developed, because they are computer graded and provide informative feedback to help the student learn and support their learning experience online.
Notebooks – online assignments and projects
Courses developed in the early stages of the CLASS™ Project were designed so that students could use Netscape Composer™ for assignments to be submitted to the distance teacher. This was at a time when Netscape dominated as the browser of choice. However, technology changed so the courses needed to operate on both Netscape™ and Internet Explorer™ using either a PC or Macintosh™ (Mac) computer. Internet Explorer did not have the same capability to send graphics, audio, and video to a teacher at a distance. And, if students created their assignments using software that their teachers did not use, the teachers could not access the students' work. Vendors of management systems are now offering more flexible packages to accomplish online writing assignments.
Browser/Platform and Software Incompatibility
Early in the CLASS™ Project, some of the software packages available to create interactive activities proved to be incompatible with Internet Explorer™ on Macintosh™ computers. For example, Authorware™ used when the courses were developed for both PC and Mac computers on Netscape™ did not operate correctly on Macs with Internet Explorer™. Test, and test again, what ever is to be used before it is actually used. Designers of content and of graphics are very creative people – but a set of tools and specific guidelines must be adhered to consistently. The message here is to stay simple – just because you “can” add something snazzy doesn’t mean you “should” add it – if it doesn’t add to the learning experience, if it is not non-browser specific, if it is too large of a file for a dial-up connection download – don’t add it to the course. Know the environment in which your users exist and develop to that maximum, and – always assume your users have less computing capability than you do.

[ home ] [ contact ]