Web-based Study Skills

home | email

CLASS™ Project Focus Group Summary – December 1998
Nine Studies and Fourteen Themes

A high school classroom was contacted to participate in a group discussion about the CLASS™ courses. Approximately ten students attended the discussion, accompanied by their teacher. Courses were projected onto an overhead screen and students commented on graphical and design features.
A primary purpose of the internal studies was to study course characteristics and experiences surrounding involvement in courses. Themes and details were expected to emerge from the studies that would be reflected across the different types of studies. From this information, recommendations were expected to be made available to the CLASS™ research and design staff both to inform future on-line course design and to serve as a research artifact of the grant.

Summary

The nine studies conducted by the CLASS™ grant uncovered a broad range of detailed information on student, administrator, and teacher experiences while interacting with web-based courses. The detailed statements and interpretations derived from surveys and interviews expressed the acuity with which students, administrators, and teachers viewed CLASS™ web-based courses. When provided a forum to speak of their experiences and voice their views, study participants provided prolific commentary which is detailed in the 14 themes that follow.

Theme 1: Teacher Experience
Teachers who are accustomed to interacting with students in a paper-based correspondence curriculum experienced considerable disruption to their daily work schedule as they learned to adopt new vehicles for teaching. These teachers were no exception. The online courses were complicated for the teachers to work with for a number of reasons. These were articulated in the Findings section of the study report. Teachers expressed strong opinions regarding their role in course development and the impact on teaching. Findings relevant to future program design include:

Teachers who are expected to engage in web-based teaching need plenty of time to learn the technology, the program in which courses are built, and to learn the new courses thoroughly.
To implement technology and expect immediate adoption of innovation is unrealistic. If such implementation is to be considered, care must be taken to ensure adequate support and training.
Teachers strongly prefer, whenever possible, to have a voice in the creation of courses that they are expected to teach.
If participation in authoring is not an option, teachers who are flexible and knowledgeable of their content like to have a means to supplement or enhance course material to fit learning needs of individual students.

Teachers’ responses support the extensive body of literature on adoption of innovation. The unique perspective of the independent study distance-learning teachers adds to this literature. When planning online course development and teacher training timelines, time and participation time requirements should be taken into account.

Theme 2: Administrator Experience

Experiences of administrators as they interact with Sales and Marketing representatives, Customer Service, and others involved with installing courses in schools and learning centers builds relationships that can maintain, strengthen, or bring failure to future expansion. The need for research on the relationship between administrator experience and technology adoption is paramount as administrator’s influence or control budgetary decisions.

Administrative challenges included the interactions of students with the technology:

Students were not familiar with computer technology;
Turnover of key staff (lab manager and AmeriCorp volunteers in one study) resulted in repeated delays;
Technology challenges presented problems for implementation;
Unique demands of working with at-risk students and families meant spending the time getting the program up and running was “a great accomplishment;”
Program administrators became aware that students with learning disabilities may not be good candidates for the web-based courses;
Customer care technical support with both CLASS and class.com was excellent;
Timing of the start of such a project is crucial. The system must be up with kinks worked out in time to begin at the start of a school year or semester break.

Expecting smooth implementation of technology innovation in schools and learning centers without thorough preliminary technical assessment is unrealistic. Should an administrator understand possible experiences that may be confronted when adopting web-based courses, realistic expectations may be framed and frustrations may be alleviated. Potential rejection of courses before experiencing full benefit of web-based courses may then be avoided.

Theme 3: Student Experience

Experiences of students as they enter and engage in online courses influence attrition rates and re-enrollment decisions. As distance education competition increases, student customers have more choices for online education. Overall quality of courses in a full curriculum or individual courses is defined by a wide range of development and support details. Students expect their total experience to be smooth, seamless, and friendly.

Most students reported little to moderate experience, depending upon the aspect of computer technology investigated. Findings indicated a connection between student experience with computer and Internet experience and frustration with the online test course under investigation.

Students with little technology experience self-reported high levels of success expectation in an online course. (Orientation)
While in the courses, an inverse relationship between technology experience and frustration emerged that influenced student learning experience. Frustration resulted from lack of fluency with experience with technology. (Orientation; Start-up Sequence)
Technology function and lack of knowledge of how to handle difficulties was the primary source of frustration for the majority of students. (Orientation; Start-up Sequence)
Technology fluency had an impact on students’ perceptions and tolerance for technology-related “glitches” within courses. (Orientation; Start-up Sequence)
Students who had less experience with computers were more easily frustrated by slow downloads, program and computer crashes, and out-of-course links that did not work. Frustration of low experience students with technical difficulties suggests a link between high frustration and a lack of knowledge of the World Wide Web and computer function in general. (Orientation; Start-up Sequence)
In studies that involved hands-on computer experiment, computers or software locked up at least one time for each student. Students who had self-identified as having higher levels of experience reported the same technical difficulties but no frustration. (Orientation)
Students report two other comments: “Good to remind students teacher is watching.” (12/2/98 Focus Group); “Teachers should reply faster.” (12/2/98 Focus Group)

Findings suggest that to reduce student frustration is to increase potential for student satisfaction, which would possibly influence re-enrollment. Lack of experience with the Internet and computer functionality presented comments that revealed frustration based upon unrealistic expectations. This finding has strong implications for curriculum and instructional designers.

Theme 4: Curriculum Design General Comments

Recent research suggests that learners negotiate hypertext differently than they do paper-based curriculum. Curriculum designers for multimedia rich web-based courses are faced with little research to inform application of appropriate curriculum design in a hypertext environment. Participants in most of the CLASS™ studies provided feedback for course designers to consider in course design and revision. Recommendations for curriculum design were interspersed in responses to other questions in the surveys and interviews. Curriculum design comments and suggestions cover a wide range of application and are derived from the Students were specific and detailed in their responses in the studies:

Students want clear, step-by-step instructions, including how to get started, where to type answers, how to work in notebooks or technical lesson submission procedures. (Millard South; 12/2/98 Focus Group; Start-up Sequence)
Design instructional strategies such as advance organizers, objectives; introductions, headers and subheads, and course outlines, to help students organize the content. (Orientation; 12/2/98 Focus Group; New Users)
Ensure that the content is organized, consistent and correct. (Orientation; 12/2/98 Focus Group; New Users; Start-up Sequence)
Divide content into small lessons. (12/2/98 Focus Group)
Present information in lists whenever possible. (12/2/98 Focus Group; New Users)
Students like callout “cool little facts,” pop-up windows, and “please explain” boxes. (12/2/98 Focus Group)
Content presentation should contain active learning, attention to student engagement. (12/2/98 Focus Group)
Summarize content periodically. (12/2/98 Focus Group)
Provide background on stories that are included as content. (12/2/98 Focus Group)
Build in opportunity for students to interact with each other and learn opinions. (12/2/98 Focus Group)
It [the content] should tell you all the work you should have completed before starting a new area. (12/2/98 Focus Group)
Include clear information about when student completes a lesson, section, unit. (12/2/98 Focus Group)
Include troubleshooting tips within content or clearly placed in the Toolbar. Troubleshooting tips should include strategies to cope with hardware or software error. (Orientation)
Students feel competent with online learning when instructional design builds access to resources, including: Glossary, Course Guide, Visible navigation buttons, Self-checks –(Orientation)
“Make it easier to begin course after Orientation.” (12/2/98 Focus Group)
Write content to respect the intellectual competency of learners. “Introduction cheesy.” “Voice sounded cheesy.” (12/2/98 Focus Group)

As critical as curriculum design is to student satisfaction, and perhaps to successful performance, attention to curricular elements must build upon each other to maximize cognitive development of hard and soft skills. Study participants interjected curriculum-related comments within surveys and interviews that did not specifically request information on this aspect of the test courses.

Theme 5: Metaphor

Within a web-based teaching and learning environment, the use of metaphor to create a context for learning, rather than presenting content to learners devoid of any storyline, is consistent with the definition of contextual teaching and learning. Contextual learning is a solid constructivist instructional approach that serves online instructional designers by situating learners within the web-based instruction, especially through the use of a metaphor or storyline as a cognitive “anchor.” Fitting contextual learning to web-based learning environment involves framing a problem into a readily recognizable context, or situation, in which the learner can build knowledge.

A metaphor serves different purposes for learners. Within a hypertext environment, free from time and space, students do not have landmarks with which to locate both themselves and new knowledge unless landmarks are built into the design of a course. A metaphor serves as one layer of landmarks within which the student can locate. By creating a metaphor that places the learner in a virtual environment that simulates a real, familiar environment such as work, home, or school, the learner can create relevance for the newly constructed knowledge, and have a context for transferring knowledge into practical life situations.

The study conducted at Millard South High School was designed to explore at-risk high school student responses to a metaphor-rich online learning environment. The course, Learning Fundamentals, was used as the curriculum for a specific group of students for whom learning to learn skills were integral to the curriculum. The Orientation study findings also offered one comment.

Students prefer a storyline (metaphor) to no story or theme to locate themselves in virtual space. (Millard South)
If a story is designed into the content, it must be related to the content. (Millard South)
Story must be relevant to the lives of the learners to encourage engagement and immediate transfer. (Millard South)
If a metaphor is included in course design, include an explanation both in the Course Guide and in another accessible location to benefit students who do not access course resources. Explanation of a connection between metaphor and course helps the student to understand course organization. (Orientation)

Students in web-based courses can benefit from being located in a context that provides a time and space while still benefiting from the rich resources available on the World Wide Web. Contextual learning enjoys a long history of documented learning effectiveness in face-to-face classroom environments. Adapting contextual learning to web-based course development offers potential for student success.

Theme 6: Glossaries

Access to definitions of unfamiliar words is an essential component of any learning environment. Face-to-face classrooms generally have a dictionary available for student use. A glossary is also an essential component of quality textbooks and curricular materials. Classrooms with access to the World Wide Web do not consistently have ready access to Internet-based reference materials due to firewalls. Web-based courses used in conjunction with a classroom face the possibility of firewall interference with access to online dictionaries. A sensible solution is inclusion of a glossary within web-based courses. Web-based curriculum design offers alternative glossary designs and access routes.

Study participants offered specific recommendations for glossary design for curriculum designers to consider:

Include multiple languages, especially in the language courses (i.e., English and Spanish). (8/00 Focus Group)
Design pop-up glossaries on page with rollover or click-ons design; some students report click-ons take too much time. (8/00 Focus Group; 12/2/98 Focus Group)
Buttons to access definitions placed within a sentence are easy to access but can be distracting. (12/2/98 Focus Group)
Include technical terms in Glossary. (Start-up Sequence)

Students request ready access to meanings of words as they move through content. To miss the meaning behind a concept due to lack of knowledge of a definition or lack of ready access to a glossary presents a troublesome barrier for students.

Theme 7: External Websites

The World Wide Web contains rich information that can be used to augment student experiences with curriculum. Access to appropriate websites can add value to a course by enriching content and providing alternative views on a topic. Observations and recommendations that emerged include:

Students generally do not click on the linked WWW pages while some students enjoyed exploring off-course web sites. (Virtual Teacher; 12/2/98 Focus Group)
Explanation is needed to tell students that they need to click on the link to go to the page. (8/00 Focus Group; Virtual Teacher)
No consistent preference for page-o-links. Some students preferred the page-o’-links. Some preferred direct access from course page to the web. (8/00 Focus Group)
On Page-o-links, group links by unit, section, lesson. (8/00 Focus Group)
List titles and URLs on page-o-links. (8/00 Focus Group)
General comment: students liked the page-o’-links to be visually appealing similar to the Introductory pages of the courses. (8/00 Focus Group)

The dynamic nature of the World Wide Web has resulted in a rise of sites that become inaccessible, or simply nonexistent. A “page-o-links” is a design convenience that allows designers to easily identify such links. Use of a page with all links included is preferred by some students, not by others.

Theme 8: Assessment

Assessments are the measure by which students demonstrate their hard and soft skills within a content area. The web-based courses examined presented different types of assessments, including self-checks, short quizzes, objective tests, and essay tests. Five of the studies provided data about assessment format. Participants from all five studies provided information based upon their experiences and expectations of assessment purpose and functionality:

Include frequent self-checks or quizzes, but not constant little quizzes that are worth only two or three points. (Millard South; 12/2/98 Focus Group; Orientation)
If information that will be tested is located on an external or cached website, clearly inform students as they tend to skip over links. (Virtual Teacher)
Include clear, concise, accurate step-by-step instructions for homework activities and exams. “I didn’t know if some questions were graded.” (8/00 Focus Group; Orientation; 12/2/98 Focus Group)
Provide clear directions to pages that contain links to exams. (Orientation)
Provide information about the nature of the assessment so students know if collaboration or access to course materials are acceptable. “Too easy to cheat.” (12/2/98 Focus Group)
Point students to location in the course where they can locate information about the questions. This may be a description of the unit and lesson or may be a link from the test question to the place in the course where the content is presented. (12/2/98 Focus Group)
Provide immediate feedback for students. “Instant answer (correct/incorrect) way cool!!” (12/2/98 Focus Group)
Allow students to change answers before submitting assessment. (12/2/98 Focus Group)
Inform students of how many times to click to enter an answer. (12/2/98 Focus Group)
Inform students of upcoming assessments. (12/2/98 Focus Group)

In an educational system that defines student success by performance, design and access to assessments is critical. Systems that are built into online quizzes that eliminate technological distractions and solidify knowledge foundation help students to achieve successful performance. Cues supplied by study participants provided direction for curriculum developers to consider while creating content and assessment presentation.

Theme 9: Assessment Feedback

Students stated that online education is not a substitute for interaction with a teacher. However, online and face-to-face settings present vastly different means for effective communication that is a normal part of learning. Within a face-to-face classroom, communication between student and teacher occurs with verbal and non-verbal messages. In an online environment there is no opportunity for students and teachers to see and interpret body language. In some formats such as independent study curriculum, there can be limited exposure to the actual teacher.

Curriculum designers can address the lack of teacher presence in several ways. Messages placed throughout a course containing specific language to increase motivation, self-efficacy, and assessment success can begin to bridge a gap between a student and a distant teacher. Feedback written directly into assessments can improve student learning and self-efficacy regarding success in an online course.

Implications for assessment feedback emerged from three studies. Study participants were high school and university students, all accustomed to assessments. They came to the studies armed with personal knowledge of what constitutes good and bad assessment methods, procedures, and presentations. The students responded with specific recommendations of how assessments can be improved to help students achieve desired results. They provided a range of suggestions for curriculum designers to consider for course development and revision:

Use “smiley” and “frowny” faces, not checks and Xs for grading. (8/00 Focus Group)
Correct answers should be given immediately. (12/2/98 Focus Group)
Encouragement—students like to be told different encouragement messages. (Virtual Teacher)
Direct, “good job” sorts of messages are requested by students who require supportive messages from teachers. (Virtual Teacher)
Include feedback with further information about the topic of the homework or evaluation questions that are correct or incorrect. (Virtual Teacher; 12/2/98 Focus Group)
Students like to know detailed information about what to expect on the tests. Students were very concerned about tests. (Virtual Teacher)
Feedback for correct and incorrect answers should be different. (8/00 Focus Group)
Feedback should not be in red if the answer is correct. (8/00 Focus Group)

Assessment feedback messages provide guidance to students as they strive to succeed. Message content construction must vary according to placement in the course and type of assessment. Encouragement messages and test answer explanations require careful attention by curriculum designers.

Theme 10: Non-content Messages

In addition to messages that clarify instruction, students request messages that are not specifically related to knowledge construction of the content. Such messages address areas of support that would normally come from a teacher or technical support person in a classroom. Teachers support students with “how-to” message to help cope with confusing technology or procedures that are unrelated to content. Non-content comments that were sifted from other survey and interview responses from six studies include.

Students like/need to have access to a “live” teacher. (Virtual Teacher; Start-up Sequence)
Messages from the teacher/teacher presence guide, direct, and encourage students to achieve success in an online course. The appropriate form for the feedback language should be informed by whether the student is intrinsically or extrinsically motivated to succeed in school. (Virtual Teacher)
Separate Mac and PC manuals and browser-specific instructions. Provide less opportunity for confusion for participants who do not always distinguish between platforms. (New Users; Start-up Sequence)
Students require immediate support help with technical issues. In a classroom, when the student’s hand is raised, the teacher responds. Online students request similar immediate attention. Include Customer Care number in frequent and conspicuous places in the text. (Millard South; Start-up Sequence; Orientation)
Students require immediate support to clarify instructions, especially for evaluations and for technical issues, such as how to submit an assignment. (Virtual Teacher; Millard South; Start-up Sequence; 12/2/98 Focus Group)
Students like having an additional audio voice with which to interact. They are not accustomed to working in silence and respond to the additional voice. (Virtual Teacher)
Students like to have sections of the course read to them (audio) directly from the page, especially information that pertains to the tests. (Virtual Teacher)
Technology-inexperienced students demand accurate information that will lead them, step-by-step, through procedures. (Virtual Teacher; 12/2/98 Focus Group; Start-up Sequence)
Students request that learning success strategies be included in the Handbook, Orientation section and strategically located within course content. (Virtual Teacher; Start-up Sequence)
Explain the purpose of the Content Viewer in visible places in the text where students have ready access to the information. (Orientation)
Include information that states how long a download will take, estimating for the slowest possible computer and modem configuration. (Orientation)
Include hints or tips throughout the courses that technology problem sometimes happen and to not be frustrated. Include ideas on what may be happening if there is a technical glitch or options for the student to do while waiting for support. (Orientation)
If students need to go to a specific place (online or handbook) and become familiar with all of the information within, instructions to do so must be clear, visible, and direct. (Orientation; Start-up Sequence)
Clarify instruction messages. For example, change the name of the Welcome section (in Handbook) to something like Start Here so students know where to start. (Start-up Sequence)

Information intended to assist students with technology-related challenges or expectations serves to reduce barriers to learning. In addition, such information must be clearly presented for all students to comprehend. When students are frustrated due to lack of knowledge and lack of information, cognitive load increases, thus reducing the student’s cognitive space to learn. As previously reported, technology issues were the major cause of frustration in online courses.

Theme 11: Instructional Design

Instructional design is distinguished from curriculum design in this research. The focus of curriculum design is on course content and presentation of content to increase learning performance. Instructional design, on the other hand, includes technical functionality of the curricular elements within course design. Study participants in four studies identified specific requirements for instructional design of web-based courses. The studies provided information for designers to consider when designing the technical presentation of curriculum:

Do not require students to go out to the web to download plug-ins and software by themselves. Only students with considerable experience can negotiate that level of technical work. Most students in the online courses are inexperienced with the technological end of computer functionality. (Start-up Sequence; New Users)
Develop a tutorial that visually details how computers function and realistic expectations regarding technology performance. Include animation demonstrating basic functions such as how to copy and paste. (Orientation; New Users; Start-up Sequence)
Terminology needs further clarification: Include section at the beginning on basic “computerese” with screen captures if possible. Include words such as "case sensitive", "desktop", "default drive." (Start -up Sequence)
Release courses following thorough Quality Assurance procedures. (Orientation)
Pre-test computer competence to measure experience and provide advanced insight for teaching staff. Follow the pretest with an optional short visual tutorial demonstrating how Windows or Mac OS work. (Orientation)
Engineer tests for maximum performance. “The tests take a long time to load.” (12/2/98 Focus Group)
Design and clearly organize pages to aid visual comprehension. “Lots of things are too small and very busy.” Avoid lines pointing across page. (12/2/98 Focus Group; New Users; Start-up Sequence)
Create a different, smaller Handbook for each course. (Start-up Sequence)
Add instructions for what to do when the CLASS icon appears on the desktop. (Start-up Sequence)

Theme 12: Colors/Graphics/Audio

Further technical design considerations were gleaned from study findings. Use of color can be used to enhance engagement and draw attention to important information in a web page. Certain uses of color, such as using red text for feedback of correct answers on assessments, delves into culturally defined symbolic meanings of color and can be confusing to students if the colors used are not familiar in the context of the learning. In addition, color combinations of text with backgrounds or backgrounds with link colors can decrease reading efficiency.

Study participants had specific recommendations for designers responsible for the construction of course web pages. Despite students’ comments on the attractive graphical presentation of some course pages, when students are required to read and understand text, best use of color for course page construction must be considered. Instructional designers may consider assigning color combinations that improve reading ease rather than for visual engagement. Comments specifically regarding use of color included:

Don’t color the text the same as the links in the text (8/00 Focus Group)
Assessment feedback should not be in red if the answer is correct (8/00 Focus Group)
When comparing courses, students stated that courses with white page backgrounds needed more color and graphics (8/00 Focus Group)
“All gray kinda dull” (12/2/98 Focus Group)
Emphasize important elements with color; “more color on things like course objectives.” (12/2/98 Focus Group)
Colorful pages at the beginnings of some of the courses attract and engage students. (8/00 Focus Group)
Use bright, high contrast colors. (8/00 Focus Group; 12/2/98 Focus Group)
Same background from page to page is “boring” (8/00 Focus Group)

Color is an integral aspect of graphical elements in web page design. Inappropriate use, including incorrect placement of images in relation to content, can distract or even disrupt learning. In addition, audio can be used to enhance graphics or highlight important concepts. Study participants tended to like graphics and animations but expressed specific recommendations to enhance their learning experiences in the test courses:

Students like moving images such as movies and animations. (8/00 Focus Group; 12/2/98 Focus Group)
Depending upon the content, pictures or cartoons may be preferred. Represent real people and places with real pictures. (8/00 Focus Group; 12/2/98 Focus Group)
Avoid sexist animation and graphics. (12/2/98 Focus Group)
Discriminate where to place and include animation. Animation can be distracting. (12/2/98 Focus Group)
Place graphics near related text, or instruct student to scroll to see the graphic. (Start-up Sequence)
Include appropriate graphics and audio as much as possible; “Love the graphics; Love the videos; Very rad video—lovin’ it”; “I like the sound, there should be more of it.” When appropriate, have graphics exactly match descriptions. (12/2/98 Focus Group; New Users; Start-up Sequence)
Some students prefer audio clips that read text so they can “listen and follow along.” (12/2/98 Focus Group)

Students in the four studies reported specific recommendations regarding use of color, graphics and audio elements in instructional design of course pages. Combinations of graphics and sound, use of “real” images versus drawn or cartoon images, use of video and animations, and placement of graphical images within text were considerations for instructional designers in development and revision of web based courses.

Theme 13: Text

Size, shape and color of text presented to readers of online course pages can determine readability or accessibility of the written word. Students who participated in the two focus groups offered opinions on text presentation:

Use a larger size, easier to read. (8/00 Focus Group)
Varied colors catch the eye. (8/00 Focus Group)
Good bold face (reference to use of bold text in selected places in the pages viewed). (12/2/98 Focus Group)

While comments specifically regarding text were few, study participants recommended that instructional designers attend to size to enhance readability, and use of color and bold text for emphasis.

Theme 14: Navigation

Navigation may be the single most important instructional design consideration that influences student comfort, understanding, or frustration with a web based course. Seamless navigation allows the student to glide through the technology while complicated, unclear directions create high levels of frustration that can deter learning. Student participants from five studies responded with strong statements of their expectations that course navigation be easy and clear. Specific features were recommended, some as requirements for convenient, accessible navigation.

Students require and appreciate clear navigation. (Millard South; 12/2/98 Focus Group; Start-up Sequence)
Students recommend sequential order of lesson information, whether course content is presented in linear or nonlinear design. Note that sequential order does not require linear order throughout the course. Students can miss content with confusing navigation and prefer to not suffer this disadvantage. (Orientation; 12/2/98 Focus Group)
Provide links to Orientation, themes and lessons to help students find their way within courses. Include back buttons on the pages. (Orientation; 12/2/98 Focus Group)
Include Back and Forward buttons on each page. (8/00 Focus Group)
Include page numbers on each page to locate within a lesson (i.e. Back 1 2 3 4 Forward). (8/00 Focus Group)
Build in visible and easy access to Course Guide information where it would be helpful for students. (Orientation)
Avoid hidden links to resources. (Orientation)
Include more instructions such as where to go when done or not to push the back button in the middle of a quiz. (12/2/98 Focus Group)
Some students want more navigation structure, less personal decision-making. (12/2/98 Focus Group; Start-up Sequence)

Access to resources, visible access to pages within lessons and units, and clear order all serve as locators within a course. Clear indicators of test locations are imperative as missing a test due to poor navigation undermines student success. While students do not consistently utilize all course resources, presence and easy access eliminates these barriers to academic success. Study participants have provided instructional designers with valuable information.

[ home ] [ contact ]