Abstract. The details of the inception, development, and implementation for an online mathematics course, Engineering Calculus I, are presented. Several of the major challenges in the design and layout of the content are given special attention and their contrast with textbook structure noted. With numerous surveys given throughout the course, we were able to continuously monitor student reaction to the venue and comprehension of materials. In pedagogy, several observations were made about how students learn from Web-based materials. These have formed a key component during the continuing development stage of this project.

The WebCalC Project, whose team members are Michael Stecher, Philip B. Yasskin, and this author, is a complete "course-in-a-box." The course is Calculus I, a typical engineering calculus course, equivalent to an AP calculus (but without a graphing calculator component). The inception of this online course was in the spring of 1997. Several months of discussion ensued, particularly over the two most essential points.

1. What should be the method/software for delivery?
2. What does an online course need to be?

Dozens of other issues, from the file management system to the creation of graphics, were ignored at first, though each eventually required careful attention. The second question has a simple solution summarized as follows:

1. Perfect mathematical typography — math should look like math
2. Great color and graphics
3. Interactive quizzes and exams
4. Internet links; fast downloads
5. Symbolic mathematics capabilities
6. Complete solutions to examples and exercises
7. Question-answer notes
8. Animation and Java
9. Sound and Video
10. Years of testing

The logic for most of these requirements has been discussed in some detail in [Allen, et al, 1998a and 1998b]. So, we make here only a few supplementary points. First of all, students are accustomed to the highest quality software, whether from the viewpoint of a word processor or spreadsheet program, or from their vast experience with computer games. Therefore, the key point of excellent mathematical typography is in the interest of student expectations as well as student comprehension, particularly the beginning student enrolled in calculus. While mathematics instructors have little difficulty reading mathematics in almost any form, the beginner does have trouble. For these students, the presentation must look as perfectly textbook-like as possible. This decision clearly limited available choices. The most direct way to achieve good mathematical typography is through GIF images created by programs such as LATEX2HTML or, say, through HTML export of a MSWord document. Both procedures generate mathematics of good appearance, but neither prints well. In addition, resulting file sizes tend to be large. This means files download slowly, and fast download times are essential. Student interest and attention fade if lengthy downloads persist. (Roughly, a consistent 20 seconds download time per page is near the maximum time students will accept.) A couple of other methods are available, but they each suffer one or more defects. In a recent paper, Robson (Robson, 1999) discusses objected oriented course design for Web-based authoring to alleviate some of the extraordinary complexity for developing such a project.

The last three requirements (items 8 through 10) were considered only after the project was underway. Indeed, we have added a few animations (animated GIF images) and Java applets. Such devices have not yet become essential course features, although students do like them. Sound and video were not feasible when we began, but thanks to new, highly compressed streaming software, they now are. This fall we will begin to add narrated slide shows to the course in select places [Levine, 1999]. The last point about testing was unrecognized only after testing began, and we began to understand how complex this aspect of the project was. Briefly, it is important to understand that learning how to use an online course is a separate skill altogether. Hypothetically, even the perfect online course could fail its field tests if the instructor is not given directions on how to administer it. Further details of this fact, vis a vis WebCalC, are described later.

Our software decision was to use Scientific Notebook, a TeX typesetting engine, with a built in Internet browser and a computer algebra system (Maple). TeX is the de facto standard for mathematical typesetting today. It has been used for more than 20 years and produces the best-appearing mathematics of any competing method. This generally makes for very small files, normally five to 10 KB excluding graphics. The browser part of the program builds the page from the TeX codes when it arrives at the client’s computer. With the Maple kernel, which is also menu driven, most students can answer many of the "what if" type questions that typically come up or produce great graphics with hardly more than the click of a button. Finally, it is exceptionally easy to typeset mathematics using Scientific Notebook, as that feature is also menu driven. Often portions of the material must be rewritten as the developer notes how students respond to it. Remember that without the lecture, there is little opportunity to clarify what has been written.

Though graphics in abundance have emerged in mathematical textbooks only in the last half of the twentieth century, they are today regarded as essential. In that connection students, having experience with the most cutting-edge programs today, are again the experts. The graphics must be excellent. We construct ours using a variety of methods, the most dominant being to use Corel Draw to generate Windows metafiles ( i.e., vector graphics). These Windows metafiles (WMF) are superior to GIF or JPEG images in almost every way except possibly file size. Most of our graphics are 30 kilobytes or less. They are in full color as needed and are in quality at the level of modern textbooks. As line speeds increase over the next few years, we look forward to generating graphics that are ever more sophisticated and interesting.

Basically, our course does meet the requirements outlined above in items 1 through 10. Importantly, the lean and lively text pages ([Douglas, 1986]) are written to cover all the essential points needed for the topic at hand. Neither lengthy introductions nor long summaries are included. It is curious that, in recent reviews of WebCalC one reviewer noted this as a negative feature but then further noted that the preponderance of students don’t read introductions and long summaries anyway. We’ve noted that, just as in the traditional course, students tend to begin with the problems and read the text only as needed. Without the lecture wherein students see examples worked and theory explained, students of online courses have less instruction than their traditional counterparts. Therefore, proceeding directly to the problems is a riskier strategy. On this basis, one fundamental observation we have noted is that learning from an online course is task oriented. Simply assigning chapters to be read and exercises to be worked is too abstract or vague to be effective. It is important to give precise tasks to the student on a daily basis. This can be accomplished through quizzes and worksheets. To drive this point home, every instructor, following the traditional method of teaching, is substantially the metronome for their course. The lecture sets the tasks, and the students respond by working the problems or reading specific material. This facet of traditional teaching needs an analogue in the online course. While there may be better methods yet undiscovered, for now the quizzes and worksheets seem to be working.

The online course requires a number of other features, and WebCalC has them.

1. Randomized quizzes
2. Homework assignments with complete answers
3. A consistent, attractive look and feel that is artistic.
4. Notes (pages within pages)
5. Symbolic mathematics functionality

Many of these requirements can be lumped under the rubric of interactivity. It is important for the course to communicate with the student by asking questions and providing answers, though not at the same time. This is accomplished in a variety of techniques. One is the randomized quiz. These quizzes are generated on the fly and provide different questions (over the same topic) each time the button is selected. For example, think of a problem involving solving a linear equation where each time the button is chosen a different linear equation appears. When answers to the question or questions have been selected, the student receives immediate feedback. The course also contains blue ball quizzes, so named because of the blue balls next to the multiple-choice answers. These quizzes are carefully crafted with feedback provided with each wrong answer and with explanation given for the correct answer. (Some students just choose buttons until the correct answer is found.)

The online course should have a nonlinear structure. Thus, the student can be maneuvered about as the links and hyperlinks determine. One particularly nonlinear useful feature is the pop-up note. For example, when explaining the solution to a problem it may be necessary to invoke a previous concept, say the Pythagorean theorem. At this point, a button is inserted and the student, by choosing it, automatically sees the needed information. This is just one use of the pop-up note. They can be used for making definitions, for reinforcing a concept, and for supplying a relevant graphic. They can also be used to enhance text information by supplying additional facts that would clutter the main page. Such notes are downloaded with the page and appear only if selected. Our most frequent use of the pop-up note has been to supply answers to exercises. Next to each exercise problem there is an answer button, which the student chooses to view the answer. Overall, such notes give a definite appearance of interactivity. Links and hyperlinks are used for larger notes and for additional information of a nature inappropriate for a small pop-up box, and of course, the next page.

WebCalC has a few animated GIF images that students like, as well a few fully interactive Java applets. One applet plots a user-supplied function and then displays a secant line evolving into the tangent line. As interesting as this is, students don’t seem to use it. We believe that some specific task must be associated with the applet for it to achieve a more prominent place within the course. By the way, some Java applets tend to download and launch slowly. A delay of a minute or more is not uncommon.

In an ideal world, pre-selecting students is the rule. Hence the question: If our world was ideal, who should and who should not take an online mathematics course? Our first choice would be to select students that are strongly motivated (self starters), intellectually mature, home-schooled, or the handicapped. Our last choices would be students that have low mathematics pretest scores, are generally unmotivated, or need the structured classroom environment. At this point of online course development, the temptation to view these courses or require them to be suitable for all students is at best misguided, at worst a recipe for disaster. At minimum, a key ingredient to the success of the online course is a clientele that have very good reading skills. A very few (about three out of 150) of our WebCalC students decided that online calculus was not for them. Every effort was made to transfer them to a traditional section.

The mode of teaching, termed the facilitator mode, has several essential factors in common with the traditional mode. In the facilitator mode, the instructor is in the classroom with the students during a regularly scheduled period. The students, however, sit at a computer terminal and learn from the online course. The teacher/facilitator

(a) answers questions in situ, (b) assigns/collects/grades homework, (c) establishes the pace, (d) creates/grades exams and quizzes, and (e) holds office hours. In brief, the facilitator performs all the traditional functions except to deliver a lecture. The time traditionally spent lecturing is spent helping individual students with individual problems, usually to explain a worked example or to explain how to answer homework questions. The facilitator is generally occupied with student questions throughout the period.

What we have noticed is that students group in pairs or sometimes trios to study/learn together. Called spontaneous group learning, this remarkable phenomenon differs from other group learning since it is not contrived (Gantt, 1998, Jensen & Sandlin, 1991). Other students work strictly alone. Many of them never ask even one question. Most groups formed within a few weeks; other individuals migrated from one group to another. While we have no precise data about the results, our impression is that well over half the class worked in some loosely or tightly formed group, and that the grades of these students were higher on average as compared with the "loners." That the more socialized students do better seems to be true, but there is a law of diminishing returns: Too much of this good thing seems to decrease grades.

We have yet to discuss the true distance-education mode, where there is no regular meeting time or place and where most communication is not face-to-face. In one sense, that is fortunate. Our experience with online courses has suggested that going to the full or true distance mode may have proven difficult had it been attempted earlier. The single most difficult question remaining is how to give proper and timely mathematical help to students as it is needed. Other documented problems include student isolation and frustration. The first, true distance-education version of WebCalC was offered during the Fall 1999 term. We are mindful that students completing Calculus I must then proceed to a traditionally formatted Calculus II. Therefore, students must be provided first quality education from WebCalC.

Some believe that because of technology, the very nature of the course, the syllabus itself, should change. This alters the burden of online course development. However, before massive changes in the course syllabus are implemented in synchrony with this philosophy, online courses should first satisfy the "proof of concept" of any new design with respect to the traditional syllabus.

The measurement of course efficacy for the traditional course has been long abandoned. Typically, students are surveyed about the instructor with a sampling of questions about the course. Traditional pedagogy is old, time-tested, and well understood. On the other hand, online courses seem to need to prove themselves to a higher standard. College and high school administrators are anxious to join the new age of online courseware but are nervous about the quality of the product. Perhaps trust for their teaching colleagues is not what it could be, and perhaps they fully comprehend the difficulties presented by this new medium; but surely they desire that online courses meet a level of quality consistent with their institution.

The developers of WebCalC are conscious as well about the quality of their new product. They typically test the quality using a panoply of surveying instruments. The most basic measures of course efficacy are surveys that measure students’ impressions and studies that measure students’ grades in subsequent courses. The surveys taken for this study do not follow precise standards of statistical sampling (Hall, Pilant and Strader, 1999) but are rather preference surveys given and collected during a class period, four times per semester. The randomness of the sample was addressed earlier.

Since this the course has been taught just five times, data have not convincingly established what we hope is true, namely, that online mathematics taught in the facilitator mode is competitive or better than the traditional mode. However, if results are just comparable after only five semesters of use, this marks a positive indicator of the potential of Web technology as the medium of the future.

A number of questions were posed to students, many of them routine in nature, many key questions of great importance to the project. Responses were on a scale from "Strongly Disagree" to Strongly Agree." The more important questions and responses are displayed in Tables 1 through 8 below. Less significant questions to the reader are those involving material comprehension. Example: I understood the material on inverse trigonometric functions well. Other questions were included to measure students’ "comfort level." Example: I am very worried about the upcoming third exam. Finally, some of the same questions were posed in alternate ways. Example: The level of the material in the course was too advanced. Or: I found that chapters adequately explained the material. These questions are similar to the question in Table 1.

The mild "disagree" response indicates the normal confusion and difficulty that many students have with a beginning college mathematics course. Overall, the surveys indicate that the level and pace of the material are acceptable, at least to the students. In Table 2 below, we inquired about the perceived challenge learning over the Web.

Should these responses be considered surprising? Not at all, because the student is doing most of the work in the class. No longer is the atmosphere passive, where the disinterested student can simply let a lecture flow around but not through his/her awareness. Students come to the lab to work. In Table 3 below, we investigate student study location habits. Do they work at home or in class?

The responses are generally neutral overall being well balanced. The indicator here points to possible results for the distance mode, which was attempted in the Fall 1999. It also demonstrates that many students feel that they accomplish much in the class period and can supplement their learning in the open computing laboratories throughout campus. The results presented in Table 4 below reveal that the power of the software package is underutilized.

The responses here were a little puzzling. Scientific Notebook which functions as the browser has a powerful menu-driven Maple engine. It is very, very simple to learn and to use. Why didn’t students use it? Probably, because it wasn’t emphasized in the classroom. Several students used their graphing calculator to help them resolve questions. A few of the academically stronger students actually taught themselves how to use the Maple engine. However, most students did not venture far from the path of assigned work. Note that the use of a CAS (computer algebra system) in the classroom is not well understood, even after a decade of attempts. (Allen, G. D., Herod, J., Holmes, M., Ervin, V., Lopez, R. L., Marlin, J., Meade, D., & Sanchez, D., 1999) In Table 5 we check the comfort level of students with respect to live help?

The responses here were gratifying, but more importantly illustrate the effectiveness of the facilitator mode. Note that students did not often come to my office for help. These responses are generated from in-class help, as it was requested. The strong "agree" and better responses indicate that, even the loner-type students seemed to feel that adequate help was available, although they may never have requested it. In the following Table, we verified what we observed, that many students believe that they are actively working with others.

The responses to this question would have been the most surprising of all, had it not been observed directly. From this table it is clear that, for a strong majority of students, the learning process was collaborative. Called spontaneous group learning, this learning mode was attractive to many students and is radically different from the traditional lecture mode. Oftentimes questions would be asked by a small group of students. Additionally, we also noticed the stronger students would find themselves helping weaker ones, though I am not convinced of the efficacy of this. The principal reason is that the stronger student, though very capable, may not be in tune with the real problems of the weaker student. He/she may give help that results in further confusion. The most tangible example of this, which may be peculiar the mathematics instruction, is to notice the better student showing the weaker one a special way of solving a problem that is apparently quite different from the methods being taught. Indeed, the facilitator mode of teaching may work best when the facilitator is a highly experienced teacher well grounded in all the subject’s methods and techniques. This point is to emphasize that the teaching component has by no means diminished; rather, it has been enhanced.

In fact, it seems that learning in this mode is partly a social process. What is even more fascinating is that the students in the spontaneous groups tended to perform better on examinations, though these were administered in the traditional way, with no student interaction permitted. However, this has not yet been measured from grade results data. Finally, we note that these spontaneous groups were somewhat self-selecting. Most groups consisted of students relatively similar in ability. With this group learning process in place, the lab is naturally less than a quiet and contemplative environment. Do the students perceive it that way?

Even though the lab atmosphere, with all the conversation would be distracting in a traditional sense, most students did not regard it so. We could have insisted on strict independence of work and silence, leaving in the lab only the hum of CPU cooling fans, but we would have lost noticing the remarkable feature of social learning in a task-oriented environment. The last question, would the student take another online course, was important feedback, from which several interpretations are possible.

WebCalC is a successful online project, principally because it works. By this we mean that students can learn Calculus I online, perform successfully on traditional examinations, and perform well in successive calculus courses. Moreover,

1. Students adapt to online learning without difficulty.
2. Students learn to "get to work" right away, making efficient use of their time.
3. Students like the task-oriented environment.
4. Students can work together with positive results.

Student solicitousness and faculty bulletproofing are particularly important until the preponderance of instructors have experience with online courseware. There are numerous pedagogical techniques that must be observed to allow the online course to work. At this juncture in online education, many instructors, if given an online course teaching assignment, will treat it as some modification of what they already know. Administering the course on the basis of traditional face-to-face teaching may lead to unsatisfactory results.

However, it will be important to measure the "value-added" component, as enhancements come at an increasingly high cost. High production costs, high development skills, and high production time schedules are each deterrents to further online course development. For example, a professional quality Java applet may require several months of programming effort. Commercial costs can run into hundreds of thousands of dollars. Then, the applet must be tested for efficacy. In contrast, textbook technology is well tested and well understood, and accurate methods exist for estimating production costs.

It is significant that no doubt many false starts and many revisions to online pedagogical designs will be needed to perfect this mode of education. If costs are excessive, the funding may not be forthcoming, certainly from public institutions.

Allen, G. D., Stecher, M. S. & Yasskin, P. (1998a, November/December). The Web-based mathematics course, a survey of the required features for an on-line math course and experiences in teaching one, Syllabus Magazine, Nov/Dec, 1998 .

Allen, G. D., Stecher, M. S., & Yasskin, P. (1998b). WebCalC I, a description of the WebCalC project, its history and features, Proceedings of the Eleventh ICTCM Conference, New Orleans, Addison-Wesley-Longman, Reading, MA.

Allen, G. D., Herod, J., Holmes, M., Ervin, V., Lopez, R. L., Marlin, J., Meade, D., & Sanchez, D., (1999). Strategies and guidelines for using a computer algebra system in the classroom, International Journal of Engineering Education, 15 (6), 1999, pp 112-134.

Bogley, W. A., Dorbolo, J., Robson, R. O. & Sechrest, J. A., (1996), New pedagogies and tools for Web-based calculus, Proceedings of the AACE WebNet96 conference, 1996. [Online]. Available: http://iq.orst.edu/papers/WebNet96

Bogley, W. A., Dorbolo, J., Robert O. Robson, R. O., and John A. S., (1998). Pedagogic Innovation in Web-Based Instruction, Proceeding of the Ninth ICTCM Conference, Addison-Wesley, Reading, MA, pp. 421-425.

Available online: http://iq.orst.edu/papers/ICTCM96.html

Hall, R. J., Pilant, M. S., & R. A. Strader, R. A. (1999), The impact of Web-based instruction on performance in an applied statistics course. In Proceedings of the International Conference on Mathematics/Science Education and Technology (M/SET 99).

Levine, Lawrence E. (1999), Mathematical slide shows, The Math/Science-Online Newsletter, Summer 1999.

Available:http://www.math.tamu.edu/~webcalc/newsletter_99b.html

Lynch, P. J. & Horton, S., (1999). Web Style Guide. Yale University Press. New Haven, Ct

Robson, R., (1999). Object-oriented Instructional Design and Web-based Authoring, submitted to the Journal of Interactive Learning Research.