ON-LINE WEB BASED TEST
Satoru Shinagawa, Kapiolani Community College <satoru@leahi.kcc.hawaii.edu>
ABSTRACT
In this paper, I am introducing my on-line web-based Japanese language vocabulary practice and testing program along with a brief discussion of feedback from students and other instructors. This computer program is made for a web browser, such as Netscape. There are two advantages to this program. One is that this program can be used with any type of computer, whether your computer is Macintosh or IBM. The other advantage is that students can use this program to study in the lab, at home or anywhere, as long as their computer is connected to the Internet. This program has two modes, a practice mode and a testing mode. A student can use the practice mode to practice vocabulary. The testing mode is designed so that it can be used for actual vocabulary testing. The computer, in a few short seconds, will automatically grade the test, and calculate the score--saving time for the instructor. Also, I will present comments from instructors and students who have used this program. Input and discussion on improvements would be welcome.
INTRODUCTION:
With the internet growing very rapidly, many colleges and universities are starting to offer on-line courses. I too have been developing an on-line web based Japanese vocabulary practice program. This paper will discuss how this program is designed, the advantages and disadvantages of this program, and some feedback from students and instructors.
WEB BASED COMPUTER PROGRAM:
The program has been co-developed with Professor Hiroshi Yokoyama and myself. Professor Yokoyama is a faculty member of the Department of Information Engineering at Akita University, Akita, Japan. He has been working as the main programmer of this project.
Before the era of Internet, many language learning programs, not only for Japanese, but also for German, French, Russian, etc. have been developed. However, almost all of the programs developed so far have an undeniable problem which is a source of headaches for instructors. If the programs were developed for Macintosh computers, usually they won't work on IBM computers, and if the programs were for IBM, they won't work on Macintosh. Some learning software has indeed been released both in Macintosh and IBM format, but such programs make up only a handful of all language learning software.
Another problem is copyright. Since almost all of the programs are copyrighted, when you purchase a language learning program, the number of computers on which the software can be used must match the number of purchased copies of the software. For example, if you purchased 1 program, you can install the software on one computer. If you needed to install the same software on another computer, you must erase the program from the first computer before installing it on another computer in order to comply with copyright laws. If one purchased one copy of the program and then had it installed on 10 computers, this would be piracy and a violation of copyright law. The penalties for breaking these laws are severe. One, in this case, must buy ten copies of the same software if one wants to use it on ten computers. Thus, as you can see, unless one is planning on using a software program on only one computer, one is left with either breaking the law or spending large sums of money on the purchase of multiple copies of the same program.
Copyrights should be and have to be honored so as to protect the intellectual rights of software developers and the software companies. However, this causes a tremendous inconvenience in the classroom where, usually, having only one copy of a software program is quite inconvenient. It is true that, when an instructor wants to simply show the software to a group of students, it may not be much of an inconvenience to have only one copy as long as you can use a large projector connected to the computer. But in reality, not many colleges and universities have the luxury of such equipment, meaning the software program must be demonstrated on a normal size computer monitor. As one might expect, in this case it is extremely hard to show a computer program to a whole class if the software is installed on just one computer
It gets worse. If an instructor wants students to actually use the software, having multiple copies is a must. Without multiple copies, students would have to take turns at the computer on which the software is installed. Thus, it would take the whole class period to finish even a small amount of computer work, if there is only one available computer on which is located the software you would like your students to use.
If a student likes and wants to use the program to study, the student is not allowed to copy the program. Therefore, he has to use it in the computer lab of the university. This means that he cannot take the program back home and use it in his own computer. If I were a student, I would feel more relaxed studying at home and would study more there than at a university computer lab.
One of my major goals in developing this web-based program was to develop a program aimed at eliminating all of the problems described above. The solution I came up with was to develop a web-based program. The web-based program eliminates all of the problems I described above. A web-based program can be used on any type of computer as long as a web browser (such as Netscape) is installed. Even though we (Prof. Yokoyama and I) have decided to own the copyright of the program, the practice mode of this program is not password protected so that students and instructors who could potentially benefit from this program can access it freely. Also a web-based program can be used by every single computer in a computer lab at the same time, which makes it feasible for all students in class to use and experience the program at the same time in the class period.
TWO MODES:
This program consists of two modes.
One is a practice mode and the other is a testing mode. The practice mode is used any time except for when the testing mode is open. We have designed the program so that when the test mode is open, the access to the practice mode is not available. This is because it is possible, even though it requires a high level of computer programming knowledge and, even then, is rather difficult, for a user to have access to the practice mode during the exam to cheat.
PRACTICE MODE:
When you go to the practice mode, you will type in the number of words you would like to practice from each lesson.
For example, if you would like to practice the 30 vocabulary on lesson 5, simply type in 30 in the lesson 5 spot.
Or you can even combine multiple lessons like 20 words from lesson 5 and 20 words from lesson 6.
Once you have chosen the number of words, you click on the "generate" button.
After the vocabulary comes out, you will enter in your answers, and when you are done with your answers, click on the "Check answers" button. The server will check your answers and show the results in a matter of few seconds. The correct answers are indicated by an O, and incorrect answers are indicated by an X--the standard Japanese style of showing correct and incorrect answers. At the bottom of this page, you will see a "try incorrect answers again" button. By clicking on this button, you can practice your incorrect answers again if you wish to do so.
TESTING MODE:
What you see in the testing mode is almost identical except for some changes necessary for tests. Such changes are that you must be a registered user to use a testing mode.
In other words, unless you are registered for the course, you cannot take the exam. Another difference is that after you are done typing in your answers, you don't see the "check answers" button. You see instead a "turn in test" button.
Click here to see the above bigger
This difference should be self-explanatory since this is a testing mode. After you turn in your test, you will receive a prompt confirming that your test was received successfully.
After making sure your test was received successfully, you can check your score by answering "Yes" to "Do you want to know your score?" question. If you answer yes, you will see your score.
If you answer "yes" to "Do you want to see your answers?" you can see the graded test.
WIZARD'S PAGE:
The wizard's page is a set of pages for controlling the whole program.
Here I will simply list the names and functions of the wizard's pages. In the wizard's page, you can open and close the test (Folder access control). You can see the results of the tests (Results). You can make new examinations (Make examination forms). You can delete examinations (Delete registered forms). You can add users (User and Group administration). And you can edit the dictionaries (Dictionaries). The wizard's page is password protected. You have to be registered as an "administrator" to be able to have access to this page. Therefore, unless you are a registered administrator, you cannot have access to this page. If you are registered as a student, you don't have permission to access this page.
I will not talk about the details of the wizard page. However, if any of you are interested in exploring the wizard's pages, please e-mail me. I would be happy to register you as an administrator so that you can surf the wizard's pages.
SECURITY MEASURES FOR THE TESTING MODE:
One advantage of on-line testing is that keeping track of log-in time and access nodes is easy, and this is used as one of the security measures to know if a student accesses properly or not. An administrator can, not only see the answers, but also see the time when the answers were received by the server and from what remote host address the answers were sent. Initially, we didn't think this type of information was necessary but, after one bad incident occurred, we realized it actually is one security measure you can use.
When the students take the test, they take it under my supervision. Therefore, I know who exactly is taking the exam, and how long it took them to finish the exam. However, some circumstances force me to give a test away from my supervision. This is because the computers in the computer lab breaks down from time to time and consequently, there are not enough computers available for all of the students to use. Fortunately, there is another computer lab (walk-in lab) on campus. I ask for volunteers to go to the other computer lab, and give them the test at a time other than the designated exam time. Once when I did this, I was checking the result of the tests and I noticed something very unusual.
If you take a look at the above "remote host" information, you can tell that the remote host name of the third row is distinctively different from other remote host names. The remote host names "comXX.kcc.hawaii.edu" are the remote host names assigned for the walk-in computer lab, where the students were supposed to take the exam. The "hula.net" is a host name by a local internet service provider in Hawaii. I checked with HulaNet and confirmed with them that the remote host name "elua24.hula.net" is indeed a dial-up remote host name. Therefore, it's clear that, in this instance, the student didn't go to the computer lab to take the exam. Even though there's no proof, it's probably the case that the student took the exam at home, and turned in the answers not knowing that I can tell whether a student really went to the lab just by checking the remote host name. When I asked the student, she told me that it wasn't she who took the test, but it was her brother who logged into the exam, punched in the password, and went through the whole process of taking the exam and sending the exam in at the designated time, and scored 100%. I could have told her that she cannot take the exam again because she didn't give me any excuses for not taking the exam (after all, it was her brother who took the exam, according to her story). But I gave her another chance to take the exam under my supervision.
STUDENTS' FEEDBACK:
In general, the feedback from the students after using this program is positive. Almost all students say that they like the program when they practice because they can get the feedback immediately and can practice 24 hours a day. There is no any negative feedback as long as the students are using this program to practice the vocabulary. But, interestingly enough, their feedback changes when it comes to taking the exam. About 95% of the students say that taking a test on computer is fine and no problem. But about 5 % of students come to see me with complaints after they see their grades.
However, their reasoning for disapproval and not liking the on-line test makes you wonder. Almost all of the complaints are related to the students lack of knowledge of the correct Japanese writing system and/or indications that they hadn't practiced with the program enough before taking the test. It is also interesting to note that those complaints are coming from those students who score 60-70% (or lower) on the test. It seems to me that, for some of these students at least, raising complaints with the computerized test may be one means of trying to get me to give them a second chance where they may be able to raise their score.
Instead of brushing aside these complaints, however, let's look at some commone ones. The most common type of complaint is "The computer cheats." What students are claiming here is that they typed a word in correctly, but the computer registered their typing incorrectly. Since I know enough about computers to realize this is highly improbable, I don't worry too much about this type of complaint.
Another common complaint is that typing an answer is different from writing answers on paper. I wasn't certain where the problem was in this case and so, in order to clarify what they meant, I talked to my students about this. It turns out that their complaint is related to their lack of the knowledge of the Japanese writing system. When you indicate a long vowel in Japanese, you have to be careful with some vowels because they are irregular. I, here, would like to avoid getting into a discussion of how long vowels should be written in Japanese because it's not relevant to our main discussion. But in short, a student has to type in "ou" instead of "oo" to indicate the long vowel. Even though this is true for writing out the Japanese words in pen or pencil as well, some students do type in "oo" instead of "ou," and their answers are graded as incorrect. The assumption here is that, if they are writing with their own pens and pencils, students would avoid making such errors. While this might be true for an occasional student, my experience is that I see the same incorrect answer from students when they write out their answers. It is also interesting to note that I get this claim mainly from Japanese 101 students who have just started to learn Japanese, and never from upper level (for example, Japanese 201 and above) students. This supports my claim that this second complaint is a reflection of unfamiliarity with the writing system and, as student progress in Japanese, this problem eventually disappears.
I realize that it may be true that for some it is easier to write out answers rather than typing out answers. However, in the end, this problem is not related to the functioning of the program itself. It is related to the human element involved in using the program. In fact, it is also true that for some it is easier to type out answers rather than writing them out.
INSTRUCTORS' FEEDBACK
I have asked my fellow instructors to use this program. I have never received any negative comments from any of my colleagues. The common comment is that this program is useful and valuable to students. The reasons they give are the following: The entire class can use the program at the same time. The students can practice either at school or at home (or anywhere) 24 hours a day. The program gives the feedback immediately. These reasons given by my fellow instructors are exactly the same as the goals for the development of this program. I believe the fact the instructors' reasoning of liking the program and my goals for the development of the program match shows that the demand for such a program is high and that most instructors are not entirely happy with the current software they use.
COMPARISON BETWEEN WRITTEN TEST AND ON-LINE TEST
Once, when the on-line test was scheduled, there was lightning a little before the exam. This caused an interruption of the power supply to the computer center where the server was located. The lab where the students were scheduled to take the on-line exam didn't suffer the interruption of the power supply. However, I was not able to give an on-line test because the server was down. The students had studied for the test, so I was not able to postpone the test. I used an old fashioned written test to find my way out of this emergency. Most students were happy with my decision of using the regular written test for this emergency situation, but some students were not happy with taking the test on paper because they believed they could score better on an on-line test. I decided to give an on-line test to those 8 students on the next day. This turned out to be a nice experiment where I could compare the score difference between the regular written test and the on-line test. For each test, the student needed to write the meanings of 100 English words in Japanese. The set of 100 words chosen was different each time. The following table is the results of both of these tests taken by these students.
|
On-line Test |
Written Test |
A |
95% |
96% |
B |
92% |
89% |
C |
91% |
92% |
D |
88% |
85% |
E |
85% |
89% |
F |
82% |
78% |
G |
76% |
80% |
H |
72% |
69% |
There is not a significant difference between the results of the 2 tests. In general, those who got an A on the on-line test got an A on the written test, and you can see the the same trend for those who got B's and C's. The percentage difference between the two tests falls somewhere between one and three percentage points. This is not very large at all. And fifty percent of the students did better on the on-line test and fifty percent did better on the written test. Overall, I believe that it is safe to conclude from this that there is no considerable difference between the result of the on-line test and the written test.
APPLICATIONS TO OTHER SUBJECTS:
This program works for any other language and subject as long as the questions you are asking require one straight forward answer. For example, if you go to this page and scroll all the way down, you will see the following. This program can be used for any language, such as French, Korean, and Spanish as is shown here.
This program can be used not just for languages, but for other subjects like Philosophy. Below you will find some sample Philosophy questions, which were made by Prof. Robin Fujikawa.
CONCLUSION:
There are many useful commercial computer assisted programs available. Due to copyright restrictions, an academic institution needs to purchase such software in bulk, should they wish to make that program widely available to their students. However, these days most institutions don't have enough funding available for such mega purchases.
I have tried to come up with an alternative software to solve this problem by creating my web-based on-line program. The feedback from the students and the instructors are good. As I mentioned in the instructors' feedback section
TCC Online Conferences
Kapi`olani Community College
University of Hawai`i
Honolulu, HI