OPPRESSION IN TESTING: AN EXAMINATION OF HOW COMPUTER ADAPTIVE TESTING ALIENATES AND OPPRESSES
Amber Sunsted, Montana State University-Billings
Daniel D. Gross, Montana State University-Billings <ca_gross@vicuna.emcmt.edu>
The recent decision by the Educational Testing Service (ETS) to eliminate the paper-and-pencil version of the Graduate Record Examinations (GRE) sheds light on an oppression and alienation exhibited by that group. A Computer Adaptive Test (CAT) has already begun to replace the traditional test, reducing the chances that certain members of society will be able to enter the professional world. Some societal members might not adapt to the technology needed to complete this exam, therefore thwarting their chances of entering graduate school and some areas of the professional world. Critical theories back this claim, giving evidence of how the ETS is taking these measures in order to save an elite consumer.
BACKGROUND ON THE GRE AND CAT
The Graduate Record Examinations Board, an organization affiliated
with the Association of Graduate Schools and the Council of Graduate
Schools in the United States, sponsors the Graduate Record
Examinations. The GRE is considered a psychometric instrument rather
than an examination. This measuring tool is prepared and administered
by Educational Testing Service, a private company in New Jersey which
writes and administers tests. According to ETS, the GRE measures
certain developed verbal, quantitative, and analytical abilities
important for academic achievement (GRE Power, 1996). Students who
have obtained a bachelor's degree must take the GRE in order to be
admitted into a graduate program.
The GRE program offers 17 tests -- a General Test measuring the abilities mentioned above, and 16 subject test in fields such as biology, chemistry, and engineering. Traditionally, students have taken the GRE in a paper-and-pencil version, but a new development now offers the Computer Adaptive Test (GRE Power, 1996). The CAT is structured differently than the paper-and-pencil version, but, according to Kaplan -- the nation's premier educational company specializing in test preparation, admissions, and career services -- the question types are exactly the same (Kaplan's Introduction, 1997). The test is called adaptive because it literally adapts to the performance of the test-taker while he or she is taking it.
Rather than having a preset mixture of easy, medium, and hard questions like the paper-and-pencil version, the CAT selects questions based on how well the test-taker is doing. When he or she begins a section, the computer assumes the student has an average score, and will present a question medium in difficulty. Each time the participant answers a question correctly, the computer adjusts the question to ones of higher difficulty. If the test-taker answers incorrectly, the computer will give easier questions until the participant begins to answer them correctly, at which time it will present harder questions (Kaplan's CAT, 1).
CLAIMED ADVANTAGES OF THE CAT
Kaplan claims the CAT offers more convenience to taking the GRE.
Listed below are some of the reasons Kaplan cites:
- Year-round testing in over 300 test centers around the world
- Easier scheduling, testing available in the afternoon * More comfortable testing environment, less hectic and quieter
- Fewer questions, you have more time to spend on each question
- Questions that are tailored to your ability level * Immediate viewing of scores on screen
- Opportunity to choose four score recipients after seeing your scores
- Official score reports mailed approximately 10-15 days after testing (GRE: Computer, 1997, p. 1)
In addition to these benefits, Kaplan raves "advancing technology means that computer-based testing is the wave of the future. The number of paper-based administrations will gradually decrease" (GRE: Computer, 1997, p. 6)
These predictions by Kaplan are backed by the plans of the ETS to make this alternative to the paper-and-pencil test mandatory. Already it is beginning to phase out the traditional test (GRE Power, 1996). And although the computer-adaptive test is gaining support, there is at least some opposition to this transition. One test preparation company, The Cambridge Review (GRE Power), believes this new form does more harm than good, and in 1996 the governor of New York passed legislation mandating regulations for this type of testing.
CLAIMED DISADVANTAGES OF THE CAT
The Cambridge Review offers various explanations for the
disadvantages of the CAT. In fact, the guide advises the
potential-tester to avoid the test at all cost.
- Guessing can hurt you. On the paper-and-pencil version of the GRE, there is a guessing bonanza. You can't lose any points for a wrong guess, and you get free points for a right guess. Not so on the CAT. Since you have to answer each item before you can go on to the next, you have to guess even if you haven't got a clue. Two or three bad guesses and the computer may just decide that you have reached the level of your ability.
- A computer screen is not a booklet. You get scratch paper at your workstation, but that means that you have to look up at the screen and then look down at the scratch paper, look up at the screen and then down at the paper, up and then down, over and over again. That is not only very annoying but tiring as well because your eyes have to adjust to the screen or paper each time. Additionally, some questions and other materials are too long to be displayed in their entirety on a single screen. Reading Comprehension passages, for example, which can be as long as 60 or 70 lines, cannot be displayed on a single screen. So you have to use a "scroll bar." With a scroll bar, finding text is very difficult and so is keeping in mind the connections between various pieces of text.
- The CAT is not accountable to you. When you take the paper-and-pencil version, you are entitled to a copy of the test booklet containing the questions used to calculate your score and your responses. You can make sure that your test was correctly scored, and you can challenge any items that you think were designed incorrectly. With the CAT, you do not have these rights.
- The CAT is much more expensive. Taking the CAT instead of the paper-and-pencil version of the GRE will cost you about $50 more (GRE Power, (GRE Power, 1996, pp. 11-12).
The Cambridge Review suggests potential graduate students take the CAT only if they are pressed for time. It is the belief of this company that the traditional version is better for almost everyone. "And we believe that most students, even those who are computer literate, will feel more comfortable with the paper-and-pencil version of the test" (GRE Power, 1996, p. 5).
LEGISLATION REGULATING THE CAT
The Governor of New York, George Pataki signed legislation in 1996 to
extend truth-in-testing coverage to university admissions exams
administered by computers. The new law requires test-makers in that
state to periodically make public their questions and answers. This
legislation is opposed by the testing industry, however.
Historically, the nation complies with New York's truth-in-testing
disclosure provisions (Computer Tests, 1996).
Beginning immediately, one complete set of items used for the computer adaptive Graduate Record Exam during the past three years must be filed with the state Commissioner of Education. In 1997, a portion of the computerized item pool administered in the past year, equivalent to one pencil-and-paper exam must be disclosed. In following years, even more items must be made public (Computer Tests, 1996, p.1).The legislation mandates that by the year 2000, test-makers must develop a process so that all students who take a computerized test will be able to review the questions and answers administered to them. In addition, New York's Commission of Higher Education must be notified, as other exams are adapted to computer administration.
TECHNOLOGY'S PLACE IN SOCIETY
As shown in the examples above, there are some who find the Computer
Adaptive Test to have flaws. However, the ETS and Kaplan would both
like their publics to believe they have found a better system, "the
wave of the future" (GRE: Computer, 1997, p. 1). Partisanship
influences this message. According to Eisenberg and Goodall (1997),
the knowledge of the figure conveying the message is shaped by
theories and interpretations to make sense of the world. The theory
that applies to the ETS and Kaplan will be discussed in detail
later.
The following quotation, taken from the writings of DeFleur and Ball-Rokeach (1989), explains how technology is ever present in our society:
It seems abundantly clear that mass communications today are a central part of our institutional structure. That is, while they are industries in their own right, they have penetrated deeply into each of the five basic social institutions of our society: For example, with their emphasis on the services and products of our commercial and industrial establishment, they are a central part of the economic institution. With their increasing role in the election process, their use in various hearings, and their focus on government in the news, they have become a significant feature of our political institution. With their heavy emphasis on entertainment and popular culture, much of which is consumed as entertainment in the home, they are indisputably an important factor in our family institution. For many, the electronic ministry has become a significant part of the religious institution. To a limited degree they are also a part of our educational institution core (p. 124).
CRITICAL THEORIES APPLIED
Hollinger (1994) explained "critical theory is an attempt to
revitalize Marx's humanism and his idea that modern society exploits
and alienates individuals" (p. 79). Western Marxism emphasizes the
themes of cultural alienation and a class struggle. Other critical
theories venture that people do not know their own minds, not knowing
when they are oppressed. Early critical theorists emphasized science
and technology are in the service of capitalism. Hollinger stated the
only way around this sort of society is to resist power.
In essence, the Educational Testing Service conforms to Marxist and critical theory beliefs. By making a computer adaptive test mandatory, the ETS is trying to alienate certain individuals. Not everyone is comfortable using computers; and although Kaplan claims no prior knowledge of computers is needed, that type of testing procedure might be intimidating and confusing, not to mention stressful (Kaplan's Top, 1997). "Some people find the testing environment intimidating. The bulk of the questions will be harder for people who score above the median. You can't skip a question you don't like, and you can't go back to fix up any errors you made" (Kaplan's Top, 1997, p. 2). In addition, despite society's move into the technological age, an adaptation to a new technology can take years. DeFleur and Ball-Rokeach (1989) argued over 30 years into the adaptation of computers, every household is still not capable of supporting a computer and the accessories necessary to support it. Not only is there a lack of funds to support this type of system, but there is not yet a necessity in every home. This situation reflects the Marxist idea of class struggle. The higher class, represented by the ETS, has the means to purchase the technology of today, and therefore are pushing out the lower class. The lower class in this case is composed of various members of society.
Potential graduate students range in age, sex, race, and social and economic background. By requiring a computerized test, the ETS eliminates and discards those not fitting the standards set by the ETS.
A number of variables explain why people do not fit these standards. First, the ETS would want to eliminate anyone not familiar or comfortable with computers because computers and technology are now an integral part of the business world. Mandating the use of computerized tests assures at least some of these people will be eliminated from potentially entering the upper class, or, in this case, the professional world. Groups that can be alienated by the use of a computer-based tests are for example older adults and the poor. "Despite such low skill requirements, it is often frightening for people, especially adults, to have to learn new ways of doing things" (DeFleur, p. 331). These groups are known to be less likely to adopt to the technology desired by a technological society. By excluding these groups, to highlight only two segments of society, the ETS is supporting a higher network of organizations. In addition, a computerized test ensures older members of society will not break into a professional world expecting the stereotype of young, technology-based go-getters.
The critical theorists' belief that people do not know their own minds can be applied in this situation. Slowly branches of society, such as the ones mentioned above, are becoming oppressed. An elite organization, such as the ETS, is dictating the future of members of society without their knowledge. In fact, society is being structured and manipulated by computers and technology. Machines determine the future of society and its members. When technology is viewed as a tool of convenience and advancement, people fail to see how they are manipulated and oppressed by that technology.
Critical theorists claim that to avoid this sort of society, one must resist power. This resistance can be achieved when society sets standards for the use of technology. When standards are set, society can better monitor and control how and when technology is used.
The examples given above provide evidence that computerized Graduate Record Examinations are allowing the Educational Testing Service to oppress those it does not consider appropriate for an elite technology society, or in this case, graduate schools and the professional world. With the established critical and Marxist theories, members of society are offered a better understanding of how they are oppressed and alienated by this elite group. However, this trend can be reversed when the type of power exhibited by elitists is resisted through regulation.
CRITICISM
DeFleur and Ball-Rokeach (1989) give evidence that the computer is
still on the move through society and will soon become a staple in
our lives. Children want computers for recreating and homework;
business persons want them for convenience. "All these developments
strongly suggest that the microcomputers' rapid rate of diffusion
will continue, and with it the increased prospect that computers will
become the cornerstone of future modes of mass communication" (p.
330). These predictions from 1989 have proven to be true. The
Internet has increased the capability of receiving information
quickly, and electronic mail is a convenient way to correspond with
people around the world.
However, cost still hinders some from benefiting from technology.
Until costs decrease, it is unlikely everyone can move into this
technological age. Therefore, the findings stated above will stand.
If the recent quantum leaps toward embracing a technological society
continue, equity can be attained only through access. Thus, measures
to provide strong funding from governmental educational agencies can
provide inexpensive access for a wide range of individuals across
society^"s strata. Though governmental programs and funding may
provide the technology, training is more difficult to attain.
Therefore, the programs adopted should include training so as to
insure equity.
REFERENCES
Computer Tests to Be Covered by Disclosure Law. (1996). [On-line]. Available: http://www.kaplan.com/gre/grecat/top40_grecat.html
DeFleur, M. L., & Ball-Rokeach, S. (1997). Theories of Mass Communication (5th ed.). New York: Logman.
Eisenberg, E. M., & Goodall, H. L., Jr. (1997). Organizational Communication (2nd ed.). New York: St. Martin's.
GRE: Computer-Based Testing (CBT). (1997). [On-line]. Available: http://www.gre.org/cbttest.html#cbt-benefits
GRE Power. (1996). New York: Macmillan.
Hollinger, R. (1994). Postmodernism and the Social Sciences. Thousand Oaks: Sage.
Kaplan's CAT Guide: How the Test Works. (1997). [On-line]. Available: http://www.kaplan.c...recat/catworks
Kaplan's Introduction to the CAT. (1997). [On-line]. Available: http://www.kaplan.com/gre/grecat/index.html
Kaplan's Top 40 GRE CAT Facts. (1997). [On-line].
Available: http://www.kaplan.com/gre/grecat/top40_grecat.html
TCC Online Conferences
Kapi`olani Community College
University of Hawai`i
Honolulu, HI