Column
On the Plus Side
Time for a New Testing Paradigm
Should traditional methods for proctoring CompTIA exams go the way of the steam engine?
by Jeff Durham
7/30/2003 -- The highest cost of adding a certification to your resume isn't always the cost of the exam. It isn't always the price of a study guide, either. Nor is it the price of a self-study test engine. For a great many people, the most expensive thing about getting certified is the time it takes to go to a testing center.
If you don't live in a city where a testing center is right next door, then you have to leave work, drive to the center, take the test, drive back, etc. While it may sound minimal on the surface, try taking several hours of a packed day -- and planning for it enough ahead of time to schedule it -- and justify it with your bosses.
The testing center model is horribly dated. It is based upon the concept that people are dishonest and can only be given exams under controlled, proctored conditions. It is also based on the idea that test centers employ the antithesis of exam candidates; employees are trustworthy, committed and righteous. Lets explore the current paradigm before moving to a new one. Though both of the main vendors in our industry, Pearson Vue and Prometric, operate in essentially the same fashion, my experience is with Pearson Vue, and I am using them for specifics.
Old Paradigm
When you schedule an exam, an encrypted Access file is downloaded to a server at the test center of your choice. This file can be downloaded anytime between when you schedule the exam and when you take it, but is usually within 24-hours of delivery. After going through the steps of showing identification and signing in to take the exam, you sit at a client machine. The client machine establishes a dumb terminal session with the local server and runs an application pulling data from the file that was downloaded to the testing center's server.
The proctoring can be accomplished in one of three acceptable ways:
1. An administrator of the test site sits in the room and watches you take the exam,
2. A window separates you and an administrator so they can watch you take the exam, or
3. A camera is used in the testing room to monitor you.
With the first two possibilities, you know if (and often when) someone is looking directly at you. With the third possibility, you don't know if there is anyone looking at the monitor at this moment, or even if the monitor is turned on at all. Not surprisingly, the third choice is the one most employed by test centers.
When you finish the exam, a score report prints out locally showing how well you did and you leave. At some point in time, the administrator at the testing center will run an update routine to send your score results up to the vendor and remove the Access file from their server. This is supposed to be done within 24-hours of your taking the test, but does not always happen with that frequency.
It is easy to see that the current model puts a lot of weight on the test center and exam administrators they employ. For carrying this burden, the test center is not paid anything unless the quantity of exams they deliver each month is high. After delivering a certain number of exams, they start to make a couple of dollars on each successive exam above that number. If the test center is not getting paid to deliver exams (and set aside their machines and rooms, etc.), then why are they doing it? Test vendors might argue that it is purely out of altruism -- wanting to provide a service to the community. I'll argue that it is solely to sell other services. If you fail an exam, maybe I can sell you on a class, or a book, or...
If we discount the motives of the testing center, then the integrity of the current system must lay with the test site administrator. What does it take to be a test site administrator? Basically, you have to be able to fog a mirror. You cannot know anything about the tests you are administering, for anyone who holds any certification from Microsoft, Cisco, etc. is immediately banished for life from being able to be a test-site administrator. You do have to pass a short test correctly identifying the brand names of the applications you will run to download tests and upload results, but little else is required.
New Paradigm
I believe that people are inherently good. There are some who cheat, and some who steal, and some who don't follow the rules, but those people are just as likely to work for a testing center as frequent one.
When you want to take a CompTIA exam, I believe you should schedule it as you currently do. At delivery time, however, I see no reason why your computer (at work, at home, etc.) cannot establish a dumb terminal session with a CompTIA server and pull the exam directly from there. This makes it possible for a user to take an exam from anywhere without the need to travel to a proctored site, and makes it possible for the vendor to immediately get pass/fail results.
Afraid that everyone taking an exam will only have access to a slow dial-up connection? Not a problem, CompTIA exams are not exactly overflowing with cutting-edge graphics. Afraid someone might sneak a peek in a book while taking the exam? Require a Web cam to be active during the exam -- the candidate will not know when or even if they are truly being monitored. Afraid that a connection might be lost during the middle of the exam? There are numerous ways to plan for this, and they are employed at every site that handles monetary transactions; the technology is there and can be ported over with little effort.
What about authentication? When you go to a test center now, you have to show two forms of ID, and everyone knows that they would never be fake (read with all intended sarcasm). Under the new model, authentication should take place online: Ask for your social security number, mother's maiden name, etc. These variables are good enough to use when making major online purchases, they are good enough to use when taking an entry-level, one-time exam from CompTIA.
What is stopping a new testing model from emerging and replacing the existing one? Two things: outcry and realization. First, there needs to be a public outcry pointing out the ridiculousness of the testing system that has now become comfortable before any serious change and can be considered. Second, CompTIA needs to realize that by implementing this model, they will be able to keep the entire testing fee for themselves and stop sharing it with the testing centers. If that isn't incentive enough, I don't know what is.
What's your take? Add your two cents by posting your comments below!
Jeff W. Durham, MCP, A+, i-Net+, Linux+, is the recent co-author of the Security+ Short Course. E-mail any questions or comments to .
|