CertCities.com -- The Ultimate Site for Certified IT Professionals
Register today for a Free Sponsored Tech Library Webcast and you could WIN! Share share | bookmark | e-mail
  Microsoft®
  Cisco®
  Security
  Oracle®
  A+/Network+"
  Linux/Unix
  More Certs
  Newsletters
  Salary Surveys
  Forums
  News
  Exam Reviews
  Tips
  Columns
  Features
  PopQuiz
  RSS Feeds
  Press Releases
  Contributors
  About Us
  Search
 

Advanced Search
  Free Newsletter
  Sign-up for the #1 Weekly IT
Certification News
and Advice.
Subscribe to CertCities.com Free Weekly E-mail Newsletter
CertCities.com

See What's New on
Redmondmag.com!

Cover Story: IE8: Behind the 8 Ball

Tech-Ed: Let's (Third) Party!

A Secure Leap into the Cloud

Windows Mobile's New Moves

SQL Speed Secrets


CertCities.com
Let us know what you
think! E-mail us at:



 
 
...Home ... Editorial ... Columns ..Column Story Saturday: April 5, 2014


 On the Plus Side  
Jeff Durham
Jeff Durham


 Time for a New Testing Paradigm
Should traditional methods for proctoring CompTIA exams go the way of the steam engine?
by Jeff Durham  
7/30/2003 -- The highest cost of adding a certification to your resume isn't always the cost of the exam. It isn't always the price of a study guide, either. Nor is it the price of a self-study test engine. For a great many people, the most expensive thing about getting certified is the time it takes to go to a testing center.

If you don't live in a city where a testing center is right next door, then you have to leave work, drive to the center, take the test, drive back, etc. While it may sound minimal on the surface, try taking several hours of a packed day -- and planning for it enough ahead of time to schedule it -- and justify it with your bosses.

The testing center model is horribly dated. It is based upon the concept that people are dishonest and can only be given exams under controlled, proctored conditions. It is also based on the idea that test centers employ the antithesis of exam candidates; employees are trustworthy, committed and righteous. Lets explore the current paradigm before moving to a new one. Though both of the main vendors in our industry, Pearson Vue and Prometric, operate in essentially the same fashion, my experience is with Pearson Vue, and I am using them for specifics.

Old Paradigm
When you schedule an exam, an encrypted Access file is downloaded to a server at the test center of your choice. This file can be downloaded anytime between when you schedule the exam and when you take it, but is usually within 24-hours of delivery. After going through the steps of showing identification and signing in to take the exam, you sit at a client machine. The client machine establishes a dumb terminal session with the local server and runs an application pulling data from the file that was downloaded to the testing center's server.

The proctoring can be accomplished in one of three acceptable ways:

1. An administrator of the test site sits in the room and watches you take the exam,

2. A window separates you and an administrator so they can watch you take the exam, or

3. A camera is used in the testing room to monitor you.

With the first two possibilities, you know if (and often when) someone is looking directly at you. With the third possibility, you don't know if there is anyone looking at the monitor at this moment, or even if the monitor is turned on at all. Not surprisingly, the third choice is the one most employed by test centers.

When you finish the exam, a score report prints out locally showing how well you did and you leave. At some point in time, the administrator at the testing center will run an update routine to send your score results up to the vendor and remove the Access file from their server. This is supposed to be done within 24-hours of your taking the test, but does not always happen with that frequency.

It is easy to see that the current model puts a lot of weight on the test center and exam administrators they employ. For carrying this burden, the test center is not paid anything unless the quantity of exams they deliver each month is high. After delivering a certain number of exams, they start to make a couple of dollars on each successive exam above that number. If the test center is not getting paid to deliver exams (and set aside their machines and rooms, etc.), then why are they doing it? Test vendors might argue that it is purely out of altruism -- wanting to provide a service to the community. I'll argue that it is solely to sell other services. If you fail an exam, maybe I can sell you on a class, or a book, or...

If we discount the motives of the testing center, then the integrity of the current system must lay with the test site administrator. What does it take to be a test site administrator? Basically, you have to be able to fog a mirror. You cannot know anything about the tests you are administering, for anyone who holds any certification from Microsoft, Cisco, etc. is immediately banished for life from being able to be a test-site administrator. You do have to pass a short test correctly identifying the brand names of the applications you will run to download tests and upload results, but little else is required.

New Paradigm
I believe that people are inherently good. There are some who cheat, and some who steal, and some who don't follow the rules, but those people are just as likely to work for a testing center as frequent one.

When you want to take a CompTIA exam, I believe you should schedule it as you currently do. At delivery time, however, I see no reason why your computer (at work, at home, etc.) cannot establish a dumb terminal session with a CompTIA server and pull the exam directly from there. This makes it possible for a user to take an exam from anywhere without the need to travel to a proctored site, and makes it possible for the vendor to immediately get pass/fail results.

Afraid that everyone taking an exam will only have access to a slow dial-up connection? Not a problem, CompTIA exams are not exactly overflowing with cutting-edge graphics. Afraid someone might sneak a peek in a book while taking the exam? Require a Web cam to be active during the exam -- the candidate will not know when or even if they are truly being monitored. Afraid that a connection might be lost during the middle of the exam? There are numerous ways to plan for this, and they are employed at every site that handles monetary transactions; the technology is there and can be ported over with little effort.

What about authentication? When you go to a test center now, you have to show two forms of ID, and everyone knows that they would never be fake (read with all intended sarcasm). Under the new model, authentication should take place online: Ask for your social security number, mother's maiden name, etc. These variables are good enough to use when making major online purchases, they are good enough to use when taking an entry-level, one-time exam from CompTIA.

What is stopping a new testing model from emerging and replacing the existing one? Two things: outcry and realization. First, there needs to be a public outcry pointing out the ridiculousness of the testing system that has now become comfortable before any serious change and can be considered. Second, CompTIA needs to realize that by implementing this model, they will be able to keep the entire testing fee for themselves and stop sharing it with the testing centers. If that isn't incentive enough, I don't know what is.

What's your take? Add your two cents by posting your comments below!


Jeff W. Durham, MCP, A+, i-Net+, Linux+, is the recent co-author of the Security+ Short Course. E-mail any questions or comments to .

 


More articles by Jeff Durham:

-- advertisement --


There are 41 CertCities.com user Comments for “Time for a New Testing Paradigm”
Page 1 of 5
7/30/03: Anonymous says: I couldn't agree more! Whereas IT is supposed to be leading the future, this is true almost everywhere but in testing. Testing sites don't use their best machines for the exams, since they need those for the paying customers in the classrooms, and the person who is the "administrator" is usually also the person who has to answer the phones, get coffee for the instructors, etc. Great article.
7/30/03: Tcat Houser from Digitial Nomad says: Three cheers! :-)
7/30/03: Anonymous says: As if the lousy, simple, over-priced CompTIA tests aren't overadvertised enough, this bit-banging writer seems to think that CompTIA needs a way to make even MORE MONEY via stupid, privacy invading methods. This guy needs to think about things a bit; not everybody has the kind of hardware it takes to do what he suggests. Has he ever seen video sent over a phone line? I saw the first attempts at video cams over phone lines back in 1993; jumpy, blurry. Sure the newer webcam stuff might be better but not by much since ya can only pack so much data over a phone line. Seesh! No wonder this clown of a writer only has lame CompTIA certs and an unnamed MCP. Heck, maybe the bit-banger writer that wrote this article should admit to being a schill for CompTIA. Then again, a glance at the cert listing for this bit-banger tells me that realized that he's spent about 2 hours in a test center. I passed my Aplus, both sections, in 20 minutes, the i-NetPlus test in 20 minutes, and the LinuxPlus test in 20 minutes. I passed my first MS test, for my MCP, in about 1 hour. So, try schilling for someone else ya mangy bit-banger of a web writer.
7/30/03: Anonymous says: I strongly believe that security around exams should be even greater by undergoing iris recognition scanning. The technology is there, why don't they use it !!!!
7/31/03: Anonymous says: C'mon, Jeff, try spending some time at a local college and talk to the instructors about how students take exams. People may be inherently good, but test candidates are inherently looking for a way to pass the exam. What does CompTIA do when six people from the same office want to take the Aplus exam, and each of them takes it with two helpers sitting right behind??? Cheating will be rampant, and the test items will be invalidated faster than they can be written. Then the cert is worthless, and what's the point? FWIW, CompTIA explored web-based delivery of the Aplus exam many years ago, rejected it for solid reasons, and nothing has changed since then. You just wrote a book on SecurityPlus...why do we need such a cert if people are inherently good? LOL
7/31/03: Victor from The Young and the Restless says: Its all about the creation of wealth. As the system currently exists, VUE and Prometric make a ton of money (they are cash cows) without any true costs - costs are incurred in exam development by the vendors and in exam delivery by the testing centers. If the exams were to change delivery format, VUE and Prometric might not be able to make as much money, and their parent companies do not want to see that happen. What is best for the person certifying is not whats best in the eyes of big business.
7/31/03: Randy Trask from Pearson VUE says: Jeff Durham's article suggesting candidates for IT certification exams self proctor is persuasive but unfortunately based on a false premise. Jeff makes the assumption that exams are proctored strictly to verify a candidates' identity and to prevent cheating. Although both functions are certainly important, the proctor also plays an instrumental role in protecting the intellectual property of the test sponsor (e.g. CompTIA). The process of building an exam that is statistically validated to discern qualified candidates is both time intensive and very expensive. Someone taking an improperly or non-proctored exam not only has the complete freedom to cheat at will, but can also compromise exam content by disclosing the questions to future test takers. Compromised exams can cost test sponsors tens-of-thousands of dollars. With a proctor present, we can positively identify that the person presenting them self to sit for an exam is in fact that person. We also know prohibited materials (e.g. learning guides) were not taken into the testing room; that the candidate was not observed engaging in any form of cheating; that the person taking the test was neither copying test questions or using a recording device; and that no one else was present with the individual during the examination. Pearson VUE has about 3,400 testing centers located in 130 countries. Add to that number the more than 3,000 examination centers the other major testing provider advertises and it is difficult to imagine that the vast majority of the population is not conveniently located to a testing center. The value of certification is directly tied to maintaining the integrity of the testing process and of the credential issued. Jeff obviously takes his career seriously. He has been awarded multiple credentials and his pride in his accomplishments is displayed after his name in the form of a listing of each credential awarded. Along with many other IT professionals, Jeff invested in his career. Along with our clients--the test sponsors--Pearson VUE remains committed to protecting Jeff's investment.
8/4/03: Anonymous says: Randy, don't forget the night deposit, because you're arguments don't carry much weight beyond that. The exam objectives are widely known, while the questions are not. Keeping the existing structure in place just to keep the questions secure is foolish, at best. A far better solution is to make the questions more relevant - i.e. hands-on, real-world-based. Red Hat knows how to make questions that real world administrators encounter and they know how to present them to administrators: as hand-on questions in a real-world lab-based scenario. So what if someone learns what the question is? To pass, they must know how to solve problems and not how to guess A on a multiple-choice exam that isn't worth the paper the certificate is printed upon. Don't justify how you need to secure something so meaningless; look for ways to make the certification more relevant. Jeff is on to something, and you're argument proves that.
8/6/03: Never Happen says: Important certs will never be delivered via the web to your desktop. Never, for the same reason you'll never take the SAT or Bar Exam at home. They will always, always, always be proctored. But what I want to know is why CompTIA is going linear with the Aplus exam on Sept 12...see their Aplus FAQ.
8/6/03: Anonymous says: Mr. Night Deposit still has a huge exposure, because all it takes to pass a non-proctored "hands-on, real-world" exam is an expert looking over your shoulder, telling you what to do. As a certification manager, I KNOW that people cheat, particularly in countries outside the US and western Europe. We must do all we can to ensure that the exams, whether hands-on or multiple-choice, remain an effective measure of an INDIVIDUAL's skill. To do that, we have to count on a proctored system.
First Page   Next Page   Last Page
Your comment about: “Time for a New Testing Paradigm”
Name: (optional)
Location: (optional)
E-mail Address: (optional)
Comment:
   

-- advertisement (story continued below) --

top