8/15/2003 -- When you prepare for a certification exam, it goes without saying that you should know the material you'll be tested on inside and out. In addition, though, you should also know how you'll be tested. Knowing the style of questions that will be presented can help you be in the proper frame of mind to avoid getting flustered and increase your odds of passing the exam.

In this article, we will look at the various types of exam question formats (and the testing implications of each) currently in use by a number of certification vendors. As we proceed through the possibilities, we'll start with the simplest type of questions first, and move to the more difficult ones.

In almost all cases, vendors make known what types of questions they use within the certification FAQ/preparation guide postings on their Web sites. It is not uncommon, however, for them to list more possible exam types than those actually employed to keep from giving away too much.

Single-Answer Multiple Choice
The simplest type of question that can be asked, and psychometrically defended, is the single-answer multiple choice question. At one point in time, most certification exams used this almost exclusively. While it has fallen out of favor with many - usually because it could be guessed by those who do not know the correct answer, it is still the most common question type used on most exams.

A sample question would be:

Radio buttons are used to allow one -- and only one -- choice to be made. In the above example, even if you have no idea what the answer is, you have a 1 in 3, or 33 percent chance of correctly guessing the answer. While three choices still exist on some older exams, most of the newer ones from CompTIA, Cisco, Microsoft, and other vendors use a minimum of four choices. With four possibilities, the odds of guessing the correct answer decrease to 25 percent; with five possibilities, the odds become 20 percent, and so on.

Multiple-Answer Multiple Choice
The second most common type of question that appears on certification exams is the multiple-answer multiple choice question. Checkboxes replace radio buttons, and you can pick anywhere from one answer up to the total number of answers possible. All vendors, including Microsoft, use these types of questions on their exams.

Two deviations are possible within this question type. The first is that you are told how many choices to choose, and the second is that you are told to choose "all correct answers." An example of the first type is:

On the surface, it may seem as if you have a 50 percent chance of correctly guessing two choices out of four possibilities, but actually your odds are not that good. The answer possibilities you could choose are: A&B, A&C, A&D, B&C, B&D, and C&D. Only one of those six possible combinations is correct meaning that you only have a 16 percent chance of guessing correctly if you don't know the answers. Increasing the number of possibilities to five instead of four decreases your chances to 10 percent.

Different vendors employ different rules during question creation, but one of the most common is that if you tell the candidate how many answers to choose, there must be at least two more answers than that possible. Thus, if you tell them to choose three, there should be five or more possible; if you tell them to choose four, there should be six or more answers to choose from, and so on.

The deviation that makes the question more difficult is as follows:

Notice that here the vendor isn't telling you how many are correct. Now the possibilities, if you are only guessing, become: A, B, C, D, A&B, A&C, A&D, B&C, B&D, C&D, A&B&C, A&B&D, B&C&D, A&B&C&D. Whereas there were six possible ways to answer the question before, now there are 14. This simple change decreases the odds of guessing the correct answer to only 7 percent.

Fill in the Blank
A staple of Linux exams, this question type does appear outside of that realm, but not too often. "Free Response" is the term most in favor these days, and means the same thing. These questions are difficult to grade, but useful when a vendor wants to verify that you know the correct syntax of a command. They work perfect in the Linux world since that operating system is case-sensitive and still employs administration from the command line on a regular basis. A sample question would be:

Novell uses free response questions occasionally, and does -- according to their latest information -- grade on case-sensitivity (did you capitalize correctly?). Misspellings are graded incorrectly by all vendors.

Ranking
At one point in time, Microsoft made a great push toward ranking questions. They have since moved away from their fascination with them, but they still exist on older exams. Ranking questions provide you with a scenario, a list of required objectives, a list of optional objectives, a proposed solution, and then ask you to rank how well the solution meets the objectives. An example would be:

Hot Area Questions
The "Hot Area" question, also known as hotspot, is a simple single-answer multiple-choice question that has replaced the A-D choices with a graphic. While there are often attempts to make it sound more complex, it truly is as simple as this. For example, as a multiple-choice question, an exam may ask:

To turn it into a "hot area" question, it simply becomes:

What has happened is that each of the possible multiple choice answers from the word question have become a hot area. Instead of making your choice by clicking a radio button, you are making the exact same choices by clicking within a defined area:

Drag-and-Drop Questions
Just as you can draw a parallel between hot area questions and single-answer multiple choice questions, a similar parallel can be drawn between drag-and-drop questions and multiple-answer multiple-choice questions. Microsoft has recently started employing more drag-and-drop based questions on their exams, and they have started cropping up on CompTIA tests as well.

With this type of question, a number of elements are presented and you must move them from one location to another. Often, you must also change the order during the move, and you are occasionally given more items to choose from than should appear in the correct answer. An example of a simple networking exam question would be:

Active Screen Questions
Active Screen questions are relatively new to Microsoft, and are combinations between hot area questions and drag-and-drop. Typically, you need to do something on a graphic but your choices are quite limited. You can drag text around and put it in the appropriate place, or select an option by clicking on a radio button or checkbox. These questions are often employed with configuration that must be done (add a user by using the template, etc.).

A deviation on this is to use the active screen only to provide information that can then be used to answer a multiple-choice question. This makes it an "active" exhibit, and comes close to being a performance-based question (discussed next), with the difference being that here you see the information, and there you change it.

Performance-Based Questions
Performance-based questions, also known as simulations, often work with an emulator of some type; Flash is popular at the moment. The emulator can come on the screen when you click the button and bring up something that looks like what the configuration information in the real product would be. This allows the test to have questions that look like you are working with the real product, without the time and overhead involved of bringing up the real product. The final result that you have accomplished is what is graded, versus the route that you took to get there; therefore if there are multiple ways to do the same thing (and the simulator allows them), what matters is that you correctly did it one of those ways.

While the emulator can load much quicker than the actual product in question, it is still very time-consuming. As such, when encountering them on an exam you should budget your time accordingly .

Most vendors, including Microsoft, toy with performance-based questions. Red Hat, Cisco and a few other vendors have taken this to the next level by skipping the emulator and making you work with the real product. This has to be done in a lab setting and significantly reduces the number of candidates who can take the exam, while greatly increasing the cost.

While it would be nice to see a lab aspect added to every certification -- thus reducing any chanting about paper certifications -- it is impractical for many. The cost and infrastructure for administering (in addition to the volume restriction) is prohibitive for all but a few high-end certifications.

Deviations
All certification exams consist of some combination of the question types discussed. Other terms are often bandied about, but they do not change the types of questions. For example, under adaptive testing (as opposed to "form testing") the first question you get is totally at random and pulled from a pool. Beyond that first question, every other question presented to you is in somehow related to how well you answered the preceding question. This reduces the number of questions you get asked, but the questions are still of the same type. There is no item review in adaptive testing (or on any Cisco exams), and there is no going back to the preceding question(s). Once you answer a question, you are done with it, and you can draw a fair conclusion of how you did by whether or not the next question is on a similar topic.

Scenario-based (also known as case study) questions use the same question types already discussed. They differ from others in that they ask their questions in indirect ways, requiring you to figure out what the question really is from a verbose story problem. The psychological field of question-writing and delivery is known as psychometrics. Within this field, question difficulty is often related in terms of Bloom's Taxonomy (two good sites to visit for more information on this are: http://faculty.washington.edu/krumme/guides/bloom.html and http://www.dlrn.org/library/dl/guide4.html). The logic, in a nutshell, is that the more difficult you make the question, the more you are testing to see if the candidate truly knows the material or not.

Questions? Comments? Post 'em Below!