This segment describes how to use filter and profiles to define which questions will be loaded on your test. Click the links below to learn about all of the different options for loading test questions.
Question Order
The question pool is displayed according to the order of test units on the Test Units page—until you press in the "Order" button. Then you see the questions only, and you see them in the order in which they will appear on the test. Initially, that order is set as follows:
If you apply a profile, VISION randomly selects questions from the pool to meet the specifications of the profile. The resulting question order is the order in which VISION randomly selected those questions. But if you manually select questions instead of using a profile, then the order is that in which you selected the questions. For example, if you have two questions, A and B, and on the Selection page you select B first and A second, then B would appear before A in the test question order.
Test Unit Arrangement
Unless the test units are themselves questions, the questions on the Selection page will be organized under the test units. For instance, if your test units are Program Training Units, then each training unit is broken down into the objectives that are sequenced into it. Similarly, for each objective you will see the questions that are linked to the objective. This allows you to see which objective a question belongs to and, in turn, which training unit the objective is sequenced into.
If you have not applied a profile to the test, you can manually select from the question pool the questions that you want to test. Even if you have applied a profile, you can still alter the selection manually afterward. But note that constitutes interfering with the profile, meaning you may be violating your own plan for the test.
A checkbox appears to the left of each question. To select or unselect a question, click on that checkbox. Whether the selection process is done manually or through a profile, you can view and modify the order in which the questions will appear on the test via the Order button.
To show or hide a particular column or row type, click your right mouse button anywhere on the Questions Pool window to check or uncheck a column or row type. Right-click on the column headers to quickly access the column-selection menu.
The Question Pool
VISION initially includes all the parts of every test unit you add to a test. For example, suppose the test unit is a Topic from the program curriculum:
•Suppose the topic has 3 lessons; in each lesson there are 10 objectives—a mixture of terminal and enabling objectives; linked to each objective there are 4 questions—a mixture of different question types. That means this test unit, the Topic, will automatically contribute 30 objectives and 120 questions to the test pool.
•However, you might not want all of the items (objectives and questions) from the topic to be included in the pool for the test. If you only want to test Enabling Objectives, and for those objectives, you only want to use Multiple Choice questions, then you would use filters to limit the pool to only the items that you want.
Why VISION always selects certain questions
Loading Questions
The Load button is used to load your question pool and to define filters. To define a filter, click the Load button, select Define Filter, and then click on the type of filter you want to use.
If a filter type is disabled, it is because that kind of filter does not apply to the type of test unit you have chosen for your test. If the test unit type is questions, then the entire Load button is disabled, since the questions have already been loaded by the act of choosing the test units.
On the Load button's menu, you can set whether practice questions will be included in the pool when it is subsequently loaded.
Once you have defined a filter, or if you choose not to define a filter, fill the question pool by clicking the Load button and choosing "Unfiltered" or "Filtered" from the menu.
Defining the Filter
A “filter” is one or more rules for limiting which subordinates of the test units are admitted into the question pool. There are three kinds of filters: Training Unit, Objective, and Question. Which of these are available on a given test depends on the kind of test unit used for that test. The larger the test unit type, the more kinds of filters are available, because there are more kinds of subordinates.
The Filter workscreen lists the Defined Filters. These are filters that have already been defined and preserved as files. On the Filter workscreen you can choose to do one of three things:
Using a defined filter, as-is
Using a defined filter, with modifications
Defining a new filter
Filter Design Tool
You can use the Filter Design tool to create a filter file or save a filter to file from the filter interface. Enter a specific title for the filter and then use the drop-down lists to select customizable options. Use the + and - buttons to add or remove filter rules and the "Not" checkbox to use the negative of a selection. When you are finished, click the OK button and then select your filter.
After creating filters
Defining a Profile
The Define Profile option on the Select button drop-down menu allows you to define a selection profile, or set of parameters, that provide VISION with the necessary information to randomly select questions from the pool to appear on the test.
To define a profile, you begin by clicking the Select button and choosing Define Profile from the menu. There you will see the Limits page, where you will decide how you want to limit your test. There is also a Weighting page that is available if your test units are Program Training Units (program units that have sequenced objectives) or Program organizers. If the Weighting page is available, you can decide if you want to use the page. It is not a requirement to weight each test unit.
Your entire question bank is the total collection of questions associated with all the objectives in your project database. Creating a test involves forming a pool of questions that are pulled together from the question bank. The pool is based on the test units that you select during the process of making the test. Every test question associated with the test units you choose will be placed by VISION into the pool, constrained by any filters that are used. You probably will not want to select every question in the pool.
For instance:
•You may want to select some items, and not others, so that each test is unique.
•You may also wish to select only enough questions to meet, but not exceed, some target such as a total point value, total number of questions, total amount of time, or some other limit.
•You may want to weigh certain units more heavily than others, to reflect the emphasis that was given in a lecture or some other factor.
Applying these criteria by hand would not be easy to do if you have a large, or even semi-large, pool of questions. That’s when you use a profile.
Understanding profiles
Limits
There are several ways you can limit your test:
•By points
•By time
•By number of questions
•By objectives
If you select "by points," "by time," or "by number of questions," you will need to enter a value for the limit in the space provided on the Limits page.
If you select "by objectives," your test will be oriented around objectives instead of questions. To apply weighting to different objective types, go to the Weighting page.
If VISION doesn’t meet the limit you want
VISION cannot always meet the value you entered on the Limits page. The following is a list of reasons why this might occur:
•There are too many questions that are not active, have zero points, or have no distractors (for Multiple Choice questions only). VISION will ignore these types of questions during the selection process.
•There are not enough questions in a test unit’s pool.
•The percentages are too small. If you have many test units and you are testing a small percentage of the questions from each test unit, the rounding errors may reduce the actual number of questions selected.
If VISION does not meet the value you entered on the Limits page, you can apply the profile again. If you need a test that exactly matches this value, you may have to select or deselect some questions manually, after applying the profile.
Weighting
The Weighting page will ONLY be available if your test units are Programs or Program Training Units (program units that have sequenced objectives). Weighting is an optional feature.
If you limit your test by points, by time, or by number of questions, you identify the percentage of the test that will be made up of questions in the selected test unit.
If you are limiting your test by "number of questions," VISION will round any fractions to the nearest whole number. So, 3.49 would be rounded to 3 and 3.50 would be rounded to 4. This is done because you cannot select a fraction of a question.
Any test unit that has 0% will be ignored when the profile is applied to the test. This means that VISION will not select any questions from the test unit’s pool of questions. If all test units have 0%, VISION will put all questions into one large pool when the profile is applied.
If you limit your test by objectives, then on the Weighting page you specify what percentage of objectives to include from each type, as well as how many questions to select from each objective of that type.
