What is a Profile and when should I use one?
For each test, there is a "pool" of questions. This is the set of all the questions (subject to any filters) that are "contained" in the particular group of test units on that test.
From this pool of questions, you select the ones that you want to actually appear on the test. You can do this selecting by hand, which might be time-consuming, or you can make VISION do it according to a selection profile that you define.
A "Profile" is a set of instructions that you enter into VISION to control the selection of questions from the test's pool of questions. When you apply the profile, VISION randomly selects questions for the test according to the profile you entered. Each time you apply the profile, different questions are selected, but always according to the same instructions.
For more information on how a profile interacts with a question pool, see: Question Pool and Test Profile.
How to create a profile by points, time or number of questions
Note: Depending on the type of test units, the profile window may or may not have a Weighting tab. The Weighting tab appears only if your test units are Programs.
To define a profile, follow these steps:
1.In the Limits tab select how you want the test limited:
b.By time
2.Enter the value for the limit in the space provided. If you choose not to enter any information on the Weighting tab, or if the Weighting tab is not visible, press the OK button and then press the "Apply Profile" button to apply the profile.
3.In the Weighting tab, enter a percent value for each test unit. The sum of values entered for each test units cannot exceed 100%.
Note: Any test unit that has 0% will be ignored when the profile is applied to the test. This means that VISION will not select any questions from that test unit's pool of questions. If all test units have 0%, VISION will put all questions into one large pool when the profile is applied.
See also Question Pool and Test Profile
4.Press the OK button.
Note: When you press the OK button, and you entered percentages on the Weighting tab, VISION will use the percentages to calculate how close it can come to matching the value entered on the Limits tab. If the number calculated is less than the value entered on the Limits tab, a message box will appear, informing you of this.
5.After you apply the profile, switch to the Summary tab. Review the "Test Summary" field to see how close VISION came to fulfilling your profile specifications.
6.Also look at the "Total Questions" field to see how many questions VISION was able to select. The total might be fewer than the number requested in the profile. The question pool and profile definition may have made it impossible to select that number.
How to create a profile by objectives
The by objectives option on the Limits tab will allow you to orient the test around objectives (not around questions). On the Weighting tab you can specify what objective types (level names) you want to test and how many questions VISION should select. Lets look at an example:
Example
Let's say you have three program units on the test: A, B and C. On the Limits tab you click the Objectives radio button. On the Weighting tab, the three test units combined contain all four objective level names (Performance Terminal, Performance Enabler, Cognitive Terminal or Cognitive Enabler). Because you only want to test Cognitive Objectives, you enter 50% Cognitive Terminal and you want two questions from each, and 25% Cognitive Enablers with one question from each.
When VISION applies this profile, it will randomly pick 50% of the Cognitive Terminal Objectives and randomly select two questions from each objective. Next, VISION randomly selects 25% of the Cognitive Enablers and randomly picks one question from each objective.
Doing the math
When you enter percentages for a test unit on the Weighting tab, VISION will give that test unit the percent you enter of the value entered on the Limits tab. For example, let's say you entered 65 in the value field on the Limits tab and on the Weighting tab you assign 5% to a test unit. To distribute the test unit's portion of 65, VISION will perform the following calculation:
.05 x 65 = 3.25
What this means is that VISION will randomly select questions from this test unit's question pool until the 3.25 value is met. So if you are limiting the test by points, VISION will look at the point value that each question is worth during the selection process. If the first question it encounters is worth 2 points, this question will be selected. If the second question encountered is worth 1 point, this question will also be selected. If there are no questions left in the test unit's pool that have a point value less than or equal to .25, VISION will not select any more questions from that pool and will move on to the next test unit.
Note: If you are limiting your test by "number of questions", VISION will round any fractions to the nearest whole number. So, 3.49 would be rounded to 3 and 3.50 would be rounded to 4. This is done because you cannot select a fraction of a question.
Why doesn’t VISION meet the limit I wanted?
VISION cannot always meet the value you entered on the Limits tab. The following is a list of reasons why this might occur:
•There are too many questions that are not active, have zero points or have no distractors (for Multiple Choice questions only). VISION will ignore these categories of questions during the selection process.
•There are not enough questions in a test unit's pool.
•The percentages are too small. If you have many test units and you are testing a small percentage of the questions from each test unit, the rounding errors may reduce the actual number of questions selected. See "Doing the Math" above.
If VISION does not meet the value you entered on the Limits tab, you can apply the profile again. If you need a test that exactly matches this value, you may have to select or deselect some questions manually.
Why does VISION always select certain questions?
In the workscreen for a question, there is a check box that reads must appear on test. If this checkbox is checked for a question and that question happens to appear in the question pool of a test, VISION will always select this question. In fact, if you have a lot of questions whose "must appear on test" checkbox is checked, the number of selected questions may even exceed the value entered on the Limits tab.
Question Pool and Test Profile
Although we usually speak of a test as having a single question pool, in the context of test profiles it sometimes makes more sense to think of a test as having multiple question pools.
In the context of a test profile, a "pool of questions" can have two meanings depending on how much information you enter in the profile. If you only enter information on the Limits tab of the test profile screen, VISION takes each pool of questions for each test unit and throws them all into one large, overall pool. Then, when you apply that profile, VISION randomly selects questions from this one large pool. Let's look at an example.
Points
Select this option to specify the desired total point value of the test. Enter the number of points in the "Value" field below.
Note: If a question's point value is zero (or its status is not Active), the question will not be selected on the test.
Time
Select this option to specify the amount of time it should take for the student to complete the test.
Enter the time (in minutes) in the space provided below. VISION uses the value entered in each question's "Time to Complete" field in order to determine how many questions to select.
Note: This choice is only practical if you have followed the practice of entering time-to-complete values for questions.
If a question's "Time to Complete" field is zero (or its point value is zero or its status is not Active), the question will not be selected on the test when using this kind of selection profile.
Number of Questions
Select this option to specify how many questions to select for the test. Enter the number of questions in the space provided below.
Note: If a question's point value is zero or its status is not Active, the question will not be selected on the test.
Objectives
Select this option to orient your test profile around objectives instead of questions. To apply weighting to different objective types, go to the Weighting tab, at the top of this window.
Note: This option is disabled, naturally, if the test is composed directly from questions (i.e., the test units are questions).
