TestMaker Users' Guide

TestMaker Users' Guide

Contents:

Introduction

With the TestMaker application you can easily create different versions of printable tests or worksheets in a wide range of domains like mathematics, physics, etc. The tests are built as a collection of parametrized problems (tasks), and based on the parameter values each test will contain randomly generated versions of the same type of problems. This is a useful feature for teachers who don't want to hand out the same copy of test to all of their students. The application can generate an unlimited number of versions, together with the answer keys corresponding to each version.

Basic concepts

  • Test. A test is a definition of the format and contents of a printable worksheet. It contains a header part (with text and/or images) and a collection of problems. The same problem may appear multiple times in a test.
  • Problem. A problem is a definition of the format and contents of a task with a solution. The task and solution part may contain text and images, and both parts may contain variables. During test generation the instances of problems will contain specific values of the variables, leading to different versions of the problem.
  • Variable. A variable is a parameter of a test. It has a name, a type (it may be either an integer, a real number or textual), a range (an enumeration of allowed values or specified by minimum and maximum values) and a method of generating values (either picking randomly from the allowed values or by picking the values sequentially). Using variables you can have different instances (versions) of the same problem type without having to create the different instances manually.
  • Constraints. In a problem definition you can impose constraints on the variables. For example you can require that two randomly distributed integer variables from the range [0..5] must not have the same values.

Installation and running the application

You need the Java Runtime Environment (at least version 1.6) installed on your machine to be able to run the application. If you don't have it, please download and install from http://www.java.com/en/download/index.jsp. (For some reason version 1.6 is sometimes called version 6, and likewise 1.7 is the same as 7, don't get confused by version numbering.)

Unzip the installation package to any directory. On Windows you can double click run.bat to start the program, on Mac double clicking the TestMaker.jar should start it. (Alternatively you can open a command prompt in the installation directory and type
java -jar TestMaker.jar
to start. This should work on any platform.)

When the program starts for the first time you need to accept the End User License Agreement. Please review the text displayed and click Accept if you are ready to use the application with the given terms.

The application stores tests and problems in a user defined directory structure under a specified base directory. By default this is the 'data' subdirectory under the application's installation directory. If, however, you would like to keep your tests elsewhere then set your preferred location of your base directory using the Edit/Preferences menu item. You may also wish to change this setting at a later time if you decide to move your data to a new location. In this case make sure that the location of problem files and images remains the same relative to the base directory.

Performed the above steps you are ready to use the application.

Creating problems

imgYou can define problems on the Problem data tab of the application's main window. The controls of this window are the following:

  • Name. An arbitrary name or short description of the problem.
  • Problem text area. The text of the task your students need to solve. The contents of this text field can be edited by clicking the Edit problem button.
  • Edit problem button. Pressing this button opens an editor window. See the description of the built in text editor.
  • Solution text area. The solution of the task. Here you can define the answer key to the problem. The contents of this text field can be edited by clicking the Edit button next to this field.
  • Edit button (for solutions). Pressing this button opens an editor window. See the description of the built in text editor.
  • Variables table. It lists the variables defined for this problem. Use the Add, Remove and Edit buttons next to this table to manage your variables. If you press the Add or Edit button to create or modify a variable then a new window appears, see managing variables.
  • Constraints. You can specify constraints the problem's variables must satisfy during problem instance generation. Note that constraints must not be specified on textual variables, only on numeric ones. The constraint text must be a boolean combination of formulas that can be evaluated to TRUE or FALSE. Use the & and | symbols to combine the formulas with AND or OR, respectively. A formula contains two expressions and exactly one of the following comparison operators between them: <, >, <=, >=, =, != . The boolean expression is evaluated from left to right, parentheses may be used only within an expression, but not to change the evaluation order of the boolean combination. A sample constraint that requires that two variables have either the same value or their difference must be greater than 10 is:
    $a=$b | abs($a-$b) > 10
    This means that in the generated problem instance the variables $a and $b may have values (5, 5), (3, 3) or (10, 30) but not (10, 11) or (10, 1).
  • Verify button. It opens a new window showing a problem instance with task and solution parts filled in with concrete values of the variables. Or, it shows an error message if there are syntactic errors in the expressions you used, or in case of undefined variables, etc. Use the Next button to see further instances of the same problem.

Saving the problems

To save the problem you created use the Save Problem or Save Problem As menu items from the File menu or press the corresponding toolbar buttons. When saving the problem, the file name by default will be generated based on the text you entered in the Name field, but you may specify any other name for your saved files.

A note for advanced users. Each problem is stored in 3 files. Beside the main problem definition file (having a .prb extension) the problem text and solution parts are stored in two HTML files, so that you may easily view their contents with your web browser or edit them with your favourite HTML editor, if you need to add some formatting beyond the features of the the built in text editor. However, for most of the times you need not worry about how problems (and tests) are stored, and I recommend editing these files only from within the TestMaker application. If you edit the files, make sure you observe the naming conventions: the xxx.prb file must have an accompanying xxx_text.html and xxx_solution.html file in the same directory.

Using the built in text editor

imgThe text and solution parts of problems and also the headers of tests are edited using a built in text editor which has these features of the usual set found in text editors (in the order of toolbar buttons found on the top of the editor window):

  • Left align paragraph
  • Center paragraph
  • Right align paragraph
  • Bold characters
  • Italic characters
  • Underlined characters
  • Subscripts
  • Superscripts
  • Insert image (see notes below)
  • View HTML source (see notes below)
  • Undo
  • Redo

Also you can set the size and type of the font. On the left side of the window there is a collection of symbols commonly found in math texts like operators and Greek letters. You can insert these into the edited text by clicking on the symbol buttons.

A note on image handling. In the current version only JPG images are supported. Also there is no support to set image properties like alignment and size. These settings can be made however by editing the HTML source of the text content, see the following note.

A note on HTML editing. There are several features of HTML beyond the capabilities of the built in text editor, for example you may want to insert tables into the edited document. Users familiar with HTML may insert hand made HTML fragments into the document to overcome the editor's limitations. Note however, that certain HTML features (for example frames) are not handled properly by Java's component that displays HTML, so even if you can write proper HTML, the end result may not be satisfactory, so all editions to the raw HTML source must be tested carefully for correctness.

As an example to the previous notes, if you insert an image into the editor using the insert image toolbar button, the resulting HTML code will be something like this:
<img alt="img" src="file:///C:\data\img\sample.jpg">
If the image takes too much space in your final output you may decide two downscale it to 300x200 pixels by editing the above HTML fragment so that it looks like this:
<img alt="img"src="file:///C:\data\img\sample.jpg" width="300" height="200">

Managing variables

The Variable definition window appears when you press the Add or Edit button next to the Variables table of a problem. The window has the following controls:

  • Name. The name of the variable. It must begin with the $ sign and must contain a letter as its 2nd character. It must be unique within the scope of a problem. Some valid variable names are $a, $a1, $myvariable. Some invalid ones are $1, myvariable, $_x.
  • Type. Choose one of the 3 allowed types.
  • Distribution.
    • Choose Random if you want to have a value chosen randomly from the variable's range. The minimum and maximum of the range is included in the distribution. For enumerated variables a random element is chosen.
    • Choose Sequence if you need that the elements in the variable's range are chosen sequentially, so when the variable is needed for the first time in the generated output then the first element is used, the next time the second element is used, etc. When the list is exhausted then the following value will be the first one from the list once again, and so on. Note that this setting cannot be used with variables of type real.
  • Values.
    • Choose Range to specify the lower and higher limits of the variable. The limits are inclusive, so a variable of integer type with low=1 and high=5 may take the values 1, 2, 3, 4 and 5. This setting can not be used for variables of type string.
    • Choose List if you would like to enumerate the allowed values for the variable.

Using variables

You can refer to a variable in the problem's text or solution part by its name. Wherever you write $variablename in the text editor, a new value of the variable will be substituted for it each time the variable is needed during test output generation, see the tutorial for a detailed example.

Variables of numeric types (integer and real) may appear in expressions (ie. within an eval(...) function), see the Using expressions section for details. In this case the current substitution value of the variable is taken and used during the evaluation of the expression, returning a numeric value. The generated output will contain the returned value.

As a quick example, suppose you want to create a simple problem for calculating the square of a small integer. Then you can create a variable with these settings: name = $n, type = Integer, range = [0..20], distribution = Random. The text of the problem may look like:
Evaluate: Square of $n = ...
And your solution is
$n squared is eval($n*$n)

In the generated output the problem may look like this:
Evaluate: Square of 11 = ...
And your answer key will contain
11 squared is 121

Creating tests

imgYou can define tests on the Test basic data tab of the application's main window. The controls of this window are the following:

  • Name. An arbitrary name or short description of the test.
  • Header text area. The text of the test header, which will appear on top of each test version. The contents of this text field can be edited by clicking the Edit header button.
  • Edit header button. Pressing this button opens an editor window. See the description of the built in text editor.
  • Problems table. List of problems the test contains. Use the buttons next to this table to manage the contents of this list.
  • Add button. Opens a file browser that you can use to insert problems into the test. Note that multiple problems can be selected by using the Shift and/or Ctrl keys.
  • Add again button. Adds one more instance of the highlighted problem to the test. This can be used to have more than one problems of the same type in the test. If the problem has variables then the concrete problem instances will be different.
  • Remove button. Removes the highlighted problem.
  • Edit button. Switches to the Problem data tab where you can edit the highlighted problem.
  • Up and Down buttons. Move the highlighted problem around in the table.

Saving the test

To save the test you created use the Save Test or Save Test As menu items from the File menu or press the corresponding toolbar buttons. When saving the test, the file name by default will be generated based on the text you entered in the Name field, but you may specify any other name for your saved files.

A note for advanced users. Similarly to problems, tests are also stored in multiple files. Beside the main test definition file (having a .tst extension) the header text is stored in a HTML file. If you edit these files, make sure you observe the naming conventions: the xxx.tst file must have an accompanying xxx_testheader.html file in the same directory.

Generating output

imgYou can generate printable test variations on the Test set generation tab of the application's main window. The controls of this window are the following:

  • Number of variations. Set the number of different test instances you need. The variations will have the same header and the same set of problem types (with different content if the problems are parametrized), (possibly) in different order. The number must be at least 1 and can be arbitrarily high.
  • Identifiers for test variations. Choose the format you would like to use as unique identifiers for test and solution variations. Make sure that the range of identifiers is greater than the number of versions you need (this is an issue only if you use a short custom list or you use letters and need more than 26 variations).
  • Each variation starts on new page check box. Whether a page break should be inserted in the generated output between variations.
  • Solution for each variation starts on new page check box. Whether a page break should be inserted in the generated output between solutions.
  • Label for variations. Your favorite term for variation. If for example you write "Group" here and use letters as identifiers then your tests will be labelled Group - A, Group - B, Group - C, etc in the header of the generated output.
  • Show version identifier check box. Whether version identifiers are included in the header of the output. Note that identifiers are always included in the solutions part.
  • Label for solutions. Your favorite term for solution. If for example you write "Answer Key" here and use letters as identifiers then your solutions will be labelled Answer Key - A, Answer Key - B, Answer Key - C, etc in the generated output.
  • Problem separator pattern. A line of symbols that separates problems within a test variation. May be left empty.
  • Shuffle problems check box. Whether the order of the problems should be randomized in each variation. The order of the problems in the tests and answer keys are always synchronized.
  • Seed for random number generation. Write any text here (and try to remember it) if you would like to have repeatable randomness in problem generation. This means that if you later run the generation process with the same test containing the same problems and use the same seed text then the result will be exactly the same. Different seeds will yield different output. If left empty then the output can not be exactly reproduced later.
  • Go! button. Do you really need this explained?
  • Output window. Status info and error messages appear here during output generation.

The result of the generation is an xxx.rtf and an xxx.html file, created in the same directory where the xxx.tst file (the one you used as the basis for the test generation) is located. You may use whichever you prefer for further editing and printing. Note that the html version contains horizontal lines instead of page breaks.

Using expressions

You can write expressions when editing the problem's text or solution, these will be evaluated to numeric values during output generation. The expressions can contain numbers (real and integer numbers), numeric variables, operators and functions. To evaluate an expression you must write it in an eval(...) block.

Assuming you have the $a and $b numeric variables defined for a problem, here are some valid examples for expressions:

  • eval($a) --replaced by the current value of $a in the output
  • eval($a+$b) --evaluates to the sum of $a and $b
  • eval(sin($a)) --example of a single argument function
  • eval(min($a,$b)) --example of a two-argument function
  • eval(sqrt($a*$b) - ($a+$b)/2) --difference of means

The supported operators (in precedence order):

  • ^ : exponentiation. 4^3 = 64
  • * : multiplication. 4*3 = 12
  • / : division. 4/3 = 1.333
  • % : modulo (works for real numbers as well). 4%3 = 1
  • + : addition. 4+3 = 7
  • - : subtraction. 4-3 = 1

The supported functions (in alphabetic order):

  • abs : absolute value. abs(-3) = 3
  • acos : arccosine.
  • asin : arcsine.
  • atan : arctangent.
  • binomial : choose k of n. binomial(4, 2) = 6
  • ceil : ceiling. ceil(4.3) = 5
  • cos : cosine.
  • exp : exp(3) = e^3
  • fact : factorial. fact(4) = 24
  • floor : floor. floor(4.3) = 4
  • log : natural logarithm. log(e) = 1
  • min : minimum. min(3,10) = 3
  • max : maximum. max(3,10) = 10.
  • neg : negation. neg(-2) = 2
  • rnd : random number generation. rnd(2.5) is a random real number in the range [0 ... 2.5)
  • sin : sine.
  • sqr : square. sqr(3) = 9
  • sqrt : square root. sqrt(25) = 5
  • tan : tangent.

 

Tutorial

In this tutorial you will learn how to create a problem, how to build a test from existing problems and how to generate printable worksheets. The end result of these steps can be found in the data/tutorial subdirectory of your TestMaker installation, however I recommend following these steps and reproducing these results on your own. In the following it is assumed that you have already installed the application and that your base directory for storing tests and problems is set to the data subdirectory of the application's installation directory. Please verify this by opening the Preferences dialog from the Edit menu, and change the setting if needed by selecting the data directory from the file browser.

Creating a problem

imgOur sample problem will be one from the domain of elementary geometry, the students have to know about similarity of triangles to solve this problem.

  1. Start TestMaker and switch to the Problem data tab. If TestMaker is already running and there is a problem displayed then you need to start creating a new one by choosing the File/New Problem menu item or pressing the corresponding toolbar button. In this case you may be requested to save your changes to the previously edited problem.

  2. Add a name to the problem, write 'Similar triangles' into the Name field.

  3. Press the Edit problem button to start the text editor.

  4. First we write the text of the problem. Write the following text into the editor's main window:

    On the figure below line CB is parallel to line ED. Let's denote the distance between points X and Y with d(X,Y). Your task is to determine d(E,D) knowing that

    d(A,B) = $ab,

    d(B,C) = $bc,

    d(B,D) = $bd.

  5. Experiment with the features of the editor, for example you can highlight the unknown term d(E,D) in the text and make it bold by pressing the appropriate toolbar button.

  6. Now we add the image to the problem's text. Navigate to the end of the text you wrote above, insert an empty line so that there will be a little space between text and image. Press the toolbar button with Mona Lisa's image on it and select the triangles.jpg file from the img folder and press Open. The image should appear in your editor window.

  7. Press Done to return to the Problem data screen. The Problem text area now contains the result of your work so far.

  8. The problem references 3 variables, we shall define these in the next steps. First define the variable representing the distance between points A and B. Press the Add button next to the Variables table, this opens the Variable definition screen. Write $ab to the name field (it can be any other name as long as it is used consistently in the problem definition text and later in the solution text). We would like the variable to be of type Integer and that it is distributed Randomly, these settings are selected by default in the Type and Distribution lists, so these don't have to be modified now. Define the variable's range by typing 20 in the Low field and 40 in the High field. Press Done to return to the problem definition screen.

  9. Repeat the above steps twice: first define a variable named $bc with the same settings as above, and one more with name $bd, again with the same settings. Now you have all 3 variables listed in the variables table, they look similar, only with different names.

  10. Now we are ready to formulate the solution to our problem. Triangles ABC and ADE are similar, so d(E,D) / d(C,B) = d(A,D) / d(A,B) = (d(A,B) + d(B,D)) / d (A,B). So the unknown d(E,D) can be written as d(C,B) * (d(A,B) + d(B,D)) / d (A,B). With our variables this is $bc*($ab+$bd)/$ab. Open the solution editor by pressing the Edit button next to the Solution text area. Write

    d(E,D) = eval($bc*($ab+$bd)/$ab)

    and press Done. Writing eval() around the expression means that the expression will be evaluated during output generation. The solution will contain the value of the expression calculated using the current values of the variables.

  11. To try whether the problem is correct press the Verify button. This opens a window showing a randomly generated instance of the problem with the corresponding solution. Press the Next> button a few times to see further instances, and then press Close.

  12. Some of the generated problem instances might be very easy to solve, for example if d(A,B) happens to be the same as d(B,D). To avoid this and also for our image to be somewhat to scale, we can set a constraint on our random variables. Let's request that d(A,B) is the largest of the 3 given parameters by typing this to the Constraint text field:

    $ab > $bc & $ab > $bd

    Press Verify to see that in all problem instances we get now, the constraint we set is met.

  13. Advanced Tip. You may want to be polite to your students and provide a problem with an integer solution. This can be achieved by modifying the constraint by checking that the solution is integer, ie, it is the same as its floor. Remember, our solution was expressed as $bc*($ab+$bd)/$ab so we must add this term to our constraint: floor($bc*($ab+$bd)/$ab) = $bc*($ab+$bd)/$ab. The whole constraint now looks like this:

    $ab > $bc & $ab > $bd & floor($bc*($ab+$bd)/$ab) = $bc*($ab+$bd)/$ab

    14. Press Verify to see this working.

  14. Now we are ready with the problem creation, save it from the File/Save Problem menu or by pressing the appropriate toolbar button. You may overwrite the default file name the program offers. You may choose the place the file will be saved to, any directory is fine as long as it is under the base directory defined in the Preferences.

Creating a test

Let's make our first test by combining the problem we created above with some other problems.

  1. Switch to the Test basic data tab of the application's main window. If there is already a test displayed then you need to start creating a new one by choosing the File/New Test menu item or pressing the corresponding toolbar button. In this case you may be requested to save your changes to the previously edited test.

  2. Give a name to your test by writing 'Sample test' or anything else of your choice into the Name field.

  3. Press the Edit header button and write a short heading into the text editor window. Press Done when you are ready.

  4. Now we shall add problems to our test. Press the Add button and locate the problem about similar triangles you have saved in the previous steps of this tutorial. Press Open. The problem appears in the Problems list.

  5. Add two more problems by pressing Add and selecting two .prb files from the 'data/math/Basic Math/whole numbers' directory. These also appear in the problem list.

  6. Select one of the problems in the Problems list and press Add again. Notice that another instance of the same problem appeared in the list.

  7. Use the Up and Down buttons to move the problems around in the list until you are satisfied with their ordering.

  8. Now you are ready with the test creation, save it from the File/Save Test menu or by pressing the appropriate toolbar button. You may overwrite the default file name the program offers. You may choose the place the file will be saved to, any directory is fine as long as it is under the base directory defined in the Preferences.

Generating printable tests

If you have followed the above steps you have everything needed to create printable worksheets. Let's do that now!

  1. Switch to the Test set generation tab of the application's main window.

  2. Set the number of variations you need. Write 2 in the appropriate field.

  3. Select your preferred format for variation identifiers. If you select custom list, then separate the values by a semicolon (;), for example write 'First;Second'.

  4. Define the pagination settings of the output, most likely you'll leave the two corresponding check boxes as they are by default.

  5. Define what you'd like to call the variations and solutions. For example write 'Group' in the first and 'Answer Key' in the second in the two corresponding text fields. Leave the Show version identifier checkbox checked.

  6. If you'd like to see a separator line between problems then write something like
    - * - * - * - * -
    to the Problem separator field. Alternatively you can leave this field empty.

  7. Leave the Shuffle problems checkbox unchecked.

  8. Leave the Seed for random number generation field empty.

  9. Press Go! The application generates the output in rtf and html format in the directory where the .tst file of your test is saved.

  10. Open the rtf document and observe that

    • There are two different versions of the test with the same type of problems.
    • In both versions there are two problems of the same type (the one you duplicated with the Add again button above), but the concrete problem is different.
    • The answer key at the end of the document contains the correct answer for all problems.

  11. Close the rtf document and regenerate the output with checking the Shuffle problems checkbox. Observe that the problems are in different order in the output but the answer keys are in sync with the problems in each version.

  12. Advanced tip. Close the rtf document and regenerate the output a couple of times, each time writing different data into the Seed for random number generation field. Observe that the output is exactly the same if the seed remains the same, but changes if the seed is changed.

That's it. By this point you have learnt all what you need to know about TestMaker to be able to create your worksheets. Have fun!

If you have feedback, please send an email to
This e-mail address is being protected from spambots. You need JavaScript enabled to view it