Exporting and Importing test specifications

(Professional version only)

To enhance testIDEA with functionality offered by other applications, we have several possibilities for exporting and importing test specifications. The usual procedure involves creation of the base test specification in testIDEA, exporting it, creating variants in external tool, and then importing of the generated tests. Then we run tests with testIDEA.

Available export formats can be seen in the export dialog:

They are described in the following two sections: The following exports are currently supported:

Spreadsheets and CSV

When we want to create a set of test specifications for the same function, but with different values of parameters, editing is easier with tools like Excel or Open Office Calc. For this purpose we can use testIDEA's export and import functionality. The general procedure is the following:

  1. Create base test specification in testIDEA. It should specify the function to be tested and any other items, which define the test. For example, the most common such items are function parameters and expected return values. testIDEA exports the following items, if defined:
    • Function parameters
    • Expected values. The expected expression is split to two parts where the split point is the last operator. The first part is written in the second row, while the second part is added to the first one on import. For example, the expression 'rv == 3' is split to 'rv ==' and '3'. If we enter values '4', '5', and '6' in the column below 'rv ==' in the exported spread sheet, derived tests will have expected expressions 'rv == 4', 'rv == '5', and 'rv == 6'.
    • Initialized variables
    • Assignments for stubs
    • Statistics requirements for coverage
    • Profiler timing and hits requirements
    • HIL parameters
    • winIDEA options
    Please note again, that the above items are exported only if they are defined in the base test specification. For example, if the base test specification does not specify HIL parameters, HIL section will not be exported. The base test specification must have test ID defined, otherwise testIDEA reports an error.
  2. Select the test specification(s) in testIDEA, then execute command File | Export. All selected test specifications get exported to the selected file. In XLS file each test specification gets its own sheet, while in CSV file they are separated by lines with test IDs. An example of exported test specification is shown below:

  3. Edit the exported file in some third party tool, like Excel, Calc, ...). The following rules must be taken into account while editing:
    • Do not edit the first row of each test specification. It defines test sections, and the naming and order are important here.
    • Edit the second row with identifier names only when renaming them, or modifying expected expression. It is preferred to rename items in testIDEA, and then export the test specification again.
    • The third row contains values of the base test specification. It is there for information only - it is not parsed on import, so any changes done here are ignored.
    • The fourth and next rows should contain values to be used for derived test specifications. If we enter formulas, the calculated values will be used.
    Empty rows in CSV files are ignored, rows which start with '#' character are treated as comments and are also ignored.
  4. An example of edited test specification is shown below:

  5. Save the file and import it in testIDEA. If we want to import derived test specifications only for some base test specifications, we can select those base test specifications and then check the 'Import data only yo selected test specifications' check box in the import dialog. All existing derived test specifications of the selected test specifications are deleted on import!
After import we can execute the tests. If we want to modify the values of derived tests parameters, we don not have to export the test specification again, but simply change values in the exported file and import them again.

CTE XL Professional

CTE XL Professional among other features provides graphical view of test cases with variations of test inputs, and enables rule based creation of tests cases. On the other hand testIDEA provides execution of test cases with analyzer measurements and HIL.
To use the benefits of both applications, we can use testIDEA's export/import functionality and share the data in both ways.

Usual workflow includes the following steps:

Note: When importing CTE XL file to testIDEA, information about rules is lost.

CTE XL Classification Tree as created by testIDEA

Information from testIDEA test specifications is divided into two parts, when exported to CTE XL. All the data, which define the execution of tests cases (test inputs), is shown in the CTE XL Classification Tree. Compositions and classifications in this tree are created according to data in the selected test specification, while classes are created according to values found in the derived test specifications.

The Classification Tree generated by testIDEA includes: If any of these sections is empty in testIDEA, it is not exported.

Other parts of test specification, for example expected results, coverage and profiler sections are exported as properties of test cases in CTE. Each testIDEA section creates one tab, which contains test specification data in YAML format. You are free to modify these data, as long as the syntax is valid.

Test cases and test groups

CTE XL supports several forms of test specifications and their grouping, while testIDEA contains only one. This section describes mapping of these data between testIDEA and CTE XL:

CTE XL TypetestIDEA Type
Test GroupBase test specification. Because Test Group in CTE XL can not be executed, it is recommended to configure base test specifications as abstract ones. These test cases do not have marks associated so them, so in CTE XL we can modify them only via properties.
Test CaseDerived test specification.
Test SequenceNot supported by export/import.
Test StepNot supported by export/import.

Editing the exported Classification Tree in CTE XL

When editing the composition tree in CTE XL, which we plan to import into testIDEA, we have to be aware that the structure and names of compositions are fixed. They may be missing, but when present they must be one of: Classifications and classes may be modified at will.

If test case does not have some section specified, it is merged from parent test group on import. For example, if we create a new test case in CTE, it does not have variables section defined. On import, testIDEA first copies the variables section from the base test group and then executes assignments specified with marks in CTE.

Function parameters

Each function parameter is shown as one CTE XL class in the Classification Tree. Since parameters must be passed to function in proper order, each class has a tag paramIdx, which determines position of parameter in parameters list. If position of parameter changes, we have to modify this value. Names of parameters are not used on import and can be freely modified.

Variable and HIL classes

For these classes names of CTE classes are used on import.

Test results

When tests are executed in testIDEA, test results are also exported to CTE XL format file. To view them in CTE XL, right click in the Test Case Tree view, and select option Show Columns | Test Result.