Exporting and Importing test specifications
(Professional version only)
To enhance testIDEA with functionality offered by other
applications, we have several possibilities for exporting and
importing test specifications. The usual procedure involves
creation of the base test specification in testIDEA, exporting it,
creating variants in external tool, and then importing of the
generated tests. Then we run tests with testIDEA.
Available export formats can be seen in the export dialog:
They are described in the following two sections:
The following exports are currently supported:
Spreadsheets and CSV
When we want to create a set of test specifications for the same
function, but with different values of parameters, editing is
easier with tools like Excel or Open Office Calc. For this purpose
we can use testIDEA's export and import functionality. The
general procedure is the following:
After import we can execute the tests. If we want to modify the
values of derived tests parameters, we don not have to export the
test specification again, but simply change values in the exported
file and import them again.
- Create base test specification in testIDEA. It should
specify the function to be tested and any other items, which
define the test. For example, the most common such items are
function parameters and expected return values. testIDEA
exports the following items, if defined:
Please note again, that the above items are exported only if
they are defined in the base test specification. For example,
if the base test specification does not specify HIL
parameters, HIL section will not be exported.
The base test specification must have test ID defined,
otherwise testIDEA reports an error.
- Function parameters
- Expected values. The expected expression is split to
two parts where the split point is the last operator. The
first part is written in the second row, while the second
part is added to the first one on import. For example, the
'rv == 3' is split to
'3'. If we enter values
'6' in the column
'rv ==' in the exported spread sheet,
derived tests will have expected expressions
'rv == '5', and
- Initialized variables
- Assignments for stubs
- Statistics requirements for coverage
- Profiler timing and hits requirements
- HIL parameters
- winIDEA options
- Select the test specification(s) in testIDEA, then execute
File | Export. All selected test
specifications get exported to the selected file. In XLS file
each test specification gets its own sheet, while in CSV file
they are separated by lines with test IDs. An example of
exported test specification is shown below:
- Edit the exported file in some third party tool, like Excel,
Calc, ...). The following rules must be taken into account while
Empty rows in CSV files are ignored, rows which start with '#'
character are treated as comments and are also ignored.
- Do not edit the first row of each test
specification. It defines test sections, and the naming and
order are important here.
Edit the second row with identifier names only when
renaming them, or modifying expected expression. It is
preferred to rename items in testIDEA, and then
export the test specification again.
The third row contains values of the base test
specification. It is there for information only - it
is not parsed on import, so any changes done
here are ignored.
The fourth and next rows should contain values to be
used for derived test specifications. If we enter
formulas, the calculated values will be used.
An example of edited test specification is shown below:
Save the file and import it in testIDEA. If we want to
import derived test specifications only for some base test
specifications, we can select those base test specifications
and then check the 'Import data only yo selected test
specifications' check box in the import dialog.
All existing derived test specifications of the selected test
specifications are deleted on import!
CTE XL Professional
CTE XL Professional among other features provides graphical view of
test cases with variations of test inputs, and enables rule based
creation of tests cases. On the other hand testIDEA provides
execution of test cases with analyzer measurements and HIL.
To use the benefits of both applications, we can use testIDEA's
export/import functionality and share the data in both ways.
Usual workflow includes the following steps:
Note: When importing CTE XL file to testIDEA, information about
rules is lost.
- Create base test specification in testIDEA. Define all sections,
which are needed for testing.
- (optional) Create few derived test specifications, which contain
typical values of test inputs (CTE XL classes)
- Select the base test specification in testIDEA and export it to
CTE format with command
File | Export.
- Open the file in CTE and generate additional test cases. Save
- Import the file to testIDEA and run tests.
- (optional) Export tests with results and view results in
CTE XL Classification Tree as created by testIDEA
Information from testIDEA test specifications is divided into two
parts, when exported to CTE XL. All the data, which define the
execution of tests cases (test inputs), is shown in the CTE XL
Classification Tree. Compositions and classifications in this tree
are created according to data in the selected test specification,
while classes are created according to values found in the derived
The Classification Tree generated by testIDEA includes:
If any of these sections is empty in testIDEA, it is not exported.
- function parameters
- test local and target global variables
- stub assignments and script function parameters
- HIL outputs
- script extension functions parameters
Other parts of test specification, for example expected results,
coverage and profiler sections are exported as properties of test
cases in CTE. Each testIDEA section creates one tab, which contains
test specification data in YAML format. You are free to modify these
data, as long as the syntax is valid.
Test cases and test groups
CTE XL supports several forms of test specifications and their
grouping, while testIDEA contains only one. This section describes
mapping of these data between testIDEA and CTE XL:
|CTE XL Type||testIDEA Type|
|Test Group||Base test specification. Because Test
Group in CTE XL can not be executed, it is recommended to
configure base test specifications as abstract ones. These test
cases do not have marks associated so them, so in CTE XL we can
modify them only via properties. |
|Test Case||Derived test specification.|
|Test Sequence||Not supported by export/import.|
|Test Step||Not supported by export/import.|
Editing the exported Classification Tree in CTE XL
When editing the composition tree in CTE XL, which we plan to import
into testIDEA, we have to be aware that the structure and names of
compositions are fixed. They may be missing, but when present they
must be one of:
Classifications and classes may be modified at will.
- the root element has a name composed of test ID and function name
- params - parameters of function under test
- variables - contains the Variables section of testIDEA
- stubs - contains stubs. Its children are compositions with
names of stubbed functions, which may have two
compositions. The assignments composition contains testIDEA
stub assignments expressions, while the scriptParams
composition contains stub script extension function parameters.
- HIL - contains HIL output parameters
- scripts - this composition may have up to four children
compositions, with names of script extension functions.
If test case does not have some section specified, it is merged from
parent test group on import. For example, if we create a new test
case in CTE, it does not have variables section defined. On import,
testIDEA first copies the variables section from the base test group
and then executes assignments specified with marks in CTE.
Each function parameter is shown as one CTE XL class in the
Classification Tree. Since parameters must be passed to function in
proper order, each class has a tag
determines position of parameter in parameters list. If position
of parameter changes, we have to modify this value. Names of
parameters are not used on import and can be freely modified.
Variable and HIL classes
For these classes names of CTE classes are used on import.
When tests are executed in testIDEA, test results are also exported
to CTE XL format file. To view them in CTE XL, right click in the
Test Case Tree view, and select option
Show Columns | Test Result.