Skip to content

Unit Testing

Jump to bottom Edit New page
Eric Domke edited this page Mar 23, 2016 · 5 revisions

Motivating Principles

  • Since you are likely already developing AML scripts in Innovator Admin, you should be able to perform tests here as well. The benefit to this is that you don't have to copy and paste AML back and forth between Innovator Admin and another environment (e.g. Visual Studio)
  • It should be easy to maintain tests. This is achieved because the result of running tests is itself a unit test.
  • When matching results to expected, everything should be verified and full details should be provided for any errors
  • It should be easy to compare the actual results to the expected results

Many of these ideas were influenced by the article SQL Server Unit Testing by Alex Kuznetsov

Developing Unit Tests

To develop and run a unit test in Innovator Admin, simply change your SOAP action from its typical value (e.g. ApplyAML, ApplyItem, etc.) to the special value of > Run Unit Tests

Test Script Structure

A basic, commented test structure is given below. In general, the flow of a test script is

  1. Any initialization steps (running AML, defining parameters, etc.)
  2. A listing of named tests to perform. Tests generally start with an AML statement and consist of one or more subsequent AML statements, AssertMatch statements, parameter definitions, etc.
  3. Any cleanup steps to perform

Note: When building tests, it is not necessary to fill out the <Expected> tag for your assertions, when you run your tests the first time, the script that is output as the result will contain these values defined.

<TestSuite>
  <!-- Any initialization steps which should be performed -->
  <Init></Init>
  <!-- The actual tests -->
  <Tests>
    <!-- An actual test -->
    <Test name='Verify Part ItemType'>
      <!-- AML to perform (item tag, sql, DownloadFile, etc.) -->
      <Item type='ItemType' action='get' id='4F1AC04A2B484F3ABA4E20DB63808A88' select='is_versionable,history_template'>
      </Item>
      <!-- Equality assertion based on the specified XPath -->
      <AssertMatch match='(//Item)[1]'>
        <Expected>
          <Item type='ItemType' typeId='450906E86E304F55A34B3C0D65C097EA'>
            <history_template keyed_name='Default' type='History Template'>3BC16EF9E52B4F9792AB76BCE0492F29</history_template>
            <is_versionable>1</is_versionable>
          </Item>
        </Expected>
      </AssertMatch>
      <!-- Save off a parameter from the result using XPath -->
      <Param name='HistId' select='(//Item)[1]/history_template'></Param>
    </Test>
    <!-- Another test -->
    <Test name='Verify History Template'>
      <!-- Another query -->
      <Item type='History Template' action='get' id='@HistId' select='name'>
      </Item>
      <!-- Another assertion -->
      <AssertMatch match='(//Item)[1]'>
        <Expected>
          <Item type='History Template' typeId='BA053C68D62E4E5293039323F10E116E'>
            <name>Default</name>
          </Item>
        </Expected>
      </AssertMatch>
    </Test>
  </Tests>
  <!-- Any cleanup actions -->
  <Cleanup></Cleanup>
</TestSuite>

Running Unit Tests

A possible output of running the above unit test script is included below. The results are nearly identical to the input with a few key differences

  • A sessionId attribute is included with the suite to add additional randomness if passwords are included in the test (see below for details on passwords)
  • A <Results> section is placed at the top of the script indicating for each test whether it passed or failed, how many milliseconds it took to complete and the date/time the test was started (in the machine's local timezone). This section is ignored if included with a test that you run.
  • An <Out> section indicates the final values of any parameters for convenience if you would like to copy these value into the <Init> section of another test script
  • Any assertions which were missing an <Expected> block will have that block filled in. (The exception to that would be if an error occurred. In this case, the error will be stored in an <Actual> block so that an error is not accidentally specified as an <Expected> value. With that said, an error can be listed as expected with the correct XPath (e.g. (//faultstring)[1]).
<TestSuite sessionId='Jsr/zXSGzieyBlrzi1+PwUl4R0zHvECO'>
  <Results>
    <Run name='Verify Part ItemType' result='Pass' elapsedMs='41' start='2016-03-23T08:25:10' />
    <Run name='Verify History Template' result='Pass' elapsedMs='22' start='2016-03-23T08:25:10' />
  </Results>
  <Out>
    <Param name='HistId'>3BC16EF9E52B4F9792AB76BCE0492F29</Param>
  </Out>
  <!-- ...
     SAME AS ABOVE
  ... -->
</TestSuite>

Passwords and Unit Tests

It is possible to embed passwords in unit tests if you want to login as a particular user (e.g. in order to verify permissions). The basic structure of such a test is as follows:

<Test name='Part Not Discoverable'>
  <Login type='Explicit' username='USER' password='PASSWORD' />
  <Item action='get' type='Part' select='item_number'>
    <item_number>1234</item_number>
  </Item>
  <AssertMatch match='(//faultstring)[1]'>
    <Expected>No items of type Part found.</Expected>
  </AssertMatch>
  <Logout />
</Test>

After the test is run, it will look something like the following. Notice that the password has been encrypted and encoded. The encryption is based on the network domain and controller of the computer along with the sessionId placed in the TestSuite.

<TestSuite sessionId='Jsr/zXSGzieyBlrzi1+PwUl4R0zHvECO'>
  <Tests>
    <Test name='Part Not Discoverable'>
      <Login type='Explicit' username='USER' password='pw:y+8w0F1nXZuGF+PrXylHg3==' />
      <Item action='get' type='Part' select='item_number'>
        <item_number>1234</item_number>
      </Item>
      <AssertMatch match='(//faultstring)[1]'>
        <Expected>No items of type Part found.</Expected>
      </AssertMatch>
      <Logout />
    </Test>
 </Tests>
</TestSuite>

Another Example

Another example unit test for further inspiration is this test which verifies basic vault upload/download functionality

<TestSuite>
  <Tests>
    <Test name='Upload/Download'>
      <Param name='newId' select='x:NewId()'> />
      <Item type='File' id='@newId' action='add'>
        <actual_data encoding='none'>This is a new file.</actual_data>
        <actual_filename>C:\Test.txt</actual_filename>
      </Item>
      <DownloadFile>
        <Item type='File' id='@newId' action='get' />
      </DownloadFile>
      <AssertMatch match='(//Item)[1]'>
        <Expected>VGhpcyBpcyBhIG5ldyBmaWxlLg==</Expected>
      </AssertMatch>
    </Test>
  </Tests>
</TestSuite>
Add a custom footer

Home

Clone this wiki locally