AntSys -- lightweight and powerful system test framwork

AntSys

Why write AntSys

I was looking for a well-know and easy to use open source tool for our system testing.
My requirements are:

Test Model, provides the concept of test suite, test case and test result

Test Method: launch command line application, check the outputs. which means should be easy to
* configure parameters
* setup environment variables
* able to redirect stand input, output, and error
* get back the process exit code

Test Assertions: should built-in with the following assertions:
* check if output file is created as expect
* check the content of output file against expected file.
* check process exit code
* check the content of system.ouput.
* check the content of system.error.

Test Result:It should generate XML and HTML report

Test Language/API: the test language should provide rich API to help users write test case:
such as create/delete directory, copy files, and even xml processing.

Extensibility: the test lanauge can be extended by end users.

Although the requirements are quite generic, I can't find a well-known OSS project. Most companies are
using their own home-grow system test framework.
Let's create a OSS tool to solve this problem then!

What is AntSys

AntSys is a lightweight and powerful system test framwork built on top of Ant.
It is designed to be easy to work and user can create system/function test cases in xml file without writing single line of code.

AntSys Tasks

AntSys provides the following Ant tasks to write a system test:

tastsuite: is used to aggregate tests into groups of test cases that should be run together.

testcase: is a set of conditions under which a tester will determine if a requirement or use case upon an application is partially or fully satisfied.

assertfileexist: check if the output file is exist as expected.

assertfileequal: check the content of output file is the same as expected test result.

assertfilegrep: check the content of output file contains expected string defined in regular expression

assertpropertyequals: check the ant property against expected value. This is offened used after 'exec' task to check the process exit code.

Example test script

Here is one example of test script to test echo command.

<project name="antsys-systemtest" basedir="." default="systemtest">
<target name="systemtest">
<!-- this test suite is using AntSys to test echo shell commands -->
<testsuite name="CommandTestSuite" failonerror="false" todir="./report" htmlreport="true">
<!-- test echo command output -->
<testcase name="echo_test">
<!-- create working dir for testing -->
<mkdir dir="work"/>
<!-- use echo to create a test file -->
<exec vmlauncher="false" executable="echo" output="work/test.dat">
<arg value="testmessage"/>
</exec>
<!-- check the file exists or not -->
<assertfileexist file="./work/test.dat" errormsg="should generate test.dat" />
<assertfilegrep file="./work/test.dat" regexp="^test" errormsg="should start with test" />
<assertfileequal expectedfile="./work/test.dat" actualfile="./work/test.dat"/>
<delete dir="work"/>
</testcase>
</target>
</project>