The VersionOne FitNesse Integration (V1Fit) imports FitNesse test
results into VersionOne, making the latest test results more visible
across the an organization. The results can be recorded at two
different levels of granularity—project and test. The aggregate
results for a suite of tests in FitNesse can be linked to a project
in VersionOne, so that everyone can see the health of their project
from a testing standpoint from within VersionOne. The results of an
individual FitNesse acceptance test can be linked to a Test in
VersionOne, so that development team members receive quick feedback
on which tests are passing and failing.
The integration point with FitNesse is the xml formatted results
file. When configuring the integration, teams must automate a
process that runs the FitNesse tests, transforms the results into
the FitNesse xml format, and stores the results in an output
directory. V1Fit watches the output directory for new files and
imports the results into VersionOne.
Once V1Fit has been installed, VersionOne users update the reference
field for test suites or tests with the path to the corresponding
suite or test page in FitNesse. A single VersionOne test suite can
be assigned to one or more VersionOne projects. The Test Run Trend
report shows the aggregated test results for the test suite which
was assigned to the project at the time of the test run. For
individiual tests, each test in VersionOne should have its own test
page in FitNesse. V1Fit considers a test passing if all of the
assertions for that test are passing. If a test fails on a story or
defect that is already closed, V1Fit can be configured to create a
defect in the current iteration, so the team is notified of the
failing test.
These installation instructions assume
FitNesse is already installed, configured, and working properly.
1. Determine Install Location
V1Fit can be installed on any server with network access to both
your VersionOne and FitNesse installations. Exact placement
should be determined by your internal software management
requirements. The integation server must meet the
System Requirements stated above.
FitNesse is a wiki that provides a powerful interface for
specifying and running acceptance tests, but it also includes
the capability to run tests from the command line. Although
FitServer implementations vary, they generally include a Test
Runner executable that can be run from the command line to
execute a suite of FitNesse tests.
The steps below show the commands that could be used to set up a
test runner for a FitNesse instance that is using .Net Fit 1.1
as its FitServer. They are intended as a process guide. The
exact commands and their parameters are dependent on your
FitServer implementation and environmental configuration.
Use
TestRunner.exe to run the tests and store the results in a text
file.
TestRunner -results Results.txt localhost 8080 SuiteAllTests *.dll
Use fitnesse.runner.FormattingOption to transform your tests
into XML.
java fitnesse.runner.FormattingOption Results.txt xml Results.xml localhost 8080 SuiteAllTests
The
FormattingOption class is part of the java
FitServer implementation and may not have been
ported to your FitServer implementation.
Automate test runs.
How you automate the test runs should be driven by
the requirements of the development team. Some teams
run them at regularly scheduled intervals, such as
nightly. Other teams run them after every successful
build. Teams with a large number of tests may do a
combination of both.
Download V1Fit using the link above and extract it into a folder
of your choice.
Open the file
VersionOne.ServiceHost.exe.config in a text editor, change the
configuration parameters listed below, and save your changes. For a
complete description of each configuration parameters, see
Hosted Service Details.
Table 1. V1Fit Configuration Parameters
|
Parameter | Description |
|---|
|
Watch | FitNesse test results output folder. Paths
are relative to the V1Fit installation folder. |
|
ApplicationUrl | VersionOne installation URL |
|
Username | User that will access VersionOne |
|
Password | Password for the username above |
|
IntegratedAuth* | VersionOne security setting |
* When using the integrated authentication security setting,
V1Fit will login to VersionOne using the Windows domain user
under which the executable is running if both the username and
password parameters are blank. If the username and password is
entered, you must specify the windows domain account in the form
"User@FullyQualifiedDomainName". If you are unsure what the
fully qualified domain name is, see the Domain name shown on the
'Computer Name' tab in the My Computer...Properties dialog.
Open up the command prompt,
navigate to your installation folder, and run the following command:
VersionOne.ServiceHost.exe
If you have configured your system properly, you should see
several [Info] messages followed by a [Startup] message.
Drop a
FitNesse results xml file into your watch folder. You should
see the following messages:
[Info] Starting Processing File: C:\PathToWatchFolder\WatchFolder\Results.xml
[Info] Finished Processing File: C:\PathToWatchFolder\WatchFolder\Results.xml
Since you haven’t referenced any tests or test suites, the
service didn’t make any updates in VersionOne. Let’s add a Test
Suite and make sure it gets updated.
Navigate to the page Administration > Projects > Test Suites in
VersionOne. Add a test suite and give it a reference to one of
your existing FitNesse suites.
In our example, the Test Suite reference is
“SuiteAllTestss.SuiteProjectA”. When processing test suites,
V1Fit concatenates the <rootpath>
and <relativepagename>
values from all tests in the results file and then aggregates
the results that meet the reference. In the
example file,
TestOne has a full path
of "SuiteAllTests.SuiteProjectA.TestOne", so it would be
included for test suites with a reference of "SuiteAllTests" or
"SuiteAllTests.SuiteProjectA".
Drop a new FitNesse results xml file that contains the results
for this suite into your watch folder. You should see the
following messages:
[Info] Starting Processing File: C:\PathToWatchFolder\WatchFolder\Results2.xml
[Info] Suite:
TestRun.Date = 1/1/2007 8:13:00 AM
TestRun.Failed = 0
TestRun.Elapsed = 0
TestRun.Description = Fit Test Suite
TestRun.Passed = 1
TestRun.NotRun = 0
TestRun.Name = SuiteAllTests.SuiteProjectA
TestRun.TestSuite = TestSuite:1069
[Info] Finished Processing File: C:\PathToWatchFolder\WatchFolder\Results2.xml
If you receive the [Info] Suite: message, then a Test Run was
inserted in VersionOne for your Test Suite.
Note that V1Fit only processes a file once, so changing the
contents of a file after it has been processed will have no
effect. If you want to reprocess a file, save it with a new
name.
Shut down the service host by pressing “Q” in the console
window.
6. Install as a Windows Service
Run the
following command from the console window:
VersionOne.ServiceHost.exe --install
This command installs the service so it will will run under the account
NT AUTHORITY\Local Service. Local Service must be given access
privileges to the directory where the V1Fit was installed so it can
store its state and write to log files. Follow the steps below to change
the security on the installation directory:
Right click the
installation folder from Windows Explorer.
Select
"properties".
Select the "Security" tab.
Click the "Add" button.
Enter "Local Service" and
click "OK".
Click the "Allow" checkbox for the
"Full Control" row .
Click "OK" to save the
changes.
For more information on how the Windows Service is configured see
Service Host Configuration.
V1Fit utilizes three hosted services within the
V1 Service Host framework in order to perform the integration.
Each service performs a specific function and is configured by
changing values in VersionOne.ServiceHost.exe.config.
The FitReaderService watches for new result files, interprets
FIT test results, and publishes the results to the
TestWriterService. FitReaderService processes XML files in the
schema generated by
fitnesse.runner.FormattingOption.
Table 2. FitReaderService Configuration Parameters
|
Parameter | Description |
|---|
|
Watch | The directory to poll for FIT test results
files. The account the service is running as (i.e. Log
On As setting) must have read rights to the directory
configured in the Watch element. |
| Filter |
The file filter to use when polling for FIT test
results files; it may contain any valid search
pattern compatible with
Directory.GetFiles() |
Example 1.
<FitReaderService class="VersionOne.ServiceHost.TestServices.Fit.FitReaderService, VersionOne.ServiceHost.TestServices">
<Watch>.\FitTestResults</Watch>
<Filter>*.xml</Filter>
</FitReaderService>
The FitReaderServiceTimer triggers the FitReaderService every N
milliseconds.
Table 3. FitReaderServiceTimer Configuration Parameters
|
Parameter | Description |
|---|
|
Interval | The interval, in milliseconds, between
events raised (via publishing the IntervalSync) |
|
PublishClass | Contains the class and its'
assembly that that the TimePublisherService will
publish at every interval. For FIT integration using
the FitServiceReader, it must be set as in the
example below. |
Example 2.
<FitReaderServiceTimer class="VersionOne.ServiceHost.Core.Services.TimePublisherService, VersionOne.ServiceHost.Core">
<Interval>60000</Interval>
<PublishClass>VersionOne.ServiceHost.TestServices.Fit.FitReaderService+IntervalSync, VersionOne.ServiceHost.TestServices</PublishClass>
</FitReaderServiceTimer>
The TestWriterService listens for test run and suite results
published by the FitReaderService and updates VersionOne with
the results.
Table 4. TestWriterService Configuration Parameters
|
Parameter | Description |
|---|
|
ApplicationUrl | The URL for your VersionOne
application |
| Username | The
VersionOne account username, if using native
VersionOne security. If using Windows Integrated
Security, leave empty. |
| Password |
The VersionOne account password, if using native
VersionOne security. If using Windows Integrated
Security, leave empty. |
| APIVersion |
The minimum application version required for this
hosted service. |
| IntegratedAuth* |
False if using VersionOne native security, true is
using Windows Integrated Security. If VersionOne is
configured to use Windows Integrated Security, the
account the service is running as must be a
configured VersionOne user with a project role of
Team Member or higher. Also, Username and Password
should be empty if IntegratedAuth is true.
|
| PassedOid | The TestStatus
asset Oid in VersionOne that indicates test a test
succeeded |
| FailedOid | The
TestStatus asset Oid in VersionOne that indicates
that a test failed |
|
TestReferenceAttribute | The cross-reference
used to associate tests in VersionOne to test in the
external test system ('Number' is the default,
'Reference' another option) |
|
ChangeComment | The change comment put into
history for any modifications the service makes in
VersionOne |
| DescriptionSuffix |
A suffix for the description of any defects that the
service creates; Embedded Rich Text (HTML) is valid
in this suffix |
| CreateDefect |
Indicates whether the TestWriterService should
create defects when automated tests fail for closed
tests in VersionOne. Valid values are:
All: creates a 'failed test' defect for any closed tests in the project
CurrentIteration: only creates a 'failed test' defect for closed tests within the current iteration
None: never creates defects
|
Example 3.
<TestWriterService class="VersionOne.ServiceHost.TestServices.TestWriterService, VersionOne.ServiceHost.TestServices">
<Settings>
<ApplicationUrl>http://localhost/VersionOne/</ApplicationUrl>
<Username>admin</Username>
<Password>admin</Password>
<APIVersion>7.1.0.0</APIVersion>
<IntegratedAuth>false</IntegratedAuth>
</Settings>
<PassedOid>TestStatus:129</PassedOid>
<FailedOid>TestStatus:155</FailedOid>
<TestReferenceAttribute>Number</TestReferenceAttribute>
<ChangeComment>Updated by VersionOne.ServiceHost</ChangeComment>
<!-- Embedded Rich Text (HTML) is valid in this suffix -->
<DescriptionSuffix>Check the external test system for more details.</DescriptionSuffix>
<CreateDefect>All</CreateDefect>
</TestWriterService>
Fit XML Results File Example
The xml content
below can be saved to a file and used for testing purposes.
<?xml version="1.0"?>
<testResults>
<host>localhost:8080</host>
<rootPath>SuiteAllTests</rootPath>
<result>
<relativePageName>SuiteProjectA.TestOne</relativePageName>
<content></content>
<counts>
<right>1</right>
<wrong>0</wrong>
<ignores>0</ignores>
<exceptions>0</exceptions>
</counts>
</result>
<finalCounts>
<right>1</right>
<wrong>0</wrong>
<ignores>0</ignores>
<exceptions>0</exceptions>
</finalCounts>
</testResults>