BizUnit - Framework for Automated Testing of Distributed Systems

New Post: Xml Validation - Check if Element Exists, Not Exists

tolandmike — Tue, 22 Sep 2009 15:38:46 GMT

I'm trying to create XML validation steps that check if an element exists or doesn't exist. But, looking at the documentation around BizUnit.XmlValidationStep, this step will only allow me to query and validation on the expected value of an existing element. Is that right?

Created Issue: Source Code Project Types Incompatible in Visual Studio 2008 [10671]

tolandmike — Tue, 22 Sep 2009 14:22:57 GMT

I can't open any of the source code projects nor any of the sample projects in VS 2008.

Created Issue: DBQueryStep - Validate Multiple Rows not working [10619]

sharathk — Fri, 11 Sep 2009 00:17:34 GMT

DBQuery Step doesnt work for multiple row validation

Row to validate is assigned the first row from DataSet in foreach loop

Source: DBQueryStep.cs
public void Execute(Context context)
{
\\\
bamDBRow = ds.Tables[0].Rows[0];
\\\
}

Need to assign correct row from the DataSet

New Post: Testing webservice with Bizunit

mandava29 — Tue, 08 Sep 2009 20:14:03 GMT

I exposed my orchestration as a webservice and trying to test the webservice using bizunit SOAPHTTPRequestResponseStep.

here is my config xml

<TestStep assemblyPath="" typeName="BizUnit.SOAPHTTPRequestResponseStep">
<WebServiceWSDLURL>http://localhost/KForce.Lookup_Proxy/KForce_Lookup_Orchestration_1_SOAPRequestRespPort.asmx?wsdl</WebServiceWSDLURL>
<ServiceName>KForce_Lookup_Orchestration_1_SOAPRequestRespPort</ServiceName>
<WebMethod>Operation_1</WebMethod>
<InputMessageTypeName>KForce.Lookup.RequestSOAP_0x30f8b4a5.xsd</InputMessageTypeName>
<MessagePayload>C:\Development\Kforcelookup.xml</MessagePayload>

    <ValidationStep assemblyPath="" typeName="BizUnit.XmlValidationStep">
        <XmlSchemaPath>C:\Development\KForce.Lookup\KForce.Lookup\ResponseSOAP.xsd</XmlSchemaPath>
        <XmlSchemaNameSpace>http://KForce.Lookup.ResponseSOAP</XmlSchemaNameSpace>
        <XPathList>
            <XPathValidation query="/*[local-name()='Root' and namespace-uri()='http://KForce.Lookup.ResponseSOAP']/*[local-name()='wtehours' and namespace-uri()='']">2009-01-07T00:00:00</XPathValidation>
        </XPathList>
    </ValidationStep>
</TestStep>

</TestExecution>

</TestCleanup>

</TestCase>

When I run it using nunit gui it gives me the following error

System.TypeLoadException Could not load type Bizunit.Proxy.KForce.Lookup.RequestSOAP_0x30f8b4a5.xsd from assemblt TestFrameWorkProxy0

<InputMessageTypeName>what should I be passing here</InputMessageTypeName>

Thanks

Anil

New Post: Bizunit Null reference exception when using with Visual studio 2005 professional

santoshbenjamin — Sat, 05 Sep 2009 16:58:40 GMT

Hi Anil,

Any chance you could post the code you have written? or drop me an email via the contact on codeplex and send it offline and i can try to help. Its difficult to say what the problem could be without looking at the test you've written

Thanks,

benjy

New Post: Bizunit Null reference exception when using with Visual studio 2005 professional

mandava29 — Fri, 04 Sep 2009 17:16:13 GMT

Hi Santosh,

It not working even in visual studio 2008 professional as well.vs 2008 prof had the test framework.

I am trying to execute bizunit test using nunit the error it gives me id null reference exception :object not set to instance of the object

thanks

anil

New Post: Bizunit Null reference exception when using with Visual studio 2005 professional

mandava29 — Fri, 04 Sep 2009 14:42:11 GMT

Hi Santosh,

It not working even in visual studio 2008 professional as well.vs 2008 prof had the test framework.

I am trying to execute bizunit test using nunit the error it gives me id null reference exception :object not set to instance of the object

Thanks

Anil

New Post: Bizunit Null reference exception when using with Visual studio 2005 professional

santoshbenjamin — Fri, 04 Sep 2009 10:31:41 GMT

Hi Anil,

BizUnit can be used with NUnit, MbUnit, MStest etc. However some framework must exist. it cannot run in an ordinary class library project.

Vs2005 Professional does not support MStest projects, but VS2008 profeessional (and of course team edition) does.

HTH,

Benjy

New Post: Bizunit Null reference exception when using with Visual studio 2005 professional

mandava29 — Thu, 03 Sep 2009 20:05:42 GMT

hi,

I created a class library project since I had not visual studio team system.

I added the test method and I tried to test a sample orchestration it gives me a null reference exception.

Does bizunit work with VS 2005 professional edition.

Someone please point me in the right direction.

The same code worked in VS 2008 team edition but not working in VS2005 professional.

can someone give me a work around

thanks

Anil

New Post: BizUnit.DotNetObjectInvokerStep

sbeamish — Fri, 21 Aug 2009 18:14:21 GMT

How do I store the output from DotNetObjectInvokerStep into a context?

I am attempting to use the created GUID and use it throughout the test...

<TypeName>System.Guid</TypeName>

<MethodToInvoke>NewGuid</MethodToInvoke>

</TestStep>

<TypeName>System.Guid</TypeName>

<MethodToInvoke>NewGuid</MethodToInvoke>

</TestStep>

New Post: Message Context Not Supported

kevinsmi — Sun, 26 Jul 2009 06:29:15 GMT

Hi,

It kind of depends what your scenario is, if you want to check the message context after a message has flowed thru a pipeline the bes tway to do that is to extend Bram's pipeline step to write the BizTalk msg ctx to the BizUnit msg ctx, then you can use a validation step to check it, you could kind of do this with the existing step though its a little more fiddly since the ctx is written to disc. If you want to check the msg ctx after an orch has been executed thats a little more tricky. would be good to know more about the scenario you want to test?

thanks,

Kevin

New Post: BizUnit and EDI testing

bveldhoen — Thu, 23 Jul 2009 11:43:49 GMT

puneetjain2710,

BizUnit 3.1 now contains ExecuteReceivePipelineStep and ExecuteSendPipelineStep, which can be used to test your custom pipelines. See the test project in the BizUnit test solution for a sample.

Bram.

New Post: Message Context Not Supported

bveldhoen — Thu, 23 Jul 2009 11:38:27 GMT

yieldvs,

The ExecuteReceivePipelineStep and ExecuteSendPipelineStep have functionality that supports creating, writing to disk, and inspecting the message context. Maybe this can be of assistence?

Bram.

Updated Wiki: Home

kevinsmi — Wed, 08 Jul 2009 20:34:00 GMT

BizUnit

The adoption of an automated testing strategy is fundamental in reducing the risk associated with software development projects, it is key to ensuring that you deliver high quality software. Often, the overhead associated with developing automated tests is seen as excessive and a reason to not adopt automated testing.

BizUnit enables automated tests to be rapidly developed. BizUnit is a flexible and extensible declarative test framework targeted that rapidly enables the automated testing of distributed systems, for example it is widely used to test BizTalk solutions. BizUnit is fully extensible. Its approach is to enable test cases to be constructed from generic reusable test steps, test cases are defined in XML which allows them to be auto-generated and also enables the ‘fixing up’ of Url’s for different environments, e.g. test, staging and production environments. Defining test cases in XML enables test cases to be auto-generated.

BizUnit v3.0 - RC1 - now available, new features include a BizUnit object, Excel test case runner, a new UI automation test step library, and pluggable logging

BizUnit is a framework and as such does not have any dependency on either NUnit of VS Unit Testing, either of these make a great way to drive BizUnit test cases, though equally you could write custom code to do the same.

Test Case Format

A test case can be represented as an Xml document or using the BizUnit object model, it is made up of three stages, test setup, test execution and test cleanup, the cleanup stage is always executed (even if the main execution stage fails) and intended to leave the platform in the same state that it started.
Each stage may consist of zero or more test steps, test steps are in general autonomous, state can be flowed between them if required using the ‘context’ object that is passed to each test step by the framework.
BizUnit also has the notion of TestGroupSetup and TestGroupTearDown, these are test cases that are executed at the beginning and end of a suite of unit tests.

The diagram below illustrates the format of a test case.

In addition to test steps, BizUnit has the notion of validation steps and context loader steps. These can be thought of as sub-steps and can in general be independantly executed from any test step. For example, an MSMQ-read step might be used to read and validate both Xml and Flat File data from a queue, the same step can be used with both the RegExValidationStep and the XmlValidationStep to validate the data read.

A test step within a test case can be marked with the attribute - runConcurrently which causes subsequent test steps to be started before it has completed. In addition test steps maybe marked with the attribute - failOnError, setting it to false cause BizUnit to ignore a failure of that test step, this is particularly useful for the setup and cleanup stages of test cases.

Lets look at an Example Scenario...

BizUnit takes a black box approach to testing solutions, lets look at the BizTalk scenario below, a BizTalk solution receives a request-response message over HTTP, the message is routed to an Orchestration which, sends a message to MSMQ and another to a FILE drop, the Orchestration waits for a FILE to be received, after which the Orchestration sends the response back to the waiting HTTP client. The solution also uses BAM, writing business data to the BAM database.

In order to test this scenario, a BizUnit test case is defined that has 5 test steps:

The HttpRequestResponseStep sends the request to the two-way receive port and waits for the response. This step is executed concurrently so that the other test steps may execute whilst it waiting for the response
The MSMQReadStep waits for a message to appear on an MSMQ queue.
- When it reads the message it uses the XmlValidationStep to perform schema validation and also execute a number of XPath expression to ensure the message contains the correct data
The FileValidateStep waits for a FILE to be written to a given directory
- When it reads the FILE it validates the data using the RegExValidationStep validation step since the FILE picked up was a flat file format
The FileCreateStep creates a new FILE in the specified directory containing the data that the backend system would typically create. This allows the Orchestration to complete and send the response back to the waiting HttpRequestResponseStep step
Finally, DBQueryStep is used to check that all of the BAM data has been successfully written to the BAMPrimaryImportDB

Test Steps

A test step is a .NET class which implements the ITestStep interface:

public interface ITestStep
{
    void Execute(XmlNode testConfig, Context context);
}

BizUnit will create and execute the test steps as dictated by the Xml test case. The test case will list the steps that need to be excuted in each stage of the test. The example below will cause BizUnit to create the Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep. BizUnit uses the assemblyPath and typeName to load and create the type:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    ...
</TestStep>

Executing Steps Concurrently

Test steps can be maked to execute concurrently by decorating them with the runConcurrently attribute:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpRequestResponseStep" runConcurrently="true">
    ...
</TestStep>

Reading Configuration from Context

BizUnit enables state to be flowed between test steps using the Context object, subsequent steps may read configuration from the Context object which was written by a previous test step, this is acheived using the takeFromCtx attribute within a steps configuration. For example:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpPostStep">
    <SourcePath>.\TestData\InDoc1.xml</SourcePath>    
    <DestinationUrl takeFromCtx="HTTPDest">http://localhost/TestFrameworkDemo/BTSHTTPReceive.dll</DestinationUrl>
    <RequestTimeout>60000</RequestTimeout>
</TestStep>

Wild Cards

BizUnit supports wild card for reading configuration, the following wild cards are supported

%DateTime% - will replace the wild card with the current date time in the format HHmmss-ddMMyyyy
%ServerName% - will replace the wild card with the name of the server BizUnit is being executed on
%Guid% - will be replaced by a new Guid

For example, for the test step configuration below:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    <SourcePath>..\..\..\TestData\InDoc1.xml</SourcePath>         
    <CreationPath>..\..\..\Rec_03\TransactionId_%Guid%_%ServerName%.xml</CreationPath>
</TestStep>

CreationPath becomes "..\..\..\Rec03\TransactionId12345678-D6AB-4aa9-A772-938972E3FD51ZEUS001.xml"_

Validation Steps

BizUnit supports the notion of validation steps which may be nested within test steps which support validation. This means that an MSMQ read step may use an XML validation step or a regular expression validation step to validate the data that it receives. Validation steps need to implement the IValidationStep interface.

public interface IValidationStep
{
    void ExecuteValidation(Stream data, XmlNode validatorConfig, Context context);
}

A test step may use the Context object utilities to execute the appropriate validation step as shown below:

public void Execute(XmlNode testConfig, Context context)
{
    XmlNode validationConfig = testConfig.SelectSingleNode("ValidationStep");
    MemoryStream data = null;

    ...

    context.ExecuteValidator( data, validationConfig );
}

The test step snippet below illustrates how a validation step is embeded in a file read step:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileValidateStep">
    <Timeout>3000</Timeout>
    <Directory>..\..\..\Rec_03\</Directory>
    <SearchPattern>TransactionId_*.xml</SearchPattern>
    <DeleteFile>true</DeleteFile>
			
    <ValidationStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.XmlValidationStep">
        <XmlSchemaPath>..\..\..\TestData\PurchaseOrder.xsd</XmlSchemaPath>
        <XmlSchemaNameSpace>http://SendMail.PurchaseOrder</XmlSchemaNameSpace>
        <XPathList>
            <XPathValidation query=
                "/*[local-name()='PurchaseOrder' and namespace-uri()='http://SendMail.PurchaseOrder']
                /*[local-name()='PONumber' and namespace-uri()='']">PONumber_0</XPathValidation>
        </XPathList>
    </ValidationStep>			
</TestStep>

Context Loader Steps

Similarly, BizUnit supports context loader steps, which are responsible for loading data into the BizUnit context. These types of need to implement the IContextLoaderStep interface as shown below:

public interface IContextLoaderStep
{
    void ExecuteContextLoader(Stream data, XmlNode contextConfig, Context context);
}

Getting Started

The best way to get started is to download the latest version, install it and then take a look at the SDK samples. All the source is currently included, so feel free to take a closer look at the code. All the test steps along with the framework itself are documented in the .CHM that is installed.

Finally, my apologies for the shameless plug :-), but Chapter 8 of Professional BizTalk Server 2006 book which I co-authored discusses in more detail how BizUnit may be used as an integral part of your development process, and how it may be used to drive stress and performance testing.

Acknowledgments

I'd like to thanks the follwing people who have contributed to BizUnit in some way, either by donating test steps, identifying bugs, or by providing requirements which have subsequently been implemented. I may have missed some people off, if so, it's not intensional, please drop me a mail to remind me:

Iain Bapty
Dharshana Kalahejagoda
Jon Fancey
Isaac Young
Mike Becker
Tanveer Rashid
Young Jun Hong
Dave Regan
Ian Cross
Greg Beach
Daren Jefford
Kevin Purcell
Karina Apostolides
Jon Bonnick
Brian Milburn
Rahmatullah Khan

Enjoy!

New Post: Can't set BizUnit instance logger?

kevinsmi — Wed, 08 Jul 2009 20:31:46 GMT

Hi,

This was a bug, it should be fixed in the latest version,

thanks,

Kevin

New Post: The test step could not be created, check the test step type and assembly path are correct

kevinsmi — Wed, 08 Jul 2009 20:25:30 GMT

Hi Martin,

BizUnit uses reflection to load test steps, you can either specify the typename or the typename + assembly path for the step. v3.0 had some out of date test cases in the "SampleSolutionFunctionalTests", specifically Test_Setup.xml wasn't correctly specifying the typename of the HostConductorStep, this should be fixed in v3.1 where the TestStep is defined as follows:

Note, you can fix up v3.0 by changing the version number above to:

Thanks,

Kevin

Created Release: 3.1.1001.0 (Jul 06, 2009)

kevinsmi — Mon, 06 Jul 2009 21:39:49 GMT

BizUnit 3.1.1001.0

The following steps have been added, thanks to Bram Veldhoen for contributing these:
BizTalkSteps/ExecuteMapStep
BizTalkSteps/ExecuteReceivePipelineStep
BizTalkSteps/ExecuteSendPipelineStep

Bug fixes:
*FileDeleteMultipleStep was not working when being executed via Xml configuration
*Bug fix: XmlValidationStepEx was not initialising the collection xPathValidations correctly
*Bug fix: MsmqWriteStep - added support to write to non-transactional queues

Released: 3.1.1001.0 (Jul 06, 2009)

Mon, 06 Jul 2009 21:39:49 GMT

BizUnit 3.1.1001.0

Commented Issue: XMLValidationStepEx ExecuteValidation throws NullReferenceException

maakku — Thu, 02 Jul 2009 13:20:27 GMT

private IList<Pair> xPathValidations gets never initialized.

As a result method public void ExecuteValidation(Stream data, XmlNode validatorConfig, Context context) always throws NullReferenceException.
Comments: ** Comment from web user: maakku **

Sorry, not always, only if there are xpath expressions involved.

Created Issue: XMLValidationStepEx ExecuteValidation throws NullReferenceException

maakku — Thu, 02 Jul 2009 13:16:36 GMT