Testing Rules with BizUnit/BizUnitExtensions

Apr 26, 2007 at 5:38 PM
Hi

BizUnit & BizUnitExtentions are excellent tools, as we are using regularly for our testing with our BizTalk projects.
I appreciate your effort for building them.

Now we started building rules, and we anticipate them to grow at large volume, and so take advantage of BizUnit again. So, we looked into it, we got struck at one point. So, we wants to see your opinion.

We need to pass bunch of facts, as any typical rule engine based solution, and all these facts are objects.
Before passing these objects as facts to rules engine, these objects are already created in prior stages of the flow and they have some state with them, few of these objects are created by other business logic dlls etc.

And on BizUnit side, the only step available is "FactBasedRuleEngineStep", we can pass facts as objects with parameter constructors only.
We can't pass already intialised objects. So, is there any way to pass-in already initialised object to BizUnit ?
But based on the interface contract ITestStep, there is no real way of passing objects to BizUnit.

Currently we are thinking of customising to some extent with help of deserialization in this scenario.

Let me know your thoughts like any alternative way or any future versions etc

I'm posting this both on BizUnit & BizUnitExtentions

Appreciate your help
Kishore
Coordinator
Apr 28, 2007 at 10:40 PM
Edited May 3, 2007 at 9:39 PM
Hi Kishore,

Afraid I don't have time to test your scenario, but from what you have described, the approach that I would take is to have your preceding steps load the facts as you describe but have them save them into the BizUnit context object. The context object is the mechanism to pass state between test steps. Use the takeFromCtx attribute in the rule step to have it use the object set in the ctx object.

Thanks,

Kevin
Coordinator
Apr 28, 2007 at 10:56 PM
Kishore,

One more thing just in case you are not aware, providing that you use the helper method on the BizUnit context object, any test step config that is decorated with the @takeFromCtx attribute will cause the helper methods to automatically extract the config from the context object instead of from the Xml config for the step.

Thanks,

Kevin
Apr 29, 2007 at 10:09 AM
Hi Kishore,
As Kevin pointed out, there is no way of passing initialised objects to BizUnit. However, with Extensions, im looking at the possibility of taking initialised objects, perhaps even entire "test case objects" and serializing them and sending that to BizUnit via some temp xml file. Its not very elegant but might work. Im still sort of chewing the fat on this, but it looks a distinct possibility. Btw, i havent tried out rules steps myself so the approach i described here is still in theory-land.

You could try something of this sort by modifying extensions yourself if you are in a hurry cos right now my priority is in getting extensions aligned with the latest BizUnit when it comes out.

Hope this helps,
cheers
benjy

Coordinator
Apr 29, 2007 at 11:59 AM
Benjy / Kishore,

No....thats not what I was saying at all, and the statement above is incorrect.

You can pass initialised objects to BizUnit, you simply add them to the context object and pass that into BizUnit, with 2.3 you can set state on the context before running the test. The approach that you suggest Benjy is definitely not one that I would recomend.

The clean way to do this is to pass them in the context object, that is it's role in life, it is the mechanism to pass state between test steps.

Thanks,

Kevin
Apr 29, 2007 at 1:48 PM
Edited Apr 29, 2007 at 1:56 PM
Kevin/Kishore,

I think there is still some confusion and this needs to be clarified .

(1) I think what Kishore wants to do is something i wanted to do ages ago, namely, write code like the following
FactBasedRuleEngineStep step = new FactBasedRuleEngineStep();
step.someParameter1 = somevalue (dont know the parameters offhand)
step.someParameter2 = somevalue
and then perhaps say
step.Execute() , or
bizUnit.Execute(step)

Kishore, pl correct me if i am wrong.

Kevin, since the above is not possible, are you saying there is an alternative way to do something like the following
Context context = new Context() (or statically use the context) and do
Context.Add("Parameter1")
Context.Add(Parameter2") and then
BizUnit.RunTest(testdata.xml) where in the xml we write
<TestStep name = FactBasedRuleEngineStep....>
<Parameter1 @takeFromCtx="Parameter1">
</TestStep>
and so on? if so, this is news to me and im sure to many others as well. pl see point 2 which is related..

(2) In terms of putting stuff in the context , from the earliest days of using it , i tried to initialise the context object in the C# code and tried to add things to the context but the compiler wouldnt let me do that. I could not find any code examples either. So i resorted to writing my own ContextPopulateStep so i could directly load things into the context. I then moved on to using the XmlContextLoad with app.config files which i now use extensively to load up file locations, database connection strings etc.

Re: the point that the context object passes state between steps, that maybe its role, but i havent seen that happen. As far as i know, unless the step at the end of its execution can be told to put something in the context, its not going to do that. For example, we had a requirement to use the WaitForFile to look for a file with a particular pattern but then for the next test step (which we used the DotNetObjectInvokerEx) we needed to pass the exact file name which we would not know before hand but the step would. Since the WaitForFile uses the FileSystemWatcher, it gets the name of the file and checks whether its of the pattern expected, but does not (or did not) explicitly store the file name in the context. So we had to write WaitForFileEx() which did this (exposed a property named WaitedFile which it put into the context) . What i would have liked to have seen is a generic declarative way in the step xml to tell it to load the result or other parameters into the context.

Im not certain what the problem is with the approach i suggested. Just as some folk like to generate the Xml from spreadsheets, some would like to do stuff like the code sample i showed above, or just fill up different step objects from various sources and pass a collection of steps to BizUnit without going via Xml. The Xml is nice and easy but it should not prove to be a block to adopting the tool for those who prefer a strong typed route. This is what i was referring to in my roadmap about providers.

Ideally i would like to see BizUnit used as follows

ITestCaseProvider provider = new XmlFileCaseProvider
provider. Load(xmlfilename);
BizUnitTestRunner runner = new BizUnitRunner(provider)
runner.ExecuteTest();

BizUnitTestRunner accepts anything of type ITestCaseProvider and in return it expects each provider to expose 3 methods namely GetTestSetupSteps() , GetTestExecutionSteps(), GetTestTearDownSteps() and each returns a TestStepCollection which it iterates through and calls execute on them. I would also like to pass an ILogger object to the TestRunner so my logger could be a ConsoleLogger, EntLibLogger or anything else.

Exposing the properties allows people to know exactly what a step needs instead of having to look at the CHM for examples of what to put in each testStep and they can use the approach i showed in point 1 (ie) explicitly setting values and then asking BizUnit to execute the steps. I wouldnt want them to call Execute themselves because only the BizUnit class is maintaining the context internally and its better to keep that control with BizUnit, but as far as setting the properties goes, why not do it directly?

In the interim, since BizUnit only accepts a xml file containing the test case, why not generate the temp file by serializing a set of steps into xml and passing that to BizUnit? Why should BizUnit care where the Xml came from?

Hope you can see what Im driving at.

Thanks,
benjy

Coordinator
May 3, 2007 at 9:30 PM
Edited May 3, 2007 at 9:41 PM
So, before we spend a load of time redesigning and re-implementing features of BizUnit I think it would be prudent to understand if / why Kishore’s scenario cannot be met with the current functionality….

First a small piece of relevant history, the FactBasedRuleEngineStep was developed for a large BizTalk solution that I worked on, we had some 500+ rules from memory that we tested with BizUnit. The facts that the rules used were reasonably complex, a mixture of objects, datatables and Xml documents. The objects were of varying levels of sophistication, ranging from simple functions to complex data structures that acted as a caching façade over an Oracle DB. The test step work very well for the varied requirements of that large and complex solution, that does not of course mean that it’ll work for your scenario Kishore’s, but it gives me a reasonable level of confidence that it handles a good many scenarios.

Kishore, from your description above you say: these objects are already created in prior stages of the flow and they have some state with them, few of these objects are created by other business logic dlls etc, I can read this two ways, prior test steps create the fact objects, or the fact objects are created before the test even executes. Around this, I’d like to understand why none of the following three approaches work:

1. Firstly, note that the fact test step already allows you to pass facts as compiled binaries, I don’t know if this meets your requirements Kishore without any modifications, if it does not can you explain why not, I’d like to understand what is missing – maybe you could send me a cut down sample? Here’s an example test step snippet, where the a ruleset is being tested which uses three facts - two objects and an Xml document:

    <Facts>
        <Fact type="object" assemblyPath="PoC.RuleExecuter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d0095d9d835ad9eb" typeName=" PoC.RuleExecuter.CommonBreFunctions" />
        <Fact type="object" assemblyPath="PoC.RuleExecuter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d0095d9d835ad9eb" typeName=" PoC.RuleExecuter.VariableList" /> 
        <Fact type="document" schemaType="PoC.Schemas.InboundDoc" instanceDocument="D:\Projects\PoC\PoC.RuleExecuter.Tests\TestMessages\Inbound\InboundTestMessage_Rule25b(i)x.xml" />
        …
    </Facts>

2. If #1 does not work, and prior test steps create the facts, the BizUnit context is the way to flow these objects to the FactBasedRuleEngineStep – the prior test step will of course need to create them and add them to the context, this will most likely require you to write a custom step to save them to the ctx. The FactBasedRuleEngineStep would need to fetch them from the context. This may or may not require some minor modifications, let me know if it does and what they are, and I will see if it is approapriate to role them back into the step.

3. If the facts are created prior to the test running, your unit test should create them and add them to the BizUnit context, the context should be passed as a constructor arg when you instantiate the BizUnit class. The FactBasedRuleEngineStep would need to fetch them from the context. Again as mentioned, the FactBasedRuleEngineStep step may or may not require some minor modifications to support fetching the facts from the context (taking a brief look at the code I suspect that it would), but this should be a very minor change. The code to create the facts and pass them into BizUnit would look something like this, (note: I made some minor changes to the programming model post 2.3 Alpha to make it cleaner to create a ctx object, you will need 2.3 Beta 1 for this context object constructor, you could do this with Alpha 1 but I wasn’t happy with the programming model since it was overly complex):

        [TestMethod]
        public void FactStuff()
        {
            Object[] factObjArray = null;
 
            // Create and Initialize facts...
 
            Context ctx = new Context();
            ctx.Add("Fact1", 32);
            ctx.Add("Fact2", factObjArray);
 
            BizUnit bizUnit = new BizUnit(@"..\..\..\Data\FactBasedRuleEngineStepTest.xml", ctx);
            bizUnit.RunTest();
        }

Kishore, let me know which approach works for you, thanks.

Benjy, regarding you comments, the ability to create the context before running the test is new to the 2.3 release, a few customers have asked me for this feature so I decided to roll it in.

Also, you have been able to set objects in the context for a number of releases now, so not sure what your scenario / problem was or where you were going wrong, but I’ve used that feature on a many projects. In fact, even the branched version of BizUnit that you have allows this :-) :-).

I can see that the ability to set properties on steps is near and dear to you, however, the value of this is diminished given that properties may be set in the ctx prior to running a test. If you consider the fact that you can create the ctx object, set values on it and then pass that into the test, when used in conjunction with the ability that all test steps have to fetch their config from the ctx by simply decorating the Xml tag with takeFromCtx attribute (providing they use the helper methods on the ctx object to fetch their configuration), you effectively get very similar functionality.

The example that you give for getting the file name is addressed by the ability to set values in the ctx prior to executing the test. Also, some steps support IContextLoaderStep which is a more generic mechanism of setting values in the context.

The power of BizUnit being declarative is that the Xml may be generated, for example on many of the projects that I have worked on, we have generated the test cases from Excel spreadsheets which represent the test matrix. You could general code instead of Xml of course, but its less flexible. Xml lowers the skills required to configure test cases, i.e. you don’t need to be a developer, business analysts and testers who don’t code are typically familiar with Xml. In addition, its easy to fix up the Xml with out the need for a recompilation.

Finally, regarding custom log sync’s, its coming, but probably won’t be there until 2.4, concurrent test steps make this slightly more complex, but its coming. To be honest, the real power of this in my opinion is not to support EntLib but instead to enable the output to be formatted as Xml. A style sheet can be used to ‘prettify’ the output, this is also a convenient way to persist the test results so they can be saved in Visual Studio Team Foundation server, test director, etc.

Enjoy!

Kevin
May 4, 2007 at 12:03 AM
Edited May 4, 2007 at 12:07 AM
Hi,
I took a look at 2.3 alpha... it definitely looks very good. Nice additions to the BizUnit and Context classes now. I like the fact that you can pass a stream to bizunit now. That, i think will probably cover my requirements to use a non xml route. I think the Context Change() method that i had written and which you havent included could also be replaced with the corrected ContextManipulatorStep which wasnt working earlier. (unless you can be persuaded to put back the Change() and HasKey() methods?)

I too dont know what the problem was with setting data into the context from the C# class, but i gave up that battle rather early. I can see how 2.3 will make things much easier.
and yes, i kinda understand how we can set all the properties into the context and pass them in.. Will have to experiment with it a bit first...

General questions
(a) the CHM documentation still refers to 2.2 and doesnt have the new structure of the libraries etc. Will that be changed ?
(b) do you have plans for more comprehensive documentation or are you continuing to rely on the CHM saying everything? For instance, the new additions to BizUnit() and Context() classes are worthy of an entire article and actually i would be quite happy to write that up.

Just saw that you have the 2.3 beta also put up . I will get extensions tested with the 2.3 beta and see if its all working... ..when you are ready with 2.3 i shall re-release a new package of extensions tested against it and without the core.

Thanks for patiently responding to all the comments ....

Rgds
Benjy
May 8, 2007 at 3:32 PM
Thank you for your prompt responses and sorry for delay from my side.
Here is a sample of my original issue.

In the foll ex, employee name and id are assigned in const, so I can use BizUnit to initialize it using

<Fact type="object" ...>

But, there is one more public var NumVacationDaysLeft, value of this var is not init'd in constructor.
It is set by passing instance to a emp_bl.CalcNumVacationDaysLeft(Employee).
At this point of time I need to call rules, which means I need to call BizUnit step here.
As of I understand, if I'm not wrong, BizUnit step may not help here.

After seeing your responses, I understand I can add Context to BizUnit constructior as you specified in
approach 3.

To try this approach, I looked for 2.3 Aplpha/Beta versions in Releases and Source Code tabs. I didn't find any.
I will try this approach as soon as I can find the binaries(or codebase).

Code
   public class Employee
    {
 
        public Employee(string name, string id)
        {
            // assign name and id here
        }
        /// <summary>
        /// Eithere sets or gets the Employee name
        /// </summary>
        public string Name
        {
            get { return ""; }
            set{}
        }
        /// <summary>
        /// Either sets or gets the employee id
        /// </summary>
        public string ID
        {
            get { return ""; }
            set{}
        }
        /// <summary>
        /// This value is populated by business logic components
        /// after population, contains non-negative value
        /// (shud have been property here than public var)
        /// </summary>
        public int NumVacationDaysLeft = -1;
    }
    public class EmployeeBL
    {
 
        /// <summary>
        /// Calculates the number of vacation days left for the give employee
        /// </summary>
        /// <param name="e"></param>
        public void CalcNumVacationDaysLeft( Employee e)
        {
            //TODO:implement logic for calculation
        }
    }
Rule
	IF  
		AND
		[Employee.ID] is not equal to [<null>]
                [Employee.NumVacationDaysLeft] is greater than [0]
        THEN	
		Action
		 [RuleValidationResults.VacationAllowed]=[True]
Thanks again for your great and prompt responses
May 8, 2007 at 3:37 PM
Thanks
Kishore
Coordinator
May 10, 2007 at 12:23 PM
Hi Kishore,

Afraid this will be brief as I'm pushed for time...the 2.3 drop of BizUnit is on the planned releases tabs since it's Beta at this time.

From your description above, you'll need to create and poplulate your facts, add them to the ctx object as below.

        [TestMethod]
        public void FactStuff()
        {
            Object[] factObjArray = new Object[1];
 
            // Create and Initialize facts...
            Employee employee = new Employee("Bill Gates", "001");
            factObjArray[0] = employee;
 
            Context ctx = new Context();
            ctx.Add("EmployeeFacts", factObjArray);
 
            BizUnit bizUnit = new BizUnit(@"..\..\..\Data\FactBasedRuleEngineStepTest.xml", ctx);
            bizUnit.RunTest();
        }

You'll then need to make a minor change to the FactBasedRuleEngineStep to fetch the facts object form the context, the code in the step will need to use the context helper methods to fetch the object from the context, rather than write your own code to do that, note the Xml will need to be decorated with the takeFromCtx attribute. There are lots of examples in the code where steps do this.

Let me know if that makes sense and how it goes. If I get time I'll make the changes, but I'm in a busy spell at work now so proably best for you to just dive in, copy the step and make the changes.

Thanks,

Kevin
May 15, 2007 at 3:20 PM
Kevin,

I'm envisioning adding facts to context doesn't require any key. Because, the idea is store all the object facts in the context object
& retrieve these facts in FactBasedRuleEngineStep code, pass to policy object and all facts needs to be passed.
After rules execution is completed, I can check object status for any updates made by rules engine. So, I believe key is not required here.
In my case, there are 6 to 10 object facts I need to pass, and I don't use this key anywhere else.
So, instead of code
            Context ctx = new Context();
            ctx.Add("EmployeeFacts", factObjArray);
I'm envisioning as below,
            Context ctx = new Context();
            ctx.Add(factObjArray);
I see that Context class uses hashtable to store these facts.

Thanks
Kishore
Coordinator
May 15, 2007 at 10:30 PM
Edited May 15, 2007 at 10:32 PM
Hi Kishore,

So, you can't add something into the ctx without a key, and similarly you can't fetch something from the ctx without a key. So, its not clear to me how your facts will be fetched from the ctx, i.e. how will the FactBasedRuleengineStep know where to get the facts from? The ctx typically contains many items, something, somewhere needs to tell what needs to fetch what items form the ctx.

So, I'm suggesting the approach is as follows, first your test code creates the ctx and adds the facts:
            Context ctx = new Context();
            ctx.Add("EmployeeFact1", employeeFact1);
            ctx.Add("EmployeeFact2", employeeFact2);
            ctx.Add("EmployeeFact3", employeeFact3);

The FactBasedRuleengineStep step will need to handle loading facts from the ctx, perhaps something like the following below, the test case fragment would be decorated with the takeFromCtx attribute:
<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FactBasedRuleEngineStep, Microsoft.Services.BizTalkApplicationFramework.BizUnit.BizTalkSteps">
	<RuleStoreName>C:\LoanProcessing.xml</RuleStoreName>
    <RuleSetInfoCollectionName>LoanProcessing</RuleSetInfoCollectionName>
    <DebugTracking>C:\outputtraceforLoanPocessing.txt</DebugTracking>
    <ResultFilePath>C:\RulesTestCases</ResultFilePath>
    <Facts >
        <Fact type="object" takeFromCtx="EmployeeFact1"/>
        <Fact type="object" takeFromCtx="EmployeeFact2"/>
        <Fact type="object" takeFromCtx="EmployeeFact3"/>
    </Facts>
</TestStep>

The following code snipet will need to be changed to handle fetched from the ctx, it'll need to fetch from ctx and add it to the fact array facts:
                switch (fact.SelectSingleNode("@type").Value)
                {
                    case "object":
                        {

You'll need to read the fact from the ctx, it's an optional load, if its there set it as the fact item:
                            object factObj = ctx.ReadConfigAsObject(fact, ".", true);
                            if(null!=fact)
                            {
                                facts[i] = factObj;
                            }
                            else
                            {

Otherwise, the object should be loaded as before:
                                string assemblyPath = fact.Attributes.GetNamedItem("assemblyPath").Value;
                                string typeName = fact.Attributes.GetNamedItem("typeName").Value;
 
                                #region Simple (string-only) constructor argument handling added by Ian Picknell
                                object[] objectArgs = null;
                                if (null != fact.Attributes.GetNamedItem("args"))
                                {
                                    objectArgs = fact.Attributes.GetNamedItem("args").Value.Split(new char[] { ',' });
                                }
                                #endregion
 
                                System.Reflection.Assembly asm;
                                if (assemblyPath.Length > 0)
                                {
                                    asm = System.Reflection.Assembly.LoadWithPartialName(assemblyPath);
                                    if (asm == null)
                                        // fail
                                        throw (new Exception("failed to create type " + typeName));
                                    type = asm.GetType(typeName, true, false);
                                }
                                else
                                {
                                    // must be in path
                                    type = Type.GetType(typeName);
                                }
                                facts[i] = Activator.CreateInstance(type, objectArgs);
                            }
                            break;
                        }

I hope that helps,

Thanks,

Kevin
Coordinator
May 16, 2007 at 10:13 PM
Edited May 16, 2007 at 10:16 PM
Benjy,

Appologies, I meant to reply to you before. Thanks, I'll be updating the .chm. In terms of additional documentation for BizUnit, I added a whole chapter around what it takes to test an enterprise class BizTalk solution in Professional BizTalk Server 2006, the chapter talks quite a bit about BizUnit and also positions it in terms of the type of testing that it is approapriate to use it for.

Of course the changes in 2.3 around the ctx are not covered in the book, if you have the time to write a paper on that be my guess, that would be great. I'd be happy to review it.

Thanks,

Kevin

May 17, 2007 at 11:14 AM
Hi Kevin,
No problem at all. I have ordered the book already and am eagerly awaiting it. By the way, im attending a Biztalk user group meeting today in London (the Biztalk Foundation) and Darren Jefford and Alan Smith are speaking there!! I''ve also been rather tied up at a busy phase in work and am trying to set aside sometime to look through the new version and write up more on the context .. As soon as i get some progress there, i'll let you know. i will most probably write it up a bit at a time and post it on the extensions wiki.

cheers,
benjy
Jul 25, 2008 at 9:08 AM
Hi,
My Biztalk application needs to test 1000 messages in a day.
Is there any way to generate xml messages based on particular schema using this tool.
I have seen Test step, filecreatesetup, which creates a file in particular folder from given location. But running that step causing error and fil of test case as it
says "Access to Output location is denied" and this location is set to recieve location of Port (FileAdapter). In fact any directory in Win xp is readonly.

Can anyone help me ?

Regards
Karan
Jul 25, 2008 at 10:47 AM
Hi Karan,
To generate volume data you should use LoadGen. BizUnit has some steps for driving LoadGen as well.
In terms of the read-only error,
(1)  if biztalk is giving the error then ensure that the biztalk user account has write access to the folder
(2) if it happens for non biztalk apps - check if XP has locked the C drive , there are some settings like "allow file sharing etc.. cant remember the exact settings off the top of my head


HTH
benjy