My beef with the ESB Toolkit 2.0 resolution framework

Lately, I have been working on an extension for the ESB Toolkit 2.0 for BizTalk Server 2009. It can be found here on CodePlex. More posts about this ESB.Extensions solution will follow later, this particular post will describe why I think the ESB Toolkit’s resolution framework should be improved.  

A short summary of how the ESB Toolkit resolution framework works. Resolvers are configured in the esb.config file using a specific name that can be coupled to a .NET type that needs to implement IResolveProvider. From the esb.config:  

<resolver name="BRE" type="Microsoft.Practices.ESB.Resolver.BRE.ResolveProvider, Microsoft.Practices.ESB.Resolver.BRE, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>

The above type implements IResolveProvider, and the ESB Toolkit provides a ResolverMgr and ResolverConfigHelper to allow for retrieving and caching the appropriate resolver types when the resolvers are retrieved from the itinerary and to be used by the itinerary service.  

Let’s look at IResolveProvider:  

public interface IResolveProvider
{
    Dictionary<string, string> Resolve(ResolverInfo resolverInfo, XLANGMessage message);
    Dictionary<string, string> Resolve(string config, string resolver, XmlDocument message);
    Dictionary<string, string> Resolve(string config, string resolver, IBaseMessage message, IPipelineContext pipelineContext);
}

IResolveProvider provides three methods that all perform more or less the same task: resolve some information into a Dictionary<string, string>. My main problem with this approach it that everything that needs to be resolved needs to be in the form of string in order to be applicable for this way of resolution. I would expect a generic resolution framework to be more generic than that!  

To make matters worse, there are limitations to which keys you are allowed to use in the resolution dictionary, if you’re using the out of the box resolvers (i.e. the BRE resolver). Basically, you’re stuck with the keys that are provided on the Microsoft.Practices.ESB.Resolver.Resolution type:  

[Serializable]
public sealed class Resolution : IContextParameters, ITransformParameters, IEndpointParameters
{
    ... private members & constructor (left out for brevity) ...  

    public string Action { get; set; }
    public string ActionField { get; set; }
    public string CacheTimeout { get; set; }
    public string DocumentSpecNameField { get; set; }
    public string DocumentSpecStrongNameField { get; set; }
    public string EndpointConfig { get; set; }
    public string EpmRRCorrelationTokenField { get; set; }
    public bool FixJaxRpc { get; set; }
    public string InboundTransportLocationField { get; set; }
    public string InboundTransportTypeField { get; set; }
    public string InterchangeIdField { get; set; }
    public string IsRequestResponseField { get; set; }
    public string MessageExchangePattern { get; set; }
    public string MessageType { get; set; }
    public string MethodNameField { get; set; }
    public string OutboundTransportCLSID { get; set; }
    public string ReceiveLocationNameField { get; set; }
    public string ReceivePortNameField { get; set; }
    public bool Success { get; set; }
    public string TargetNamespace { get; set; }
    public string TransformType { get; set; }
    public string TransportLocation { get; set; }
    public string TransportNamespace { get; set; }
    public string TransportType { get; set; }
    public string WindowUserField { get; set; }
}

Where the three Interfaces (IContextParameters, ITransformParameters, IEndpointParameters) all define a seperate set of properties (which are implemented by the Resolution type, see above).  

The resolvers use this Resolution type to fill the (more generic) Dictionary<string, string> type using ResolverMgr.SetResolverDictionary:  

public static void SetResolverDictionary(Resolution resolution, Dictionary<string, string> ResolverDictionary)
{
    try
    {
        ResolverDictionary.Add("Resolver.Action", resolution.Action ?? string.Empty);
        ResolverDictionary.Add("Resolver.ActionField", resolution.ActionField ?? string.Empty);  

        ... more calls to ResolverDictionary.Add("Resolver.Xxx", resolution.Xxx ?? string.Empty); (removed for brevity) ...   

        ResolverDictionary.Add("Resolver.TransportType", resolution.TransportType ?? string.Empty);
        ResolverDictionary.Add("Resolver.WindowUserField", resolution.WindowUserField ?? string.Empty);
    }  

   ... some exception handling ...  

}

This basically means that you can only work with the properties that the Resolution type provides. I see that as another limitation, because it means that the current resolvers are limited to only be able to resolve those properties. If I want to resolve a string value, that does not correspond to one of the properties of the Resolution type, I have two options:  

  • Abuse one of the properties of the Resolution type. I.e. put a pipeline type name in the property TransformType. I started with this workaround, but it became a problem because I needed to resolve more complex objects than just strings (more on that later).
  • Write a custom Resolver. For instance, this means that I would need to implement a custom BRE resolver, that does resolution using the BRE in exactly the same way as the standard BRE resolver, just because of the different property.

The conclusion is that at the moment, the resolvers are coupled to what those resolvers are actually resolving. That should be decoupled if you ask me.  

I took a stab at improving this situation by implementing a resolution framework that uses the following interface:  

    public interface IResolutionProvider
    {
        ResolutionDictionary Resolve(ResolverInfo resolverInfo, XLANGMessage message);
        ResolutionDictionary Resolve(string config, string resolver, XmlDocument message);
        ResolutionDictionary Resolve(string config, string resolver, IBaseMessage message, IPipelineContext pipelineContext);
    }  

    [Serializable]
    public class ResolutionDictionary : Dictionary<string, object>, ISerializable
    {
        public ResolutionDictionary()
        {
        }  

        public void SetValue(string key, object value)
        {
            base.Add(key, value);
        }  

        public object GetValue(string key)
        {
            if (base.ContainsKey(key))
            {
                return base[key];
            }
            return null;
        }  

        public string GetString(string key)
        {
            return GetValue(key) as string;
        }
    }

This dictionary allows resolvers to resolve anything, for as long as the resolution objects are serializable. It also allows for using any key value (string). The code, that’s calling this kind of resolution, will be responsible for casting the object to the type that it expects.  

The ESB.Extensions resolution framework, that uses the above IResolutionProvider and ResolutionDictionary, is still compatible with the esb configuration and the itinerary designer.  

A full example can be found the in the ESB.Extensions solution on codeplex. I use the resolution framework there to resolve parameters needed in a couple of services: ReceivePipelineService, SendPipelineService and Resequencer services. I implemented some BRE vocabularies to enable for creating the resolution objects.  

Off course, any feedback is welcome!

Posted in BizTalk | Tagged , , | Leave a comment

Promoted property bug in XmlDisAsm component?

I have just uploaded a bug report (#477852 on Microsoft Connect) regarding a fairly specific scenario using the XmlReceive pipeline. It involves having an envelope and a document schema, in which the document schema contains an element (or attribute) that is promoted both as a distinguished AND as a promoted property. In this case, after processing by the XmlReceive pipeline, the property is only present in the message context as a distinguished property, and NOT as a promoted property. Some other people may have experienced this same issue:

I reproduced this behaviour both on a BizTalk Server 2009 deployment and in standalone mode with the ExecuteRecievePipelineStep, found in BizUnit 3.1. See the screenshots below.

Schema containing both promoted and distinguished property

Message Context missing the promoted property

For BizUnit ExecuteReceivePipelineStep intimi: I used test T_015, added both a distinguished and a promoted property to the Child1Attribute1 attribute (which is of type string) from schema0.xsd, and pointed it to promoted Property2 (which is of type string); then I ran the test. The screenshots above show that the promoted Property2 property isn’t there.

I have not tested this with the Flat File Disassembler pipeline components.

A workaround for this bug is to define the schema to only use the promoted property, and use the promoted property syntax from within your orchestrations.

Posted in BizTalk | Tagged , , , , , | Leave a comment

BizTalk ESB Guidance 2.0 – Fixing the Itinerary OnRamp?

First off, for a very good presentation about what an ESB could be, watch Mark Richards here.

Recently, I have taken a look at the new-and-improved ESB Guidance 2.0 for BizTalk Server 2009 Beta. The ESB Guidance contains some interesting and potentially powerful concepts. Where BizTalk provides ESB-functionality out of the box (routing, transformation, etc., etc.), the ESB Guidance allows for a much more dynamic approach to realize these functionalities. With ESB Guidance, the majorities of these functionalities (i.e. dynamic routing and/or dynamic transformation) can be used by the client applications of the ESB (the systems sending messages to BizTalk) in the form of an ESB Guidance specific soap header called an Itinerary header. These Itinerary headers describe the processing steps that the message should go through within the ESB, and this Itinerary header could be different per message if that were necessary.

This approach offers a great deal of flexibility, because per each separate message coming from one of the ESB clients, it’s path through the ESB can be set individually. However, it also means that those clients will need to be aware of the ESB Guidance Itinerary concept and have some way to add the Itinerary headers in the soap header of the message. Next to that, if there are changes to or within the ESB (configuration changes, system replacements), there’s a chance that the Itinerary header will have to be changed as well.

In order to circumvent introducing this dependency of the ESB clients to the Itinerary concept, wouldn’t it be a good idea to have the ESB be responsible for assigning the Itinerary headers? The most obvious place I can think of doing just that would be in a custom pipeline component, which can be configured with the appropriate Itinerary information, and which will just add this Itinerary header to the message as it comes in into the ESB. Taking this approach, the ContextAdder pipeline component, written by Jon Flanders, comes to mind. Using the ContextAdder, together with Saravana Kumar’s article on design-time properties for custom pipeline components, I came up with a sample that does as I described above. You can download it here. BTW, the pipeline component in this sample writes or promotes any set of properties you want it to, it’s not specific to Itineraries.

The sample needs BizTalk 2009 and ESB Guidance 2.0. I worked with the ESB Guidance 2.0 October 2008 release, so there may be some discrepancies when using it with the January 2009 release. Configuring the pipeline component is somewhat tedious, as the xml syntax needs to be very specific (especially regarding &amp;lt;, &lt; and <). If using it for real, a way to programmatically generate the config would be a good enhancement.

The sample uses FILE receive locations only and uses a very simple Itinerary that uses the orchestration routing and transformation services, configured with the STATIC resolvers. A screenshot of the pipeline configuration is displayed below.

Fixed Itinerary OnRamp Pipeline Configuration

I’d be very interested in other’s thoughts on this approach, so drop me a comment!

[27-02-2009] Nick Hauenstein pointed out that the January 2009 CTP2 release of ESB Guidance 2.0 contains a much better way to achieve the same result: static Itineraries, which can be set within a pipeline component. Check out his post!

Posted in BizTalk | Tagged , , , , | Leave a comment

Combining BizTalk Software Factory and BizTalk Deployment Framework

Recently I have finished a preliminary version of an adaptation of the BizTalk Software Factory guidance automation package, owned by Jean-Paul Smit and Dick Dijkstra (BSF), where I have extended the functionality of the BizTalk Software Factory with support for Scott Colestock’s BizTalk Deployment Framework (BDF). The latest changeset (17321) of the BSF source contains these changes, so you can testdrive it yourself!

The main advantage of using BSF and BDF in conjunction is that you should be able to produce a full BizTalk compilable and deployable solution within minutes using a few mouseclicks (ok, and some keyboard entries), leaving a situation where you have a repeatable development and deployment process from day 1, whilst maintaining strict naming conventions. After generating the solution, you will still be able to enjoy the full functionality of the BDF.

During the wizard pages of the BSF, you will be prompted to enter some BDF specific settings:

Wizard Page Options

PreProcess Bindings: If unchecked, you will get a regular bindings file that will not be processed by ElementTunnel.exe or XmlPreProcess.exe. If checked, you will get a bindingsMaster file, that will be processed by the two mentioned exe’s.

Add VDirList: If unchecked, you will not get a VDirList file (so you won’t be exposing webservices). If checked, you will get a VDirList file, that you will have to edit manually to contain your vdir mappings.

Company Name: The company name that is used in the generation of the Windows Installer (msi).

Copy DeployTools, Copy WiXSetup: If unchecked, no files will be copied. If checked, BSF will copy the BDF DeployTools and WiXSetup directories into your solution root folder. To keep things simple, the files in these directories will not be added to your solution.

The BDF offers many many more settings to play with, but I decided to keep things simple and limit the configurable settings to those that would alter the solution structure or the files within the solution.

The BSF will generate the following solution artifacts:

BizTalk Solution, created with BSF

So after preparing your environment for BDF, you should be able to deploy your (completely empty) solution to BizTalk via the generated msi! (To do this: select the *.WiXSetup.build file in solution explorer, then in VS2005 Tools execute “NAnt Current Target”).

To prepare your environment, you basically only have to run MakeExternalTools_VS2005.vbs and make sure all paths in this .vbs file are correct (see BizTalkSoftwareFactory\BDF\Info), but refer to the BDF docs for complete documentation.

Posted in BizTalk | Tagged , , | Leave a comment

Unit testing BizTalk artifacts using BizUnit

First off, credit where credit’s due: Tomas Restrepo has done an amazing job making pipeline testing way easier with the pipeline testing library. Read all about it here.

Tomas’ library inspired me to write some test steps for the BizUnit framework that would allow for true unit testing of BizTalk components: being able to test BizTalk components without having to deploy the components to a BizTalk server, making testing roundtrips easier and overall less painfull. As additional advantage, you can have all tests use a similar approach/syntax, and use the already present validation steps to validate the results. I have implemented the following steps for BizUnit:
– ExecuteMapStep
– ExecuteReceivePipelineStep
– ExecuteSendPipelineStep

Obviously, the latter two are using Tomas’ pipeline testing library.

The ExecuteMapStep step can be used to test biztalk maps, using an input xml file. An example of a BizUnit test, that executes a map and validates the resulting output xml file (via the BizUnit context), is displayed below.

<TestCase testName="T_006_ExecuteMapStepWithValidationWithContextLoader">
  <TestExecution>
    <TestStep assemblyPath="BizUnit.BizTalkSteps.dll" typeName="BizUnit.BizTalkSteps.ExecuteMapStep">
      <Map assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.MapSchema1ToSchema2"/>
      <Source>..\..\..\Test\BizUnit.BizTalkTestArtifacts\Instances\Schema1.xml</Source>
      <Destination>Schema2.006.xml</Destination>
      <ContextLoaderStep assemblyPath="" typeName="BizUnit.XmlContextLoader">
        <XPath contextKey="testValue">/*[local-name()='Schema2Root' and namespace-uri()='http://BizUnit.BizTalkTestArtifacts.Schema2']/*[local-name()='Child1' and namespace-uri()='']/@*[local-name()='Child1Attribute1' and namespace-uri()='']</XPath>
      </ContextLoaderStep>
      <ValidationStep assemblyPath="" typeName="BizUnit.ContextValidationStep">
        <Context keyName="testValue">1</Context>
      </ValidationStep>
    </TestStep>
  </TestExecution>
</TestCase>

The ExecuteReceivePipelineStep and ExecuteSendPipelineStep let you test receive or send pipelines. The input for both the ExecuteReceivePipelineStep and ExecuteSendPipelineStep (optionally) consists of:
– Per-instance pipeline configuration
– Document specs
– Input xml file (multiple input files can be specified for the ExecuteSendPipelineStep)
– Message context input file(s): an xml file, containing all context properties (name, namespace, value, and whether these properties are promoted or not)

ExecuteReceivePipelineStep:

<TestStep assemblyPath="BizUnit.BizTalkSteps.dll" typeName="BizUnit.BizTalkSteps.ExecuteReceivePipelineStep">
  <Pipeline assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.ReceivePipeline1">
    <InstanceConfigFile>..\..\..\Test\BizUnit.BizTalkSteps.Tests\TestData\ReceivePipeline3.InstanceConfig.xml</InstanceConfigFile>
    <DocSpecs>
      <DocSpec assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.Schema0" />
      <DocSpec assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.Schema3Env" />
    </DocSpecs>
  </Pipeline>
  <InputContextFile>..\..\..\Test\BizUnit.BizTalkSteps.Tests\TestData\Context.0.xml</InputContextFile>
  <Source>..\..\..\Test\BizUnit.BizTalkTestArtifacts\Instances\Schema3Env.xml</Source>
  <DestinationFileFormat>Output015.{0}.xml</DestinationFileFormat>
  <OutputContextFileFormat>Context015.{0}.xml</OutputContextFileFormat>
</TestStep>

ExecuteSendPipelineStep:

<TestStep assemblyPath="BizUnit.BizTalkSteps.dll" typeName="BizUnit.BizTalkSteps.ExecuteSendPipelineStep">
  <Pipeline assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.SendPipeline1">
    <InstanceConfigFile>..\..\..\Test\BizUnit.BizTalkSteps.Tests\TestData\SendPipeline3.InstanceConfig.xml</InstanceConfigFile>
    <DocSpecs>
      <DocSpec assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.Schema0" />
      <DocSpec assemblyPath="..\..\..\Test\BizUnit.BizTalkTestArtifacts\bin\Development\BizUnit.BizTalkTestArtifacts.dll" typeName="BizUnit.BizTalkTestArtifacts.Schema3Env" />
    </DocSpecs>
  </Pipeline>
  <InputContextDir>..\..\..\Test\BizUnit.BizTalkSteps.Tests\TestData</InputContextDir>
  <InputContextSearchPattern>Context*.xml</InputContextSearchPattern>
  <SourceDir>..\..\..\Test\BizUnit.BizTalkTestArtifacts\Instances\</SourceDir>
  <SearchPattern>Child*.xml</SearchPattern>
  <Destination>Output.025.xml</Destination>
  <OutputContextFile>Context.024.xml</OutputContextFile>
</TestStep>

Per-instance pipeline configuration:

The test steps also allow for using a per-instance pipeline configuration. Below the per-instance pipeline configuration of a pipeline, containing only the Xml Disassembler pipeline component.

<?xml version="1.0" encoding="utf-8"?>
<Root xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Stages>
    <Stage CategoryId="9d0e4105-4cce-4536-83fa-4a5040674ad6">
      <Components>
        <Component Name="Microsoft.BizTalk.Component.XmlDasmComp">
          <Properties>
            <EnvelopeSpecNames vt="8">BizUnit.BizTalkTestArtifacts.Schema3Env</EnvelopeSpecNames>
            <EnvelopeSpecTargetNamespaces vt="8">http://BizUnit.BizTalkTestArtifacts.Schema3Env</EnvelopeSpecTargetNamespaces>
            <DocumentSpecNames vt="8">BizUnit.BizTalkTestArtifacts.Schema0</DocumentSpecNames>
            <DocumentSpecTargetNamespaces vt="8">http://BizUnit.BizTalkTestArtifacts.Schema0</DocumentSpecTargetNamespaces>
            <AllowUnrecognizedMessage vt="11">false</AllowUnrecognizedMessage>
            <ValidateDocument vt="11">false</ValidateDocument>
            <RecoverableInterchangeProcessing vt="11">false</RecoverableInterchangeProcessing>
            <HiddenProperties vt="8">EnvelopeSpecTargetNamespaces,DocumentSpecTargetNamespaces</HiddenProperties>
          </Properties>
        </Component>
      </Components>
    </Stage>
  </Stages>
</Root>

Context xml file:

In order to be able to verify the context of a BizTalk message after execution of the pipeline, the test steps allow for using an xml file, that contains the context of the BizTalk message(s).

<MessageInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ContextInfo PropertiesCount="1">
    <Property Name="Property1" Namespace="http://BizUnit.BizTalkTestArtifacts.PropertySchema1" Value="10" Promoted="true" />
  </ContextInfo>
</MessageInfo>

The output of the steps consist of:
– Xml output file
– Message context xml file, with the same format as the input context xml file.

The example below displays a BizUnit validation step snippet that checks if a certain property was promoted and has value 10:

<ValidationStep assemblyPath="" typeName="BizUnit.XmlValidationStepEx">
  <XPathList>
    <XPathValidation query="MessageInfo/ContextInfo/Property[@Name='Property1' and @Namespace='http://BizUnit.BizTalkTestArtifacts.PropertySchema1' and @Promoted='true']/@Value">10</XPathValidation>
  </XPathList>
</ValidationStep>

The ExecuteMapStep, ExecuteReceivePipelineStep and ExecuteSendPipelineStep have been included in BizUnit 3.1, which is now available at codeplex. Thanks, Kevin!

Posted in BizTalk | Tagged , , , | Leave a comment

Messaging-only request-response correlation

Recently I was asked if it was possible to “couple” two services via BizTalk without using orchestrations. Let’s look at that required scenario more closely:

1. A request message (ReqA) is received on the WCF-BasicHttp Receive Adapter, and it is posted to the MsgBox.
2. A solicit-response send port is activated due to a subscribtion to ReqA.
3. ReqA is mapped to ReqB.
4. ReqB is sent via the WCF-BasicHttp Send Adapter
5. RespB is received by the WCF-BasicHttp Send Adapter.
6. RespB is mapped to RespA and it is posted to the MsgBox.
6. RespB is picked up by the appropriate instance of the request-response receive port due to an instance subscription.
7. RespA is sent via the WCF-BasicHttp Receive Adapter.

The first questions that came up with me were:
1. How can the response of the outgoing webservice call be correlated to the incoming one? In the above scenario, that question would be “How can response B be correlated to response A?”. I certainly didn’t know that that comes out of the box with BizTalk. How? We’ll see that later on.
2. What happens if we configure multple (outgoing) solicit-response send ports, subscribing to messages coming in on the one request-response receive port? Will timing issues start to play a role? Or will an exception be thrown?

Before answering these questions, I would like to say that SoapUI is simply AWESOME when developing/working with web services (yes, including WCF web services). True, it’s JAVA application, but still… 🙂 It blows other webservice related stuff I’ve seen out of the water. Give it a try! In the above example, I used SoapUI to submit requests/receive responses to/from the WCF service, coupled to the request-response receive location.

In order to answer question 1, we could check the instance subscription generated when a message is received on the request-response receive port (by stopping -but not unenlisting- the subscribing sendport  and submitting a request), and it’s as follows:

BTS.EpmRRCorrelationToken == Svr:WIN2003EER2SP2;PID:3092;RRUid:{9731CF80-DDCF-4BB8-9CCA-03651243AE3E} And
BTS.RouteDirectToTP == True

Documentation on those properties is hard to come by. The BizTalk documentation states:

BTS.EpmRRCorrelationToken

Promoted by the Messaging Engine on request-response message execution. The property is promoted before messages are submitted into the MessageBox database. Used internally by the Messaging Engine. Specifies the Server Name, Process ID and a unique GUID for a request response stream of messages.

BTS.RouteDirectToTP

Promoted by the Messaging Engine on messages for loop back or request-response execution. The property is promoted before messages are submitted into the MessageBox database. Used internally by the Messaging Engine to enable loop back and request-response scenarios.

Anyone will agree that <censored>. However, from the fact that the above example is indeed working, we can assume that the EpmRRCorrelationToken property value is copied to the contexts of all messages involved in the communication flow. This implies that the EpmRRCorrelationToken property “survives” maps, the WCF solicit-response adapter and the XmlReceive pipeline. If you’d check the message context of the response message, published by the solicit-response port, you’ll find those properties present and promoted.

Question 2 can be answered fairly quickly. Configure & start a second solicit-response send port, and then fire a message into the request-response receive port (i.e. using SoapUI). This will give you a routing failure in BizTalk and the following soap fault on the soap client:

“The message found multiple request response subscriptions. A message can only be routed to a single request response subscription.”

Which basically means that a message just won’t get processed in this scenario. Makes sense, but would you have predicted that? Searching the web found a hotfix (KB923632), that enables scenarios where multiple request response subscriptions should be allowed. If you can think of a decent scenario where this would be needed, leave a comment!

How could we use this information?
One scenario could be to have an orchestration that uses two separate direct bound orchestration ports (one receive and one send), instead of one receiving request-response orchestration port to process the request and to submit a response. So instead of having an original request-response port, bound to a physical request-response port (“specify later”), like this:

…we create an orchestration with two separate direct bound ports like this:

Where the receive shape (Receive_1) has the following filter expression:
BTS.MessageType == http://Blah.BlahBlah.Schema1#Schema1Root&#8221;

The SetContext expression shape looked like this:
Message_2(*) = Message_1(*);

The result was that the response message (actually, better to say the transformed message, because we’re intentionally trying to avoid request-response terminology here) couldn’t be routed back to the physical request-response port (routing failure). Checking the message context revealed that the EpmRRCorrelationToken and RouteDirectToTP were present (due to copying over the properties), but they were not promoted, which is correct (because setting properties in an expression shape within an orchestration does not promote them). This can be solved by adding a correlation type and set, using these two context properties, and setting the send shape to initialize this correlation set. The result was that the message was correctly picked up by the subscribing request-response port, but another error occurred (event id 5813):

A response message for two-way receive port “ReceivePort1” is being suspended as the messaging engine could not correlate the response to an existing request message. This usually happens when the host process has been recycled.

Hmmmm, what’s all this about? Apparently some required properties, that are used internally by BizTalk for request response correlation, weren’t satisfied. After some fiddling around with context properties, performing a lot of WinMerge actions to compare message contexts (of the messages coming from the solicit-response ports (see above) and the messages coming from the orchestration), and most importantly, by finding this excellent post by Paolo Salvatori, describing exactly the property set needed, I changed the expression shape and the correlation type into the following:

Expression shape:
Message_2(*) = Message_1(*);
Message_2(BTS.RouteDirectToTP) = true;
Message_2(BTS.IsRequestResponse) = true;

Correlation Type:
BTS.CorrelationToken,BTS.EpmRRCorrelationToken,BTS.IsRequestResponse,
BTS.ReqRespTransmitPipelineID,BTS.RouteDirectToTP

The result was that the response was correctly picked up & processed by the instance-subscribing physical request-response port.

Conclusion

A much overlooked feature of BizTalk is the ability to route responses without the need for orchestrations by using messaging only. This post described what happens internally in such a scenario and also described how that information can be used to create an orchestration, in which the request and response can be communicated over separate direct bound orchestration ports instead of over one request-response port, which can be helpful (for instance) for testing the orchestration using physical FILE ports.

Posted in BizTalk | Tagged , , , | 1 Comment

Post-receive debatching without orchestrations in BizTalk Server

This post describes a way to perform debatching of a message after the original message has already been received and submitted to the BizTalk message box. This can be the case when for instance there are subscribers defined in BizTalk that subscribe to:
– The original message before debatching. To my knowledge, it isn’t possible to post both the original message and the debatched messages to the BizTalk MsgBox.
– A mapped version of the original message. To my knowledge, it isn’t possible to post both the original and the mapped messages to the BizTalk MsgBox.

So how do we handle a message in the BizTalk MsgBox, that needs to be debatched into seperate messages (without orchestrations)? Below a schematic representation of the solution used in this sample.

Post-receive debatching send port using loopback adapter

 

org = Original Message: contains some elements that occur 0..n.
arr = Array Message: a non-envelope message containing 0..n entities to debatch
env = Envelope Message: BizTalk envelope message with identical structure to array message
msg 1, msg 2, …, msg n = Debatched Messages

The steps in the above scenario:
1. A solicit-response send port has a subscription filter for the original message.
2. An outbound map is performed, transforming the original message into an array message.
3. Within the NamespaceReplacing pipeline component, the namespace for the array message is replaced with the namespace of the envelope message (this is in fact a transformation, though not via a BizTalk map).
4. The Loopback Adapter bounces the envelope message.
5. The XmlDisassemble component in the XmlReceive pipeline debatches (splits) the envelope message into its seperate messages.
6. The seperate messages are posted to the BizTalk MsgBox for further processing.

In order to make this work, there are some specific requirements to how the BizTalk schemas are setup. Below the representation of the used BizTalk schemas:

Envelope and entity schemas

 

org = Original Message = OrdersRequest
arr = Array Message = InternalOrders
env = Envelope Message = InternalOrdersEnvelope
msg 1, msg 2, …, msg n = Debatched Messages = InternalOrder

Key thing here is that the array schema has an xml structure similar to the envelope schema. In fact, the envelope schema imports the array schema, and the envelope schema’s root element type is set to be the same as the root element type of the array schema. The only differences between the array schema and the envelope schema are:
– The root namespace.
– The envelope schema is configured to be a BizTalk envelope schema, while the array schema is not.

So why the Namespace replacing?
You may have noticed that the above could also be done by mapping from the original message to the envelope message directly, and not (have to) use the namespace replacing component. However, if you will try this, you will find that BizTalk doesn’t allow to map TO envelope schemas, when that map is configured as inbound or outbound map on a BizTalk physical port. You’ll get an error like (event id 5782):

The Messaging Engine failed while executing the outbound map for the message going to the destination URL "LOOPBACK://cfc0607f-e119-4e6b-ae34-8fae2a92b084" with the Message Type "http://Samples.Messaging.Schemas.OrdersRequest#OrdersRequest". Details:"Document type "http://Samples.Messaging.Schemas.InternalOrder#InternalOrder" does not match any of the given schemas. "

I will send a chocky bar of choice to the first one to find the solution for this error, wherever you are!

In order to work around this issue, I have used the Namespace replacer component to perform a 1:1 map from the array schema to the envelope schema by replacing the root namespace. A true BizTalk map, with all elements from the array schema ‘connected’ to the corresponding elements in the envelope schema, should have had the same effect (were it not for the fact that BizTalk doesn’t allow that on ports).
The Namespace replacer component has been implemented by using a lot of code from the Microsoft.Practices.ESB Namespace pipeline components (RemoveNamespace component). The Namespace replacer component takes two configuration parameters: OldNamespace and NewNamespace, and it just substitues the OldNamespace on all elements and attributes it encounters with the NewNamespace.

The XmlReceive pipeline configuration in this sample is pretty standard: debatching using an envelope schema, exactly like you would do on a ‘normal’ receive port.

This sample provides a way to debatch (or normalize) a message on a single port without the need for orchestrations. Think of it as a debatching port or normalizing port. There are some limitations to this approach:
– Error handling isn’t possible on a per-debatched-message basis. If debatching fails (on one debatched message), I assume you will get one suspended envelope message. When using orchestrations for debatching, the error handling could be done per debatched message. This could (to some extent) be mitigated by turning on Recoverable Interchange Processing on the XmlReceive Pipeline in combination with Failed Message Routing.
– …? any additions are welcome.

The sample runs on BizTalk Server 2006 R2. You can download it here.

Some other techniques for debatching messages are described here. You can find the Loopback adapter here.

BTW, If you’re interested in the namespace replacement only, you’re much better off writing a custom pipeline component that uses the Microsoft.BizTalk.Component.NamespaceTranslatorStream class, found in Microsoft.BizTalk.Pipeline.Components.

Posted in BizTalk | Tagged , , | 1 Comment