What do you include in your BizTalk project?

 I personally only have the actual BizTalk items (schemas, pipelines, maps, orchestrations). And of course, the solution might also have other projects like custom external DLLs, custom pipeline components, test projects, .NET 3.X projects, all the stuff that will eventually end up in a DLLs or running in IIS.

But my question is, do you also include the binding files, strong name key files, documentation, etc. in your project? An email from a friend has prompted me to write this. I personally think that while these files definitely have to be kept in source control together with the solution files, they shouldn’t be added to the projects.

Expose an orchestration as a web service difference between BizTalk 2004 and BizTalk 2006

Another issue we ran into while upgrading a client’s BizTalk 2004 solution to BizTalk 2006 R2 is related to exposing an orchestration as a web service. Our client has a website that calls a web service exposed orchestration. The website sends to BizTalk the information filled out on a “Sign Up” page by a potential new customer, and BizTalk maps and forwards the info to our customer’s internal CRM system.

 This web service call was working in BizTalk 2004, but once we deployed the BizTalk 2006 version of the same orchestration, the following error was thrown by the web service:

[NullReferenceException: Object reference not set to an instance of an object.] 

System.Web.Services.Protocols.SoapHeaderHandling.SetHeaderMembers(SoapHeaderCollection headers, Object target, SoapHeaderMapping[] mappings, SoapHeaderDirection direction, Boolean client)

System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters) 

The error stack continues to include the web service’s operation and the application’s Page_Load method. The null reference exception is an error that we all love isn’t it? It explains so much!

Anyways, during our investigation my friend and co-worker Mark realised that the operation description between the BizTalk 2004 and the BizTalk 2006 versions of the web service were different for some fields. All of the fields with enumeration had the correct enumeration in 2004 but show up as bytes in 2006.

So, say this is the  URL for the operation:

http://localhost/MyTestProject_Proxy/MyTestOrchestration_MyTestPort.asmx?op=MyTestServiceOperation

In 2006 the SOAP sample shown was, for example (notice the NewCustomer field):

<?xml version=”1.0″ encoding=”utf-8″?>

<soap:Envelope xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” xmlns:soap=”http://schemas.xmlsoap.org/soap/envelope/”>

  <soap:Body>

    <MyTestServiceOperation xmlns=”http://tempuri.org/”>

      <MyTest xmlns=”http://BizTalk”>

        <MessageHeader xmlns=”">

          <MessageName>string</MessageName>

          <TimeStamp>dateTime</TimeStamp>

          <NewCustomer>bytes</NewCustomer>

          <UserId>string</UserId>

        </MessageHeader>

        …

But in 2004 the SOAP sample has the correct enumeration:

<?xml version=”1.0″ encoding=”utf-8″?>

<soap:Envelope xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” xmlns:soap=”http://schemas.xmlsoap.org/soap/envelope/”>

  <soap:Body>

    <MyTestServiceOperation xmlns=”http://tempuri.org/”>

      <MyTest xmlns=”http://BizTalk”>

        <MessageHeader xmlns=”">

          <MessageName>string</MessageName>

          <TimeStamp>dateTime</TimeStamp>

          <NewCustomer>Y or N</NewCustomer>

          <UserId>string</UserId>

        </MessageHeader>

        …

And that happened to all the enumerated fields on the schema exposed by the orchestration. We  had to fix this, even if it doesn’t end up fixing the null reference error. I knew I’d seen how to change this behaviour somewhere before… But where? Well, google gave me the answer. My own blog! I had run into this while writing this article on how to test an orchestration with WCF.

  This behaviour is caused by a change on the way the web services wizard generates a class from the schema exposed by the orchestration. Try using xsd.exe to generate a class from the same schema in .NET 1.1 and .NET 2.0 and you will see that all enumerated fields in .NET 2.0 have an XML type attribute with AnonymousType set to true.

This generated class created by the BizTalk 2006 web services wizard sits on the App_Code folder under the root of the web service, and the file is called DataTypes.cs.

So, in my example, the code generated for the NewCustomer enumeration inside DataTypes.cs was:

[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "2.0.50727.42")]

[System.SerializableAttribute()]

[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, Namespace=http://BizTalk)]

public enum MyTestMessageHeaderNewCustomer {

        /// <remarks/>    Y,

        /// <remarks/>    N,

}

Once we removed AnonymousType=true from all the enumerations inside DataTypes.cs the SOAP sample started showing the correct enumeration values and the website call worked. Phhew!

Yossi Dahan explains here in more detail what that attribute does to the WSDL.

 Note (!!!): Don’t use namespaces like http://BizTalk or web service operation default namespace like http://tempuri.org/ in real implementations! Bad bad bad.

Kauaeranga Valley

View from The Pinnacles, Coromandel, New Zealand.

Flat file disassembler difference between BizTalk 2004 and 2006

  We are updating a medium/large sized solution from BizTalk 2004 to BizTalk 2006 R2 for a client at the moment. We have run into a couple of interesting and unexpected issues which I’ll describe on this post and the next. The first “issue” we ran into is that the flat file disassembler in BizTalk 2006 and R2 now complain when there is unaccounted data (not described in the schema) at the end of a flat file. I put double quotes around issue because it isn’t really one; it’s doing what it should. It’s BizTalk 2004 that lets the extra data get away with it.

  A lot of changes have been made to the flat file disassembler pipeline component that ships with BizTalk 2006 and BizTalk 2006 R2 when compared to its BizTalk 2004 version. Most of it is related to the recoverable interchange feature.

 While everything was working fine in BizTalk 2004 we kept getting the following error message raised by the flat file disassembler component in BizTalk 2006:

“The remaining stream has unrecognizable data“

  So, let’s have a closer look at what we have here. Let’s say we have the following flat file schema:

 Where Record1 has max occurs unbounded and min occurs of 0, and Record2 has max occurs of 1 and min occurs of 0. All four elements are restricted with a max length of 8. The default child delimiter is ; and both Record1 and Record2 use the default child delimiter. The default child order is Postfix but the child order of Record1 and Record2 is Infix. The tag name of Record1 is “Record1;” and tag name of Record2 is “Record2;”. Here’s the schema code:

<?xml version=“1.0“ encoding=“utf-16“?>

<xs:schema xmlns=“http://TestBizTalkProject.TestSchema“ xmlns:b=“http://schemas.microsoft.com/BizTalk/2003“ targetNamespace=“http://TestBizTalkProject.TestSchema“ xmlns:xs=“http://www.w3.org/2001/XMLSchema“>

    <xs:annotation>

        <xs:appinfo>

            <b:schemaInfo count_positions_by_byte=“false“ parser_optimization=“complexity“ lookahead_depth=“3“ suppress_empty_nodes=“false“ generate_empty_nodes=“true“ allow_early_termination=“false“ standard=“Flat File“ root_reference=“Root“ child_delimiter_type=“char“ default_child_delimiter=“;“ default_child_order=“postfix“ />

            <schemaEditorExtension:schemaInfo namespaceAlias=“b“ extensionClass=“Microsoft.BizTalk.FlatFileExtension.FlatFileExtension“ standardName=“Flat File“ xmlns:schemaEditorExtension=“http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions“ />

        </xs:appinfo>

    </xs:annotation>

    <xs:element name=“Root“>

        <xs:annotation>

            <xs:appinfo>

                <b:recordInfo structure=“delimited“ preserve_delimiter_for_empty_data=“true“ suppress_trailing_delimiters=“false“ sequence_number=“1“ child_order=“postfix“ child_delimiter_type=“default“ />

            </xs:appinfo>

        </xs:annotation>

        <xs:complexType>

            <xs:sequence>

                <xs:annotation>

                    <xs:appinfo>

                        <b:groupInfo sequence_number=“0“ />

                    </xs:appinfo>

                </xs:annotation>

                <xs:element minOccurs=“0“ maxOccurs=“unbounded“ name=“Record1“>

                    <xs:annotation>

                        <xs:appinfo>

                            <b:recordInfo sequence_number=“1“ structure=“delimited“ preserve_delimiter_for_empty_data=“true“ suppress_trailing_delimiters=“false“ tag_name=“Record1;“ child_delimiter_type=“default“ child_order=“infix“ />

                        </xs:appinfo>

                    </xs:annotation>

                    <xs:complexType>

                        <xs:sequence>

                            <xs:annotation>

                                <xs:appinfo>

                                    <b:groupInfo sequence_number=“0“ />

                                </xs:appinfo>

                            </xs:annotation>

                            <xs:element name=“Element1“>

                                <xs:annotation>

                                    <xs:appinfo>

                                        <b:fieldInfo sequence_number=“1“ justification=“left“ />

                                    </xs:appinfo>

                                </xs:annotation>

                                <xs:simpleType>

                                    <xs:restriction base=“xs:string“>

                                        <xs:maxLength value=“8“ />

                                    </xs:restriction>

                                </xs:simpleType>

                            </xs:element>

                            <xs:element name=“Element2“>

                                <xs:annotation>

                                    <xs:appinfo>

                                        <b:fieldInfo sequence_number=“2“ justification=“left“ />

                                    </xs:appinfo>

                                </xs:annotation>

                                <xs:simpleType>

                                    <xs:restriction base=“xs:string“>

                                        <xs:maxLength value=“8“ />

                                    </xs:restriction>

                                </xs:simpleType>

                            </xs:element>

                        </xs:sequence>

                    </xs:complexType>

                </xs:element>

                <xs:element minOccurs=“0“ maxOccurs=“1“ name=“Record2“ nillable=“true“>

                    <xs:annotation>

                        <xs:appinfo>

                            <b:recordInfo sequence_number=“2“ structure=“delimited“ preserve_delimiter_for_empty_data=“true“ suppress_trailing_delimiters=“false“ child_order=“infix“ child_delimiter_type=“default“ tag_name=“Record2;“ />

                        </xs:appinfo>

                    </xs:annotation>

                    <xs:complexType>

                        <xs:sequence>

                            <xs:annotation>

                                <xs:appinfo>

                                    <b:groupInfo sequence_number=“0“ />

                                </xs:appinfo>

                            </xs:annotation>

                            <xs:element name=“Element3“>

                                <xs:annotation>

                                    <xs:appinfo>

                                        <b:fieldInfo sequence_number=“1“ justification=“left“ />

                                    </xs:appinfo>

                                </xs:annotation>

                                <xs:simpleType>

                                    <xs:restriction base=“xs:string“>

                                        <xs:maxLength value=“8“ />

                                    </xs:restriction>

                                </xs:simpleType>

                            </xs:element>

                            <xs:element name=“Element4“>

                                <xs:annotation>

                                    <xs:appinfo>

                                        <b:fieldInfo sequence_number=“2“ justification=“left“ />

                                    </xs:appinfo>

                                </xs:annotation>

                                <xs:simpleType>

                                    <xs:restriction base=“xs:string“>

                                        <xs:maxLength value=“8“ />

                                    </xs:restriction>

                                </xs:simpleType>

                            </xs:element>

                        </xs:sequence>

                    </xs:complexType>

                </xs:element>

            </xs:sequence>

        </xs:complexType>

    </xs:element>

</xs:schema>

Now let’s say we have a text file with the following data:

Record1;Element1;Element2;Record2;Element3;Element4;

Copy the schema file (say Test.xsd) and the text file above (say Test.txt) to the PipelineTools folder in the BizTalk sdk. Run the following in the visual studio command prompt:

FFDasm.exe Test.txt -bs Test.xsd

The above tests the flat file disassembler. It works for both versions of BizTalk. Now if we change the data to:

Record1;Element1;Element2;Record2;Element3;Element4;gibberish

 and test again you will see that BizTalk 2004 still processes the message with no errors and disregards the gibberish at the end of the file, where BizTalk 2006 throws the “The remaining stream has unrecognizable data” error.

  This difference in behaviour is inside the “GetNext2″ method of the FFDasmComp class inside Microsoft.BizTalk.Pipeline.Components.dll. In BizTalk 2004 that error message would only be thrown when there is a trailer schema supplied and it can’t probe the extra data against the schema. It disregards any extra data if there’s no trailer schema supplied. In BizTalk 2006 that extra data must be accounted for as it has an extra if statement checking for it in the code.

 We only ran into the issue because there is actually trailer data in the messages coming into our client’s BizTalk Server and the original developers did not define that trailer in the schema or as a trailer schema (probably because BizTalk 2004 didn’t complain about it).

 The fix to this in BizTalk 2006 is simple: create a catch all flat file schema (with a record and a string element with no restrictions) and define it in the flat file disassembler as the trailer schema. This has already been described here and here.

So for example create the following schema:

<?xml version=“1.0“ encoding=“utf-16“?>

<xs:schema xmlns=“http://BizTalk_Server_Project1.Trailer“ xmlns:b=“http://schemas.microsoft.com/BizTalk/2003“ targetNamespace=“http://BizTalk_Server_Project1.Trailer“ xmlns:xs=“http://www.w3.org/2001/XMLSchema“>

    <xs:element name=“Root“>

        <xs:complexType>

            <xs:sequence>

                <xs:element name=“Element1“ type=“xs:string“ />

            </xs:sequence>

        </xs:complexType>

    </xs:element>

</xs:schema>

Say it’s called “Trailer.xsd”. If you run the following it will work in BizTalk 2006 correctly:

FFDasm.exe Test.txt -bs Test.xsd -ts Trailer.xsd

Note that this is not picked up by validating the sample file against the schema inside visual studio, hence the use of FFDasm.exe.

External harddrive packed with BizTalk goodies

Microsoft has kindly sent our company an 80gb laptop external harddrive packed full with videos, presentations, whitepapers, etc. on BizTalk and SOA. Most of it you can find online, but the harddrive is great as it’s all already downloaded and categorized into folders.

Whenever I have time I’m going through the presentations, it’s great stuff!

Back after 3 months!

You know when there is a lot of work before the Christmas/New Year’s break? Well, this year was different, there’s quite a lot more work AFTER the break, I’m involved in a few projects at the same time. Keep tuned as I’m finally back!