[INACTIVE BLOG] Connected Thoughts – Thiago Almeida

November 20, 2009

Insight into BizTalk Server message tracking

Filed under: BizTalk — Thiago Almeida @ 7:41 am

I had to go through this a few times recently so I thought I’d document it:: when does BizTalk Server save tracking information to the tracking database, and how does it all link together?

Basically BizTalk will track information about a messages in the following locations:

  • On a receive port right after the adapter receives the message and before the receive pipeline or any maps are executed
  • On a receive port right after the pipeline and the maps are executed – this means the message type tracked after the receive port is the message type after the map execution
  • On an orchestration as a message is received
  • On an orchestration as a message is sent
  • On a send port right before any maps or the send pipeline is executed
  • On a send port right after the pipeline is executed just before the adapter sends it out

Let’s consider a typical scenario: a message is received via a receive port with the file adapter. No disassembly is made (i.e. there is only one document in the message). An orchestration subscribes to the message, does some processing, and sends out only one message. A send port bound to the orchestration sends the message out.

This diagram shows the stages where BizTalk would record information about the message to the tracking database:

image

Note that BizTalk records that the message has gone through those stages even if you haven’t enabled property or message body tracking on the ports or orchestration. This is recorded online by the Tracking Data Decode Service to the dta_MessageInOutEvents and other tables in the BizTalk Tracking database. If message content and context tracking is enabled on the particular service then the BizTalk SQL Agent  jobs will take care of moving this from the Message Box database to the Tracking database on a scheduled basis (every minute with the default setting).

Now that we’ve established the basics of where BizTalk records the message information, how does it link together? How do you know the sequence of what happened after BizTalk has processed the messages? If you look back at the diagram you will see that there is a direction and a different message GUIDs at each message tracking point, and there is a different Service Instance GUID on receive ports, send ports and orchestrations.  Service instances are the specific instance of the service (receive pipeline, send pipeline, orchestration) that handled that particular message. A service instance will receive a message with a message GUID and send a message with a different message GUID, with the exception of when failures occur. So a receive port service instance receives a message with message GUID 1. It then sends one or more messages out, each with their own message GUIDs. The next service instance (send port or orchestration) that subscribes to those messages will have their own service GUID, but the received message GUID is the same as the message GUID sent by the receive port. Then if this orchestration or send port service instance sends out one or more messages these will have their own message GUIDs as well, and so forth.

To make it clear, let’s look at a scenario where a receive port receives an XML message and disassembles it into two messages; then two instances of an orchestration that subscribes to the disassembled messages performs some operation on them; then the orchestration instances send out a final message via a send port. With this you will have this GUID tree structure in the tracking database:

  • Receive Port Service Instance 69dee51d-21c0-4f85-ba80-58fdb08da6ec
    • Receive Message 2b220b2d-a07a-4752-ad23-4cfd9f769c25 << Unparsed Interchange (ie original unchanged message)
    • Send Message 56684d73-5dac-4bd8-a2ce-faba9bc89b6c
    • Send Message 7553db2f-3ea6-47a1-9cf4-fdb02d6d3c37
    • Orchestration Service Instance f3dfe962-3718-4995-bcfb-d3f0cdf82772
      • Receive Message 56684d73-5dac-4bd8-a2ce-faba9bc89b6c
      • Send Message dade4b4b-cc39-4464-b587-d8fa09772796
        • Send Port Service Instance a96660f7-9665-4a05-b440-6929679eafd5
          • Receive Message dade4b4b-cc39-4464-b587-d8fa09772796
          • Send Message 854a04da-d1b1-4c26-94df-7e36fbe8cb3b << end of tree
    • Orchestration Service Instance 380185b9-0c0e-4b21-afd5-94afebc11dfb
      • Receive Message 7553db2f-3ea6-47a1-9cf4-fdb02d6d3c37
      • Send Message 36baa5ba-3cb8-45aa-afe3-1d0e0b60469a
        • Send Port Service Instance 68041f1b-242f-4043-8f95-a6227b82dd12
          • Receive Message 36baa5ba-3cb8-45aa-afe3-1d0e0b60469a
          • Send Message 67e51bf5-e530-47d9-90b5-48ffeaca8ad0 << end of tree

As you can see there is an initiating tracking entry with an unparsed interchange receive and the tree builds from there with any number for nodes. If you enable message body tracking on the receive port, orchestration or send port BizTalk will also track the message content. It is with the information from the GUID values that the ‘message flow’ option in the BizTalk Administration Console is built. If you open the BizTalk Administration Console and look at the Tracked Message Events you can see the direction in the Event Type column, the schema name (with unparsed interchange as the first one), and the service class/service name to determine the service that was handling the message. With the above scenario you will get a result like the below: a received unparsed interchange and two sends from the same pipeline service instance. Two orchestration instances receive each message and send a message out. And two send pipeline instances receive the messages from the orchestration and sends them out:

image

If you right click on the unparsed interchange entry and select ‘message flow’ you get the good old HAT view of the message flow, that gives us a lot of information: the ‘Instance ID’ at the top is the service instance id of the receive pipeline. Each entry on the table has an in or out event depicted by the envelope with an arrow coming in or an arrow going out. Each entry also shows the message id and has a clickable link to the next or previous step in the tree if there is one. I’ve highlighted in red the parts of interest:

image

From the above you can then click on one the orchestration links and the message flow will show the flow from that orchestration instance’s perspective (where you can also switch to the orchestration debugger but that’s another story):

image

This way you can traverse the entire tree of what happened after the original message was received. Another way of following this structure is by querying the dtav_MessageFacts view on the BizTalk Tracking database and building a common table expression query (with MUCH caution) around the three values [MessageInstance/InstanceID], [ServiceInstance/InstanceID], [Event/Direction].

I hope with this insight into message tracking in BizTalk you can now follow the tracking of messages and events through the BizTalk Administration Console with ease!

Regards,

Thiago Almeida

About these ads

6 Comments

  1. Hi Thiago,

    Good article! Thanks!

    Just a comment:
    In the Query result there is an ambiguous “Event type” column.
    For Orchestration it is clear, all operations are with the MessageBox on the other side. “Receive” equals “Receive from MB” and “Send” equals “Send to MB”. But for the Pipeline-adapter services it is not so. No column that gives us idea what is the service. Is it a Send Pipeline-adapter or is it a Receive Pipeline-adapter. Without this information the Event type means … you don’t know without investigation.
    For example, for the the Receive Pipeline-adapter the “Receive” event type means the raw original message before all processing. For the Send Pipeline-adapter the “Receive” event type means the Xml message from the MB.
    Seems the same word used in defferent places for opposite meanings.

    Leonid Ganeline [BizTalk MVP]

    Comment by Leonid Ganeline — February 9, 2010 @ 7:28 am

    • Hi Leonid,

      Thanks! Yes, you have to determine if it is a receive location or send port before you can determine if the Receive means receive from the outside world (receive ports) or if Receive means receive from the message box (send ports). I currently use the Port Name column to figure out if it is a receive or send port, but maybe a Port Type column would be useful.

      Regards,
      Thiago

      Comment by Thiago Almeida — February 9, 2010 @ 1:52 pm

  2. [...] Original Post : Find it here [...]

    Pingback by BizTalk Server Message Tracking « Raheem's Tech Diary — April 14, 2010 @ 8:49 am

  3. Hi Thiago,

    I am quiet new to Biztalk and this article helps me to get a good view to the send and recieve handling and all what’s happening there.

    Thanks,
    Theo verhage

    Comment by Theo Verhage — July 15, 2011 @ 7:09 pm

  4. Hi Thiago,

    It is really a nice article.
    I have a question, if i use passthrough pipeline, will there be still two MessageID’s (one unparsed, right after the message arrived and one before publising to message box) ?

    Thanks
    Guru.

    Comment by kumaragurus — December 28, 2011 @ 1:32 am

    • Hi, it’s been a while but I think with a pass through pipeline it’s the same message id

      Comment by Thiago Almeida — December 28, 2011 @ 5:36 am


RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

The WordPress Classic Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 39 other followers

%d bloggers like this: