Feb 032012
 

Earlier in the week I spent some time trying to accomplish what I thought was going to be a trivial task – mapping from a sequence structure to a flat file structure using BizTalk Development Tools.

The main idea is to take a looping structure on the left hand side and map it to a flat structure on the right hand side.

As it turns out, it’s pretty easy to do, and not complicated.  I spent a lot of time trying to do something more dramatic, if you’re looking for this info, I think you’ll be surprised by how logical the solution is.

Take a schema like the following:

image

Which defines a structure along the lines of this:

<ns0:Account xmlns:ns0="http://Mapping_Project.Inbound">
  <FirstName>Jed</FirstName>
  <LastName>Bartlett</LastName>
  <Address>
    <Line1>1 Constitution Ave</Line1>
    <Line2></Line2>
    <Suburb>Lincoln</Suburb>
    <State>Nebraska</State>
    <Country>USA</Country>
    <PostCode>90210</PostCode>
    <Type>Home</Type>
  </Address>
  <Address>
    <Line1>Suite 10</Line1>
    <Line2>1090 Eggplant Blvd</Line2>
    <Suburb>Washington</Suburb>
    <State>Nebraska</State>
    <Country>USA</Country>
    <PostCode>90211</PostCode>
    <Type>Work</Type>
  </Address>
</ns0:Account>

We’re going to try and transform it into something ugly, like this:

image

Which means flattening the data from the left hand side to the right hand side – but how?

image

First up, generate some test data. It’s a very good habit to get into, and it’s easy as pie. Right click on a schema and select “Generate Instance”. Modify the values so they’re easy to debug with and you’re set.

image

Let’s look at how we transform this simple XML.  The answer, as it turns out is blindingly simple.

Open up your map..

Now, let’s assume we want to apply a condition to how we separate the data on the left hand side.  Let’s assume there will be two types of addresses (Home and Work).  Our aim is to assign all the “Address 1” address details as “Home”, and all the “Address 2” details as “Work”.

Drop two Logical EQUALs functoids onto the map and join them to the “Type” attribute.  Then, open each functoid and we’ll set the comparison value to ‘Home’ and ‘Work’ respectively (omit the quotes).  This applies the logic to split the output.

1. image 2. image

Now, assign all the attributes directly across to their destination.

image

Next, connect each equals functoid to the destination it reflects, i.e. connect the functoid for “Home” to each of the “Address 1” properties, etc.  Don’t forget to map the “Type” too, if you want to capture that detail.

image

Given the data I’ve specified above – let’s give it a test!  With the map open in Visual Studio, open the Properties Window and take a look at the following details:

image

Change the “Test Map Input” to “XML” and then you can specify the path to a test XML document (which you might have created earlier).  Here’s how mine ended up:

image 
image

To test, right click on the map file in Solution Explorer and select “Test Map”.  You should get some results in the output window:

TestMap used the following file: <file:///C:\Dev\BizTalk\Mapping\Mapping Project\Test Data\Inbound_Data.xml> as input to the map.
Test Map success for map file C:\Dev\BizTalk\Mapping\Mapping Project\Maps\InboundToOutbound.btm. The output is stored in the following file: <file:///C:\Dev\BizTalk\Mapping\Mapping Project\Test Output\Output.xml>
Component invocation succeeded.

Which provides a generated instance which looks like this:

<ns0:AccountDetails xmlns:ns0="http://Mapping_Project.Outbound">
  <Name>Jed Bartlett</Name>
  <Address1_Line1>1 Constitution Ave</Address1_Line1>
  <Address1_Line2 />
  <Address1_Suburb>Lincoln</Address1_Suburb>
  <Address1_State>Nebraska</Address1_State>
  <Address1_Country>USA</Address1_Country>
  <Address1_PostCode>90210</Address1_PostCode>
  <Address1_Type>Home</Address1_Type>
  <Address2_Line1>Suite 10</Address2_Line1>
  <Address2_Line2>1090 Eggplant Blvd</Address2_Line2>
  <Address2_Suburb>Washington</Address2_Suburb>
  <Address2_State>Nebraska</Address2_State>
  <Address2_Country>USA</Address2_Country>
  <Address2_PostCode>90211</Address2_PostCode>
  <Address2_Type>Work</Address2_Type>
</ns0:AccountDetails>

There were some warnings about multiple mappings, but I didn’t find the absence of a looping functoid to be a problem.  I realise this is a pretty obvious and simple mapping, but when I went looking for examples about how to solve this type of mapping, I couldn’t find much at all (including from e-books).

There’ll be more on BizTalk mapping and other fun stuff in some upcoming articles.

Jan 032012
 

Introduction

As I alluded to at the very end of 2011, I’ve been spending more and more time with BizTalk 2010. It’s been a little while since I’ve had to support/build/maintain BizTalk artefacts, but it doesn’t take long to reacquire “the groove”.

To gently thrust you into the dark underbelly of the land of BizTalk development, I thought I’d start off by exploring some fairly basic BizTalk 101 type experiences (and solutions!).  This article focuses on generic issues with XML schemas, with an eye to BizTalk’s usage of said schema.

Sadly, some of what you might read here (and later) might smack a tad of ‘What?  You didn’t know that?”, but I’m willing to swallow some pride if it helps out other Architects and Programmers.

What you need (to follow along)

This article is (IMHO) decent enough for the generalist (non-BizTalk specialist) and especially may be handy for those unfamiliar in working with XML schemas.

There is a big part where I’m playing with BizTalk’s Developer Tools (in Visual Studio 2010), so if you’d like to give it a go, I’d suggest you pick up a copy of BizTalk Server 2010 Developer Edition (click here for more info on this edition of BizTalk), which – I believe – you need to have to install the developer tools.

If I write subsequent article on BizTalk, the BizTalk developer tools will likely play a significant role, so it might be worth investigating.  As always, it’s worth bookmarking the BizTalk Development Center link.

The Joys of Schema

If you know anything about BizTalk, you’ll know that it’s all about the schemas. Everything ends up mapping to a document schema type whether it’s inbound or outbound. Schemas (XSD) can be an acquired taste, especially if you’ve been off doing something more exciting.

Last month, handwritten schemas tripped me up quite a bit. I was used to using Wizards and Adapters to generate a lot of the schemas for me, so when I went to design some basic input and output schemas, I encountered some “fun” issues.

Something funny happened…

Now, I was putting together some very basic sample schema to use as an example for demonstrating a really neat technique with maps (transforms).

Just in case someone else encounters this (or, like me, found it mildly amusing) I thought I’d document some cause and effect of why you might get some head scratching error messages when designing in Visual Studio 2010 with the BizTalk designer.

Given the following (flawed) schema, which I built within Visual Studio:

<?xml version="1.0" encoding="utf-16"?>

<xs:schema xmlns="http://Sandbox.InputSchema" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" targetNamespace="http://Sandbox.InputSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="Root">

    <xs:complexType>

      <xs:sequence>

        <xs:element minOccurs="2" maxOccurs="3" name="Customer">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Address">

                <xs:complexType>

                  <xs:attribute name="Street" type="xs:string" />

                  <xs:attribute name="State" type="xs:string" />

                  <xs:attribute name="Country" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="ID" type="xs:integer" />

            <xs:attribute name="FirstName" type="xs:string" />

            <xs:attribute name="Surname" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Which, when loaded in the BizTalk Schema Designer looks like this:

clip_image002

Has an unfortunately obvious flaw if you consider I was expecting the schema to facilitate the following format of XML data:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

In other words, I was expecting to store multiple “Customer” elements in my XML document.

Debugging

When I loaded the document and tested it in Visual Studio 2010, I received the following, very weird error message: “The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’.” – huh?

clip_image004

The problem, I thought, might be related to not expressly prefixing the namespace directive to the child element, so I made some edits.

This resulted in an even more ambiguous error message, funny even:

“The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’ in namespace ‘http://Sandbox.InputSchema’. List of possible elements expected: ‘Customer’.”

or, in generic terms:

“The element ‘<element>’ in namespace ‘<namespace>’ has invalid child element ‘<child element>’ in namespace ‘<namespace>’. List of possible elements expected: ‘<child element>’.”

WTF?

clip_image006
You may have to squint to read the text…

At this point, I felt like I was going backwards. So I undid my changes and took a look at the schema definition – which, upon second inspection was clearly wrong.

The Solution

See if you can spot the difference?

clip_image008

My mistake was in not declaring a sequence after the root element. Thus rectified, my XML now matched the fixed schema, and I was able to continue on my merry way.

clip_image009

Filtering in a Transform/Map

This all brings us back to the original intention of my article – to highlight the use of functoids for filtering multiple nodes in a transform.

Given that we want to transform our input schema (from before) into another format, with filtered data, here’s the schema for the output:

clip_image010

The Mapping

We can construct a map which assigns the fields we want per the following example:

clip_image012
A basic map

Now, if we wanted to only take Customers who live in the state of New South Wales, we need to validate each element in the input XML recursively.

A Functoid for all occasions

The easiest means is to use a equality functoid, in this example I’ve used an equals functoid, as per below. You must still map the field directly to the output schema, but also to the functoid – which is, in turn, mapped to the logical root of the target (per the example above).

clip_image013

When the functoid returns a true value, the element is copied to the output. If the result is fale, the entire element (children etc) are skipped. Thus, from the following input XML:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

When run through this map (using trusty ‘Test Map’ functionality), will generate the following output, filtering out that pesky Queensland (QLD) customer:

<ns0:Root xmlns:ns0="http://Sandbox.OutputSchema">

  <Address Street="12 Jump Street" State="NSW" Country="Australia" CustomerID="100" />

</ns0:Root>

Helpful Support in Visual Studio 2010

Before you start despairing about schema development and BizTalk maps, there are a bunch of very helpful tools built into Visual Studio 2010 (and prior versions).

Generating Schema Instances

Given you have Schema files in your project, you can right click them and select “Generate Instance”. This will create an XML file using intelligent default values, according to the schema definition. These files can be modified and used to test out other artefacts who use this schema.

Validating Schema Instances

If you want to test if a particular XML file is valid, you can do the reverse – by clicking on the schema properties, you can set the “input XML” for a given schema. Then, when right clicking on the Schema in Solution Explorer, you can select “Validate Instance”. This will highlight any issues with your schema.

clip_image014

Testing a Map/Transform

Once you have some valid XML for your schema, you can then use it as test input data for various things including maps.

The following shows how to establish test data parameters for a map within Visual Studio 2010:

clip_image016

When you have configured a map to use a specified XML input, you can then right click on the map (in Solution Explorer) and select “Test Map”.

Other Options – Xml Schema/Data Type Utility

A handy note for future reference – if you have an XML structure in mind, but (like me) you are rusty with your XSD designing skillz, there’s another way forward which may prove to be better.

There is a tool which helps you work with XML and XSD document formats, and it ships by default with a number of Microsoft products and SDKs, it is called “Xml Schemas/DataTypes support utility” or XSD.exe.

Important Disclaimer: In nearly almost all circumstances, it is considered a better practice to produce or design the XML schema first. Generating a schema from XML data can be fraught with peril, especially if your XML sample doesn’t accurately reflect the full spectrum of supported formats.

For example, min/max occurrences, type lengths and other constraints, vague data types (where the type or precision is “guessed”), elements supported by the schema which are not required – and thus missing from your sample XML – to name but a few.

Where Do I Find This Tool?

Chances are that you have it already. I’ve located it installed in the following locations (check the (x86) folder as well if you are on a 64 bit machine):

Directory of C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\x64

What does it do?

Keeping in mind that this utility was really meant for the days of evil DataSets, it still can be quite useful.

Using the following (poor) sample XML structure:

<?xml version=”1.0” encoding=”utf-8”?>
<Country Name="Australia">
     <State Name="NSW">
           <Capital Name="Sydney"/>
     </State>
     <State Name="QLD">
           <Capital Name="Brisbane"/>
     </State>
     <State Name="WA">
           <Capital Name="Perth"/>
     </State>
     <State Name="TAS">
           <Capital Name="Hobart"/>
     </State>
     <State Name="VIC">
           <Capital Name="Melbourne"/>
     </State>
</Country>

You can use the XSD.exe utility to generate an approximation of the schema, like so:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xml

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.xsd’.

Which would produce the following output:

<?xml version="1.0" encoding="utf-8"?>

<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">

  <xs:element name="Country">

    <xs:complexType>

      <xs:sequence>

        <xs:element name="State" minOccurs="0" maxOccurs="unbounded">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Capital" minOccurs="0" maxOccurs="unbounded">

                <xs:complexType>

                  <xs:attribute name="Name" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="Name" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

      <xs:attribute name="Name" type="xs:string" />

    </xs:complexType>

  </xs:element>

  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">

    <xs:complexType>

      <xs:choice minOccurs="0" maxOccurs="unbounded">

        <xs:element ref="Country" />

      </xs:choice>

    </xs:complexType>

  </xs:element>

</xs:schema>

Now, given I mentioned that bit about the DataSet support.. You’ll need to remove the element called “NewDataSet”. This is included because the utility assumes you want to generate a schema for use with DataSets.

Removing the element, renders a nice XSD in Visual Studio:

clip_image018

But wait.. There’s more

Given a valid XSD schema, you can now use the same utility (isn’t it wonderful?) to generate valid C# classes by using the following syntax:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xsd /c

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.cs’.

The result of executing this command produces the following classes (which you could now serialize into and out of XML):

clip_image019

Summary

You may not be able to teach a dog new tricks, but I’m willing to bet he remembers a bunch of old ones!

Although I’m a bit rusty, I’m able to overcome obstacles reasonably quickly. This article might not make me look like Einstein, but I thought I’d write up some of my experiences in the hope it helps others.

We’ve only just touched the tip of the iceberg with BizTalk 2010. The development environment is challenging and prickly (which makes for great sources of inspiration for blog articles) so you can be sure there will be more here soon, so check back!