Schema Fun with BizTalk Server 2010 (and more!) 4


Introduction

As I alluded to at the very end of 2011, I’ve been spending more and more time with BizTalk 2010. It’s been a little while since I’ve had to support/build/maintain BizTalk artefacts, but it doesn’t take long to reacquire “the groove”.

To gently thrust you into the dark underbelly of the land of BizTalk development, I thought I’d start off by exploring some fairly basic BizTalk 101 type experiences (and solutions!).  This article focuses on generic issues with XML schemas, with an eye to BizTalk’s usage of said schema.

Sadly, some of what you might read here (and later) might smack a tad of ‘What?  You didn’t know that?”, but I’m willing to swallow some pride if it helps out other Architects and Programmers.

What you need (to follow along)

This article is (IMHO) decent enough for the generalist (non-BizTalk specialist) and especially may be handy for those unfamiliar in working with XML schemas.

There is a big part where I’m playing with BizTalk’s Developer Tools (in Visual Studio 2010), so if you’d like to give it a go, I’d suggest you pick up a copy of BizTalk Server 2010 Developer Edition (click here for more info on this edition of BizTalk), which – I believe – you need to have to install the developer tools.

If I write subsequent article on BizTalk, the BizTalk developer tools will likely play a significant role, so it might be worth investigating.  As always, it’s worth bookmarking the BizTalk Development Center link.

The Joys of Schema

If you know anything about BizTalk, you’ll know that it’s all about the schemas. Everything ends up mapping to a document schema type whether it’s inbound or outbound. Schemas (XSD) can be an acquired taste, especially if you’ve been off doing something more exciting.

Last month, handwritten schemas tripped me up quite a bit. I was used to using Wizards and Adapters to generate a lot of the schemas for me, so when I went to design some basic input and output schemas, I encountered some “fun” issues.

Something funny happened…

Now, I was putting together some very basic sample schema to use as an example for demonstrating a really neat technique with maps (transforms).

Just in case someone else encounters this (or, like me, found it mildly amusing) I thought I’d document some cause and effect of why you might get some head scratching error messages when designing in Visual Studio 2010 with the BizTalk designer.

Given the following (flawed) schema, which I built within Visual Studio:

<?xml version="1.0" encoding="utf-16"?>

<xs:schema xmlns="http://Sandbox.InputSchema" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" targetNamespace="http://Sandbox.InputSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="Root">

    <xs:complexType>

      <xs:sequence>

        <xs:element minOccurs="2" maxOccurs="3" name="Customer">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Address">

                <xs:complexType>

                  <xs:attribute name="Street" type="xs:string" />

                  <xs:attribute name="State" type="xs:string" />

                  <xs:attribute name="Country" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="ID" type="xs:integer" />

            <xs:attribute name="FirstName" type="xs:string" />

            <xs:attribute name="Surname" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Which, when loaded in the BizTalk Schema Designer looks like this:

clip_image002

Has an unfortunately obvious flaw if you consider I was expecting the schema to facilitate the following format of XML data:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

In other words, I was expecting to store multiple “Customer” elements in my XML document.

Debugging

When I loaded the document and tested it in Visual Studio 2010, I received the following, very weird error message: “The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’.” – huh?

clip_image004

The problem, I thought, might be related to not expressly prefixing the namespace directive to the child element, so I made some edits.

This resulted in an even more ambiguous error message, funny even:

“The element ‘Root’ in namespace ‘http://Sandbox.InputSchema’ has invalid child element ‘Customer’ in namespace ‘http://Sandbox.InputSchema’. List of possible elements expected: ‘Customer’.”

or, in generic terms:

“The element ‘<element>’ in namespace ‘<namespace>’ has invalid child element ‘<child element>’ in namespace ‘<namespace>’. List of possible elements expected: ‘<child element>’.”

WTF?

clip_image006
You may have to squint to read the text…

At this point, I felt like I was going backwards. So I undid my changes and took a look at the schema definition – which, upon second inspection was clearly wrong.

The Solution

See if you can spot the difference?

clip_image008

My mistake was in not declaring a sequence after the root element. Thus rectified, my XML now matched the fixed schema, and I was able to continue on my merry way.

clip_image009

Filtering in a Transform/Map

This all brings us back to the original intention of my article – to highlight the use of functoids for filtering multiple nodes in a transform.

Given that we want to transform our input schema (from before) into another format, with filtered data, here’s the schema for the output:

clip_image010

The Mapping

We can construct a map which assigns the fields we want per the following example:

clip_image012
A basic map

Now, if we wanted to only take Customers who live in the state of New South Wales, we need to validate each element in the input XML recursively.

A Functoid for all occasions

The easiest means is to use a equality functoid, in this example I’ve used an equals functoid, as per below. You must still map the field directly to the output schema, but also to the functoid – which is, in turn, mapped to the logical root of the target (per the example above).

clip_image013

When the functoid returns a true value, the element is copied to the output. If the result is fale, the entire element (children etc) are skipped. Thus, from the following input XML:

<ns0:Root xmlns:ns0="http://Sandbox.InputSchema"> 

  <Customer ID="100" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="12 Jump Street" State="NSW" Country="Australia" />

  </Customer>

  <Customer ID="101" FirstName="FirstName_1" Surname="Surname_2">

    <Address Street="22 Fish Street" State="QLD" Country="Australia" />

  </Customer> 

</ns0:Root>

When run through this map (using trusty ‘Test Map’ functionality), will generate the following output, filtering out that pesky Queensland (QLD) customer:

<ns0:Root xmlns:ns0="http://Sandbox.OutputSchema">

  <Address Street="12 Jump Street" State="NSW" Country="Australia" CustomerID="100" />

</ns0:Root>

Helpful Support in Visual Studio 2010

Before you start despairing about schema development and BizTalk maps, there are a bunch of very helpful tools built into Visual Studio 2010 (and prior versions).

Generating Schema Instances

Given you have Schema files in your project, you can right click them and select “Generate Instance”. This will create an XML file using intelligent default values, according to the schema definition. These files can be modified and used to test out other artefacts who use this schema.

Validating Schema Instances

If you want to test if a particular XML file is valid, you can do the reverse – by clicking on the schema properties, you can set the “input XML” for a given schema. Then, when right clicking on the Schema in Solution Explorer, you can select “Validate Instance”. This will highlight any issues with your schema.

clip_image014

Testing a Map/Transform

Once you have some valid XML for your schema, you can then use it as test input data for various things including maps.

The following shows how to establish test data parameters for a map within Visual Studio 2010:

clip_image016

When you have configured a map to use a specified XML input, you can then right click on the map (in Solution Explorer) and select “Test Map”.

Other Options – Xml Schema/Data Type Utility

A handy note for future reference – if you have an XML structure in mind, but (like me) you are rusty with your XSD designing skillz, there’s another way forward which may prove to be better.

There is a tool which helps you work with XML and XSD document formats, and it ships by default with a number of Microsoft products and SDKs, it is called “Xml Schemas/DataTypes support utility” or XSD.exe.

Important Disclaimer: In nearly almost all circumstances, it is considered a better practice to produce or design the XML schema first. Generating a schema from XML data can be fraught with peril, especially if your XML sample doesn’t accurately reflect the full spectrum of supported formats.

For example, min/max occurrences, type lengths and other constraints, vague data types (where the type or precision is “guessed”), elements supported by the schema which are not required – and thus missing from your sample XML – to name but a few.

Where Do I Find This Tool?

Chances are that you have it already. I’ve located it installed in the following locations (check the (x86) folder as well if you are on a 64 bit machine):

Directory of C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools

Directory of C:\Program Files\Microsoft SDKs\Windows\v7.0A\Bin\x64

What does it do?

Keeping in mind that this utility was really meant for the days of evil DataSets, it still can be quite useful.

Using the following (poor) sample XML structure:

<?xml version=”1.0” encoding=”utf-8”?>
<Country Name="Australia">
     <State Name="NSW">
           <Capital Name="Sydney"/>
     </State>
     <State Name="QLD">
           <Capital Name="Brisbane"/>
     </State>
     <State Name="WA">
           <Capital Name="Perth"/>
     </State>
     <State Name="TAS">
           <Capital Name="Hobart"/>
     </State>
     <State Name="VIC">
           <Capital Name="Melbourne"/>
     </State>
</Country>

You can use the XSD.exe utility to generate an approximation of the schema, like so:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xml

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.xsd’.

Which would produce the following output:

<?xml version="1.0" encoding="utf-8"?>

<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">

  <xs:element name="Country">

    <xs:complexType>

      <xs:sequence>

        <xs:element name="State" minOccurs="0" maxOccurs="unbounded">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="Capital" minOccurs="0" maxOccurs="unbounded">

                <xs:complexType>

                  <xs:attribute name="Name" type="xs:string" />

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="Name" type="xs:string" />

          </xs:complexType>

        </xs:element>

      </xs:sequence>

      <xs:attribute name="Name" type="xs:string" />

    </xs:complexType>

  </xs:element>

  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">

    <xs:complexType>

      <xs:choice minOccurs="0" maxOccurs="unbounded">

        <xs:element ref="Country" />

      </xs:choice>

    </xs:complexType>

  </xs:element>

</xs:schema>

Now, given I mentioned that bit about the DataSet support.. You’ll need to remove the element called “NewDataSet”. This is included because the utility assumes you want to generate a schema for use with DataSets.

Removing the element, renders a nice XSD in Visual Studio:

clip_image018

But wait.. There’s more

Given a valid XSD schema, you can now use the same utility (isn’t it wonderful?) to generate valid C# classes by using the following syntax:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin>xsd Input.xsd /c

Microsoft (R) Xml Schemas/DataTypes support utility

[Microsoft (R) .NET Framework, Version 2.0.50727.3038]

Copyright (C) Microsoft Corporation. All rights reserved.

Writing file ‘C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\Input.cs’.

The result of executing this command produces the following classes (which you could now serialize into and out of XML):

clip_image019

Summary

You may not be able to teach a dog new tricks, but I’m willing to bet he remembers a bunch of old ones!

Although I’m a bit rusty, I’m able to overcome obstacles reasonably quickly. This article might not make me look like Einstein, but I thought I’d write up some of my experiences in the hope it helps others.

We’ve only just touched the tip of the iceberg with BizTalk 2010. The development environment is challenging and prickly (which makes for great sources of inspiration for blog articles) so you can be sure there will be more here soon, so check back!


About Rob Sanders

IT Professional and TOGAF 9 certified architect with nearly two decades of industry experience, 18 years in commercial software development and 11 years in IT consulting. Check out the "About Rob" page for more information.


Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

4 thoughts on “Schema Fun with BizTalk Server 2010 (and more!)

  • Joel

    Old blog post I know, but I just wanted to point out that:

    1. You *don’t* need to declare nested sequences in order to have a repeating element; that is what minOccurs and maxOccurs are for, no need for the extra nested sequence-tags. Your original sample schema is not flawed.

    2. Your “WTF” error is perfectly reasonable and expected. Unless you explicity set the elementFormDefault of a schema to “qualified”, locally declared child elements of the root are assigned to the null (empty) namespace. Therefore, your attempt to fix your first error by prefixing the “Customer” element with “ns0″ is invalid. According to the schema, “Customer” is associated with the null namespace (due to elementFormDefault=”unqualified”) and not “http://Sandbox.InputSchema”. Therefore, if elementFormDefault is “unqualified”, the “Customer” element can appear as:

    or

    As for why Visual Studio was giving you an error, VS might have had an old version of the schema cached, perhaps before you modified minOccurs/maxOccurs, which default to ‘1’ and would have led to the error you received due to the second “Customer” element.

    Just for a sanity check, I recreated your sample schema using the BizTalk Schema Editor and built-in VS XML/XSD editor and did not encounter the error you described, which was expected as the original schema is valid. :)

    • Rob Sanders Post author

      Hi Joel,

      Thanks for the clarifications! Quite right, I should have set the elementFormDefault. Chalk it up to being rusty, I guess :)
      The error makes sense, although I think the compiler/designer error probably could have been a little more helpful – it hadn’t occurred to me that the child element would be assigned to a null namespace.

      Anyhow.. mystery solved..

      • Joel

        I decided a long time ago that elementFormDefault is one of those annoying subtleties of XML Schema and always explicity set it in any new schema that I create. I understand why the “unqualified” option is there (though disagree with it being the default), but I’ve never found a strong enough argument to use it, so for me, “qualified” it is!

        Sorry for the comment on an old post but I was looking for some info on how BizTalk’s XmlDasmComp loads and resolves schemas from BizTalk assemblies (I’ve resorted to Reflector for now) and this blog post came up. :)

        Take care.