Decoding JSON Data Using the BizTalk Server 2013 R2 JSONDecode Pipeline Component

By Nick Hauenstein

This is the fourth in a series of posts exploring What’s New in BizTalk Server 2013 R2. It is also the second in a series of three posts covering the enhancements to BizTalk Server’s support for RESTful services in the 2003 R2 release.

In my last post, I wrote about the support for JSON Schemas in BizTalk Server 2013 R2. I started out with a small project that included a schema generated from the Yahoo Finance API and a unit test to verify the schema model. I was going to put together a follow-up article last week, but spent the week traveling, in the hospital, and then recovering from being sick.

However, I am back and ready to tear apart the next installment that already hit the github repo a few days back.

Pipeline Support for JSON Documents

In BizTalk Server 2013 R2, Microsoft approached the problem of dealing with JSON content in a way fairly similar to the approach that we used in the previous version with custom components – performing the JSON conversion as an operation in the Decode stage of the pipeline, thus requiring the Disassemble stage to include an XMLDisassemble component for property promotion.

The official component Microsoft.BizTalk.Component.JsonDecoder takes in two properties Root Node and Root Node Namespace that help determine how the XML message will be created.

Finally, there isn’t a JSONReceive pipeline included in BizTalk Server 2013 R2 – only the pipeline component was included. In other words, in order to work with JSON, you will need a custom pipeline.

image

Creating a Pipeline for Receiving JSON Messages

Ultimately, I would like to create a pipeline that is going to be reusable so that I don’t have to create a new pipeline for each and every message that will be received. Since BizTalk message types are all about the target namespace and root node name, it’s not reasonable to set that value to be the same for every message – despite having different message bodies and content. As a result, it might be best to leave the value blank and only set it at design time.

This is also an interesting constraint, because if we are receiving this message not necessarily just as a service response, we might end up needing to create a fairly flexible schema (i.e., with a lot more choice groups) depending on the variety of inputs / responses that may be received – something that will not be explored within this blog post, but would be an excellent discussion to bring up during one of QuickLearn’s BizTalk Server Developer Deep Dive classes.

In order to make the pipeline behave in a way that will be consistent with typical BizTalk Server message processing, I decided to essentially take what we have in the XMLReceive pipeline and simply add a JsonDecoder in the Decode stage, with none of its properties set at design time.

image

Testing the JSONReceive Pipeline

In the same vein as my last post, I will be creating automated tests for the pipeline to verify its functionality. However, we cannot use the built-in support for testing pipelines in this case – because properties of the pipeline were left blank, and the TestablePipelineBase class does not support per instance configuration. Luckily, the Winterdom PipelineTesting library does support per instance configuration – and it has a nifty NuGet package as of June.

Unfortunately, the per-instance configuration is not really pretty. It requires an XML configuration file that resembles the guts of a bindings file in the section dedicated to the same purpose. In other words, it’s not as easy as setting properties on the class instance in code in any way. To get around that to some degree, and to be able to reuse the configuration file with different property values, I put together a template with tokens in place of the actual property values.

image

NOTE: If you’re copying this approach for some other pipeline components, the vt attribute is actually very important in ensuring your properties will be read correctly. See KB884981 for details.

From there, the per-instance configuration is a matter of XML manipulation and use of the ReceivePipelineWrapper class’ ApplyInstanceConfig method:

[sourcecode language=”csharp”]
private void configureJSONReceivePipeline(ReceivePipelineWrapper pipeline, string rootNode, string namespaceUri)
{
string configPath = Path.Combine(TestContext.DeploymentDirectory, "pipelineconfig.xml");

var configDoc = XDocument.Load(configPath);

configDoc.Descendants("RootNode").First().SetValue(rootNode);
configDoc.Descendants("RootNodeNamespace").First().SetValue(namespaceUri);

configDoc.Save(configPath);

pipeline.ApplyInstanceConfig(configPath);
}
[/sourcecode]

The final test code includes a validation of the output against the schema from last week’s post. As a result, we’re really dealing with an integration test here rather than a unit test, but it’s a test nonetheless.

[sourcecode language=”csharp”]
[TestMethod]
[DeploymentItem(@"Messages\sample.json")]
[DeploymentItem(@"Configuration\pipelineconfig.xml")]
public void JSONReceive_JSONMessage_CorrectValidXMLReturned()
{

string rootNode = "ServiceResponse";
string namespaceUri = "http://schemas.finance.yahoo.com/API/2014/08/";

string sourceDoc = Path.Combine(TestContext.DeploymentDirectory, "sample.json");
string schemaPath = Path.Combine(TestContext.DeploymentDirectory, "ServiceResponse.xsd");
string outputDoc = Path.Combine(TestContext.DeploymentDirectory, "JSONReceive.out");

var pipeline = PipelineFactory.CreateReceivePipeline(typeof(JSONReceive));

configureJSONReceivePipeline(pipeline, rootNode, namespaceUri);

using (var inputStream = File.OpenRead(sourceDoc))
{
pipeline.AddDocSpec(typeof(ServiceResponse));
var result = pipeline.Execute(MessageHelper.CreateFromStream(inputStream));

Assert.IsTrue(result.Count > 0, "No messages returned from pipeline.");

using (var outputFile = File.OpenWrite(outputDoc))
{
result[0].BodyPart.GetOriginalDataStream().CopyTo(outputFile);
outputFile.Flush();
}

}

ServiceResponse schema = new ServiceResponse();
Assert.IsTrue(schema.ValidateInstance(outputDoc, Microsoft.BizTalk.TestTools.Schema.OutputInstanceType.XML),
"Output message failed validation against the schema");

Assert.AreEqual(XDocument.Load(outputDoc).Descendants("Bid").First().Value, "44.97", "Incorrect Bid amount in output file");

}
[/sourcecode]

After giving it a run, it looks like we have a winner.

image

Coming Up in the Next Installment

In the next installment of this series, I will actually put to use what we have here, and build out a more complete integration that allows us to experience sending JSON messages as well, using the new JsonEncoder component.

Take care until then!

If you would like to access sample code for this blog post, you can find it on github.

Why Updated Platform Support Matters

By Nick Hauenstein

This post is the fifth in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

Each new version of BizTalk brings with it support for the latest Microsoft Operating System, Database Engine, and Integrated Development Environment. It is listed as a new feature each time, and many tend to skip over that feature on the new features list. However, this is actually very powerful – especially when considering the change over time in those underlying platforms.

In this post, I’m going to examine how the Microsoft Fakes framework (found in Visual Studio 2012.2 Premium) can enable interesting testing scenarios for code that wasn’t built specifically for test. In the process, I’m also going to be using the Winterdom  PipelineTesting library – which is easily upgraded to 2013 with some re-referencing of BizTalk assemblies.

Testing the Un-Testable

The Microsoft Fakes framework (available in Visual Studio 2012 Ultimate, or 2012.2 Premium edition and above) provides the ability to create both stubs and shims of any .NET interface or class respectively. While the stubs capability is pretty neat, they don’t feel nearly as magical as the shims.

Consider that you have developed a pipeline component with a little bit of logic to inject a context property based on the current date/time. The scenario (although contrived) that I have come up with is that of an order processing system in which rush orders are all received through the same receive location. In that location is a custom pipeline that must promote a property to route to the warehouse that will be able to get the order picked and shipped the quickest (based on which warehouse is picking orders at the current hour, or will be doing so next). When testing the code, we will need to verify the behavior of the code at specific times.

While we could write the code in such a way that the retrieval of the current date/time was done through a special date/time provider class (which implements some interface specialized to that purpose), the logic is being re-used from another location – one which will not be modified.

BizTalk Server 2013 will rise to this occasion due to relying on Visual Studio 2012 as its development environment. We can use the included Microsoft Fakes framework to create a shim that will inject logic for all calls to DateTime.UtcNow to return a fixed DateTime object of our choosing.

getting Started with Shims

In order to get started, we need to instruct Visual Studio to generate some helper classes for the assembly or assemblies of our choosing. These classes will enable us to generate Stubs/Shims for classes within these assemblies.

For our solution, we will be working with the DateTime class, found in System. We can add a fakes assembly in two clicks via the context menu:

AddFakes

After adding the assembly, our project, and its references, look like this:

AfterAddingFakes

Now we will have to write the code in such a way that calls to DateTime.UtcNow will return the value of our choosing. In order to hook into those calls only within a specific portion of code, we will need to create a ShimsContext. Because the ShimsContext class implements IDisposable, we are able to wrap a using statement around it to provide better visibility of the scope it will control.

[sourcecode language=”csharp”]
[TestMethod]
public void RushOrderMarker_FirstPartOfDay_FirstWarehouseReturned()
{

// Arrange
var pipeline = GeneratePipeline();
var testMessage = GenerateTestMessage();

DateTime fakeTime = new DateTime(
year: 2013,
month: 01,
day: 01,
hour: 1, minute: 0, second: 0);

using (ShimsContext.Create())
{

System.Fakes.ShimDateTime.UtcNowGet = () => { return fakeTime; };
var expectedId = WarehouseIdentifiers[0];

// Act
var output = pipeline.Execute(testMessage);

// Assert
var warehouseId = output[0].Context.Read(WAREHOUSE_ID_PROPERTY_NAME, WAREHOUSE_ID_PROPERTY_NAMESPACE);
Assert.AreEqual(expectedId, Convert.ToString(warehouseId), "Incorrect warehouse id returned");

}

}
[/sourcecode]

 

Here we assign logic directly to System.Fakes.ShimDateTime.UtcNowGet that will execute when any code within the scope after ShimsContext.Create() was called attempts to call DateTime.UtcNow. We then submit a message through our pipeline and verify that the correct warehouse id was promoted into the context of the message (based on the current time).

Know your Tools

If you’ve just made the move up to BizTalk Server 2013, it really is important that you not only know what feature set you have in the platform, but also what is available to you in the underlying platform, and the tools you use to create your applications.

If you want to learn BizTalk Server 2013 in the environment it was designed for, check out one of our upcoming BizTalk 2013 Developer Immersion, or BizTalk Server 2013 Deep Dive classes. The hands-on-lab environment is running the latest and greatest versions of Windows Server, SQL Server, and Visual Studio, so that you have the optimal BizTalk Server development experience.

If you would like to access sample code for this blog post, you can find it on github.