Publishing a WCF Service via Service Bus Relay in BizTalk Server 2013 (Part 2)

By Nick Hauenstein

This post is the eighth in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013. It is also the second part of a two-part series on the Service Bus Relay features in BizTalk Server 2013 (Click here for part 1).

Last week, we examined how BizTalk Server 2013 (and even 2010 in that case) can use the Windows Azure Service Bus Relay to make a service endpoint available in the cloud without otherwise directly exposing the machine, and while being able to take advantage of the claims-based authentication and authorization framework provided by the Access Control Service. This was done in such a way that the service was hosted in IIS rather than within an In Process BizTalk Host Instance. This week, we are going to look at the other side – a service endpoint that is ultimately hosted in IIS.

In this case, we have been provided with a few new adapters to help us along the way (although, if we were really adventurous, WCF-Custom could have still done the trick for us in the past). These two adapters are the WCF-BasicHttpRelay adapter and the WCF-NetTcpRelay adapter:

ServiceBusRelayInProc

Last week, we published a NetTcpRelay endpoint that required some additional work on the client to authenticate with the service. This week, we’re going to go to the other extreme, and publish a WCF-BasicHttpRelay endpoint that will be more or less transparent to the client.

Starting the Game of Telephone

We’re going to start out with the BizTalk version of Hello World, a simple (technically unnecessary, but convenient for illustrative purposes) orchestration. This orchestration receives a simple single node XML message (the single node containing the text “Hello”), and returns a simple single node XML message (containing the word “World”) as the response:

HelloOrch

The Setup

With this, we’re going to go through the typical process of creating an in-process receive location, and then walk through the BizTalk WCF Service Publishing wizard to publish the service. Then we’re going to consume the service as it sits. Once that is complete, we will change up the end point so that it is listening over a Service Bus Relay, and see that the client can still call the service.

We’ll start with the Receive Location. In order to keep this in-process, we’re actually using the WCF-Custom adapter (since WCF-BasicHttp will force us to an Isolated Host):

BasicHttpEP

Now that we have an endpoint to receive messages, we need to create an endpoint that informs clients what messages we’re even interested in – our MEX endpoint. For that, we will go back to the BizTalk WCF Service Publishing wizard, and select the second option – the path less traveled:

MEXPublish_Step1

We will walk through this wizard, ignoring everything that mentions Service Bus publishing for now. Ultimately, what we will end up with is a local endpoint that is exposing the WSDL necessary to consume the service. We can use the svcutil.exe utility to download and share this metadata offline, or even generate proxy classes for re-use later if we’d like. In this case, we will be consuming it in place, and generating a client that can be used to call the service no matter where it lives.

Since we started with an orchestration, and that will define the shape of the incoming/outgoing data, we will select the Publish BizTalk orchestrations as WCF service option. Do note the plural orchestrations in that sentence, I see way too many single operation services out there. Winking smile

MEXPublish_Step3

After browsing for and selecting an orchestration, you should be presented with a list of all public port types within the assembly containing the orchestration. If they Type Modifier property on the Port Type is not set to public, it will not be listed, and you will be sad:

selectport

After you click through a few more screens, you will be prompted for the location where you would like this metadata hosted. In this case, it will be hosted within IIS on the local machine:

MEXPublish_Location

A few more clicks through the wizard, and the MEX endpoint is created and hosted in IIS – though the service endpoint itself is still hosted in-process.

A visit to IIS manage demonstrates that the MEX endpoint is now:

http://localhost/IOServiceMEX/WcfService_QuickLearn_BizTalk_Process.svc?wsdl

Consuming the Service

So far, we haven’t really done anything earth shattering, so consuming the service is equally simple. In this case, I just fired up Visual Studio, used the Add Service Reference context menu option in Solution Explorer, and then provided the MEX endpoint to the wizard. With the (local, not relayed) Service Reference in place, I added the following code to my console app:

[sourcecode language=”csharp”]using System;

namespace QuickLearn.WCF.Client
{
class Program
{
static void Main(string[] args)
{
var client = new IOServiceReference.WcfService_QuickLearn_BizTalk_ProcessClient("BasicHttpBinding_ITwoWayAsync",
"http://localhost:81/Sample/IOService.svc");

var response = client.ProcessMessage(new IOServiceReference.InOutRequest()
{
Contents = "Hello"
});

Console.WriteLine(response.Contents);
Console.ReadLine();
}
}
}[/sourcecode]

 

In the code above, the only moderately odd thing that I did was to use an overload of the client constructor that included an endpoint address. The reason for that was so I could more easily demonstrate the only change necessary as we switch to the WCF-BasicHttpRelay adapter.

To the Cloud

Since my client code is currently working and happy, I am ready to move this to the cloud. I am actually going to maintain my local endpoint (Receive Location) as well and simply add another Receive Location that will open the relay. The configuration of this one will be a little bit different, and certainly more involved.

NOTE: Before you continue, please be sure that you have read part 1.

I’m going to re-use my Azure Service Bus namespace from last week (though I have a new key for this week). I will start by creating a new receive location, selecting the WCF-BasicHttpRelay adapter, and then opening the adapter configuration. The first thing I am prompted for is the listen URI for the endpoint. This URI must incorporate my Service Bus Namespace, and should also add some identifier for the service after:

RelayEP_Config

The only other page of the configuration that I must visit is the Security page. This is where I get to determine if the client needs to provide some sort of authentication information in order to invoke the service, if the existence of the service is going to be discoverable by hitting the root of the namespace, and this is also where I set the authentication details for the access control service – so that BizTalk can prove to Azure that it can listen at that address and receive messages.

Since we don’t want to make changes to our client (other than the endpoint address), we will set the client authentication type to None. Then we need to click Edit under Access Control Service to bring up this dialog:

acs_credentials

In this dialog, we need to re-iterate our namespace (leaving the –sb suffix in-tact), and then then we also need to provide our super secret key which we can look up through the Azure Management Portal (see the previous post for details).

Once that is all in place, we’re set to Enable our Receive Location and then update the client to point to the relay.

Updating the Client

I mentioned before that we really only want to update the endpoint in the client. While this is true, the change of endpoint does (in this case) require a single simple change in configuration. The reason has nothing to do with the Service Bus Relay itself, but instead the change of scheme from http to https. An http address does not imply any security added in by the transport, it is just clear text over the wire. An https address, on the other hand, implies transport level security, as a result we need this small tweak to the otherwise auto-generated config file:

TransportSecurity

And then an update to our endpoint in code (everything else remains):

[sourcecode language=”csharp”]using System;

namespace QuickLearn.WCF.Client
{
class Program
{
static void Main(string[] args)
{
var client = new IOServiceReference.WcfService_QuickLearn_BizTalk_ProcessClient("BasicHttpBinding_ITwoWayAsync",
"https://uniquename.servicebus.windows.net/IOService/");

var response = client.ProcessMessage(new IOServiceReference.InOutRequest()
{
Contents = "Hello"
});

Console.WriteLine(response.Contents);
Console.ReadLine();
}
}
}[/sourcecode]

This gives us the very happy result:

consoleapp_result

Clearly this is one of the most robust, secure, and complex ways to write the word “World” out to the console.

The Take-away

The take-away here for the last two posts is that BizTalk Server 2013 gives you some new options for exposing select endpoints over the internet without touching firewall configuration, without involving IIS (if you don’t want to involve it), and without exposing the whole box out to the internet. All the while giving you the ability to work with clients that are aware of the setup (in the case of the NetTcpRelay) or are blissfully ignorant of the magic behind the scenes (in the case of the BasicHttpRelay).

Windows Azure Service Bus Queues + BizTalk = Out of Body Experiences

By Nick Hauenstein

This post is the third in a weekly series that will highlight things that you need to know about new features in BizTalk Server 2013. One of the major new features of BizTalk Server 2013, is the native ability to integrate with Windows Azure Service Bus through the new SB-Messaging adapter. This week we will be covering what can happen if you’re trying to integrate with applications that are loading up your Windows Azure Service Bus Queues with Brokered Messages made purely of properties.

Out of Context

Context properties are not a foreign concept to BizTalk developers. Indeed, BizTalk messages themselves are made up of both a message context and body parts. The same is true to some extent of Service Bus Brokered Messages – though the implementation differs greatly.

To deal with context properties of Service Bus Messages, the SB-Messaging adapter provides us the ability to promote these properties to the context of the BizTalk message generated upon receipt of a Brokered Message. The adapter configuration that controls this behavior looks like this:

sb-promote-props

One major difference between the properties of a Brokered Message and the context properties of a BizTalk message, is that BizTalk message context properties have a namespace to disambiguate properties that share the same name. Brokered messages have no such thing, and no concept of a property schema for that matter (which allows BizTalk to have an awareness of all possible properties that can exist in the context).

The adapter configuration setting pictured above is a bridge between those worlds that allows the Brokered Messages properties to be understood by BizTalk Server. Of course, you will have to define a property schema for those properties on the BizTalk side. Here’s an example of a really simple one that defines two custom properties:

propertyschema

All of this will function pretty perfectly – assuming that the message sent to the Queue has a body.

Out of Body

At last year’s TechEd NA conference in Orlando, one of the presenters (I believe it was Clemens Vasters) was discussing what Brokered Messages were, and how they were exchanged. In the talk, he mentioned that it was perfectly fine, and even preferable in some cases, to create a Brokered Message that was purely properties without any body content. Certainly this was a light-weight operation that relied only on HTTP headers, and required no overhead for serialization of body content. And at the time, I accepted this, and was happy.

As a result, a lot of my code interacting with the Service Bus involved snippets like this:

[sourcecode language=”csharp”]
var client = Microsoft.ServiceBus.Messaging.QueueClient.CreateFromConnectionString(connString, queueName);
var message = new Microsoft.ServiceBus.Messaging.BrokeredMessage();
message.Properties.Add("CustomProperty", "Value");
message.Properties.Add("CustomProperty2", "Value2");
client.Send(message);
[/sourcecode]

Notice here, we’re just passing around properties, and not dealing with a message body. That’s fine when I’m receiving the message in .NET code living somewhere else, but when moving into the BizTalk world, it became problematic. When attempting to receive such a message (even with the property schema deployed, and everything else otherwise happy), I would receive this (using the PassThruReceive pipeline):

errormsg

For those trying to use Google-fu to solve the same problem, the error message reads:

“There was a failure executing the receive pipeline: ‘Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “Pipeline” Receive Port: “WontMatchWhatYouHaveAnyway” URI: “sb://namespacehere-ns.servicebus.windows.net/somequeuenamehere” Reason: The Messaging Engine encountered an error whlie reading the message stream.

If you came to this article because this error message appeared, and you’re not trying to receive an empty message from Azure Service Bus, just head right back to the Bing results page, and check the next result. Everyone else, stick with me.

Let’s take a step back for a second. BizTalk Server 2013 is designed for integration between systems. I typically should not be adjusting my behavior outside of BizTalk to work with BizTalk. So here I want to take the same approach.

What are our options here? Well clearly we have a failure in the pipeline while processing an entirely empty message body, which doesn’t allow us to get the point where we are processing a map. That removes our ability to use the excellent ContextAccessor functoid to pull the properties into the body of the message upon receipt.

Instead we are left with building a custom solution.

PropertyMessageDecoder Pipeline component

What is this custom solution I speak of? For now, I’m calling it the PropertyMessageDecoder pipeline component. It is a custom pipeline component that creates a message body based on the context properties of a BizTalk message. If you already have a message body, this pipeline component will not help you, since it blindly overwrites the body if one exists. Instead, this is a purpose-built pipeline component for the specific scenario addressed by this blog post.

I started development of this component by using Martijn Hoogendoorn’s BizTalk Server Pipeline Component Wizard. Unfortunately, it hasn’t yet been updated to work with BizTalk Server 2013, so I had to first create/upload a patch for BizTalk 2013.

Once I was able to create a custom pipeline component project, I created a simple pipeline component that took the input message, and dumped the context properties into the message bodies. I also included some configurable properties that allowed one to opt-out of properties from the BizTalk system properties namespace, or to filter out all properties except those from a specific namespace (especially helpful in this scenario).

Here’s the bulk of the Execute method for that component:

[sourcecode language=”csharp”]
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
Stream dataStream = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
pc.ResourceTracker.AddResource(dataStream);

using (XmlWriter writer = XmlWriter.Create(dataStream))
{
// Start creating the message body
writer.WriteStartDocument();
writer.WriteStartElement("ns0", ROOT_NODE_NAME, TARGET_NAMESPACE);

for (int i = 0; i < inmsg.Context.CountProperties; i++)
{
// Read in current property information
string propName = null;
string propNamespace = null;
object propValue = inmsg.Context.ReadAt(i, out propName, out propNamespace);

// Skip properties that we don’t care about due to configuration (default is to allow all properties)
if (ExcludeSystemProperties && propNamespace == SYSTEM_NAMESPACE) continue;
if (!String.IsNullOrWhiteSpace(CustomPropertyNamespace)
&& propNamespace != CustomPropertyNamespace) continue;

// Create Property element
writer.WriteStartElement(PROPERTY_NODE_NAME);

// Create attributes on Property element
writer.WriteStartAttribute(NAMESPACE_ATTRIBUTE);
writer.WriteString(propNamespace);
writer.WriteEndAttribute();

writer.WriteStartAttribute(NAME_ATTRIBUTE);
writer.WriteString(propName);
writer.WriteEndAttribute();

// Write value inside property element
writer.WriteString(Convert.ToString(propValue));
writer.WriteEndElement();
}

// Finish out the message
writer.WriteEndElement();
writer.WriteEndDocument();
writer.Flush();

}

dataStream.Seek(0, SeekOrigin.Begin);
var outmsg = Utility.CloneMessage(inmsg, pc);
outmsg.BodyPart.Data = dataStream;

return outmsg;
} [/sourcecode]

The output will end up conforming to this schema:

propertymessageschema

And will look a little something like this:

[sourcecode language=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<ns0:PropertyMessage xmlns:ns0="http://schemas.quicklearn.com/PropertyMessage/2013/06/">
<Property Namespace="http://schemas.example.org/" Name="CustomProperty1">Value1</Property>
<Property Namespace="http://schemas.example.org/" Name="CustomProperty2">Value2</Property>
</ns0:PropertyMessage>
[/sourcecode]

When included in a pipeline alongside the XML disassembler component, we are finally able to take those pesky “empty” messages from Azure Service Bus and process them as XML documents. The message context properties are still retained in the context for routing, but our custom properties are also made available for use directly in maps (through the liberal use of Value Mapping (Flattening) and Logical functoids), and further processing.

To follow this through to the end, I created a quick proof-of-concept map:

samplemap

Essentially we’re looping through the properties from the context (hence using the Value mapping flattening functoid) and only mapping the value stored inside the Property node whenever the namespace and name of the property match those that we care about. Since the namespace for all of our custom properties is the same, we have a single Equals functoid that we’re sharing across all (two) properties.

After processing a message all the way through the bus, custom pipeline, and map, this should result in this beauty as opposed to a nasty error message:

sampleoutput

un-Borked brokered messages

Remember, that none of this stuff is necessary if you’re working with Brokered Messages that have a message body. You only need to go this deep when you’re dealing with Brokered Messages made purely of properties (which is a recommended practice when possible).

By bringing out-of-the-box first class support for Windows Azure Service Bus Queues, BizTalk Server 2013 continues to prove that with a solid and extensible architecture in place, any type of integration can be made possible.

If you would like to learn more about extending BizTalk Server 2013 to meet your integration challenges, check out one of our upcoming BizTalk 2013 Developer Deep Dive classes.

If you would like to access sample code for this blog post, you can find it on github.