Complex Event Processing (CEP) solution with Business Rule Engine

I have implemented CEP solution with BizTalk and focused on real time passenger flow through the airport scenario. Imagine that information of passengers are flowing through the pipeline and you want to define some criteria to filter passenger data over pipeline. In CEP concept, collection of criteria or rules is assumed as a event. You may want to collect many different events data from multiple sources simultaneously. As a result, the aim of the complex event processing is to identify meaningful opportunities, threads etc.

solution
Let’s take a look at CEP implementation. The solution has two orchestration which Detects event and process event orchestration. The detect event orchestration takes passenger info and checks whether any event occurs or not. Every detected event is stored to event table in database. Also the event which reaches the max occurrence criterion is published to BizTalk MessageBox. The second orchestration catches the published event(thread or opportunity) and tries to process.

But where is the Business rule engine in this solution? Business rule engine has responsibility to execute event rules in detect orchestration. Can we define new event(s) without deploying the solution again? Yes, the events can be defined using with Business Rule Composer.

First, we need to define our schema to represent passenger and event information.

Schema_PassengerInfo Schema_Event

Then I have created two events that are called Turkish and German Passenger Events with Business Rule Composer.
Event_German

Event_Turkish

Business Rule Engine takes passenger info and executes defined rules of event over passenger info. Then If passenger info complaint with rules, BRE sets the values that are event name, max occurrence of event and description. This is also tricky point BRE determines events dynamically depends on the passenger info.

After defining rules, we can construct our orchestration. Passenger info comes from Port_PI receive port. And the event info is initialized in expression shape.

Orch1

We send passenger and event info to Business Rule Engine. Instead of using call rules shape in orchestration, there is a rule executer helper in the solution.
Orch2

namespace BT.POC.RuleExecuter
{
    [Serializable]
    public class PolicyUtility
    {
        public string Execute(XLANGMessage msgPassenger, XLANGMessage msgEvent)
        {
            string output = String.Empty;
            XmlDocument xmlDocPassenger = (XmlDocument)msgPassenger[0].RetrieveAs(typeof(XmlDocument));
            XmlDocument xmlDocEvent = (XmlDocument)msgEvent[0].RetrieveAs(typeof(XmlDocument));

            TypedXmlDocument txdPassenger = new TypedXmlDocument("ibrahimoguz.BiztalkPOC.CEP.Schemas.PassengerInfo", xmlDocPassenger);
            TypedXmlDocument txdEvent = new TypedXmlDocument("ibrahimoguz.BiztalkPOC.CEP.Schemas.EventSchema", xmlDocEvent);

            Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver breDriver = new Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver();
            Microsoft.RuleEngine.RuleStore breStore = breDriver.GetRuleStore();

            Microsoft.RuleEngine.RuleSetInfoCollection colPolInfo = null;
            colPolInfo = breStore.GetRuleSets(RuleStore.Filter.All);

            string nameXPath = "/*[local-name()='EventInfo' and namespace-uri()='http://ibrahimoguz.BiztalkPOC.CEP.EventSchema']/*[local-name()='Name' and namespace-uri()='']";
            foreach (RuleSetInfo pInfo in colPolInfo)
            {
                Policy p = new Policy(pInfo.Name);
                object[] facts = new object[2];
                facts[0] = txdPassenger;
                facts[1] = txdEvent;
                p.Execute(facts);

                XmlNode nameNode = txdEvent.Document.SelectSingleNode(nameXPath);
                if (nameNode != null && !String.IsNullOrWhiteSpace(nameNode.InnerText))
                {
                    output = txdEvent.Document.OuterXml;
                    break;
                }
            }

            return output;
        }
    }
}

The PolicyUtility class has execute method that takes two orchestration passenger and event messages. This utility fetches all policies from business rule engine rules store and executes sequentially. If any event occurrence returns event information to orchestration.

eventIf

If the event is occurred then the number of occurrence of this event checked from database.
eventcheck

public int GetCountEventOccurrence(string eventName)
{
   SqlConnection con = new SqlConnection() { ConnectionString = conString };
   SqlCommand command = new SqlCommand()
   {
      Connection = con,
      CommandType = System.Data.CommandType.Text,
      CommandText = String.Format("SELECT COUNT(*) FROM Event WHERE EventName='{0}' AND IsProcessed = 0", eventName)
    };

    con.Open();
    int result = (int)command.ExecuteScalar();
    con.Dispose();
    return result;
}

The passenger info mapped to event into before persisting event info to database.
Map

persist

maxOccurrence

OrchDetectLast

When max occurrence of event is reached then isProcessed field of event is set to true.
setEventProcessed

Then message is published into messageBox to trigger EventProcessing work flow.
EventProcessing

This is just testing purpose therefore event processing orchestration not have business logic. Let’s make some tests. I have used following passenger message. I have sent same message to reach max event occurrence number.

<ns0:PassengerInfo>
  <Passenger>
   <Name>Name_0</Name>
   <Surname>Surname_0</Surname>
   <FlightNo>FlightNo_0</FlightNo>
   <Departure>Frankfurt Airport</Departure>
   <Arrival>Vienna Airport</Arrival>
   <Date>2015-03-14</Date>
   <Airline>Lufthansa Airlines</Airline>
   <ScheduledTime>1999-05-31T13:20:00.000-05:00</ScheduledTime>
   <EstimatedTime>1999-05-31T13:20:00.000-05:00</EstimatedTime>
   <Age>10</Age>
   <Genre>F</Genre>
   <Nationality>DEUTCH</Nationality>
  </Passenger>
</ns0:PassengerInfo>

Eventlogs are as follows;
t1

t2

t3

t4

t45PNG

t6

t7

t8

Mapping BizTalk Xsd Types to .NET Types

The following table shows how Wcf svcUtils.exe transforms the schema data types to .NET types. The detailed explanation can be found in the SOA Patterns with BizTalk Server 2009 book of Ricard Seroter.

BizTalk XSD Type Description .NET Type
anyURI Can be any absolute or relative Uniform Resource Identifier Reference System.String
base64Binary Holds Base64-encoded arbitrary binary data System.Byte[]
boolean Logical value(0/1 or true/false) System.Boolean
byte Holds an 8-bit value System.SByte
date Object with year, month, and day properties System.DateTime
dateTime Object with year, month, day, hour, minute, second, and timezone properties System.DateTime
decimal Contains a subset of real numbers with support for at a minimum of 18 decimal digits System.Decimal
double A double precision 64-bit floating point type System.Double
duration Represents a duration of time consisting of year, month, day, hour, minute and second System.String
ENTITIES Separated list of ENTITY references System.String
ENTITY Unparsed entity that may include non-XML content System.String
float A single precision 32-bit floating point type System.Single
gDay Equal to a day recurring each month System.String
gMonth Equal to a month recurring each year System.String
gMonthDay A calendar date (month + day) recurring each year System.String
gYear A period of a single year System.String
gYearMonth A particular calendar month in a specific year System.String
hexBinary Represents arbitrary hex-encoded binary data System.Byte[]
ID Definition of document-global unique identifiers System.String
IDREF A reference to a unique identifier (ID) System.String
IDREFS Separated list of IDREF references System.String
int A 32-bit signed integer System.Int32
integer A signed integer of arbitrary length System.String
language Set of language codes called out in RFC 3066 (e.g. en-US) System.String
long 64-bit signed integer System.Int64
Name XML string with no whitespace System.String
NCName Name that conforms to namespace standard (e.g. no colons) System.String
negativeInteger Encompasses all strictly negative integers System.String
NMTOKEN Set of XML “name tokens” excluding spaces or commas System.String
NMTOKENS Separated list of NMTOKENS System.String nonNegativeInteger Encompasses all positive integers (including zero) System.String
nonPositiveInteger Encompasses all negative integers (including 0) System.String
normalizedString Contains whitespace-replaced strings (meaning all carriage returns, tabs, etc have been replaced) System.String
positiveInteger Encompasses all strictly positive integers System.String
QName Qualified name as a combination of namespace name and part name XmlQualifiedName
short Set of 16-bit integers System.Int16

Service Versioning (WCF Routing)

When we built up service versioning mechanism we need to consider the following items;

– service contract changes
– data contract changes
– service binding changes

I have tried to cover all possible scenarios which consist of various service changes. The ways of satisfying backward compatibility as follows:

1. Service address changes are acceptable

If the service address can be changed, the version management is so easy. All the versions of the service can be hosted different addresses which contains version information of the service.

https://application-address:application-port/Services/TestServiceV1.svc
https://application-address:application-port/Services/TestServiceV2.svc

2. Service address changes are NOT acceptable

In this scenario, we can use WCF Routing mechanism to serve many versions over the same address. We need to have a router service that is placed in the following address.

https://application-address:application-port/Services/TestService.svc

This router service sends requests to TestServiceV1 or TestServiceV2 depends on the filters defined in configuration part of router service. Each client application will insert the filter information into the message header and router will this version-specific information contained in the message header to route the incoming message to the appropriate version of the service.

6

The filter types are as follows:

2.1.    Endpoint Name: A custom message filter for use solely in the Routing Service, which filters messages based on the name of the service endpoint.

2.2.    XPath: Used to match header or body elements of SOAP message to a particular value.

2.2.1.     The version info can be kept in the header

2.2.2.     The username can be placed in header therefore version can be done based on user over a username-version mapping.

2.3.    Action: Used to route based on the SOAP action of a message.

2.4.    And: Used to combine filters before a match can be satisfied.

2.5.    Custom: Used to supply a custom filtering component where you have programmatic control of the filter result.

3. Commercial Products in the Market

There are two major products which are generally called XML gateway or firewall. The first one is that leading product Layer7 XML Gateway (http://www.layer7tech.com/products/industry-leading-xml-gateway-overview). It is java based and running over the Linux server. It cannot be applicable for .NET environment. The second one .Net based product which is called Sentinet produced by Nevatech Inc. (https://www.nevatech.com/sentinet/soa-management). It is extension of Microsoft MSE (Managed Services Engine) framework.

Finally I have decided to keep version information in SOAP message header. WCF Router service will route the messages depends on their SOAP header version value. I have made a solution that can be seen following pic.

5

 

 

Let’s begin with implementation of TestServiceV1. The interface of TestServiceV1:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;

namespace WcfPoc.Services.TestServiceV1
{
 [ServiceContract(Namespace="http://WcfPoc.wcfRouting.int/Increment1")]
 public interface ITestService
 {
 [OperationContract]
 List<Query> GetSavedQueries(string missionCode);
 }

[DataContract]
 public class Query
 {
 [DataMember(Order = 1)]
 public string Name { get; set; }

[DataMember(Order = 2)]
 public string Description { get; set; }

[DataMember(Order = 3)]
 public string TestType { get; set; }
 }
}

The second version of TestServiceV1 contains one more method which is called WriteTestEntities and it uses the TestEntity class that is newly added. Finally we have more than one changes which are service contract and data contract changes.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;
using System.Text;

namespace WcfPoc.Services.TestServiceV2
{
 [ServiceContract(Namespace = "http://WcfPoc.wcfRouting.int/Increment1")]
 public interface ITestService
 {
 [OperationContract]
 List<Query> GetSavedQueries(string missionCode);

[OperationContract]
 bool WriteTestEntities (string missionCode, TestEntity entity);
 }

[DataContract]
 public class Query
 {
 [DataMember(Order = 1)]
 public string Name { get; set; }

[DataMember(Order = 2)]
 public string Description { get; set; }

[DataMember(Order = 3)]
 public string TestType { get; set; }

[DataMember(Order = 3)]
 public string TestTypeIdentifier { get; set; }
 }

[DataContract]
 public class TestEntity
 {
 [DataMember(Order = 1)]
 public string Capacity { get; set; }

[DataMember(Order = 2)]
 public string AirRefuelingType { get; set; }

[DataMember(Order = 3)]
 public string TestType { get; set; }

[DataMember(Order = 3)]
 public string TestTypeIdentifier { get; set; }
 }
}

After implementation of services, we need to host these services. In order to host TestServices I have used console applciation. TestServiceV1 and TestServiceV2 host applications as follows:

Host Application of TestServiceV1


using System;
using System.ServiceModel;
using WcfPoc.Services.TestServiceV1;
using WcfPoc.Host.Common;
using System.ServiceModel.Description;

namespace WcfPoc.Host.TestServiceV1
{
class Program
{
static void Main(string[] args)
{
var host = new ServiceHost(typeof(TestService));
ServiceEndpoint endpoint = host.AddServiceEndpoint(typeof(WcfPoc.Services.TestServiceV1.ITestService),
new BasicHttpBinding("TestServiceBinding"),
"http://localhost:8081/TestV1/TestService");
endpoint.Behaviors.Add(new ServiceMessagLogger());

try
{
host.Open();
Console.WriteLine("TestServiceV1 have started.");
Console.WriteLine("Address: http://localhost:8081/TestV1/TestService.svc");
Console.ReadLine();
host.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
host.Abort();
Console.ReadLine();
}
}
}
}

Host Application of TestServiceV2

using System;
using System.ServiceModel;
using System.ServiceModel.Description;
using WcfPoc.Host.Common;
using WcfPoc.Services.TestServiceV2;

namespace WcfPoc.Host.TestServiceV2
{
 class Program
 {
 static void Main(string[] args)
 {
 var host = new ServiceHost(typeof(TestService));
 ServiceEndpoint endpoint = host.AddServiceEndpoint(typeof(WcfPoc.Services.TestServiceV2.ITestService),
 new BasicHttpBinding("TestServiceBinding"),
 "http://localhost:8082/TestV2/TestService");
 endpoint.Behaviors.Add(new ServiceMessagLogger());

 try
 {
 host.Open();
 Console.WriteLine("TestServiceV2 have started.");
 Console.WriteLine("Address: http://localhost:8082/TestV2/TestService.svc");
 Console.ReadLine();
 host.Close();
 }
 catch (Exception ex)
 {
 Console.WriteLine(ex.Message);
 host.Abort();
 Console.ReadLine();
 }
 }
 }
}

The TestService versions are not known by the end user. The user just know Router service that I am gonna tell the details. The user knows interfaces of TestService V1 and V2. The users who are using both v1 and v2 send the requests to the same address not to http://localhost:8081/TestV1 or http://localhost:8082/TestV2.

Host Application of Router Service

using System;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.ServiceModel.Routing;
using WcfPoc.Host.Common;

namespace WcfPoc.Host.TestRouterService
{
 class Program
 {
 static void Main(string[] args)
 {
 var host = new ServiceHost(typeof(RoutingService));
 ServiceEndpoint endpoint = host.AddServiceEndpoint(typeof(System.ServiceModel.Routing.IRequestReplyRouter),
 new BasicHttpBinding(),
 "http://localhost:8080/TestService/router");
 endpoint.Behaviors.Add(new ServiceMessagLogger());

 try
 {
 host.Open();
 Console.WriteLine("TestRoutingService have started.");
 Console.WriteLine("Address: http://localhost:8080/TestService.svc");
 Console.ReadLine();
 host.Close();
 }
 catch (Exception ex)
 {
 Console.WriteLine(ex.Message);
 host.Abort();
 Console.ReadLine();
 }
 }
 }
}

As you can see Router service not so much different from TestService, it is implementation of System.ServiceModel.Routing.IRequestReplyRouter interface and hosted in http://localhost:8080/TestService.svc. The tricky point and routing mechanism are in the RouterService configuration file.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
 <system.serviceModel>
 <services>
 <service name="System.ServiceModel.Routing.RoutingService" 
 behaviorConfiguration="routerConfig">
 <!--<endpoint address="http://localhost:8080/TestService/router"
 binding="basicHttpBinding"
 contract="System.ServiceModel.Routing.IRequestReplyRouter"
 name="reqReplyEndpoint" />-->
 </service>
 </services>
 <behaviors>
 <serviceBehaviors>
 <behavior name="routerConfig">
 <routing filterTableName="RoutingTable" routeOnHeadersOnly="false" />
 <serviceDebug includeExceptionDetailInFaults="true"/>
 </behavior>
 </serviceBehaviors>
 </behaviors>
 <client>
 <endpoint name="TestServiceV1"
 address="http://localhost:8081/TestV1/TestService"
 binding="basicHttpBinding"
 contract="*" />
 <endpoint name="TestServiceV2" 
 address="http://localhost:8082/TestV2/TestService" 
 binding="basicHttpBinding" 
 contract="*" />
 </client>
 <routing>
 <namespaceTable>
 <add prefix="s" namespace="http://schemas.xmlsoap.org/soap/envelope/"/>
 <add prefix="cn" namespace="http://WcfPoc.wcfRouting.int/Increment1"/>
 </namespaceTable>
 <filters>
 <filter name="firstVersionFilter" 
 filterType="XPath" 
 filterData="/s:Envelope/s:Header/cn:Version ='v1.0'"/>
 <filter name="secondVersionFilter" 
 filterType="XPath" 
 filterData="/s:Envelope/s:Header/cn:Version ='v2.0'"/>
 </filters>
 <filterTables>
 <filterTable name="RoutingTable">
 <add filterName="firstVersionFilter" endpointName="TestServiceV1"/>
 <add filterName="secondVersionFilter" endpointName="TestServiceV2"/>
 </filterTable>
 </filterTables>
 </routing>
 </system.serviceModel>
</configuration>

In the filters part there are two filters are defined as firversionFilter and secondVersionFilter. If the request has Version info is equal to v1.0 than the router service will route the request to the TestServiceV1. Also I handle the case that the request doesnt have version info inth request header. In that case the router service will throw an exception to the user. The router service usage scenarios are as follows;

1. TestClientV1 and TestClientV2 are sending request to router service.
3
2. The request has no header info.

4

Have nice coding…

BizTalk Mapping – Inline XSLT usage

The BizTalk Mapper has many functoids that are capable of transforming operations. There are different type of functoids such as Advanced functoids(Value mapping, Table Looping, Assert etc.), String functoids.

2

 

In some cases, the functoids that are listed in toolbox are not capable of solving our transformation issues such as sorting, grouping etc. In order to solve complex and advance transformation issues we have to use inline xslt using with scripting functoid. It is shown in figure below.

1

 

I have tried to list most used xslt commands and i have used following source xml in the examples.


SOURCE XML
----------
<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
 <InputMessagePart_0>
  <ns1:OrderDoc xmlns:ns1="http://MessageEnrichment.Schema.OrderDoc">
   <Id>10</Id>
   <CustomerName>CustomerName_0</CustomerName>
  </ns1:OrderDoc>
 </InputMessagePart_0>
 <InputMessagePart_1>
  <ns2:GetOrderDetailsResponse xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo">
   <ns2:StoredProcedureResultSet0>
    <ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
     <ns3:OrderId>10</ns3:OrderId>
     <ns3:OrderItemId>10</ns3:OrderItemId>
     <ns3:ProductId>10</ns3:ProductId>
     <ns3:Quantity>10</ns3:Quantity>
     <ns3:Name>10_Name</ns3:Name>
     <ns3:Description>10_Description</ns3:Description>
     <ns3:Price>10.4</ns3:Price>
    </ns3:StoredProcedureResultSet0>
    <ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
     <ns3:OrderId>11</ns3:OrderId>
     <ns3:OrderItemId>11</ns3:OrderItemId>
     <ns3:ProductId>11</ns3:ProductId>
     <ns3:Quantity>11</ns3:Quantity>
     <ns3:Name>11_Name</ns3:Name>
     <ns3:Description>11_Description</ns3:Description>
     <ns3:Price>11.4</ns3:Price>
    </ns3:StoredProcedureResultSet0>
    <ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
     <ns3:OrderId>12</ns3:OrderId>
     <ns3:OrderItemId>12</ns3:OrderItemId>
     <ns3:ProductId>12</ns3:ProductId>
     <ns3:Quantity>12</ns3:Quantity>
     <ns3:Name>12_Name</ns3:Name>
    <ns3:Description>12_Description</ns3:Description>
    <ns3:Price>12.4</ns3:Price>
    </ns3:StoredProcedureResultSet0>
  </ns2:StoredProcedureResultSet0>
  <ns2:ReturnValue>15</ns2:ReturnValue>
  </ns2:GetOrderDetailsResponse>
 </InputMessagePart_1>
</ns0:Root>

 

**********************************************************************
                      ASSIGN SELECTED TO VARIABLE
**********************************************************************

XSLT
------
<xsl:variable name="vOrderName"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0/ns3:StoredProcedureResultSet0" />

<OrderName>
  <xsl:for-each select="$vOrderName" xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
    <Name>
      <xsl:element name="Description"><xsl:value-of select="ns3:Description"/></xsl:element>
      <xsl:element name="Quantity"><xsl:value-of select="ns3:Quantity"/></xsl:element>
      <xsl:element name="Price"><xsl:value-of select="ns3:Price"/></xsl:element>
      <xsl:element name="OrderItemId"><xsl:value-of select="ns3:OrderItemId"/></xsl:element>
    </Name>
  </xsl:for-each>
</OrderName>

&nbsp;

TARGET XML
----------
<OrderName>
  <Name>
    <Description>10_Description</Description>
    <Quantity>10</Quantity>
    <Price>10.4</Price>
    <OrderItemId>10</OrderItemId>
  </Name>
  <Name>
    <Description>11_Description</Description>
    <Quantity>11</Quantity>
    <Price>11.4</Price>
    <OrderItemId>11</OrderItemId>
  </Name>
  <Name>
    <Description>12_Description</Description>
    <Quantity>12</Quantity>
    <Price>12.4</Price>
    <OrderItemId>12</OrderItemId>
  </Name>
</OrderName>

 

**********************************************************************
                                 IF
**********************************************************************

XSLT
------
<xsl:variable name="vOrderName"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0" />

<OrderName>
  <xsl:for-each select="$vOrderName" xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo" xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
  <StoredProcedureResultSet0_ns2>
    <xsl:if test="count(ns2:StoredProcedureResultSet0)>0">
      Node sayisi <xsl:value-of select="count(ns2:StoredProcedureResultSet0)"/>
    </xsl:if>
  </StoredProcedureResultSet0_ns2>
  <StoredProcedureResultSet0_ns3>
    <xsl:if test="count(ns3:StoredProcedureResultSet0)>0">
      Node sayisi <xsl:value-of select="count(ns3:StoredProcedureResultSet0)"/>
    </xsl:if>
  </StoredProcedureResultSet0_ns3>
 </xsl:for-each>
</OrderName>

&nbsp;

TARGET XML
----------
<OrderName>
  <StoredProcedureResultSet0_ns2 />
  <StoredProcedureResultSet0_ns3>
    Node sayisi 3
  </StoredProcedureResultSet0_ns3>
</OrderName>

 

**********************************************************************
                            CHOOSE, WHEN
**********************************************************************

XSLT
------
<xsl:variable name="vOrderName"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0/ns3:StoredProcedureResultSet0" />

<OrderName>
  <xsl:for-each select="$vOrderName" xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
    <xsl:variable name="quantityValue" select="ns3:Quantity" />
    <xsl:if test="$quantityValue=10">
      <xsl:element name="Quantity">If case - <xsl:value-of select="ns3:Quantity"/></xsl:element>
    </xsl:if>
    <xsl:choose>
      <xsl:when test="$quantityValue>10">
        <xsl:element name="Quantity"><xsl:value-of select="ns3:Quantity"/></xsl:element>
      </xsl:when>
      <xsl:otherwise>
        <xsl:element name="Quantity">Not enough - <xsl:value-of select="$quantityValue" /></xsl:element>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:for-each>
</OrderName>

&nbsp;

TARGET XML
----------
<OrderName>
  <Quantity>If case - 10</Quantity>
  <Quantity>Not enough - 10</Quantity>
  <Quantity>11</Quantity>
  <Quantity>12</Quantity>
</OrderName>

 

**********************************************************************
                                SORTING
**********************************************************************

XSLT
------
<xsl:variable name="vOrderName"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0/ns3:StoredProcedureResultSet0" />

<OrderName>
  <xsl:for-each select="$vOrderName" xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
    <xsl:sort select="ns3:Description" order="descending"/>
    <xsl:element name="Description"><xsl:value-of select="ns3:Description"/></xsl:element>
  </xsl:for-each>
</OrderName>

TARGET XML
----------
<OrderName>
  <Description>12_Description</Description>
  <Description>11_Description</Description>
  <Description>10_Description</Description>
</OrderName>

XSLT
------
<xsl:variable name="vOrderName"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0/ns3:StoredProcedureResultSet0" />

<OrderName>
  <xsl:for-each select="$vOrderName" xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
    <xsl:sort select="ns3:Quantity" order="descending" data-type="number"/>
    <xsl:element name="Quantity"><xsl:value-of select="ns3:Quantity"/></xsl:element>
  </xsl:for-each>
</OrderName>

TARGET XML
----------
<OrderName>
  <Quantity>12</Quantity>
  <Quantity>11</Quantity>
  <Quantity>10</Quantity>
</OrderName>

 


**********************************************************************
                             GROUPING
**********************************************************************

SOURCE XML
-----------
<ns0:Root xmlns:ns0="http://schemas.microsoft.com/BizTalk/2003/aggschema">
<InputMessagePart_0>
<ns1:OrderDoc xmlns:ns1="http://MessageEnrichment.Schema.OrderDoc">
<Id>10</Id>
<CustomerName>CustomerName_0</CustomerName>
</ns1:OrderDoc>
</InputMessagePart_0>
<InputMessagePart_1>
<ns2:GetOrderDetailsResponse xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo">
<ns2:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>10</ns3:OrderId>
<ns3:OrderItemId>10</ns3:OrderItemId>
<ns3:ProductId>10</ns3:ProductId>
<ns3:Quantity>10</ns3:Quantity>
<ns3:Name>10_Name</ns3:Name>
<ns3:Description>10_Description</ns3:Description>
<ns3:Price>10.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>10</ns3:OrderId>
<ns3:OrderItemId>10</ns3:OrderItemId>
<ns3:ProductId>10</ns3:ProductId>
<ns3:Quantity>10</ns3:Quantity>
<ns3:Name>10_Name</ns3:Name>
<ns3:Description>10_Description</ns3:Description>
<ns3:Price>10.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>11</ns3:OrderId>
<ns3:OrderItemId>11</ns3:OrderItemId>
<ns3:ProductId>11</ns3:ProductId>
<ns3:Quantity>11</ns3:Quantity>
<ns3:Name>11_Name</ns3:Name>
<ns3:Description>11_Description</ns3:Description>
<ns3:Price>11.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>11</ns3:OrderId>
<ns3:OrderItemId>11</ns3:OrderItemId>
<ns3:ProductId>11</ns3:ProductId>
<ns3:Quantity>11</ns3:Quantity>
<ns3:Name>11_Name</ns3:Name>
<ns3:Description>11_Description</ns3:Description>
<ns3:Price>11.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>12</ns3:OrderId>
<ns3:OrderItemId>12</ns3:OrderItemId>
<ns3:ProductId>12</ns3:ProductId>
<ns3:Quantity>12</ns3:Quantity>
<ns3:Name>12_Name</ns3:Name>
<ns3:Description>12_Description</ns3:Description>
<ns3:Price>12.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
<ns3:StoredProcedureResultSet0 xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails">
<ns3:OrderId>13</ns3:OrderId>
<ns3:OrderItemId>13</ns3:OrderItemId>
<ns3:ProductId>13</ns3:ProductId>
<ns3:Quantity>13</ns3:Quantity>
<ns3:Name>13_Name</ns3:Name>
<ns3:Description>13_Description</ns3:Description>
<ns3:Price>13.4</ns3:Price>
</ns3:StoredProcedureResultSet0>
</ns2:StoredProcedureResultSet0>
<ns2:ReturnValue>15</ns2:ReturnValue>
</ns2:GetOrderDetailsResponse>
</InputMessagePart_1>
</ns0:Root>

XSLT
----
<xsl:variable name="vStoredProcedureResultSet0"
              xmlns:ns2="http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//InputMessagePart_1/ns2:GetOrderDetailsResponse/ns2:StoredProcedureResultSet0/ns3:StoredProcedureResultSet0" />

<xsl:variable name="vQuantity"
              xmlns:ns3="http://schemas.microsoft.com/Sql/2008/05/ProceduresResultSets/dbo/GetOrderDetails"
              select="//ns3:StoredProcedureResultSet0[not(ns3:Quantity=preceding-sibling::ns3:StoredProcedureResultSet0/ns3:Quantity)]/ns3:Quantity" />

<OrderName>
  <xsl:for-each select="$vQuantity">
    <Quantity><xsl:value-of select="."/></Quantity>
  </xsl:for-each>
</OrderName>

TARGET XML
----------
<OrderName>
  <Quantity>10</Quantity>
  <Quantity>11</Quantity>
  <Quantity>12</Quantity>
  <Quantity>13</Quantity>
</OrderName>

 

**********************************************************************
                              PIVOT
**********************************************************************

SOURCE XML
----------

<Data Header="AAA" date="2008-10-28" Name="a1" Value="1.0" />
<Data Header="AAA" date="2008-10-28" Name="a2" Value="2.0" />
<Data Header="AAA" date="2008-10-28" Name="a3" Value="3.0" />
<Data Header="BBB" date="2008-10-28" Name="a1" Value="1.0" />
<Data Header="BBB" date="2008-10-28" Name="a2" Value="2.0" />
<Data Header="BBB" date="2008-10-28" Name="a3" Value="3.0" />

XSLT
------
<xsl:template name="NameValueTemplate">
  <xsl:param name="param1" />
  <xsl:for-each select="//Data[@Header=$param1]">
    <xsl:element name="Name"><xsl:value-of select="@Name" /></xsl:element>
    <xsl:element name="Value"><xsl:value-of select="@Value" /></xsl:element>
  </xsl:for-each>
</xsl:template>

<xsl:element name="Data">
  <xsl:for-each select="Data[not(@Header=preceding-sibling::Data/@Header)]">
    <xsl:element name="Header"><xsl:value-of select="@Header" /></xsl:element>
    <xsl:element name="date"><xsl:value-of select="@date" /></xsl:element>
    <xsl:element name="Record">
      <xsl:call-template name="NameValueTemplate">
        <xsl:with-param name="param1" select="string(@Header)" />
      </xsl:call-template>
    </xsl:element>
  </xsl:for-each>
</xsl:element>

&nbsp;

TARGET XML
----------
<Data>
  <Header>AAA</Header>
  <date>2008-10-28</date>
  <Record>
    <Name>a1</Name>
    <Value>1.0</Value>
    <Name>a2</Name>
    <Value>2.0</Value>
    <Name>a3</Name>
    <Value>3.0</Value>
  </Record>
  <Header>BBB</Header>
  <date>2008-10-28</date>
  <Record>
    <Name>a1</Name>
    <Value>1.0</Value>
    <Name>a2</Name>  
    <Value>2.0</Value>
    <Name>a3</Name>
    <Value>3.0</Value>
  </Record>
</Data>

 

BizTalk Server 2013 Hot fixes

While configuring BAM section, BizTalk configuration tool completes the action with error that is detailed in log file. The error is classic access problem to SQL database. “A network-related or instance-specific error occurred” is irrelevant because the users that will do operation in sql database are defined on the database. I have searched the problem through the internet and have found the solution which is all related with a bug. Microsoft has launched the hot fixes in order to solve that kind of problems.

 

To apply this cumulative update package, you must have BizTalk Server 2013 installed.

Cumulative update package 1 for BizTalk Server 2013 – here

  • FIX: User cannot perform certain database-related operations in BizTalk Server 2013 
  • FIX: The vertical scroll bar on the target schema does not work correctly when you use Visual Studio to design a BizTalk Server 2013 map
  • FIX: BAM tools cannot be configured in a multi-node BizTalk Server 2013 environment

 

Cumulative update package 2 for BizTalk Server 2013 – here

  • FIX: The value for the “maxOccurs” attribute is invalid when you validate the BizTalk X12 EDI 835 schema in a BizTalk EDI application in BizTalk Server 2010 or in in BizTalk Server 2013
  • FIX: The memory that a BizTalk host process consumes increases when there are no messages in the queue in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: “The InnerText property is write only” error message when you use System.Xml.XmlDocument in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: Pipeline on receive locations reverts to pass-through after you update the schema assembly in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: “Could not load file or assembly ‘Microsoft.BizTalk.Interop.SSOClient Version=5.0.1.0” error when you use the ESB configuration tool in BizTalk Server 2013
  • FIX: SQL connection leak when you use WCF-based SQL Adapter one-way send port to return a result set from a database in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: 2-way WCF-BasicHttp receive port does not respond to a WCF client when the “failed message routing” feature is enabled in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: Message is suspended when you try to receive an encrypted AS2 message together with asynchronous MDN in BizTalk Server 2010 or in BizTalk Server 2013
  • FIX: An update for Message Box Viewer is available in Microsoft BizTalk Server 2013 cumulative update 2
  • FIX: Support for HIX EDI transactions 005010×306 (820) and 005010×220 (834) for BizTalk Server 2013, BizTalk Server 2010, and BizTalk Server 2009
  • FIX: XpathMutatorStream class does not work as expected when you execute Xpath statements against XmlDocuments in BizTalk Server 2013
  • FIX: BizTalk Server 2013 cannot receive the messages from the view in document library or from the SharePoint online library
  • FIX: High CPU usage after you install BizTalk Server 2013 and SQL Server 2012 on a computer
  • FIX: “System.ArgumentException” error when you browse WCF services from ESB Toolkit 2.2 in BizTalk Server 2013

BizTalk high performance tracing/logging – CAT, ETW

Every BizTalk developer needs and uses the tracing and logging feature especially development environment. Some circumstances we need to log and trace something in production environment even though this negatively impacts the performance of production environment. Basically System.Diagnostic.Trace in the .NET Framework uses for tracing but it is so primitive and doesn’t give us the flexibility of start and stop trace capability. Additionally  log4net and Enterprise library are flexible and have more features. This great article compares the performance of  tracing components which are System.Diagnostic.Trace, Entireprise library tracing, log4net and Event Tracing for Windows.

Benchmarked Components Performance Results

ETW has the most performance and it is 25 times greater than Log4net Tracing Component. The performance of System.Diagnostic component is the least one and it shouldn’t be used in production environment if the performance is required.

The BizTalk CAT Instrumentation Framework is a high performance tracing/logging framework for BizTalk that builds upon the Event Tracing for Windows (ETW) infrastructure. The BizTalk CAT Instrumentation Framework is available here. The CAT Tracing component can be started or stopped using with BizTalk CAT Instrumentation Framework Controller.

8

I have developed the POC application for examining features of BizTalk CAT Tracing Component.

4

3

9

No Entity Framework provider found for the ADO.NET provider with invariant name ‘System.Data.SqlClient’

Working with Entity Framework in the service that is consumed by BizTalk server, I realized that if we didn’t install entity framework required packages the EF throws an exception which is “No Entity Framework provider found for the ADO.NET provider with invariant name ‘System.Data.SqlClient“. The error details are as follows:

2

Open Package Manager(PM):  Tools -> Library Package Manager -> Package Manager Console.

3

You can select different projects using Default projects’ dropdown list. Type the following command


PM> Install-Package EntityFramework

4

If this solution doesn’t solve your problem, you try to remove “App=EntityFramework” from connection string. Be carefull when deleting this part, I had been deleted with &quot. I have faced of another exception which is Format of the initialization string does not conform to specification starting at index 120.

1