Mulesoft recently released some new products, totally different from each other but both very welcome.
The first one is MuleStudio. Mule Studio is an Eclipse-based developer tool, that allows to graphically create and test Mule ESB Flows. It is currently in beta but I think it might be a good reason to go back to Eclipse. Last few years I have been using Netbeans for my Mule/Java development because I liked the combination between Netbeans and Maven better.

The other ‘product’ that is released is Mule iON. I think a graphical development tool is nice to have but this iON thing is something that can really become big. It makes you use the cloud as the platform for your integration software. They call it the iPAAS (Integration Platform as a Service).

The coming days I will have a look at both and posts my findings here so stay tuned!

As promised I’m going to show a test case for which I have used Citrus to implement it. The case is like this:
In a predefined folder on a machine a character-separated file is placed. This file is picked up by our Mule ESB application. The content is transformed to XML and this XML document is posted to another application over HTTP. This application responds with XML document telling the document was processed correctly.
To test this interface with Citrus we have to perform the following steps:

1. Copy the input test-file to the predefined folder so it is picked up by the test instance of our Mule ESB

Our Mule ESB will poll a directory each second to check if there is placed a new file to process. Our test case starts by having Citrus putting a test file in the directory .

2. Have Citrus listen on a predefined HTTP port to receive the XML message over HTTP

Inside our Mule ESB the file is transformed to XML message and posted on a HTTP port to be processed by the ‘ORDER’ system. We have configured Citrus to listen on this specific HTTP port to pick up the XML message.

3. Reply to the received message with a predefined XML message

This steps mimics the ‘ORDER’ system as it would normally process the incoming XML message and respond with another XML message. In this case Citrus will send back a predefined XML message as response.

The citrus-config.xml for this test case can be found here. I will explain the content of the config file next:

1
2
3
4
5
6

<bean id="citrusJMSConnectionFactory"
         class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="vm://localhost:61616" />
        <property name="userName" value="user"/>
        <property name="password" value="password"/>
</bean>

This part defines a ‘in-memory’ JMS provider to be used by Citrus. We will see later how we use it to check the incoming HTTP request (as described here in the Citrus reference guide).

1
2
3
4
5
6
7

<bean id="namespaceContextBuilder" class="com.consol.citrus.xml.namespace.NamespaceContextBuilder">
        <property name="namespaceMappings">
            <props>
                <prop key="ns1">http://www.redstream.nl/invoice</prop>
            </props>
        </property>
</bean>

Here we define the namespaces that are used in the xml messages in our test. We will use this namespace in the test to check values of certain XML elements.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<bean name="redstreamIntegrationTests"
         class="com.consol.citrus.TestSuite">
        <property name="tasksBefore">
            <list>
                <bean class="com.consol.citrus.actions.ExecuteSQLAction">
                    <property name="dataSource" ref="rsDataSource"/>
                    <property name="statements">
                        <list>
                            <value>DELETE FROM RS_LOGS</value>
                        </list>
                    </property>
                </bean>
            </list>
        </property>
</bean>

With this part I have Citrus clean up my log table in the database of my Mule ESB. We log all incoming and outgoing messages in our ESB to the database.

1
2
3
4
5
6
7
8
9
10
11
12
13

<!-- manadatory if XML files are used in a test -->
<bean id="xmlMessageValidator" class="com.consol.citrus.validation.xml.DomXmlMessageValidator"/>
    <bean id="schemaRepository"
           class="com.consol.citrus.xml.XsdSchemaRepository">
        <property name="schemas">
            <list>
                <bean class="org.springframework.xml.xsd.SimpleXsdSchema">
                    <property name="xsd"
                               value="classpath:schemas/rs_invoice_0.3.xsd"/>
                </bean>
            </list>
        </property>
</bean>

When you want to work with XML messages in your test (and that is very likely nowadays) you will need to define an ‘xmlMessageValidator’. And to be able to validate the message against an XML schema you will need to define the schemas in the ‘schemaRepository’ as I have done here.

1
2
3
4
5
6
7

<bean id="rsDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="url" value="jdbc:oracle:thin:@test-vm:1521:XE"/>
        <property name="username" value="user"/>
        <property name="password" value="password"/>
        <property name="driverClassName" value="oracle.jdbc.xa.client.OracleXADataSource"/>
        <property name="removeAbandoned" value="true"/>
</bean>

This is the datasource that Citrus can use to access the database that is used by our Mule ESB application. We use this datasource to clean up the logtable and can use it to perform validations after messages have been processed.

1
2
3
4
5

 <citrus:message-channel-sender id="invoiceFileSender" channel="fileOutboundChannel"/>
<file:outbound-channel-adapter id="invoice-files-out" channel="fileOutboundChannel"
        directory="file://Volumes/staging/invoice" filename-generator="test-filename-generator" />
<bean id="test-filename-generator" class="nl.redstream.citrus.TestFileNameGenerator" />
<si:channel id="fileOutboundChannel" />

This is a mix of Citrus with spring-integration configuration. With this configuration I am able to send files to my test VM with Citrus. It also shows how easy it is to mix Citrus with spring-integration.

1
2
3
4
5
6
7

<!-- receive Invoice HTTP request -->
<citrus-http:server id="orderHttpServer"
               port="8060"
               uri="/services/post-invoice"
               deamon="false"
               message-handler="jmsForwardingMessageHandler"
               auto-start="true"/>

Definition of the HTTP Server in Citrus which listens on port 8060 for a request of our Mule ESB.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

 <bean id="jmsForwardingMessageHandler"
         class="com.consol.citrus.adapter.handler.JmsConnectingMessageHandler">
        <property name="destinationName" value="Citrus.order-response"/>
        <property name="connectionFactory" ref="citrusJMSConnectionFactory" />
        <property name="replyTimeout" value="6000"/>
    </bean>
 
    <!-- Process all received requests -->
    <citrus:jms-sync-message-receiver id="orderRequestReceiver"
                            destination-name="Citrus.order-response"
                            connection-factory="citrusJMSConnectionFactory" />
    <citrus:jms-reply-message-sender id="orderResponseSender"
                            reply-destination-holder="orderRequestReceiver"
                            connection-factory="citrusJMSConnectionFactory" />

THis part forwards the incoming HTTP request to a JMS endpoint. In our test we can perform tests on this incoming JMS message and check if it is valid.

Next thing I want to explain is the test-config.xml itself. I think it is quite self-explainatory:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

<?xml version="1.0" encoding="UTF-8"?>
<spring:beans xmlns="http://www.citrusframework.org/schema/testcase"
              xmlns:ws="http://www.citrusframework.org/schema/ws/testcase"
              xmlns:spring="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://www.springframework.org/schema/beans
              http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                  http://www.citrusframework.org/schema/ws/testcase
                  http://www.citrusframework.org/schema/ws/testcase/citrus-ws-testcase.xsd
                  http://www.citrusframework.org/schema/testcase
                  http://www.citrusframework.org/schema/testcase/citrus-testcase.xsd">
 
    <testcase name="Testcase001Test">
        <meta-info>
            <author>Redstream Development Team</author>
            <creationdate>2011-04-24</creationdate>
            <status>DRAFT</status>
        </meta-info>
 
        <description>Tests Testcase001</description>
 
        <actions>
            <send with="invoiceFileSender">
                <message>
                    <resource file="classpath:data/testcase001.txt"/>
                </message>
            </send>
            <receive with="orderRequestReceiver">
 
                <message schema-validation="true">
                    <validate path="//ns1:post-invoice/ns1:CompanyCode" value="1"/>
                    <validate path="string:count(//ns1:post-invoice/ns1:InvoiceLine/ns1:DocID)" value="3"/>
                </message>
                <header>
                    <element name="rs_address" value="order-purchase-invoice-queue" />
                </header>
                <extract>
                    <message path="count(//ns1:post-invoice/ns1:InvoiceLine/ns1:DocID)" variable="${nrOfInvoiceLines}"/>
                </extract>
            </receive>
 
            <echo>
                <message>${nrOfInvoiceLines}</message>
            </echo>
            <send with="orderResponseSender">
                <message>
                    <resource file="classpath:data/response-testcase001.xml"/>
                </message>
            </send>
        </actions>
    </testcase>
</spring:beans>

The first action that is done is sending a file to my Mule ESB. The next step is that I wait for the response from my ESB. When I received that I check the contents of XML message at some points and check if it is valid against the schema.
Then after echoing the number of invoice lines I have Citrus return the HTTP request with a fixed XML response so the Mule ESB ends this action successfully.
That completes this example of how to use Citrus to test your ESB implemention. I hope it is more clear now how you can use Citrus for your (automated) integration tests with this real life example.

Recently I got into a situation where I had to put ‘metadata’ values (which were part of the Mule Message) into the XML result of an XSLT transformer. Let me explain the situation with an example. Imagine you have an ‘order’ xml that needs to be translated into an ‘invoice’ xml. Here is a simple ‘order.xsd’:

Here is the ‘invoice.xsd’:

The mapping for the transformation looks like this in MapForce:

Now the invoice number that is used in the invoice should be the MuleMessage correlationID…. (I know, weird situation, but it is only an example). The first thing that came to mind was to create a new transformer that would run after the XSLT transformer. This new transformer would add the element ‘invoicenr’ with the correlationID as value to the payload of the MuleMessage. But it can be done much easier! Here is the modified mapping that I created with MapForce:

In MapForce you can add extra input parameters for the XSL mapping as shown above. Now generate the XSLT and use that as basis for your Mule XSLT Transformer. In the Mule config you add the context-property to your XSL Transformer configuration to tell Mule with which value the parameter in the XSLT file should be filled:

1
2
3
4
5
6
7
8
9
10
11
12
13

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
     xmlns:xm="http://www.mulesource.org/schema/mule/xml/2.2"
     ...
     xsi:schemaLocation="
      http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/mule-xml.xsd
     ...
     ">
    <xm:xslt-transformer name="orderToInvoiceTransformer" xsl-file="xsl/order-to-invoice.xsl" outputEncoding="UTF-8">
        <xm:context-property key="invoiceNr" value="#[message:correlationId]"/>
    </ship:xslt-transformer>
    ...
</mule>

This makes it a very clean and powerful way to add metadata to your payload might you ever need this.

At a project for one of our customers we migrated from Mule CE to Mule EE edition (version 2.2.7). The customer had various good reasons to do this and it offered us new functionalities in Mule ESB to use. One of those ‘new’ things is RetryPolicy for some of the Mule Connectors. I quoted ‘new’ because there is also an open source variant that you can use in the community edition but since the customer paid for it I wanted to use this version that comes with Mule EE out-of-the-box.
To make use of this EE version I performed the normal steps that you would do when you add a new module to you (Maven) Mule project (as I described here). So in short:

• Add the namespace to the Mule config:

  1 2 3 4 5

  <mule xmlns="http://www.mulesource.org/schema/mule/core/2.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ee="http://www.mulesource.org/schema/mule/ee/core/2.2" xsi:schemaLocation="http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd http://www.mulesource.org/schema/mule/ee/core/2.2 http://www.mulesource.org/schema/mule/ee/core/2.2/mule-ee.xsd">

• Add the dependencies to the pom.xml:

  1 2 3 4 5 6 7 8 9 10

  <dependency> <groupId>com.mulesource.muleesb</groupId> <artifactId>mule-core-ee</artifactId> <version>2.2.7</version> </dependency> <dependency> <groupId>com.mulesource.muleesb.modules</groupId> <artifactId>mule-module-retry-ee</artifactId> <version>2.2.7</version> </dependency>

• Add the retry-element to the connector:

  1 2 3 4 5 6 7 8 9 10

  <jms:activemq-connector name="JMSConnector" specification="1.1" dynamicNotification="true" brokerURL="failover:(${jms.jndiProviderUrl})" acknowledgementMode="AUTO_ACKNOWLEDGE" maxRedelivery="3" disableTemporaryReplyToDestinations="true" username="${jms.activeMQUser}" password="${jms.activeMQPassword}" persistentDelivery="true"> <ee:retry-forever-policy frequency="30000"/> </jms:activemq-connector>

Unfortunately, this seemed not enough since I was getting the following stacktrace:

ERROR 2010-12-22 11:55:34,467 [main] org.mule.config.spring.SpringXmlConfigurationBuilder: Configuration with “org.mule.config.spring.SpringXmlConfigurationBuilder” failed.
org.mule.api.lifecycle.InitialisationException: Initialisation Failure: Configuration problem: Failed to import bean definitions from URL location [classpath:config/rs-global-config.xml]
Offending resource: URL [jar:file:/Users/pascal/.m2/repository/nl/redstream/framework/rs-test-unit/2.3-SNAPSHOT/rs-test-unit-2.3-SNAPSHOT.jar!/config/rs-core-test-config.xml]; nested exception is org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 70 in XML document from class path resource [config/rs-global-config.xml] is invalid; nested exception is org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘ee:retry-forever-policy’. One of ‘{“http://www.mulesource.org/schema/mule/core/2.2″:abstract-retry-policy, “http://www.mulesource.org/schema/mule/core/2.2″:service-overrides}’ is expected.
at org.mule.registry.AbstractRegistry.initialise(AbstractRegistry.java:76)
at org.mule.config.spring.SpringXmlConfigurationBuilder.createSpringRegistry(SpringXmlConfigurationBuilder.java:110)
at org.mule.config.spring.SpringXmlConfigurationBuilder.doConfigure(SpringXmlConfigurationBuilder.java:73)
at org.mule.config.builders.AbstractConfigurationBuilder.configure(AbstractConfigurationBuilder.java:39)
at org.mule.config.builders.AbstractResourceConfigurationBuilder.configure(AbstractResourceConfigurationBuilder.java:78)
at org.mule.context.DefaultMuleContextFactory.createMuleContext(DefaultMuleContextFactory.java:79)
at org.mule.tck.AbstractMuleTestCase.createMuleContext(AbstractMuleTestCase.java:474)
at org.mule.tck.AbstractMuleTestCase.setUp(AbstractMuleTestCase.java:443)
at junit.framework.TestCase.runBare(TestCase.java:132)
at org.mule.tck.AbstractMuleTestCase.runBare(AbstractMuleTestCase.java:295)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at org.mule.tck.AbstractMuleTestCase.run(AbstractMuleTestCase.java:274)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
at ….. (much much more)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.401 sec <<< FAILURE!

After some investigation it seemed I missed a dependency:

1
2
3
4
5

            <dependency>
                <groupId>com.mulesource.muleesb.modules</groupId>
                <artifactId>mule-module-spring-config-ee</artifactId>
                <version>2.2.7</version>
            </dependency>

The problem was that this was not documented (at least not in the places I looked ;-)). Luckily we solved it and is working fine now.

Some of our Mule ESB applications make use of content-based routing when processing a message. To implement this in Mule we use a filtering-router with a jaxen-filter. The most simple example looks like this:

1
2
3
4
5
6
7
8

...
<outbound>
  <filtering-router>
    <vm:outbound-endpoint ref="out-queue" />
     <xm:jaxen-filter pattern="//firstname" expectedValue="Pascal"/>
  </filtering-router>
</outbound>
...

You can also use a XPath function in your jaxen-filter like this (if you want to check a certain element is in an XML message regardless the value of the element):

1
2
3
4
5
6
7
8

...
<outbound>
  <filtering-router>
    <vm:outbound-endpoint ref="out-queue" />
    <xm:jaxen-filter pattern="local-name(//Relations)" expectedValue="Relations"/>
  </filtering-router>
</outbound>
...

The problem starts if you have to declare namespaces in your XPath expression. You can use namespaces in your XPath expression by declaring a namespace-manager in your mule-config, like this:

1
2
3
4
5
6
7
8
9
10
11
12

...
 <xm:namespace-manager includeConfigNamespaces="false">
   <xm:namespace prefix="ns1" uri="http://www.redstream.nl/relations/v1_1"/>
 </xm:namespace-manager>
...
   <outbound>
     <filtering-router>
       <vm:outbound-endpoint ref="out-queue" />
       <xm:jaxen-filter pattern="local-name(//ns1:Relations)" expectedValue="Relations"/>
     </filtering-router>
   </outbound>
...

But if you run this example you will get the following exception:

org.jaxen.UnresolvableException: No Such Function {http://www.mulesource.org/schema/mule/core/2.2}:local-name
at org.jaxen.SimpleFunctionContext.getFunction(SimpleFunctionContext.java:127)
at org.jaxen.ContextSupport.getFunction(ContextSupport.java:242)
at org.jaxen.Context.getFunction(Context.java:216)
at org.jaxen.expr.DefaultFunctionCallExpr.evaluate(DefaultFunctionCallExpr.java:172)
at org.jaxen.expr.DefaultXPathExpr.asList(DefaultXPathExpr.java:102)
at org.jaxen.BaseXPath.selectNodesForContext(BaseXPath.java:674)
at org.jaxen.BaseXPath.selectSingleNodeForContext(BaseXPath.java:698)
at org.jaxen.BaseXPath.stringValueOf(BaseXPath.java:279)
at org.jaxen.BaseXPath.valueOf(BaseXPath.java:254)
at org.mule.module.xml.filters.JaxenFilter.accept(JaxenFilter.java:184)
at org.mule.module.xml.filters.JaxenFilter.accept(JaxenFilter.java:118)
at org.mule.routing.outbound.FilteringOutboundRouter.isMatch(FilteringOutboundRouter.java:119)
at org.mule.routing.outbound.DefaultOutboundRouterCollection. route(DefaultOutboundRouterCollection.java:72)
at org.mule.service.AbstractService.dispatchToOutboundRouter(AbstractService.java:867)
at org.mule.model.seda.SedaService.dispatchToOutboundRouter(SedaService.java:561)
at org.mule.model.seda.SedaService$ComponentStageWorker.doRun(SedaService.java:583)
at org.mule.work.AbstractMuleEventWork.run(AbstractMuleEventWork.java:41)
at org.mule.work.WorkerContext.run(WorkerContext.java:310)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor. runWorker(ThreadPoolExecutor.java:1061)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker. run(ThreadPoolExecutor.java:575)
at java.lang.Thread.run(Thread.java:637)

Although it is logged as warning it makes the filter to return no matching values and the XML message is discarded (although I use the jaxen-filter here this also goes for the jxpath-filter and the xpath-filter).

The solution for this is quite simple: just give the XPath function a unique prefix (one that is not mapped to a namespace). So if you modify the last example and make the jaxen-filter like:
<xm:jaxen-filter pattern="fn:local-name(//ns1:Relations)" expectedValue="Relations"/>
or
<xm:jaxen-filter pattern="abc:local-name(//ns1:Relations)" expectedValue="Relations"/>
it will work.
You just have to make sure the prefix you use isn’t used in the declared namespace-manager because then the function is looked up (and not found) in the supplied XML document.

Last week I had to implement an interface for a customer in which an XML file had to be loaded into the database. But … the database table needed to be truncated first. At first, I thought that a chaining router would suffice. I soon found out that the behaviour of this router was not desirable at all.

Mule ESB contains multicasting and chaining routers to join a set of outbound endpoints. They differ in behaviour. When using a multicasting-router a message is send to multiple endpoints in a parallel way. So, you’re not sure if endpoint A has processed the message when endpoint B is about to start.

The chaining-router sends the message to the outbound endpoint collection in a sequential way, but the outcome of endpoint A will be send to endpoint B as payload. In other words, the endpoint will not process the same MuleMessage.

So, what to use if both types are not suitable?

Mule allows you to setup multiple routers (inbound as well as outbound). Class org.mule.routing.outbound.DefaultOutboundRouterCollection represents the collection of outbound routers in your Mule configuration. By using the (optional) matchAll attribute on the outbound router collection, the MuleMessage will be copied sent to *all* outbound routers (look at the public MuleMessage route(final MuleMessage message, final MuleSession session) method).

The following code snippet shows hows this should be implemented. It’s not a complete config .. but you’ll get the point. Notice the matchAll attribute!

1
2
3
4
5
6
7
8
9
10
11
12
13

<service name="foobar">
  <inbound>
    <vm:inbound-endpoint path="do.it"/>
  </inbound>
  <outbound matchAll="true">
    <pass-through-router>
      <jdbc:outbound-endpoint queryKey="truncateCustomers"/>
    </pass-through-router>
    <expression-splitter-router evaluator="xpath" expression="//Customer">
       <jdbc:outbound-endpoint queryKey="insertCustomer"/>
    </expression-splitter-router>
  </outbound>
</service>

Suppose an XML document containing Customers is offered to the inbound endpoint. What happens is that the Customers database table is truncated first and next the XML document is split up using the xpath expression. The end result is that the Customer table is replaced with the contents of the XML document.

In short: there are numerous situations where you would like to offer the MuleMessage to the complete outbound router collection. Use the matchAll attribute in these situation.

Happy Mule’ing !!

UPDATE: seems that there’s an even simpler solution to the problem that I’ve overlooked. (although my described solution works as advertised).

Using the following code snippet will do the trick as well:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<service name="foo">
...
<outbound>
  <multicasting-router>
    <jdbc:outbound-endpoint queryKey="truncateCustomers" synchronous="true"/>
    <vm:outbound-endpoint path="processCustomers" synchronous="true"/>
  </multicasting-router>
</outbound>
</service>
 
<service name="bar">
  <inbound>
    <vm:inbound-endpoint path="processCustomers"/>
  </inbound>
  <outbound>
    <expression-splitter-router evaluator="xpath" expression="//Customer">
      <jdbc:outbound-endpoint queryKey="insertCustomer"/>
    </expression-splitter-router>
  </outbound>
</service>

Notice that the matchAll attribute has been replaced with synchronous. This last attribute results in the fact that the incoming message will *not* move on to the next outbound router in the collection until it has been consumed by a previous one.

With the introduction of Mule 3, this will probably all change (for the better).

When you are using a certain version of Mule ESB you are going to run into an existing bug sooner or later. That isn’t really a problem, I would call it inevitable (at least based on my own experience). The big advantage of the open source idea is that you can dive into the source code and see where the problem is caused and fix it ( or at least work around it). I remember the time I was using Oracle Forms Developer to build applications. When you ran into a bug you had to wait until Oracle was ready to fix/patch it (or, of course, convince the client to not want that specific feature in their application).
Although its true you can find all known bugs for Mule in Jira, you still have to discover yourself your application is having a problem with that known issue. I recently implemented a new interface in our application and I started to make use of the CXF transport (until now it was mainly based on JMS and VM). The exception I got when processing a message was

java.io.IOException: Attempted read from closed stream.

The weird thing was, I didn’t get this exception when running my unit tests and it sometimes even worked in my deployed application.
As I said we got this exception when we added the CXF endpoint, although we did it as described in the manual. Well, to make a long story short, it appeared we were running into this issue. In our deployed version we make use of the Mule Notifications. We have extended the default Mule endpoints so they can indicate if a message they are processing should be logged. This logging is done by receiving the Notification and extracting the message from it. Then we log the payload of the MuleMessage as text in our database.
This works great as long the payload of the message isn’t an InputStream! in case of an InputStream the stream is read to make a String of it and since a InputStream can only be read once, the rest of the application cannot read it again (hence the ‘Attempted read from closed stream‘). This behaviour is described in Jira but you can image that it took some time before we realized we were running into this. Especially because the stack trace isn’t of much use in this case since it is pointing to some deep down CXF code that tries to read the input stream.
Next time someone has a similar issue and you are using Notification Handlers you might want to check these out and this way save yourself a lot of time looking for the cause of this issue.

Mule ESB is a versatile service bus that contains numerous usable components out-of-the-box. In this blog I’ll show you how to process an incoming XML document, break it up into multiple parts and insert them into the database without using any custom Java classes.

In this example we will be using this (simple) XML document as source input. The document contains a collection of Customers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

<?xml version="1.0" encoding="UTF-8"?>
<customers>
   <customer>
      <firstName>Steve</firstName>
      <lastName>Jobs</lastName>
      <address>
         <street>1 Infinite Loop</street>
         <city>Cupertino</city>
         <country>USA</country>
      </address>
   </customer>
   <customer>
      <firstName>Bill</firstName>
      <lastName>Gates</lastName>
      <address>
         <street>1 Microsoft Way</street>
         <city>Washington</city>
         <country>USA</country>
      </address>
   </customer>
</customers>

After breaking the XML up into multiple Customer entities, we will insert them into the following database structure:

1
2
3
4
5
6
7

create table CUSTOMERS (
   first_name VARCHAR2(32)
,  last_name VARCHAR2(64)
,  street VARCHAR2(128)
,  city VARCHAR2(128)
,  country VARCHAR2(3)
);

To split the XML document into multiple Customer entities, we’ll use an outbound splitting router like this one:

1
2
3

<expression-splitter-router evaluator="xpath" expression="//Customer">
   <stdio:outbound-enpdoint system="OUT"/>
</expression-splitter-router>

The xpath evaluator mentioned in the expression-splitter-router excepts both Dom and String objects. This means we can just use plain XML files (no need to create a w3c.dom.Document first. It’ll be parsed on-the-fly.

Once the message has been split, we want each Customer to be inserted into a database. The trick here is to use the JDBC transport. We’ll need to configure a connector in which
“named queries” can be configured. A “named query” is a PODS (~Plain Old Dml Statement) identified by a key.

1
2
3
4

    <jdbc:connector name="jdbcConnector" dataSource-ref="jdbcDataSource">
        <jdbc:query key="insertIntoFoobar"
                    value="insert into FOOBAR (id, code, description) VALUES (#[map-payload:ID], #[map-payload:CODE], #[map-payload:DESCRIPTION])"/>
    </jdbc:connector>

In this example, a Map payload is expected from which the values are derived. When using XML, an XPATH expression evaluator seems more appropriate (see final solution).

PS: It might be nice to know that unnamed queries are also supported.

I’ve introduced the basics, now let’s have a look at the complete Mule configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
      xmlns:spring="http://www.springframework.org/schema/beans"
      xmlns:vm="http://www.mulesource.org/schema/mule/vm/2.2"
      xmlns:xm="http://www.mulesource.org/schema/mule/xml/2.2"
      xmlns:jdbc="http://www.mulesource.org/schema/mule/jdbc/2.2"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xmlns:context="http://www.springframework.org/schema/context"
      xsi:schemaLocation="
      http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd
      http://www.mulesource.org/schema/mule/vm/2.2 http://www.mulesource.org/schema/mule/vm/2.2/mule-vm.xsd
      http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
      http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/mule-xml.xsd
      http://www.mulesource.org/schema/mule/jdbc/2.2 http://www.mulesource.org/schema/mule/jdbc/2.2/mule-jdbc.xsd
      http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd">
 
    <context:property-placeholder location="classpath:my.properties"/>
 
    <spring:bean id="jdbcDataSource" class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
        <spring:property name="driverName" value="${db.driverName}"/>
        <spring:property name="url" value="${db.url}"/>
    </spring:bean>
 
    <jdbc:connector name="jdbcConnector" dataSource-ref="jdbcDataSource">
        <jdbc:query key="insertCustomer"
                    value="insert into CUSTOMERS (first_name, last_name, street, city, country) VALUES (#[xpath://Customer/FirstName],#[xpath://Customer/LastName],#[xpath://Customer/Address/Street],#[xpath://Customer/Address/City],#[xpath://Customer/Address/Country])"/>
    </jdbc:connector>
 
    <xm:namespace-manager includeConfigNamespaces="true">
        <xm:namespace prefix="cus" uri="http://www.redstream.nl/schema/customer/1.0"/>
    </xm:namespace-manager>
 
    <model>
 
        <service name="processCustomers">
            <inbound>
                <vm:inbound-endpoint path="processCustomersService"/>
            </inbound>
 
            <outbound>
                <expression-splitter-router evaluator="xpath" expression="//Customer">
                    <jdbc:outbound-endpoint queryKey="insertCustomer"/>
                </expression-splitter-router>
            </outbound>
        </service>
 
    </model>
 
</mule>

Et voila … using the above configuration, you will be able to process incoming XML and insert the contents in a database.

Some final notes:

• Use a property-placeholder to configure the database credentials.
• When incoming XML contains namespaces, they could globally be configured using a namespace-manager
• Notice that the jdbc:outbound-endpoint refers to the named query by key
• The named query will be “injected” with a single Customer entity, accessible by the XPATH expression evaluator

As promised in our previous blog about Altova Mapforce we will show in this post how to combine the result of the generated code of Altova Mapforce with a Mule application. As we have shown before, the Java code that is generated by MapForce is divided in two parts:

• one is Altova generic, so will be reused for each mapping
• one is mapping specific so is the actual part that changes with the mapping (this package name is configurable)

We first jarred the Altova generic code and added it to our Maven repository (Artifactory) so we can include this in our Mule project as dependency. We removed the generated mapping-specific Java code and used the included Ant build script to create the jar file. Here is a screenshot of how it looks in Artifactory:

Now if we want to use Mapforce generated code, we add the following dependency to the pom:

1
2
3
4
5

<dependency>
  <groupId>com.altova</groupId>
  <artifactId>mapforce-core</artifactId>
  <version>2010R2</version>
</dependency>

The next step is to add the generated mapping-specific code to the project. We do this by copying the generated mapping specific code to our Mule project (in MapForce you can configure the package name to be used for the generated code so that makes it easier fitting in
with the rest of the code). Here is a screenshot of the code to copy in this case:

Then we created a Transformer class in the Mule project that calls the generated mapping code and returns the result.
Here is an example of a Transformer which uses the generated code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

package nl.redstream.transformers;
 
import com.altova.io.StreamInput;
import com.altova.io.StringInput;
import com.altova.io.StringOutput;
import java.io.InputStream;
import nl.redstream.mapping.MappingMapTocustomer2;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractTransformer;
 
/**
 *
 * @author pascal
 */
public class CustomerCsvToXmlTransformer extends AbstractTransformer {
 
    public CustomerCsvToXmlTransformer() {
        registerSourceType(InputStream.class);
        registerSourceType(String.class);
        setReturnClass(String.class);
    }
 
    @Override
    protected Object doTransform(Object payload, String encoding)
            throws TransformerException {
 
        MappingMapTocustomer2 mappingToXml = new MappingMapTocustomer2();
        StringOutput output = new StringOutput();
 
        if (payload instanceof String) {
            StringInput input = new StringInput((String) payload);
            try {
                mappingToXml.run(input, output);
            } catch (Exception ex) {
                throw new TransformerException(this, ex);
            }
 
        } else if (payload instanceof InputStream) {
            StreamInput sInput = new StreamInput((InputStream) payload);
            try {
                mappingToXml.run(sInput, output);
            } catch (Exception ex) {
                throw new TransformerException(this, ex);
            }
 
        }
        return output.getString().toString();
    }
}

That’s it! Define the transformer as a ‘custom-transformer’ in your Mule config and your done. You can now create and maintain all your mappings visually with MapForce.
We are now looking into the capabilities that SPL (Spy Programming Language) offer, because it looks like we can generate the Mule Transformer straight from MapForce, that would even boost the producivity more!

As I promised before here is an example of how to use Smooks in combination with Mule. In this example I transform a CSV file to a plain XML file. This XML file can then be transformed with an XSLT transformation to the desired XML format.
I start with the Smooks config file ‘smooks-csv-config.xml’:

1
2
3
4
5
6
7
8
9
10

<?xml version="1.0" encoding="UTF-8"?>
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
                      xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd">
    <csv:reader fields="order_no?trim,cust_no?trim,prod_no?trim,amount?trim"
    separator="|" quote="'" skipLines="0" rootElementName="orders" recordElementName="order" indent="true" />
 
    <resource-config selector="global-parameters">
        <param name="stream.filter.type">SAX</param>
    </resource-config>
</smooks-resource-list>

The important part is in the ‘csv:reader fields’ attribute. Here I define the fields that can be expected in my input CSV file. As you can see I apply ‘?trim’ to the fields so additional spaces are removed.
My Mule config file ‘smooks-csv-config’ that is used for this test can be found here. The important part looks like:

1
2
3
4
5
6

<smooks:transformer
        name="csvToXmlSmooksTransformer"
        configFile="/transforms/smooks-csv-config.xml"
        resultType="STRING"
        reportPath="target/smooks-report/report.html"
        />

In the transformer the Smooks config file to be used is defined, so in this case I refer to the one I showed in at the beginning of this post. The next step is to create the test class.:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

package nl.redstream.mule.test;
 
import java.io.File;
import java.io.InputStream;
import java.util.Locale;
import java.util.TimeZone;
 
import org.junit.Test;
import org.mule.DefaultMuleMessage;
import org.mule.api.MuleMessage;
import org.mule.module.client.MuleClient;
import org.mule.tck.FunctionalTestCase;
import org.mule.util.IOUtils;
 
public class SmooksCsvTest extends FunctionalTestCase
{
	@Override
	protected String getConfigResources() {
		return "config/smooks-csv-config.xml";
	}
 
	@Test
	public void testSmooks() throws Exception {
		InputStream in = IOUtils.getResourceAsStream("test-order.csv", this.getClass());
 
		MuleClient client = new MuleClient();
		MuleMessage reply = client.send("vm://test-csv-to-xml",new DefaultMuleMessage(in));
 
		assertNotNull(reply);
		assertNotNull(reply.getPayload());
 
		Object payload = reply.getPayload();
		assertTrue("The message payload is not an instance of String", payload instanceof String);
		assertTrue("The report file wasn't created", getReportFile().exists());
	}
 
	private File getReportFile() {
		return new File("target/smooks-report/report.html");
	}
	private void deleteReportFile() {
		getReportFile().delete();
	}
 
	/* (non-Javadoc)
	 * @see org.mule.tck.AbstractMuleTestCase#doSetUp()
	 */
	@Override
	protected void doSetUp() throws Exception {
		super.doSetUp();
 
		TimeZone.setDefault(TimeZone.getTimeZone("EST"));
		Locale.setDefault(new Locale("en","IE"));
		deleteReportFile();
	}
 
	/* (non-Javadoc)
	 * @see org.mule.tck.AbstractMuleTestCase#doTearDown()
	 */
	@Override
	protected void doTearDown() throws Exception {
		super.doTearDown();
		deleteReportFile();
	}
}

In the file I read the CSV file and put it on the inbound endpoint. The Smooks transformer is then applied to it and the result is passed to the STDIO outbound. So as a result I expect XML output in my console. The CSV file I used for testing looks like:

1888852| 21625|02745011|31|
1888853| 21625|02745011|71|
1888854| 21625|02745011| 3|

And the result in my console contains:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<orders>
        <order number="1">
                <order_no>1888852</order_no>
                <cust_no>21625</cust_no>
                <prod_no>02745011</prod_no>
                <amount>31</amount>
        </order>
        <order number="2">
                <order_no>1888853</order_no>
                <cust_no>21625</cust_no>
                <prod_no>02745011</prod_no>
                <amount>71</amount>
        </order>
        <order number="3">
                <order_no>1888854</order_no>
                <cust_no>21625</cust_no>
                <prod_no>02745011</prod_no>
                <amount>3</amount>
        </order>
</orders>

Of course this generated XML is very basic but it can be the base for the next (XSLT) transformation step as I said earlier in this post to create a more advanced XML matching your needs.