Four things you need to know about the new JBI cluster engine in ServiceMix 4

My thanks to Gert Vanthienen, who helped clear the light today as I dug my way through some odd behaviour from the new JBI clustering mechanism in ServiceMix 4 - odd behaviour that was down to some configuration 'gotchas'. I've written about this clustering mechanism before on an earlier blog; there are, however, a number of tricks you need to know about the way to use this mechanism, and I think it's worth mentioning here.

Make sure you register all JBI endpoints when using the OSGi packaging

.

Cluster the endpoint the sends to the NMR, not the endpoint that receives (this is counter-intuitive!)

.

If using a network of brokers, disable conduitSubscriptions on the ActiveMQ network connector.

.

Give each SMX instance a unique clusterName

.


Note: I still haven't fully made my peace with JBI; however, I must say that I'm mellowing a little now that ServiceMix 4 allows you to do simpler packaging (as bundles rather than as service-assemblies) and provides some optimizations to get rid of excessive NMR traffic. Anyway: on to that list:

First, make sure you register your endpoints as JBI endpoints. If you're using the new OSGi packaging of JBI endpoints (which I recommend!) then make sure you add the appropriate EndpointExport to your Spring context. You would of course be doing this for non-clustered JBI endpoints, but you do also need to be doing this for clustered endpoints as well.

<bean class="org.apache.servicemix.common.osgi.EndpointExporter" />

Second, cluster the producer, not the consumer. You would think that 'clustering' would involve some kind of configuration for each of the replicated endpoints 'listening' on the NMR. However, in the SMX 4 implementation, it goes the other way. You need to cluster the endpoint that's putting the information onto the NMR. For example, if you have a file poller endpoint, and you want to send the incoming file to a cluster of bean consumers, you need to 'cluster config' the file-poller using the OsgiSimpleClusterRegistration bean. This is entirely non obvious; here's how to do it. On the file-poller, do:

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:file="http://servicemix.apache.org/file/1.0"
 xmlns:clu="http://fusesource.com/clusterdemo"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://servicemix.apache.org/file/1.0 http://servicemix.apache.org/schema/servicemix-file-2009.01.xsd
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

 <file:poller id="filePoller" service="clu:file-poller" endpoint="endpoint"
  targetService="clu:payload-receiver" file="/tmp/incomingXML" />

 <bean
  class="org.apache.servicemix.jbi.cluster.engine.OsgiSimpleClusterRegistration">
  <property name="endpoint" ref="filePoller" />
 </bean>

  <bean class="org.apache.servicemix.common.osgi.EndpointExporter" />
  
</beans>

and for the 'clustered' bean endpoint, you can omit the OsgiSimpleClusterRegistration bean:

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:bean="http://servicemix.apache.org/bean/1.0" xmlns:clu="http://fusesource.com/clusterdemo"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://servicemix.apache.org/bean/1.0 http://servicemix.apache.org/schema/servicemix-bean-2010.01.0-fuse-01-00.xsd
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

 <bean:endpoint id="beanConsumer" service="clu:payload-receiver" endpoint="endpoint"
  bean="#myBean" />

 <bean id="myBean" class="org.file.processor.su.MyBean" />

  <bean class="org.apache.servicemix.common.osgi.EndpointExporter" />
 
</beans>

Third, and this is the trickiest, disable conduit subscriptions when using networked brokers. If you are using a network of embedded brokers for your internal cluster queue, you must disable conduit subscriptions to ensure that the use of message selectors is respected across the different consumers listening on the cluster queue. How do do this? Simply set conduitSubscriptions=false on your network connector; something like this:

        <networkConnectors>
            <networkConnector name="brokerA" uri="static://(tcp://localhost:61616)" duplex="true"
            conduitSubscriptions="false"/>

And, lastly, you do need to give each instance of SMX a unique name for clustering purposes. You can do this be dropping a file called 'org.apache.servicemix.jbi.cluster.engine.config.cfg' into the etc/ directory of your ServiceMix 4 instance. In this properties file, you can provide a variable called 'clusterName' to identify the this ServiceMix instance in the cluster.

clusterName = smx1

And that's it!