People are, in general, blown away by Camel. Hearts melt and grown men weep with joy when they learn how easy it is to write an integration flow, package it as an OSGi bundle, and then drop it into the FUSE ESB 4 container. But when the emotions have subsided and rationality comes again to the fore, I often hear the question ‘if Camel can do so much so easily, then why do I need JBI anymore?’. The answer is, that in many cases, you *don’t*.
This is hard for some ServiceMix users to stomach, and delightful to others. I have written before about my ‘love affair’ with JBI on this blog, a love affair that was soon to be blighted by some hardened home truths about what really goes on on the NMR. With my JBI relationship in almost tatters, I embraced Camel with open arms. Am I on the rebound? Hardly. Camel is simply a better technology choice when implementing ESB solutions than JBI. What makes it better? A great DSL (domain specific language) for implementing integration patterns. Extensible beyond Java into other scripting languages and technologies. No canonical intermediary format (XML). Dynamic routing. On-demand type conversion. Parameterizable routes (do *that* with JBI!). Support for unit testing of routing logic with ‘mock’ endpoints. An extensive list of technology components, with easy-to-use URI configuration. All these things make for a faster way to produce more with less.
With all these things going for it, then why continue to support JBI in ServiceMix/ FUSE ESB 4? There are a number of good reasons, and, I like to think that while my amorous intentions for JBI are firmly in the past, we do now remain “Just Good Friends’. There are many users out there who currently rely on JBI solutions, and there are a certain number of users for whom the JBI architecture is a cornerstone of a larger enterprise architecture. For these users, JBI will always be important: one of the reasons why is the promise of a standards-based modular architecture. This is good in large organizations because the adherence to a standard (i.e., the JBI 1.0 specification), you know that a component written by one of your teams in, say, Texas, will work when deployed in JBI a solution developed by a team in Hyderabad. Or, alternatively, a component written by a third-party for a CRM System will plug right into your JBI framework.
Of course, you can get this kind of plug’n’play with Camel - the Camel architecture and APIs are open source! However, some enterprises are going to be wary of working to an API that could change at the whim of a community, and prefer the solidity and comfort that only a standards-based specification can afford. I can appreciate this argument. However, I must respond that, if I were Chief Architect, I’d rather join in with a thriving open source community than a specification that’s been largely untouched and unrevised for some time, such as JBI 1.0.
All that aside, it’s clear that what’s right for some organizations is typically not right for others. ServiceMix (and FUSE) is all about supporting the integration community and, most importantly, offering choice. The advice I give my customers is to use Camel as the integration technology of choice, with an underpinning of ActiveMQ for reliable, asynchronous messaging and a touch of CXF for providing SOA or RESTful facades - all deployed in FUSE ESB 4 using OSGi. For me, the JBI container that used to be at the centre of ServiceMix / FUSE ESB is now a simply a feature that can be used to integrate with third-party or existing components that conform to the JBI specification. Camel, through the ‘jbi:’ and ‘nmr:’ components, can interoperate with such components as necessary. You get the best of all worlds.
To the JBI warriors out there, I salute you. You were not wrong. It’s just that there’s a new ‘right’ in town.
Monday, August 10, 2009
Friday, April 10, 2009
OSGi and dOSGi - from the source
Very excited about some upcoming webinars on fusesource.com: two legends of OSGi will be talking about why, what and how of OSGi technology. On April 14th (my birthday :)), Eric Newcomer, co-chair of the OSGi Enterprise Expert Group, is going to give an overview of OSGi for the enterprise. Then, on April 28th, the legendary David Bosschaert, one of our Fellow engineers at Progress, and based here in Dublin, will talk about Disributed OSGi. dOSGi answers the natural question asked by OSGi newcomers when the learn about OSGi services: is there any way we share the service registry across a number of JVMs, so that a consumer in one JVM can lookup and invoke on a service in another JVM transparently? David, a committer on Apache CXF, has implemented early prototypes on how to make this happen under the covers, and, I believe, has set CXF up to be part of the reference implementation for dOSGi.
I'm gonna grab me a sandwich log in to these webinars: they're too good to miss!
I note as well that registration at the webinars entitles you to a 10% discount on our online, live training sessions on Apache Camel & Apache ServiceMix 4. It's all good. We ran our first delivery of this course this week, and I was genuinely surprised with how our students coped with - and enjoyed! - the online learning experience. Two days of webinars, live coding, exercises, demonstration, chat via IRC, and access to our tech support guys. Of course, I was very lucky to have Hadrian Zbarcea, PMC chair of the Apache Camel project, with me for the duration, piping in with some great input where Camel is going. Nice.
But anyway: for now, I'm off for Easter...
I'm gonna grab me a sandwich log in to these webinars: they're too good to miss!
I note as well that registration at the webinars entitles you to a 10% discount on our online, live training sessions on Apache Camel & Apache ServiceMix 4. It's all good. We ran our first delivery of this course this week, and I was genuinely surprised with how our students coped with - and enjoyed! - the online learning experience. Two days of webinars, live coding, exercises, demonstration, chat via IRC, and access to our tech support guys. Of course, I was very lucky to have Hadrian Zbarcea, PMC chair of the Apache Camel project, with me for the duration, piping in with some great input where Camel is going. Nice.
But anyway: for now, I'm off for Easter...
Thursday, March 26, 2009
Muscle meets Might
Just saw the article on Red Hat Adds Muscle to Apache CXF. Hurrah! This is a great thumbs up and vote of confidence for the great work that has been done by the Apache CXF community. Bringing in this muscle, and combining it with the might of the many committers of the CXF project, lead by fusesource.com's Dan Kulp, is going to establish CXF further as the de facto implementation for open-source web services.
The new JMS flow in ServiceMix 4
I wandered down to the hotel bar last night and bumped into Guillaume Nodet; we got chatting about the new support for JMS flows in ServiceMix 4. I've written about JMS / JCA flows in a previous blog entry, arguing that, if not correctly configured, they can lead to far too much distribution and a consequent impact on performance. Back then, I argued that using explicit JMS queues - through ActiveMQ or whatever JMS broker you want - can provide just as effective a mechanism for distribution, message persistence, reliability and location transparency.
The latest stuff in SMX4 introduces some new stuff that makes me feel a lot happier about the concept of implementing the NMR with a JMS flow. First, if you want a JBI endpoint to be clustered, you have to explicitly mark it as clustered. This I like, as it gives the developer/designer back control of where the distribution points are in a flow. Second, the JBI flow now uses a single distributed queue for clustering. Previously in ServiceMix 3 a seperate queue was created for each JBI endpoint using the JMS flow; the new approach means less queues and less threads required to support the JMS flow. Different JBI endpoints will listen off the "NMR" queue using optimized message selectors to ensure that messages get consumed by the correct target services. Third, you can now optimize the JBI messaging layer to omit the use of JBI DONE messages: so, if you're implementing a one-way interaction over the JMS flow, you don't end up having to deal with an DONE that you simply don't need. Finally, Guillaume has made a cleaner separation between the JMS flow and the underlying JMS transport. So, you can use ActiveMQ if you want, or, alternatively, plug in another JMS broker to handle your JBI traffic.
I'm looking forward to playing with this, and perhaps documenting these new features. One clear point is that if you've made use of JMS flows in your architecture for ServiceMix 3, then you'll have that same support in ServiceMix 4, with just a little tweaking of configuration.
Good work, Guillaume! And, thanks again for the sandwich!
The latest stuff in SMX4 introduces some new stuff that makes me feel a lot happier about the concept of implementing the NMR with a JMS flow. First, if you want a JBI endpoint to be clustered, you have to explicitly mark it as clustered. This I like, as it gives the developer/designer back control of where the distribution points are in a flow. Second, the JBI flow now uses a single distributed queue for clustering. Previously in ServiceMix 3 a seperate queue was created for each JBI endpoint using the JMS flow; the new approach means less queues and less threads required to support the JMS flow. Different JBI endpoints will listen off the "NMR" queue using optimized message selectors to ensure that messages get consumed by the correct target services. Third, you can now optimize the JBI messaging layer to omit the use of JBI DONE messages: so, if you're implementing a one-way interaction over the JMS flow, you don't end up having to deal with an DONE that you simply don't need. Finally, Guillaume has made a cleaner separation between the JMS flow and the underlying JMS transport. So, you can use ActiveMQ if you want, or, alternatively, plug in another JMS broker to handle your JBI traffic.
I'm looking forward to playing with this, and perhaps documenting these new features. One clear point is that if you've made use of JMS flows in your architecture for ServiceMix 3, then you'll have that same support in ServiceMix 4, with just a little tweaking of configuration.
Good work, Guillaume! And, thanks again for the sandwich!
Wednesday, March 25, 2009
HamsterJam, ApacheCon, and building businesses out of Apache Software License
Just thought I'd check in from ApacheCon EU 2009 in Amsterdam. This definitely one of the most engaging conferences I've had the good fortune to attend. Yesterday I gave a one day workshop on building web- and RESTful services using Apache CXF, showing also how to plug in the JMS transport into CXF so that you can do XML/JMS or SOAP/JMS from JAX-WS code. I was blessed with a really good audience, which made for a fun day. Had an interesting scrape though: throughout the day my MacBookPro kept on 'freezing' out of the blue! Not the kind of thing you want when you're doing live-coding: took me eight hours to figure that this was happening because I'd left my bluetooth mouse on in my bag, and it was periodically taking control of my machine whenever it could make a connection. Ouch!
Today I broke with my usual tradition of doing technical talks to join the business track, where I got to follow up after a great panel discussion with my own talk, Adopting Open Source in the Enterprise (follow the link to pick up PDF of the slides). I also really enjoyed Paul Fremantle's talk on building business out of Apache Software License, which agreed with my own observations on the need for ASL-based business to give real business value in terms of their support, services, training and consulting. In my own talk I had talked about Ricardo's Law of Competitive Advantage, and how organizations will support themselves in their adoption of open-source unless a vendor - like Progress with FUSE - can demonstrate that they can provide that support cheaper, better, faster than the organization can do themselves. For me, a vendor offering services around Apache-licensed community software acts as a 'mountain guide', helping users to get most value out of the commons code while minimizing risk.
Right now I'm in a talk by Andreas Gies (a colleague from Progress) who has such great experience in adoption open source for distributed team development. He's a Maven committer, and, in my eyes, an absolute legend. Go Andreas, Go!
Today I broke with my usual tradition of doing technical talks to join the business track, where I got to follow up after a great panel discussion with my own talk, Adopting Open Source in the Enterprise (follow the link to pick up PDF of the slides). I also really enjoyed Paul Fremantle's talk on building business out of Apache Software License, which agreed with my own observations on the need for ASL-based business to give real business value in terms of their support, services, training and consulting. In my own talk I had talked about Ricardo's Law of Competitive Advantage, and how organizations will support themselves in their adoption of open-source unless a vendor - like Progress with FUSE - can demonstrate that they can provide that support cheaper, better, faster than the organization can do themselves. For me, a vendor offering services around Apache-licensed community software acts as a 'mountain guide', helping users to get most value out of the commons code while minimizing risk.
Right now I'm in a talk by Andreas Gies (a colleague from Progress) who has such great experience in adoption open source for distributed team development. He's a Maven committer, and, in my eyes, an absolute legend. Go Andreas, Go!
Tuesday, March 10, 2009
On the question of 'ServiceMix vs Camel'
There was an interesting discussion on the ServiceMix users list on how to choose between implementing integration solutions using ServiceMix or camel; thought I'd post my contribution to that discussion here. ServiceMix should really be thought of as a container, that facilitates the deployment of JBI-based integration solutions and Camel-based solutions for enterprise integration. And, of course, ServiceMix 4 is an OSGi container, which extends the concept of what can be deployed - JBI service assemblies, spring configuration, OSGi bundles, etc.
The question really becomes "Should I use the ServiceMix JBI components (-eip, -file, -ftp, etc.) or should I just use camel components?" I think the decision around this will depend on whether the JBI architecture forms an important underpinning of your solution: if so, then it makes sense to use the JBI components. If, however, you have no desire to go down the pure JBI route, then you can choose to adopt either a pure Camel or a more blended approach.
As Chris Custine and Jean-Baptiste have both pointed out, Camel integrates very effectively with JBI, so you can get the best of both worlds. You might find that the JBI servicemix-ftp component is more robust than the camel-ftp component, and choose to use the former as part of a Camel route. Or, you might prefer the servicemix-eip Aggregator's persistence of state over that of the aggregator in Camel.
So: my advice is, in a nutshell, move forward embracing a blend of both JBI and camel components, and, consider 'ServiceMix' to be a container for both.
The question really becomes "Should I use the ServiceMix JBI components (-eip, -file, -ftp, etc.) or should I just use camel components?" I think the decision around this will depend on whether the JBI architecture forms an important underpinning of your solution: if so, then it makes sense to use the JBI components. If, however, you have no desire to go down the pure JBI route, then you can choose to adopt either a pure Camel or a more blended approach.
As Chris Custine and Jean-Baptiste have both pointed out, Camel integrates very effectively with JBI, so you can get the best of both worlds. You might find that the JBI servicemix-ftp component is more robust than the camel-ftp component, and choose to use the former as part of a Camel route. Or, you might prefer the servicemix-eip Aggregator's persistence of state over that of the aggregator in Camel.
So: my advice is, in a nutshell, move forward embracing a blend of both JBI and camel components, and, consider 'ServiceMix' to be a container for both.
Friday, February 27, 2009
Me and Guillaume, talking our heads off about stuff...
... on a podcast on ZDNet, done with Dana Gardner... enjoy!
Wednesday, February 25, 2009
FUSE adoption: is it the economy, or just the sheer love of innovation?
There's a nice post on the adoption of FUSE ESB on ZDNet; in the last few months I've observed an increase in activity and excitement around FUSE. Why might that be? As the entry's quote from Debbie suggests, economic conditions certainly are prompting users to ask questions of their existing close-source providers, and evaluate in more earnest the open-source options and alternatives that are available.
From my perspective though, it's more than that: at a technical level I think users are seeing that the Apache projects that FUSE is based on - ServiceMix, ActiveMQ, Camel and CXF - offer a new way to rapidly create solutions for integration, SOA, and, REST. The potential for developers to rapidly create integration flows using Camel leaves most battle-hardened integration architects and developers bowled over with excitement.
And users are excited by OSGi too; I remember the day when Eric Newcomer, ex CTO of IONA, who was pivotal in bringing OSGi forward into the Enterprise computing space, presented to a group of us in Boston on how this OSGi could change the way we think about deploying Java solutions. David Bosschaert, a colleague of mine at Progress, has continued the charge of OSGi, and has put some very interesting stuff on OSGi and on distributed OSGi up on his blog, <coderthoughts/>. An of course, the illustrious and prolific Guillaume Nodet, also at Progress on the fusesource team, has done great work on driving ServiceMix 4 to an integration-focused OSGi based runtime.
So whatever the reason - economic pressure or technical advantage - I welcome the surge in FUSE activity :)
From my perspective though, it's more than that: at a technical level I think users are seeing that the Apache projects that FUSE is based on - ServiceMix, ActiveMQ, Camel and CXF - offer a new way to rapidly create solutions for integration, SOA, and, REST. The potential for developers to rapidly create integration flows using Camel leaves most battle-hardened integration architects and developers bowled over with excitement.
And users are excited by OSGi too; I remember the day when Eric Newcomer, ex CTO of IONA, who was pivotal in bringing OSGi forward into the Enterprise computing space, presented to a group of us in Boston on how this OSGi could change the way we think about deploying Java solutions. David Bosschaert, a colleague of mine at Progress, has continued the charge of OSGi, and has put some very interesting stuff on OSGi and on distributed OSGi up on his blog, <coderthoughts/>. An of course, the illustrious and prolific Guillaume Nodet, also at Progress on the fusesource team, has done great work on driving ServiceMix 4 to an integration-focused OSGi based runtime.
So whatever the reason - economic pressure or technical advantage - I welcome the surge in FUSE activity :)
Wednesday, February 18, 2009
Free! Free! Free! Screencasts on how to use FUSE ESB 4 (based on Apache Servicemix 4)
As part of my bi-weekly "Getting Started with FUSE ESB 4" webinar series, I've put together some screencasts of how to do all the good stuff you want to do with FUSE ESB 4. Check out the links at the bottom of the fusesource.com Training Videos page.
Right now the links will download quicktime movies, which you can then run on your machine. The movies range in size from 90Mb to 180Mb, so kick off those downloads, grab a nice cup of coffee, and the sit back and enjoy the ride as I cover how to build and install FUSE ESB 4, create and deploy an OSGi bundle from scratch, create and deploy a web service using CXF, deploy an EIP using a Camel route, and finally, deploy a JBI-based solution into the FUSE ESB 4 runtime.
If you like the videos, or have feedback, please leave a comment!
Right now the links will download quicktime movies, which you can then run on your machine. The movies range in size from 90Mb to 180Mb, so kick off those downloads, grab a nice cup of coffee, and the sit back and enjoy the ride as I cover how to build and install FUSE ESB 4, create and deploy an OSGi bundle from scratch, create and deploy a web service using CXF, deploy an EIP using a Camel route, and finally, deploy a JBI-based solution into the FUSE ESB 4 runtime.
If you like the videos, or have feedback, please leave a comment!
Wednesday, February 11, 2009
The White Source
Ah lovely: a colleague just pointed out an open letter urging Obama To Consider Open Source. I chuckle with content at the thought of Obama hacking away from the oval office - "Not now Mr. Vice President, I'm in the middle of a maven build. Nearly done. Just one... more... compile..." ;)
While it's great to see so many players in the open source community willing to step up to the plate, the use of open source in government is not news; it's a trend that started some time ago; the FUSE suite, for example, has a number of prominent government users already.
Still though, it's great to see the open source movement move forward in a unified way. Who knows: maybe that letter will make it's way onto Obama's desk in the white house...
While it's great to see so many players in the open source community willing to step up to the plate, the use of open source in government is not news; it's a trend that started some time ago; the FUSE suite, for example, has a number of prominent government users already.
Still though, it's great to see the open source movement move forward in a unified way. Who knows: maybe that letter will make it's way onto Obama's desk in the white house...
Tuesday, February 10, 2009
The day we brought down the internet :(
Today we had the first of our FUSE ESB 4 Webinar series; we had a fabulous triple digit attendance at the session. We dived right into some live coding, showing how to deploy bundles and other artifacts into FUSE ESB, and were all set to tear away with web services, REST, and a whole lot more... until, unfortunately, after an hour of noble service, some machine in some datacentre decided to give up the ghost, leading to the termination of the webinar.
While it's fun to think that our community's enthusiasm managed to melt some motherboard somewhere, I'm deeply disappointed by this: not just because were about to get the really cool stuff FUSE can do, but also because we had such enormous participation from the attendees who were all very keen to learn more. We'll be working hard to make sure the outage doesn't happen on the next session on February 23rd, when we'll present at 10am GMT and again at 1pm EST. Additionally, I'l be recording some screen casts on each of the six use-cases and we'll post them on fusesource.com.
Till then, thanks to all our attendees for taking part - we hope you enjoyed our session and look forward to hearing from you again.
While it's fun to think that our community's enthusiasm managed to melt some motherboard somewhere, I'm deeply disappointed by this: not just because were about to get the really cool stuff FUSE can do, but also because we had such enormous participation from the attendees who were all very keen to learn more. We'll be working hard to make sure the outage doesn't happen on the next session on February 23rd, when we'll present at 10am GMT and again at 1pm EST. Additionally, I'l be recording some screen casts on each of the six use-cases and we'll post them on fusesource.com.
Till then, thanks to all our attendees for taking part - we hope you enjoyed our session and look forward to hearing from you again.
JAX-RS in FUSE ESB 4.0.0.3
There's a line in RadioHead song "Phew, for a minute there, I lost myself..." I nearly did loose myself yesterday as I tried to reproduce some of Roberto Rojas's work on deploying a JAX-RS service in FUSE ESB 4.0.0.3 (based on Apache Servicemix 4). Roberto's blog entry refers to a preview edition of FUSE ESB 4, and things have moved on since then.
Luckily, I was able to ping off some of the good folk on the CXF project - Dan Kulp, Sergey Beryozkin, and David Bosschaert, and they got me going again. There's been some reworking of the JAX-RS support in SMX4, and this you need to update the cxf-bundle in FUSE 4.0.0.3, as per below:
servicemix> osgi list | utils grep CXF
[ 89] [Active ] [ 50] ServiceMix :: CXF Binding Component (2008.01.0.fuse)
[ 105] [Active ] [ 50] Apache CXF Runtime JBI Binding (2.1.3.fuse)
[ 106] [Active ] [ 50] Apache CXF Runtime JBI Transport (2.1.3.fuse)
[ 107] [Active ] [ 50] ServiceMix :: CXF Service Engine (2008.01.0.fuse)
[ 142] [Active ] [ 50] Apache CXF Bundle Jar (2.1.3.fuse)
servicemix> osgi uninstall 142
servicemix> osgi install -s mvn:org.apache.cxf/cxf-bundle/2.1.3.3-fuse
FUSE ESB 4.0.0.4, released just last night (Feb 9th 2009), already uses version 2.1.3.3-fuse of cxf-bundle, so if you're using 4.0.0.4 then you shouldn't need this step.
With this in place, you can now go and deploy your restful service. For example, I threw together a bundle with a JAX-RS 0.8 service: to install it, I need to install a few dependencies into FUSE ESB 4:
First, we need the JAX-RS (JSR 311) api. I'm using version 0.8:
servicemix> osgi install -s wrap:mvn:javax.ws.rs/jsr311-api/0.8
Next, you need to install the specs locator and the servicemix jsr311-api bundle so that FUSE can locate and load up the spec and implementation correctly.
servicemix> osgi install -s mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.locator
servicemix> osgi install -s mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.jsr311-api-0.8
While my example isn't using JSON, there is currently a runtime dependency on Jettison, so you need to install that too. You can use the bundle from SpringSource like this:
servicemix> osgi install -s mvn:org.codehaus.jettison/com.springsource.org.codehaus.jettison/1.0.1
Or, alternatively, you can use the cool wrap: functionality to wrap a plain-old java jar
servicemix> osgi install -s mvn:org.codehaus.jettison/jettison/1.0.1
Hurray! Now with all the prerequisites in place, I can install my JAX-RS service.
servicemix> osgi install -s mvn:com.progress.pso/simple-restful-service
And now my RESTful service works a treat :)
Luckily, I was able to ping off some of the good folk on the CXF project - Dan Kulp, Sergey Beryozkin, and David Bosschaert, and they got me going again. There's been some reworking of the JAX-RS support in SMX4, and this you need to update the cxf-bundle in FUSE 4.0.0.3, as per below:
servicemix> osgi list | utils grep CXF
[ 89] [Active ] [ 50] ServiceMix :: CXF Binding Component (2008.01.0.fuse)
[ 105] [Active ] [ 50] Apache CXF Runtime JBI Binding (2.1.3.fuse)
[ 106] [Active ] [ 50] Apache CXF Runtime JBI Transport (2.1.3.fuse)
[ 107] [Active ] [ 50] ServiceMix :: CXF Service Engine (2008.01.0.fuse)
[ 142] [Active ] [ 50] Apache CXF Bundle Jar (2.1.3.fuse)
servicemix> osgi uninstall 142
servicemix> osgi install -s mvn:org.apache.cxf/cxf-bundle/2.1.3.3-fuse
FUSE ESB 4.0.0.4, released just last night (Feb 9th 2009), already uses version 2.1.3.3-fuse of cxf-bundle, so if you're using 4.0.0.4 then you shouldn't need this step.
With this in place, you can now go and deploy your restful service. For example, I threw together a bundle with a JAX-RS 0.8 service: to install it, I need to install a few dependencies into FUSE ESB 4:
First, we need the JAX-RS (JSR 311) api. I'm using version 0.8:
servicemix> osgi install -s wrap:mvn:javax.ws.rs/jsr311-api/0.8
Next, you need to install the specs locator and the servicemix jsr311-api bundle so that FUSE can locate and load up the spec and implementation correctly.
servicemix> osgi install -s mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.locator
servicemix> osgi install -s mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.jsr311-api-0.8
While my example isn't using JSON, there is currently a runtime dependency on Jettison, so you need to install that too. You can use the bundle from SpringSource like this:
servicemix> osgi install -s mvn:org.codehaus.jettison/com.springsource.org.codehaus.jettison/1.0.1
Or, alternatively, you can use the cool wrap: functionality to wrap a plain-old java jar
servicemix> osgi install -s mvn:org.codehaus.jettison/jettison/1.0.1
Hurray! Now with all the prerequisites in place, I can install my JAX-RS service.
servicemix> osgi install -s mvn:com.progress.pso/simple-restful-service
And now my RESTful service works a treat :)
Monday, February 2, 2009
Free live online tutorial on FUSE ESB 4!
Come see me and my fusesource.com colleague Jack Britton give a live two-hour online tutorial on "getting started with FUSE ESB 4" on February 10th! We'll be covering everything from building the source, to implementing EIPs, to writing and deploying web-services, to deploying JBI artifacts. There'll be lots of live code, probably a few hiccups, and a whole-lot-of-fun.
Register here!
Register here!
Friday, January 9, 2009
Message Selectors and ActiveMQ
Having just completed a consultancy on FUSE Message Broker (Enterprise Apache ActiveMQ), I just thought I'd write some notes about the use of message selectors in JMS, and, in particular, how FUSE Message Broker implements them.
Message selectors are easy to use, allowing a consumer to specify a filter, using an SQL-like syntax, describing the messages of interest. This filter, example "customerStatus = 'Gold'" instructs the broker to only send messages to the consumer that have a customerStatus header equal to "Gold".
While message selectors are attractive and conceptually very nice, they do place a burden on the broker: consider the case where you have a large number of consumers, say n, on a queue: for each message the broker must evaluate between 1 and n message selectors before delivering the message. In the case of topic, all n message selectors will have to be evaluated for each message. Ouch: that's a lot of work to be putting on the broker, particularly if you're looking for high-throughput.
But apart from this, ActiveMQ's memory paging architecture has an optimisation that can flummox you when using message selectors. When messages are sent persistently, ActiveMQ will only page a portion of the messages in memory, but leave the rest on disk. This is very reasonable: however, one side effect of this is that selectors are only evaluated on those messages in memory, not those in the store. So, if your page size is 100, and you have 100 messages for which there is no current matching consumer, you end up with a "hung" queue. For example, consider the case where you have two consumers, selecting on "color = 'red'" and "color = 'blue'" respectively - here, "color" is a custom JMS message header. Let's say that the first (red) consumer is currently not in operation, and we receive 100 messages for red. Blue is waiting, waiting, waiting, but does not get any messages as there are no matching ones present. Now, we get message 101 - a nice blue message. However, our selectors are only matching against the first 100 messages (all red!) so blue still hangs, waiting, despite the fact that there's a perfectly matching message.
That example is perhaps contrived, but it does demonstrate a potential "hanging consumer" problem. For me though it's a natural side-effect of wanting to limit the broker-side overhead of having to evaluate message selectors. You can of course modify the page size to reduce the chance of the problem happening, but you're just reducing the probability of the lock happening.
My customer had come up with a conceptually elegant callback design, whereby as messages are forwarded through a server, a consumer is created (for each message) to listen on a queue for a subsequent response, using a unique message identifier within a message selector. While conceptually nice as a design, the overhead (in terms of dynamic consumer creation and message selector processing) just isn't worth it. We proposed an alternative architecture using named queues and a fixed pool of response consumers that's going to work way, way better.
Message selectors are easy to use, allowing a consumer to specify a filter, using an SQL-like syntax, describing the messages of interest. This filter, example "customerStatus = 'Gold'" instructs the broker to only send messages to the consumer that have a customerStatus header equal to "Gold".
While message selectors are attractive and conceptually very nice, they do place a burden on the broker: consider the case where you have a large number of consumers, say n, on a queue: for each message the broker must evaluate between 1 and n message selectors before delivering the message. In the case of topic, all n message selectors will have to be evaluated for each message. Ouch: that's a lot of work to be putting on the broker, particularly if you're looking for high-throughput.
But apart from this, ActiveMQ's memory paging architecture has an optimisation that can flummox you when using message selectors. When messages are sent persistently, ActiveMQ will only page a portion of the messages in memory, but leave the rest on disk. This is very reasonable: however, one side effect of this is that selectors are only evaluated on those messages in memory, not those in the store. So, if your page size is 100, and you have 100 messages for which there is no current matching consumer, you end up with a "hung" queue. For example, consider the case where you have two consumers, selecting on "color = 'red'" and "color = 'blue'" respectively - here, "color" is a custom JMS message header. Let's say that the first (red) consumer is currently not in operation, and we receive 100 messages for red. Blue is waiting, waiting, waiting, but does not get any messages as there are no matching ones present. Now, we get message 101 - a nice blue message. However, our selectors are only matching against the first 100 messages (all red!) so blue still hangs, waiting, despite the fact that there's a perfectly matching message.
That example is perhaps contrived, but it does demonstrate a potential "hanging consumer" problem. For me though it's a natural side-effect of wanting to limit the broker-side overhead of having to evaluate message selectors. You can of course modify the page size to reduce the chance of the problem happening, but you're just reducing the probability of the lock happening.
My customer had come up with a conceptually elegant callback design, whereby as messages are forwarded through a server, a consumer is created (for each message) to listen on a queue for a subsequent response, using a unique message identifier within a message selector. While conceptually nice as a design, the overhead (in terms of dynamic consumer creation and message selector processing) just isn't worth it. We proposed an alternative architecture using named queues and a fixed pool of response consumers that's going to work way, way better.
Thursday, January 8, 2009
Looking forward to ApacheCon Europe 2009
This year I'm thrilled to be doing a number of talks at ApacheCon Europe 2009, and a full day-long tutorial on building web and REST services with Apache CXF (FUSE Service Framework). Abstracts below - see you there!
Talk: Tales from the Front Line: how ServiceMix, ActiveMQ & CXF are being used to solve real problems. In this session we discuss how application integration and middleware-focussed Apache projects - like ServiceMix, ActiveMQ, Camel & CXF - are being used to build innovative solutions that add real value to organizations in health-care, government, airline-solutions and finance. For each use-case, we discuss both logical and deployment architecture concerns. We also discuss the motives for adopting open-source in these solutions, and why these Apache projects where chosen above other open source alternatives.
Talk: Adopting Open Source in the Enterprise.
For some organizations, the decision to adopt open source software
is not taken lightly, and, once made, can require significant shifts
in thinking and organization culture. This session discusses the
issues organizations face when strategically adopting open source,
including how to select open source projects, how to match open source
involvement & contribution with corporate strategy, how to select &
manage professional open source software vendors, what needs to be
done in terms of licensing and legal, and what kind of infrastructure
you need to support your open source strategy.
Tutorial (full day): Apache CXF - Developing and Deploying Open Source SOA Endpoints
This one-day tutorial will show how to design, build and deploy RESTful and Web-based web services using Apache CXF. The session will cover code-first and WSDL first services design for JAX-WS, and show how to use JAX-RS for restful services. We'll show how to swap different transports (HTTP, JMS) with different payload types (XML, SOAP, JSON). And, we'll show how CXF's support for JavaScript can be used to create client- and serverside components for AJAX applications. This workshop will combine a mix of lectures, whiteboarding, and hands-on exercise sessions.
Talk: Tales from the Front Line: how ServiceMix, ActiveMQ & CXF are being used to solve real problems. In this session we discuss how application integration and middleware-focussed Apache projects - like ServiceMix, ActiveMQ, Camel & CXF - are being used to build innovative solutions that add real value to organizations in health-care, government, airline-solutions and finance. For each use-case, we discuss both logical and deployment architecture concerns. We also discuss the motives for adopting open-source in these solutions, and why these Apache projects where chosen above other open source alternatives.
Talk: Adopting Open Source in the Enterprise.
For some organizations, the decision to adopt open source software
is not taken lightly, and, once made, can require significant shifts
in thinking and organization culture. This session discusses the
issues organizations face when strategically adopting open source,
including how to select open source projects, how to match open source
involvement & contribution with corporate strategy, how to select &
manage professional open source software vendors, what needs to be
done in terms of licensing and legal, and what kind of infrastructure
you need to support your open source strategy.
Tutorial (full day): Apache CXF - Developing and Deploying Open Source SOA Endpoints
This one-day tutorial will show how to design, build and deploy RESTful and Web-based web services using Apache CXF. The session will cover code-first and WSDL first services design for JAX-WS, and show how to use JAX-RS for restful services. We'll show how to swap different transports (HTTP, JMS) with different payload types (XML, SOAP, JSON). And, we'll show how CXF's support for JavaScript can be used to create client- and serverside components for AJAX applications. This workshop will combine a mix of lectures, whiteboarding, and hands-on exercise sessions.
Subscribe to:
Posts (Atom)
