I'll be at the FUSE Community Day, Paris event on October 14th! Feel free to sign up and come along - If the London event in June is anything to by I know it's going to be a great day out :) We've got some great speakers lined up (including all of our FUSE rock stars like Claus Ibsen, Rob Davies, and James Strachan.
Rockin'!
Monday, September 13, 2010
Wednesday, September 8, 2010
Survival of the fittest: the evolution of the SOA registry / repository concept
I've decided to write a little about SOA registries and repositories, as a few customers have brought up these issues recently and I wanted to clear the air (and, in some sense, clear my own head too). I dragged in some sanity checks from my fellow consultants at FUSE just to get their feeling on the adoption of SOA registry tooling, so hopefully what follows will make some sense.
There was a flurry of activity some years ago (say, around 2007/2008) about SOA (Service-Oriented Architecture) Registries and Repositories. SOA evangelists, architects and "thinkers' waxed lyrical about how real SOA experts (with capital letters) used registries and repositories, and that the 'little-SOA' folk just didn't get the concept, god bless them. Vendors rushed to provide (sometimes expensive) SOA registry / repository tools, a move probably motivated more from the realization that underlying SOA infrastructure and middleware was becoming a commodity, rather than the fact that customers really did need a registry.
Funny that.
Because in many of the SOA customers I've worked with, they simply haven't needed a repository/registry. In fact, the mandate from on top to get a SOA registry or repository has been a distraction! Now, don't get me wrong: I'm not saying the SOA registries are a bad idea. Not at all. I'm just saying that, as a tool, they solve a particular problem. And, if you don't have that problem, then you don't need the tool.
Let's consider what a SOA registry / repository might do for you.
The 'registry' problem is sometimes solved in software: for example, the Artix Locator, integrated with Apache CXF, allows services to register themselves automatically with a well-known-registry, which clients can then look up at runtime. Nice! I've seen other CXF users write their own registries to achieve something similar - it's really not rocket science (although, you can make your registry as complicated as you like!). I would like to see an open-source implementation of such a runtime-registry, perhaps making use of recent innovative projects like Apache Zookeeper to keep track of who's up and who's down. As an additional note on registries: one customer noted that the OSGi registry, particular in the context of distributed OSGi (dOSGi), adds another dimension to the idea of dynamic, transparent lookup of services. And, if you're acting smart in a JBI or ServiceMix 4 world, you can achieve location transparency through the NMR or ActiveMQ POQs (plain-old-queues).
Interestingly though, if you have your network folk in the room, they'll argue that they can solve this whole lookup and 'location transparency' problem in a heart-beat in hardware with a network switch: think about it: I can go to www.google.com every day - in fact, so can my mam - and never worry about what machine my search gets directed too. So, often people can achieve the 'registry' functionality easy, sometimes with zero code.
So, while there is certainly some argument for using registries and repositories for storage and retrieval of information about what services you have on the network, there are often many options available to you that don't need a dedicated software tool. And, keep in mind, if you have only a small number of services - say up to 20 - you really don't have a management problem - so ad hoc approaches like Wiki's may be perfectly adequate.
With this in mind, I think that 'SOA registry-repository' offerings begin to offer value when they tackle SOA Governance issues, and I think this is the evolutionary route these products are taking. IT owners want to know that security policies are being enforces - these policies should be configurable, not 'designed into' the fabric and code of a service where they're difficult to change. They want to know if the response times are being sluggish. They want a record of who accessed what, and when. This could be for a legal audit trail, but is probably more useful internally as a 'proof-point' to show that service re-use is happening, and, more importantly, a way for a core SOA Centre of Competence to demonstrate the value that its services are offering the rest of the organization. Now this, for me, is a place where these tools can offer real value. As a Progress employee I am biased, and will of course recommend Actional which does an awesome job in this regard - but, as a fair and open-minded person, I have seen users of CXF propose to use HP SOA Centre to achieve something architecturally similar. Actional acts as a 'gateway'. You register your services with it (hey, a touch of 'registry!'). You apply your security policies. And away you go: all SOA requests get passed through gateway and SLAs are enforced and reported. Governance made easy. The only problem with this architecture, from my perspective, is that it creates a potential 'gateway bottleneck; however, with appropriate load-balancing in place this hasn't yet been a problem in practice.
At the end of the day, the tooling is evolving to the point of value around the concepts of governance - I welcome this. However, I still believe that if you're embarking on creation or adoption of SOA concepts, you shouldn't get stalled in the early stages by selection of SOA registry / repository. First, understand your needs - your real needs - in this area, and then, go look for right tool for the job.
There was a flurry of activity some years ago (say, around 2007/2008) about SOA (Service-Oriented Architecture) Registries and Repositories. SOA evangelists, architects and "thinkers' waxed lyrical about how real SOA experts (with capital letters) used registries and repositories, and that the 'little-SOA' folk just didn't get the concept, god bless them. Vendors rushed to provide (sometimes expensive) SOA registry / repository tools, a move probably motivated more from the realization that underlying SOA infrastructure and middleware was becoming a commodity, rather than the fact that customers really did need a registry.
Funny that.
Because in many of the SOA customers I've worked with, they simply haven't needed a repository/registry. In fact, the mandate from on top to get a SOA registry or repository has been a distraction! Now, don't get me wrong: I'm not saying the SOA registries are a bad idea. Not at all. I'm just saying that, as a tool, they solve a particular problem. And, if you don't have that problem, then you don't need the tool.
Let's consider what a SOA registry / repository might do for you.
- Store contracts, service level agreements, code, deployable artifacts, documentation (this is the 'repository' nature of the tool)
- Act as a run-time lookup for physical location of services - thus abstracting a services physical network address from a 'logical' network address. (this is the 'registry' nature of the tool)
- Keep a track of who is using what services, and how much (this is the 'governance' nature).
- Enforce service level agreements (again, this is more governance: making sure that the services are available and doing what they should do in a timely fashion)
The 'registry' problem is sometimes solved in software: for example, the Artix Locator, integrated with Apache CXF, allows services to register themselves automatically with a well-known-registry, which clients can then look up at runtime. Nice! I've seen other CXF users write their own registries to achieve something similar - it's really not rocket science (although, you can make your registry as complicated as you like!). I would like to see an open-source implementation of such a runtime-registry, perhaps making use of recent innovative projects like Apache Zookeeper to keep track of who's up and who's down. As an additional note on registries: one customer noted that the OSGi registry, particular in the context of distributed OSGi (dOSGi), adds another dimension to the idea of dynamic, transparent lookup of services. And, if you're acting smart in a JBI or ServiceMix 4 world, you can achieve location transparency through the NMR or ActiveMQ POQs (plain-old-queues).
Interestingly though, if you have your network folk in the room, they'll argue that they can solve this whole lookup and 'location transparency' problem in a heart-beat in hardware with a network switch: think about it: I can go to www.google.com every day - in fact, so can my mam - and never worry about what machine my search gets directed too. So, often people can achieve the 'registry' functionality easy, sometimes with zero code.
So, while there is certainly some argument for using registries and repositories for storage and retrieval of information about what services you have on the network, there are often many options available to you that don't need a dedicated software tool. And, keep in mind, if you have only a small number of services - say up to 20 - you really don't have a management problem - so ad hoc approaches like Wiki's may be perfectly adequate.
With this in mind, I think that 'SOA registry-repository' offerings begin to offer value when they tackle SOA Governance issues, and I think this is the evolutionary route these products are taking. IT owners want to know that security policies are being enforces - these policies should be configurable, not 'designed into' the fabric and code of a service where they're difficult to change. They want to know if the response times are being sluggish. They want a record of who accessed what, and when. This could be for a legal audit trail, but is probably more useful internally as a 'proof-point' to show that service re-use is happening, and, more importantly, a way for a core SOA Centre of Competence to demonstrate the value that its services are offering the rest of the organization. Now this, for me, is a place where these tools can offer real value. As a Progress employee I am biased, and will of course recommend Actional which does an awesome job in this regard - but, as a fair and open-minded person, I have seen users of CXF propose to use HP SOA Centre to achieve something architecturally similar. Actional acts as a 'gateway'. You register your services with it (hey, a touch of 'registry!'). You apply your security policies. And away you go: all SOA requests get passed through gateway and SLAs are enforced and reported. Governance made easy. The only problem with this architecture, from my perspective, is that it creates a potential 'gateway bottleneck; however, with appropriate load-balancing in place this hasn't yet been a problem in practice.
At the end of the day, the tooling is evolving to the point of value around the concepts of governance - I welcome this. However, I still believe that if you're embarking on creation or adoption of SOA concepts, you shouldn't get stalled in the early stages by selection of SOA registry / repository. First, understand your needs - your real needs - in this area, and then, go look for right tool for the job.
Versioning WSDL interfaces in an OSGi world
Some time ago (2007, I think) I wrote a short paper with Oliver Wulff on WSDL versioning. Recently, I shared this paper with some smart OSGi-savvy ServiceMix users, and it raised a discussion about how does WSDL interface versioning mix with the versioning concepts in OSGi? Below is an extract of an email response I sent back to them: for me, the bottom line is that WSDL interface versioning (in fact, any interface versioning) is a separate concern to the way we version implementations.
For anyone out there who is implementing Web Services using OSGi runtimes like FUSE ESB (built on Apache ServiceMix and Apache Karaf), you might find the thoughts below of interest.
From my side, I think the most important point is that my paper concerns WSDL versioning, that is, the versioning of an *interface*; the versioning of an OSGi bundle is, from a service consumer's perspective, an *implementation* detail and a totally separate concern. I believe you should consider the versioning of the interface and the implementation as two *separate* evolutionary paths.
So, I could have, say, version 1 of an 'foo' web service, in namespace http://www.myorg.com/services/foo/v1, defined in foo-v1.0.wsdl. We might generate the Java JAX-WS/JAXB-B classes, and then package as a bundle foo-ws-api-1.0.jar. I could implement it today in bundle foo-v1.0-impl-1.0.0.jar, and then fix a bug later in foo-v1.0-impl-1.0.1.jar, add a cool new feature in foo-v1.0-impl-1.1.0.jar, and then do a completely new, high-performance implementation in foo-v1.0-impl-2.0.0.jar. All of these implementation bundles would implement *the same* version of the WSDL!!
Now, imagine we need to do a minor change to the web service interface: say add a new operation, or add a new parameter to a method. We can do this *without* breaking on the wire interoperability by creating the new PortType in foo-v1.1.wsdl, but still in the namespace http://www.myorg.com/services/foo/v1. We would deploy the JAX-WS content for this in foo-ws-api-1.1.jar. We could do a new implementation, and put the implementation of this in foo-v1.0-impl-1.0.0.jar. Now, we can deploy the new version of both the interface and the implementation, and, if we've been careful, the new deployment will handle requests existing consumers of version 1.0 of the interface. KEEP IN MIND THAT WHILE THIS IS POSSIBLE TO DO ... (sorry for shouting!) it's actually quite tricky to understand and I think that the complexity overweighs the benefits for most users. So, it's probably better to try to keep WSDL interface's versioning at major release numbers.
Now, with this in mind, we can consider what happens when we deploy a whole new version of the WSDL interface! We can deliver foo-v2.0.wsdL, with portTypes and all in namespace http://www.myorg.com/services/foo/v2. We can deploy this API as bundle foo-ws-api-2.0.jar, and deliver our first implementation bundle as foo-v2.0-impl-1.0.0.jar. And, later, when we fix some bugs or add some features, we can deploy the new implementation as foo-v2.0-impl-1.0.1.jar.
Get the picture? For me, the rate of change of an interface should be different to that of an implementation - ideally, it should be significantly slower, changing maybe on a yearly base. We need to apply different versioning semantics, and evolve them at different rates to the implementation bundles.
Hope that helps!
Best,
Ade.
For anyone out there who is implementing Web Services using OSGi runtimes like FUSE ESB (built on Apache ServiceMix and Apache Karaf), you might find the thoughts below of interest.
From my side, I think the most important point is that my paper concerns WSDL versioning, that is, the versioning of an *interface*; the versioning of an OSGi bundle is, from a service consumer's perspective, an *implementation* detail and a totally separate concern. I believe you should consider the versioning of the interface and the implementation as two *separate* evolutionary paths.
So, I could have, say, version 1 of an 'foo' web service, in namespace http://www.myorg.com/services/foo/v1, defined in foo-v1.0.wsdl. We might generate the Java JAX-WS/JAXB-B classes, and then package as a bundle foo-ws-api-1.0.jar. I could implement it today in bundle foo-v1.0-impl-1.0.0.jar, and then fix a bug later in foo-v1.0-impl-1.0.1.jar, add a cool new feature in foo-v1.0-impl-1.1.0.jar, and then do a completely new, high-performance implementation in foo-v1.0-impl-2.0.0.jar. All of these implementation bundles would implement *the same* version of the WSDL!!
Now, imagine we need to do a minor change to the web service interface: say add a new operation, or add a new parameter to a method. We can do this *without* breaking on the wire interoperability by creating the new PortType in foo-v1.1.wsdl, but still in the namespace http://www.myorg.com/services/foo/v1. We would deploy the JAX-WS content for this in foo-ws-api-1.1.jar. We could do a new implementation, and put the implementation of this in foo-v1.0-impl-1.0.0.jar. Now, we can deploy the new version of both the interface and the implementation, and, if we've been careful, the new deployment will handle requests existing consumers of version 1.0 of the interface. KEEP IN MIND THAT WHILE THIS IS POSSIBLE TO DO ... (sorry for shouting!) it's actually quite tricky to understand and I think that the complexity overweighs the benefits for most users. So, it's probably better to try to keep WSDL interface's versioning at major release numbers.
Now, with this in mind, we can consider what happens when we deploy a whole new version of the WSDL interface! We can deliver foo-v2.0.wsdL, with portTypes and all in namespace http://www.myorg.com/services/foo/v2. We can deploy this API as bundle foo-ws-api-2.0.jar, and deliver our first implementation bundle as foo-v2.0-impl-1.0.0.jar. And, later, when we fix some bugs or add some features, we can deploy the new implementation as foo-v2.0-impl-1.0.1.jar.
Get the picture? For me, the rate of change of an interface should be different to that of an implementation - ideally, it should be significantly slower, changing maybe on a yearly base. We need to apply different versioning semantics, and evolve them at different rates to the implementation bundles.
Hope that helps!
Best,
Ade.
Subscribe to:
Posts (Atom)
