|By Dana Gardner||
|October 22, 2012 07:30 AM EDT||
To find out why, BriefingsDirect recently gathered an international panel of experts to explore the concept of "architecture is destiny," especially when it comes to hybrid services delivery and management. The panel shows how SOA is proving instrumental in allowing the needed advancements over highly distributed services and data, when it comes to scale, heterogeneity support, and governance.
The panel consists of Chris Harding, Director of Interoperability at The Open Group, based in the UK; Nikhil Kumar, President of Applied Technology Solutions and Co-Chair of the SOA Reference Architecture Projects within The Open Group, and he's based in Michigan, and Mats Gejnevall, Enterprise Architect at Capgemini and Co-Chair of The Open Group SOA Work Group, and he's based in Sweden. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]
Here are some excerpts:
Gardner: Why this resurgence in the interest around SOA?
Harding: My role in The Open Group is to support the work of our members on SOA, cloud computing, and other topics. We formed the SOA Work Group back in 2005, when SOA was a real emerging hot topic, and we set up a number of activities and projects. They're all completed.
In fact, we've started two new projects and we're about to start a third one. So, it's very clear that there is still an interest, and indeed a renewed interest, in SOA from the IT community within The Open Group.
Gardner: Nikhil, do you believe that this has to do with some of the larger trends we're seeing in the field, like cloud Software as a Service (SaaS)? What's driving this renewal?
Kumar: What I see driving it is three things. One is the advent of the cloud and mobile, which requires a lot of cross-platform delivery of consistent services. The second is emerging technologies, mobile, big data, and the need to be able to look at data across multiple contexts.
The third thing that’s driving it is legacy modernization. A lot of organizations are now a lot more comfortable with SOA concepts. I see it in a number of our customers. I've just been running a large enterprise architecture initiative in a Fortune 500 customer.
At each stage, and at almost every point in that, they're now comfortable. They feel that SOA can provide the ability to rationalize multiple platforms. They're restructuring organizational structures, delivery organizations, as well as targeting their goals around a service-based platform capability.
So legacy modernization is a back-to-the-future kind of thing that has come back and is getting adoption. The way it's being implemented is using RESTful services, as well as SOAP services, which is different from traditional SOA, say from the last version, which was mostly SOAP-driven.
Gardner: Mats, do you think that what's happened is that the marketplace and the requirements have changed and that’s made SOA more relevant? Or has SOA changed to better fit the market? Or perhaps some combination?
Gejnevall: I think that the cloud is really a service delivery platform. Companies discover that to be able to use the cloud services, the SaaS things, they need to look at SOA as their internal development way of doing things as well. They understand they need to do the architecture internally, and if they're going to use lots of external cloud services, you might as well use SOA to do that.
Also, if you look at the cloud suppliers, they also need to do their architecture in some way and SOA probably is a good vehicle for them. They can use that paradigm and also deliver what the customer wants in a well-designed SOA environment.
Gardner: Let's drill down on the requirements around the cloud and some of the key components of SOA. We're certainly seeing, as you mentioned, the need for cross support for legacy, cloud types of services, and using a variety of protocol, transports, and integration types. We already heard about REST for lightweight approaches and, of course, there will still be the need for object brokering and some of the more traditional enterprise integration approaches.
This really does sound like the job for an Enterprise Service Bus (ESB). So let's go around the panel and look at this notion of an ESB. Some people, a few years back, didn’t think it was necessary or a requirement for SOA, but it certainly sounds like it's the right type of functionality for the job.
Harding: I believe so, but maybe we ought to consider that in the cloud context, you're not just talking about within a single enterprise. You're talking about a much more loosely coupled, distributed environment, and the ESB concept needs to take account of that in the cloud context.
Gardner: Nikhil, any thoughts about how to manage this integration requirement around the modern SOA environment and whether ESBs are more or less relevant as a result?
Kumar: In the context of a cloud we really see SOA and the concept of service contracts coming to the fore. In that scenario, ESBs play a role as a broker within the enterprise. When we talk about the interaction across cloud-service providers and cloud consumers, what we're seeing is that the service provider has his own concept of an ESB within its own internal context.
If you want your cloud services to be really reusable, the concept of the ESB then becomes more for the routing and the mediation of those services, once they're provided to the consumer. There's a kind of separation of concerns between the concept of a traditional ESB and a cloud ESB, if you want to call it that.
The cloud context involves more of the need to be able to support, enforce, and apply governance concepts and audit concepts, the capabilities to ensure that the interaction meets quality of service guarantees. That's a little different from the concept that drove traditional ESBs.
That’s why you're seeing API management platforms like Layer 7, Mashery, or Apigee and other kind of product lines. They're also coming into the picture, driven by the need to be able to support the way cloud providers are provisioning their services. As Chris put it, you're looking beyond the enterprise. Who owns it? That’s where the role of the ESB is different from the traditional concept.
Most cloud platforms have cost factors associated with locality. If you have truly global enterprises and services, you need to factor in the ability to deal with safe harbor issues and you need to factor in variations and law in terms of security governance.
The platforms that are evolving are starting to provide this out of the box. The service consumer or a service provider needs to be able to support those. That's going to become the role of their ESB in the future, to be able to consume a service, to be able to assert this quality-of-service guarantee, and manage constraints or data-in-flight and data-at-rest.
Gardner: Mats, are there other aspects of the concept of ESB that are now relevant to the cloud?
Gejnevall: One of the reasons SOA didn’t really take off in many organizations three, four, or five years ago was the need to buy the entire stack of SOA products that all the consultancies were asking companies to buy, wanting them to buy an ESB, governance tools, business process management tools, and a lot of sort of quite large investments to just get your foot into the door of doing SOA.
These days you can buy that kind of stuff. You can buy the entire stack in the cloud and start playing with it. I did some searches on it today and I found a company that you can play with the entire stack, including business tools and everything like that, for zero dollars. Then you can grow and use more and more of it in your business, but you can start to see if this is something for you.
In the past, the suppliers or the consultants told you that you could do it. You couldn’t really try it out yourself. You needed both the software and the hardware in place. The money to get started is much lower today. That's another reason people might be thinking about it these days.
Gardner: It sounds as if there's a new type of on-ramp to SOA values, and the componentry that supports SOA is now being delivered as a service. On top of that, you're also able to consume it in a pay-as-you-go manner.
Harding: That's a very good point, but there are two contradictory trends we are seeing here. One is the kind of trend that Mats is describing, where the technology you need to handle a complex stack is becoming readily available in the cloud.
And the other is the trend that Nikhil mentioned: to go for a simpler style, which a lot of people term REST, for accessing services. It will be interesting to see how those two tendencies play out against each other.
Kumar: I'd like to make a comment on that. The approach for the on-ramp is really one of the key differentiators of the cloud, because you have the agility and the lack of capital investment (CAPEX) required to test things out.
But as we are evolving with cloud platforms, I'm also seeing with a lot of Platform-as-a-Service (PaaS) vendor scenarios that they're trying the ESB in the stack itself. They're providing it in their cloud fabric. A couple of large players have already done that.
Gejnevall: Another interesting thing is that they could get a whole environment that's pre-integrated. Usually, when you buy these things from a vendor, a lot of times they don't fit together that well. Now, there’s an effort to make them work together.
But some people put these open-source tools together. Some people have done that and put them out on the cloud, which gives them a pretty cheap platform for themselves. Then, they can sell it at a reasonable price, because of the integration of all these things.
Gardner: The cloud model may be evolving toward an all-inclusive offering. But SOA, by its definition, advances interoperability, to plug and play across existing, current, and future sets of service possibilities. Are we talking about SOA being an important element of keeping clouds dynamic and flexible -- even open?
Kumar: We can think about the OSI 7 Layer Model. We're evolving in terms of complexity, right? So from an interoperability perspective, we may talk SOAP or REST, for example, but the interaction with AWS, Salesforce, SmartCloud, or Azure would involve using APIs that each of these platforms provide for interaction.
So you could have an AMI, which is an image on the Amazon Web Services environment, for example, and that could support a lab stack or an open source stack. How you interact with it, how you monitor it, how you cluster it, all of those aspects now start factoring in specific APIs, and so that's the lock-in.
From an architect’s perspective, I look at it as we need to support proper separation of concerns, and that's part of [The Open Group] SOA Reference Architecture. That's what we tried to do, to be able to support implementation architectures that support that separation of concerns.
There's another factor that we need to understand from the context of the cloud, especially for mid-to-large sized organizations, and that is that the cloud service providers, especially the large ones -- Amazon, Microsoft, IBM -- encapsulate infrastructure.
If you were to go to Amazon, Microsoft, or IBM and use their IaaS networking capabilities, you'd have one of the largest WAN networks in the world, and you wouldn’t have to pay a dime to establish that infrastructure. Not in terms of the cost of the infrastructure, not in terms of the capabilities required, nothing. So that's an advantage that the cloud is bringing, which I think is going to be very compelling.
The other thing is that, from an SOA context, you're now able to look at it and say, "Well, I'm dealing with the cloud, and what all these providers are doing is make it seamless, whether you're dealing with the cloud or on-premise." That's an important concept.
Now, each of these providers and different aspects of their stacks are at significantly different levels of maturity. Many of these providers may find that their stacks do not interoperate with themselves either, within their own stacks, just because they're using different run times, different implementations, etc. That's another factor to take in.
From an SOA perspective, the cloud has become very compelling, because I'm dealing, let's say, with a Salesforce.com and I want to use that same service within the enterprise, let's say, an insurance capability for Microsoft Dynamics or for SugarCRM. If that capability is exposed to one source of truth in the enterprise, you've now reduced the complexity and have the ability to adopt different cloud platforms.
What we are going to start seeing is that the cloud is going to shift from being just one à-la-carte solution for everybody. It's going to become something similar to what we used to deal with in the enterprise context. You had multiple applications, which you service-enabled to reduce complexity and provide one service-based capability, instead of an application-centered approach.
You're now going to move the context to the cloud, to your multiple cloud solutions, and maybe many implementations in a nontrivial environment for the same business capability, but they are now exposed to services in the enterprise SOA. You could have Salesforce. You could have Amazon. You could have an IBM implementation. And you could pick and choose the source of truth and share it.
So a lot of the core SOA concepts will still apply and are still applying.
Gardner: Perhaps yet another on-ramp to the use of SOA is the app store, which allows for discovery, socialization of services, but at the same time provides overnance and control?
Kumar: We're seeing that with a lot of our customers, typically the vendors who support PaaS solution associate app store models along with their platform as a mechanism to gain market share.
The issue that you run into with that is, it's okay if it's on your cellphone or on your iPad, your tablet PC, or whatever, but once you start having managed apps, for example Salesforce, or if you have applications which are being deployed on an Azure or on a SmartCloud context, you have high risk scenario. You don't know how well architected that application is. It's just like going and buying an enterprise application.
When you deploy it in the cloud, you really need to understand the cloud PaaS platform for that particular platform to understand the implications in terms of dependencies and cross-dependencies across apps that you have installed. They have real practical implications in terms of maintainability and performance. We've seen that with at least two platforms in the last six months.
Governance becomes extremely important. Because of the low CAPEX implications to the business, the business is very comfortable with going and buying these applications and saying, "We can install X, Y, or Z and it will cost us two months and a few million dollars and we are all set." Or maybe it's a few hundred thousand dollars.
They don't realize the implications in terms of interoperability, performance, and standard architectural quality attributes that can occur. There is a governance aspect from the context of the cloud provisioning of these applications.
There is another aspect to it, which is governance in terms of the run-time, more classic SOA governance, to measure, assert, and to view the cost of these applications in terms of performance to your infrastructural resources, to your security constraints. Also, are there scenarios where the application itself has a dependency on a daisy chain, multiple external applications, to trace the data?
In terms of the context of app stores, they're almost like SaaS with a particular platform in mind. They provide the buyer with certain commitments from the platform manager or the platform provider, such as security. When you buy an app from Apple, there is at least a reputational expectation of security from the vendor.
What you do not always know is if that security is really being provided. There's a risk there for organizations who are exposing mission-critical data to that.
The second thing is there is still very much a place for the classic SOA registries and repositories in the cloud. Only the place is for a different purpose. Those registries and repositories are used either by service providers or by consumers to maintain the list of services they're using internally.
There are two different paradigms. The app store is a place where I can go and I know that the gas I am going to get is 85 percent ethanol, versus I also have to maintain some basic set of goods at home to make that I have my dinner on time. These are different kind of roles and different kind of purposes they're serving.
Above all, I think the thing that's going to become more and more important in the context of the cloud is that the functionality will be provided by the cloud platform or the app you buy, but the governance will be a major IT responsibility, right from the time of picking the app, to the time of delivering it, to the time of monitoring it.
Gardner: How is The Open Group allowing architects to better exercise SOA principles, as they're grappling with some of these issues around governance, hybrid services delivery and management, and the use and demand in their organizations to start consuming more cloud services?
Harding: The architect’s primary concern, of course, has to be to meet the needs of the client and to do so in a way that is most effective and that is cost-effective. Cloud gives the architect a usability to go out and get different components much more easily than hitherto.
There is a problem, of course, with integrating them and putting them together. SOA can provide part of the solution to that problem, in that it gives a principle of loosely coupled services. If you didn’t have that when you were trying to integrate different functionality from different places, you would be in a real mess.
What The Open Group contributes is a set of artifacts that enable the architect to think through how to meet the client’s needs in the best way when working with SOA and cloud.
For example, the SOA Reference Architecture helps the architect understand what components might be brought into the solution. We have the SOA TOGAF Practical Guide, which helps the architect understand how to use TOGAF in the SOA context.
We're working further on artifacts in the cloud space, the Cloud Computing Reference Architecture, a notational language for enabling people to describe cloud ecosystems on recommendations for cloud interoperability and portability. We're also working on recommendations for cloud governance to complement the recommendations for SOA governance, the SOA Governance Framework Standards that we have already produced, and a number of other artifacts.
The Open Group’s real role is to support the architect and help the architect to better meet the needs of the architect client.
From the very early days, SOA was seen as bringing a closer connection between the business and technology. A lot of those promises that were made about SOA seven or eight years ago are only now becoming possible to fulfill, and that business front is what that project is looking at.
We're also producing an update to the SOA Reference Architectures. We have input the SOA Reference Architecture for consideration by the ISO Group that is looking at an International Standard Reference Architecture for SOA and also to the IEEE Group that is looking at an IEEE Standard Reference Architecture.
We hope that both of those groups will want to work along the principles of our SOA Reference Architecture and we intend to produce a new version that incorporates the kind of ideas that they want to bring into the picture.
We're also thinking of setting up an SOA project to look specifically at assistance to architects building SOA into enterprise solutions.
So those are three new initiatives that should result in new Open Group standards and guides to complement, as I have described already, the SOA Reference Architecture, the SOA Governance Framework, the Practical Guides to using TOGAF for SOA.
We also have the Service Integration Maturity Model that we need to assess the SOA maturity. We have a standard on service orientation applied to cloud infrastructure, and we have a formal SOA Ontology.
Those are the things The Open Group has in place at present to assist the architect, and we are and will be working on three new things: version 2 of the Reference Architecture for SOA, SOA for business technology, and I believe shortly we'll start on assistance to architects in developing SOA solutions.
You may also be interested in:
- The Open Group Trusted Technology Forum is Leading the Way to Securing Global Supply Chains
- Corporate Data, Supply Chains Remain Vulnerable to Cyber Crime Attacks Says Open Group Conference Speaker
- Open Group Conference Speakers Discuss the Cloud: Higher Risk or Better Security?
- Capgemini's CTO on Why Cloud Computing Exposes the Duality Between IT and Business
- San Francisco Conference observations: Enterprise transformation, enterprise architecture, SOA and a splash of cloud computing
- MIT's Ross on how enterprise architecture and IT more than ever lead to business transformation
- Overlapping criminal and state threats pose growing cyber security threat to global Internet commerce, says Open Group speaker
Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. In his session at @DevOpsSummit at 18th Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, will show how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningfu...
May. 6, 2016 07:00 AM EDT Reads: 1,217
Last week I had the pleasure of speaking on a panel at Sapphire Ventures Next-Gen Tech Stack Forum in San Francisco. Obviously, I was excited to join the discussion, but as a participant the event crystallized not only where the larger software development market is relative to microservices, container technologies (like Docker), continuous integration and deployment; but also provided insight into where DevOps is heading in the coming years.
May. 6, 2016 07:00 AM EDT Reads: 259
Wow, if you ever wanted to learn about Rugged DevOps (some call it DevSecOps), sit down for a spell with Shannon Lietz, Ian Allison and Scott Kennedy from Intuit. We discussed a number of important topics including internal war games, culture hacking, gamification of Rugged DevOps and starting as a small team. There are 100 gold nuggets in this conversation for novices and experts alike.
May. 6, 2016 05:30 AM EDT Reads: 1,051
The notion of customer journeys, of course, are central to the digital marketer’s playbook. Clearly, enterprises should focus their digital efforts on such journeys, as they represent customer interactions over time. But making customer journeys the centerpiece of the enterprise architecture, however, leaves more questions than answers. The challenge arises when EAs consider the context of the customer journey in the overall architecture as well as the architectural elements that make up each...
May. 6, 2016 04:15 AM EDT Reads: 2,091
In a crowded world of popular computer languages, platforms and ecosystems, Node.js is one of the hottest. According to w3techs.com, Node.js usage has gone up 241 percent in the last year alone. Retailers have taken notice and are implementing it on many levels. I am going to share the basics of Node.js, and discuss why retailers are using it to reduce page load times and improve server efficiency. I’ll talk about similar developments such as Docker and microservices, and look at several compani...
May. 6, 2016 03:00 AM EDT Reads: 947
Admittedly, two years ago I was a bulk contributor to the DevOps noise with conversations rooted in the movement around culture, principles, and goals. And while all of these elements of DevOps environments are important, I’ve found that the biggest challenge now is a lack of understanding as to why DevOps is beneficial. It’s getting the wheels going, or just taking the next step. The best way to start on the road to change is to take a look at the companies that have already made great headway ...
May. 6, 2016 02:45 AM EDT Reads: 545
In 2006, Martin Fowler posted his now famous essay on Continuous Integration. Looking back, what seemed revolutionary, radical or just plain crazy is now common, pedestrian and "just what you do." I love it. Back then, building and releasing software was a real pain. Integration was something you did at the end, after code complete, and we didn't know how long it would take. Some people may recall how we, as an industry, spent a massive amount of time integrating code from one team with another...
May. 6, 2016 01:30 AM EDT Reads: 1,167
From the conception of Docker containers to the unfolding microservices revolution we see today, here is a brief history of what I like to call 'containerology'. In 2013, we were solidly in the monolithic application era. I had noticed that a growing amount of effort was going into deploying and configuring applications. As applications had grown in complexity and interdependency over the years, the effort to install and configure them was becoming significant. But the road did not end with a ...
May. 6, 2016 12:30 AM EDT Reads: 781
I have an article in the recently released “DZone Guide to Building and Deploying Applications on the Cloud” entitled “Fullstack Engineering in the Age of Hybrid Cloud”. In this article I discuss the need and skills of a Fullstack Engineer with relation to troubleshooting and repairing complex, distributed hybrid cloud applications. My recent experiences with troubleshooting issues with my Docker WordPress container only reinforce the details I wrote about in this piece. Without my comprehensive...
May. 5, 2016 11:30 PM EDT Reads: 1,084
As the software delivery industry continues to evolve and mature, the challenge of managing the growing list of the tools and processes becomes more daunting every day. Today, Application Lifecycle Management (ALM) platforms are proving most valuable by providing the governance, management and coordination for every stage of development, deployment and release. Recently, I spoke with Madison Moore at SD Times about the changing market and where ALM is headed.
May. 5, 2016 07:45 PM EDT Reads: 1,647
The goal of any tech business worth its salt is to provide the best product or service to its clients in the most efficient and cost-effective way possible. This is just as true in the development of software products as it is in other product design services. Microservices, an app architecture style that leans mostly on independent, self-contained programs, are quickly becoming the new norm, so to speak. With this change comes a declining reliance on older SOAs like COBRA, a push toward more s...
May. 5, 2016 05:15 PM EDT Reads: 1,743
Small teams are more effective. The general agreement is that anything from 5 to 12 is the 'right' small. But of course small teams will also have 'small' throughput - relatively speaking. So if your demand is X and the throughput of a small team is X/10, you probably need 10 teams to meet that demand. But more teams also mean more effort to coordinate and align their efforts in the same direction. So, the challenge is how to harness the power of small teams and yet orchestrate multiples of them...
May. 5, 2016 05:00 PM EDT Reads: 620
Many private cloud projects were built to deliver self-service access to development and test resources. While those clouds delivered faster access to resources, they lacked visibility, control and security needed for production deployments. In their session at 18th Cloud Expo, Steve Anderson, Product Manager at BMC Software, and Rick Lefort, Principal Technical Marketing Consultant at BMC Software, will discuss how a cloud designed for production operations not only helps accelerate developer...
May. 5, 2016 05:00 PM EDT Reads: 1,387
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus inter...
May. 5, 2016 02:30 PM EDT Reads: 1,496
In the world of DevOps there are ‘known good practices’ – aka ‘patterns’ – and ‘known bad practices’ – aka ‘anti-patterns.' Many of these patterns and anti-patterns have been developed from real world experience, especially by the early adopters of DevOps theory; but many are more feasible in theory than in practice, especially for more recent entrants to the DevOps scene. In this power panel at @DevOpsSummit at 18th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists will dis...
May. 5, 2016 01:30 PM EDT Reads: 1,158
Much of the discussion around cloud DevOps focuses on the speed with which companies need to get new code into production. This focus is important – because in an increasingly digital marketplace, new code enables new value propositions. New code is also often essential for maintaining competitive parity with market innovators. But new code doesn’t just have to deliver the functionality the business requires. It also has to behave well because the behavior of code in the cloud affects performan...
May. 5, 2016 03:15 AM EDT Reads: 1,521
Struggling to keep up with increasing application demand? Learn how Platform as a Service (PaaS) can streamline application development processes and make resource management easy.
May. 4, 2016 09:00 PM EDT Reads: 2,273
If there is anything we have learned by now, is that every business paves their own unique path for releasing software- every pipeline, implementation and practices are a bit different, and DevOps comes in all shapes and sizes. Software delivery practices are often comprised of set of several complementing (or even competing) methodologies – such as leveraging Agile, DevOps and even a mix of ITIL, to create the combination that’s most suitable for your organization and that maximize your busines...
May. 3, 2016 07:30 PM EDT Reads: 1,964
Digital means customer preferences and behavior are driving enterprise technology decisions to be sure, but let’s not forget our employees. After all, when we say customer, we mean customer writ large, including partners, supply chain participants, and yes, those salaried denizens whose daily labor forms the cornerstone of the enterprise. While your customers bask in the warm rays of your digital efforts, are your employees toiling away in the dark recesses of your enterprise, pecking data into...
May. 3, 2016 04:45 PM EDT Reads: 1,173
You deployed your app with the Bluemix PaaS and it's gaining some serious traction, so it's time to make some tweaks. Did you design your application in a way that it can scale in the cloud? Were you even thinking about the cloud when you built the app? If not, chances are your app is going to break. Check out this webcast to learn various techniques for designing applications that will scale successfully in Bluemix, for the confidence you need to take your apps to the next level and beyond.
May. 3, 2016 12:15 PM EDT Reads: 1,667