|By Dana Gardner||
|October 22, 2012 07:30 AM EDT||
To find out why, BriefingsDirect recently gathered an international panel of experts to explore the concept of "architecture is destiny," especially when it comes to hybrid services delivery and management. The panel shows how SOA is proving instrumental in allowing the needed advancements over highly distributed services and data, when it comes to scale, heterogeneity support, and governance.
The panel consists of Chris Harding, Director of Interoperability at The Open Group, based in the UK; Nikhil Kumar, President of Applied Technology Solutions and Co-Chair of the SOA Reference Architecture Projects within The Open Group, and he's based in Michigan, and Mats Gejnevall, Enterprise Architect at Capgemini and Co-Chair of The Open Group SOA Work Group, and he's based in Sweden. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]
Here are some excerpts:
Gardner: Why this resurgence in the interest around SOA?
Harding: My role in The Open Group is to support the work of our members on SOA, cloud computing, and other topics. We formed the SOA Work Group back in 2005, when SOA was a real emerging hot topic, and we set up a number of activities and projects. They're all completed.
In fact, we've started two new projects and we're about to start a third one. So, it's very clear that there is still an interest, and indeed a renewed interest, in SOA from the IT community within The Open Group.
Gardner: Nikhil, do you believe that this has to do with some of the larger trends we're seeing in the field, like cloud Software as a Service (SaaS)? What's driving this renewal?
Kumar: What I see driving it is three things. One is the advent of the cloud and mobile, which requires a lot of cross-platform delivery of consistent services. The second is emerging technologies, mobile, big data, and the need to be able to look at data across multiple contexts.
The third thing that’s driving it is legacy modernization. A lot of organizations are now a lot more comfortable with SOA concepts. I see it in a number of our customers. I've just been running a large enterprise architecture initiative in a Fortune 500 customer.
At each stage, and at almost every point in that, they're now comfortable. They feel that SOA can provide the ability to rationalize multiple platforms. They're restructuring organizational structures, delivery organizations, as well as targeting their goals around a service-based platform capability.
So legacy modernization is a back-to-the-future kind of thing that has come back and is getting adoption. The way it's being implemented is using RESTful services, as well as SOAP services, which is different from traditional SOA, say from the last version, which was mostly SOAP-driven.
Gardner: Mats, do you think that what's happened is that the marketplace and the requirements have changed and that’s made SOA more relevant? Or has SOA changed to better fit the market? Or perhaps some combination?
Gejnevall: I think that the cloud is really a service delivery platform. Companies discover that to be able to use the cloud services, the SaaS things, they need to look at SOA as their internal development way of doing things as well. They understand they need to do the architecture internally, and if they're going to use lots of external cloud services, you might as well use SOA to do that.
Also, if you look at the cloud suppliers, they also need to do their architecture in some way and SOA probably is a good vehicle for them. They can use that paradigm and also deliver what the customer wants in a well-designed SOA environment.
Gardner: Let's drill down on the requirements around the cloud and some of the key components of SOA. We're certainly seeing, as you mentioned, the need for cross support for legacy, cloud types of services, and using a variety of protocol, transports, and integration types. We already heard about REST for lightweight approaches and, of course, there will still be the need for object brokering and some of the more traditional enterprise integration approaches.
This really does sound like the job for an Enterprise Service Bus (ESB). So let's go around the panel and look at this notion of an ESB. Some people, a few years back, didn’t think it was necessary or a requirement for SOA, but it certainly sounds like it's the right type of functionality for the job.
Harding: I believe so, but maybe we ought to consider that in the cloud context, you're not just talking about within a single enterprise. You're talking about a much more loosely coupled, distributed environment, and the ESB concept needs to take account of that in the cloud context.
Gardner: Nikhil, any thoughts about how to manage this integration requirement around the modern SOA environment and whether ESBs are more or less relevant as a result?
Kumar: In the context of a cloud we really see SOA and the concept of service contracts coming to the fore. In that scenario, ESBs play a role as a broker within the enterprise. When we talk about the interaction across cloud-service providers and cloud consumers, what we're seeing is that the service provider has his own concept of an ESB within its own internal context.
If you want your cloud services to be really reusable, the concept of the ESB then becomes more for the routing and the mediation of those services, once they're provided to the consumer. There's a kind of separation of concerns between the concept of a traditional ESB and a cloud ESB, if you want to call it that.
The cloud context involves more of the need to be able to support, enforce, and apply governance concepts and audit concepts, the capabilities to ensure that the interaction meets quality of service guarantees. That's a little different from the concept that drove traditional ESBs.
That’s why you're seeing API management platforms like Layer 7, Mashery, or Apigee and other kind of product lines. They're also coming into the picture, driven by the need to be able to support the way cloud providers are provisioning their services. As Chris put it, you're looking beyond the enterprise. Who owns it? That’s where the role of the ESB is different from the traditional concept.
Most cloud platforms have cost factors associated with locality. If you have truly global enterprises and services, you need to factor in the ability to deal with safe harbor issues and you need to factor in variations and law in terms of security governance.
The platforms that are evolving are starting to provide this out of the box. The service consumer or a service provider needs to be able to support those. That's going to become the role of their ESB in the future, to be able to consume a service, to be able to assert this quality-of-service guarantee, and manage constraints or data-in-flight and data-at-rest.
Gardner: Mats, are there other aspects of the concept of ESB that are now relevant to the cloud?
Gejnevall: One of the reasons SOA didn’t really take off in many organizations three, four, or five years ago was the need to buy the entire stack of SOA products that all the consultancies were asking companies to buy, wanting them to buy an ESB, governance tools, business process management tools, and a lot of sort of quite large investments to just get your foot into the door of doing SOA.
These days you can buy that kind of stuff. You can buy the entire stack in the cloud and start playing with it. I did some searches on it today and I found a company that you can play with the entire stack, including business tools and everything like that, for zero dollars. Then you can grow and use more and more of it in your business, but you can start to see if this is something for you.
In the past, the suppliers or the consultants told you that you could do it. You couldn’t really try it out yourself. You needed both the software and the hardware in place. The money to get started is much lower today. That's another reason people might be thinking about it these days.
Gardner: It sounds as if there's a new type of on-ramp to SOA values, and the componentry that supports SOA is now being delivered as a service. On top of that, you're also able to consume it in a pay-as-you-go manner.
Harding: That's a very good point, but there are two contradictory trends we are seeing here. One is the kind of trend that Mats is describing, where the technology you need to handle a complex stack is becoming readily available in the cloud.
And the other is the trend that Nikhil mentioned: to go for a simpler style, which a lot of people term REST, for accessing services. It will be interesting to see how those two tendencies play out against each other.
Kumar: I'd like to make a comment on that. The approach for the on-ramp is really one of the key differentiators of the cloud, because you have the agility and the lack of capital investment (CAPEX) required to test things out.
But as we are evolving with cloud platforms, I'm also seeing with a lot of Platform-as-a-Service (PaaS) vendor scenarios that they're trying the ESB in the stack itself. They're providing it in their cloud fabric. A couple of large players have already done that.
Gejnevall: Another interesting thing is that they could get a whole environment that's pre-integrated. Usually, when you buy these things from a vendor, a lot of times they don't fit together that well. Now, there’s an effort to make them work together.
But some people put these open-source tools together. Some people have done that and put them out on the cloud, which gives them a pretty cheap platform for themselves. Then, they can sell it at a reasonable price, because of the integration of all these things.
Gardner: The cloud model may be evolving toward an all-inclusive offering. But SOA, by its definition, advances interoperability, to plug and play across existing, current, and future sets of service possibilities. Are we talking about SOA being an important element of keeping clouds dynamic and flexible -- even open?
Kumar: We can think about the OSI 7 Layer Model. We're evolving in terms of complexity, right? So from an interoperability perspective, we may talk SOAP or REST, for example, but the interaction with AWS, Salesforce, SmartCloud, or Azure would involve using APIs that each of these platforms provide for interaction.
So you could have an AMI, which is an image on the Amazon Web Services environment, for example, and that could support a lab stack or an open source stack. How you interact with it, how you monitor it, how you cluster it, all of those aspects now start factoring in specific APIs, and so that's the lock-in.
From an architect’s perspective, I look at it as we need to support proper separation of concerns, and that's part of [The Open Group] SOA Reference Architecture. That's what we tried to do, to be able to support implementation architectures that support that separation of concerns.
There's another factor that we need to understand from the context of the cloud, especially for mid-to-large sized organizations, and that is that the cloud service providers, especially the large ones -- Amazon, Microsoft, IBM -- encapsulate infrastructure.
If you were to go to Amazon, Microsoft, or IBM and use their IaaS networking capabilities, you'd have one of the largest WAN networks in the world, and you wouldn’t have to pay a dime to establish that infrastructure. Not in terms of the cost of the infrastructure, not in terms of the capabilities required, nothing. So that's an advantage that the cloud is bringing, which I think is going to be very compelling.
The other thing is that, from an SOA context, you're now able to look at it and say, "Well, I'm dealing with the cloud, and what all these providers are doing is make it seamless, whether you're dealing with the cloud or on-premise." That's an important concept.
Now, each of these providers and different aspects of their stacks are at significantly different levels of maturity. Many of these providers may find that their stacks do not interoperate with themselves either, within their own stacks, just because they're using different run times, different implementations, etc. That's another factor to take in.
From an SOA perspective, the cloud has become very compelling, because I'm dealing, let's say, with a Salesforce.com and I want to use that same service within the enterprise, let's say, an insurance capability for Microsoft Dynamics or for SugarCRM. If that capability is exposed to one source of truth in the enterprise, you've now reduced the complexity and have the ability to adopt different cloud platforms.
What we are going to start seeing is that the cloud is going to shift from being just one à-la-carte solution for everybody. It's going to become something similar to what we used to deal with in the enterprise context. You had multiple applications, which you service-enabled to reduce complexity and provide one service-based capability, instead of an application-centered approach.
You're now going to move the context to the cloud, to your multiple cloud solutions, and maybe many implementations in a nontrivial environment for the same business capability, but they are now exposed to services in the enterprise SOA. You could have Salesforce. You could have Amazon. You could have an IBM implementation. And you could pick and choose the source of truth and share it.
So a lot of the core SOA concepts will still apply and are still applying.
Gardner: Perhaps yet another on-ramp to the use of SOA is the app store, which allows for discovery, socialization of services, but at the same time provides overnance and control?
Kumar: We're seeing that with a lot of our customers, typically the vendors who support PaaS solution associate app store models along with their platform as a mechanism to gain market share.
The issue that you run into with that is, it's okay if it's on your cellphone or on your iPad, your tablet PC, or whatever, but once you start having managed apps, for example Salesforce, or if you have applications which are being deployed on an Azure or on a SmartCloud context, you have high risk scenario. You don't know how well architected that application is. It's just like going and buying an enterprise application.
When you deploy it in the cloud, you really need to understand the cloud PaaS platform for that particular platform to understand the implications in terms of dependencies and cross-dependencies across apps that you have installed. They have real practical implications in terms of maintainability and performance. We've seen that with at least two platforms in the last six months.
Governance becomes extremely important. Because of the low CAPEX implications to the business, the business is very comfortable with going and buying these applications and saying, "We can install X, Y, or Z and it will cost us two months and a few million dollars and we are all set." Or maybe it's a few hundred thousand dollars.
They don't realize the implications in terms of interoperability, performance, and standard architectural quality attributes that can occur. There is a governance aspect from the context of the cloud provisioning of these applications.
There is another aspect to it, which is governance in terms of the run-time, more classic SOA governance, to measure, assert, and to view the cost of these applications in terms of performance to your infrastructural resources, to your security constraints. Also, are there scenarios where the application itself has a dependency on a daisy chain, multiple external applications, to trace the data?
In terms of the context of app stores, they're almost like SaaS with a particular platform in mind. They provide the buyer with certain commitments from the platform manager or the platform provider, such as security. When you buy an app from Apple, there is at least a reputational expectation of security from the vendor.
What you do not always know is if that security is really being provided. There's a risk there for organizations who are exposing mission-critical data to that.
The second thing is there is still very much a place for the classic SOA registries and repositories in the cloud. Only the place is for a different purpose. Those registries and repositories are used either by service providers or by consumers to maintain the list of services they're using internally.
There are two different paradigms. The app store is a place where I can go and I know that the gas I am going to get is 85 percent ethanol, versus I also have to maintain some basic set of goods at home to make that I have my dinner on time. These are different kind of roles and different kind of purposes they're serving.
Above all, I think the thing that's going to become more and more important in the context of the cloud is that the functionality will be provided by the cloud platform or the app you buy, but the governance will be a major IT responsibility, right from the time of picking the app, to the time of delivering it, to the time of monitoring it.
Gardner: How is The Open Group allowing architects to better exercise SOA principles, as they're grappling with some of these issues around governance, hybrid services delivery and management, and the use and demand in their organizations to start consuming more cloud services?
Harding: The architect’s primary concern, of course, has to be to meet the needs of the client and to do so in a way that is most effective and that is cost-effective. Cloud gives the architect a usability to go out and get different components much more easily than hitherto.
There is a problem, of course, with integrating them and putting them together. SOA can provide part of the solution to that problem, in that it gives a principle of loosely coupled services. If you didn’t have that when you were trying to integrate different functionality from different places, you would be in a real mess.
What The Open Group contributes is a set of artifacts that enable the architect to think through how to meet the client’s needs in the best way when working with SOA and cloud.
For example, the SOA Reference Architecture helps the architect understand what components might be brought into the solution. We have the SOA TOGAF Practical Guide, which helps the architect understand how to use TOGAF in the SOA context.
We're working further on artifacts in the cloud space, the Cloud Computing Reference Architecture, a notational language for enabling people to describe cloud ecosystems on recommendations for cloud interoperability and portability. We're also working on recommendations for cloud governance to complement the recommendations for SOA governance, the SOA Governance Framework Standards that we have already produced, and a number of other artifacts.
The Open Group’s real role is to support the architect and help the architect to better meet the needs of the architect client.
From the very early days, SOA was seen as bringing a closer connection between the business and technology. A lot of those promises that were made about SOA seven or eight years ago are only now becoming possible to fulfill, and that business front is what that project is looking at.
We're also producing an update to the SOA Reference Architectures. We have input the SOA Reference Architecture for consideration by the ISO Group that is looking at an International Standard Reference Architecture for SOA and also to the IEEE Group that is looking at an IEEE Standard Reference Architecture.
We hope that both of those groups will want to work along the principles of our SOA Reference Architecture and we intend to produce a new version that incorporates the kind of ideas that they want to bring into the picture.
We're also thinking of setting up an SOA project to look specifically at assistance to architects building SOA into enterprise solutions.
So those are three new initiatives that should result in new Open Group standards and guides to complement, as I have described already, the SOA Reference Architecture, the SOA Governance Framework, the Practical Guides to using TOGAF for SOA.
We also have the Service Integration Maturity Model that we need to assess the SOA maturity. We have a standard on service orientation applied to cloud infrastructure, and we have a formal SOA Ontology.
Those are the things The Open Group has in place at present to assist the architect, and we are and will be working on three new things: version 2 of the Reference Architecture for SOA, SOA for business technology, and I believe shortly we'll start on assistance to architects in developing SOA solutions.
You may also be interested in:
- The Open Group Trusted Technology Forum is Leading the Way to Securing Global Supply Chains
- Corporate Data, Supply Chains Remain Vulnerable to Cyber Crime Attacks Says Open Group Conference Speaker
- Open Group Conference Speakers Discuss the Cloud: Higher Risk or Better Security?
- Capgemini's CTO on Why Cloud Computing Exposes the Duality Between IT and Business
- San Francisco Conference observations: Enterprise transformation, enterprise architecture, SOA and a splash of cloud computing
- MIT's Ross on how enterprise architecture and IT more than ever lead to business transformation
- Overlapping criminal and state threats pose growing cyber security threat to global Internet commerce, says Open Group speaker
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, will compare the Jevons Paradox to modern-day enterprise IT, e...
Sep. 27, 2016 04:30 AM EDT Reads: 1,892
As applications are promoted from the development environment to the CI or the QA environment and then into the production environment, it is very common for the configuration settings to be changed as the code is promoted. For example, the settings for the database connection pools are typically lower in development environment than the QA/Load Testing environment. The primary reason for the existence of the configuration setting differences is to enhance application performance. However, occas...
Sep. 27, 2016 04:15 AM EDT Reads: 779
More and more companies are looking to microservices as an architectural pattern for breaking apart applications into more manageable pieces so that agile teams can deliver new features quicker and more effectively. What this pattern has done more than anything to date is spark organizational transformations, setting the foundation for future application development. In practice, however, there are a number of considerations to make that go beyond simply “build, ship, and run,” which changes ho...
Sep. 27, 2016 04:00 AM EDT Reads: 2,354
SYS-CON Events announced today the Kubernetes and Google Container Engine Workshop, being held November 3, 2016, in conjunction with @DevOpsSummit at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA. This workshop led by Sebastian Scheele introduces participants to Kubernetes and Google Container Engine (GKE). Through a combination of instructor-led presentations, demonstrations, and hands-on labs, students learn the key concepts and practices for deploying and maintainin...
Sep. 27, 2016 04:00 AM EDT Reads: 2,679
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
Sep. 27, 2016 03:45 AM EDT Reads: 2,554
While DevOps promises a better and tighter integration among an organization’s development and operation teams and transforms an application life cycle into a continual deployment, Chef and Azure together provides a speedy, cost-effective and highly scalable vehicle for realizing the business values of this transformation. In his session at @DevOpsSummit at 19th Cloud Expo, Yung Chou, a Technology Evangelist at Microsoft, will present a unique opportunity to witness how Chef and Azure work tog...
Sep. 27, 2016 02:15 AM EDT Reads: 1,720
When scaling agile / Scrum, we invariable run into the alignment vs autonomy problem. In short: you cannot have autonomous self directing teams if they have no clue in what direction they should go, or even shorter: Alignment breeds autonomy. But how do we create alignment? and what tools can we use to quickly evaluate if what we want to do is part of the mission or better left out? Niel Nickolaisen created the Purpose Alignment model and I use it with innovation labs in large enterprises to de...
Sep. 27, 2016 01:45 AM EDT Reads: 1,305
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
Sep. 27, 2016 01:15 AM EDT Reads: 2,000
If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
Sep. 27, 2016 12:30 AM EDT Reads: 1,634
SYS-CON Events announced today that Tintri Inc., a leading producer of VM-aware storage (VAS) for virtualization and cloud environments, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Tintri VM-aware storage is the simplest for virtualized applications and cloud. Organizations including GE, Toyota, United Healthcare, NASA and 6 of the Fortune 15 have said “No to LUNs.” With Tintri they mana...
Sep. 26, 2016 11:45 PM EDT Reads: 2,760
Analysis of 25,000 applications reveals 6.8% of packages/components used included known defects. Organizations standardizing on components between 2 - 3 years of age can decrease defect rates substantially. Open source and third-party packages/components live at the heart of high velocity software development organizations. Today, an average of 106 packages/components comprise 80 - 90% of a modern application, yet few organizations have visibility into what components are used where.
Sep. 26, 2016 11:30 PM EDT Reads: 648
Throughout history, various leaders have risen up and tried to unify the world by conquest. Fortunately, none of their plans have succeeded. The world goes on just fine with each country ruling itself; no single ruler is necessary. That’s how it is with the container platform ecosystem, as well. There’s no need for one all-powerful, all-encompassing container platform. Think about any other technology sector out there – there are always multiple solutions in every space. The same goes for conta...
Sep. 26, 2016 09:30 PM EDT Reads: 1,086
Let's recap what we learned from the previous chapters in the series: episode 1 and episode 2. We learned that a good rollback mechanism cannot be designed without having an intimate knowledge of the application architecture, the nature of your components and their dependencies. Now that we know what we have to restore and in which order, the question is how?
Sep. 26, 2016 07:00 PM EDT Reads: 1,238
Digitization is driving a fundamental change in society that is transforming the way businesses work with their customers, their supply chains and their people. Digital transformation leverages DevOps best practices, such as Agile Parallel Development, Continuous Delivery and Agile Operations to capitalize on opportunities and create competitive differentiation in the application economy. However, information security has been notably absent from the DevOps movement. Speed doesn’t have to negat...
Sep. 26, 2016 04:30 PM EDT Reads: 2,145
Your business relies on your applications and your employees to stay in business. Whether you develop apps or manage business critical apps that help fuel your business, what happens when users experience sluggish performance? You and all technical teams across the organization – application, network, operations, among others, as well as, those outside the organization, like ISPs and third-party providers – are called in to solve the problem.
Sep. 26, 2016 04:00 PM EDT Reads: 2,545
Enterprise IT has been in the era of Hybrid Cloud for some time now. But it seems most conversations about Hybrid are focused on integrating AWS, Microsoft Azure, or Google ECM into existing on-premises systems. Where is all the Private Cloud? What do technology providers need to do to make their offerings more compelling? How should enterprise IT executives and buyers define their focus, needs, and roadmap, and communicate that clearly to the providers?
Sep. 26, 2016 03:00 PM EDT Reads: 1,567
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management solutions, helping companies worldwide activate their data to drive more value and business insight and to transform moder...
Sep. 26, 2016 02:45 PM EDT Reads: 2,651
Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. In his session at @DevOpsSummit at 18th Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, showed how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningful f...
Sep. 26, 2016 02:30 PM EDT Reads: 2,620
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
Sep. 26, 2016 01:45 PM EDT Reads: 2,601
The many IoT deployments around the world are busy integrating smart devices and sensors into their enterprise IT infrastructures. Yet all of this technology – and there are an amazing number of choices – is of no use without the software to gather, communicate, and analyze the new data flows. Without software, there is no IT. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the protocols that communicate data and the emerging data analy...
Sep. 26, 2016 01:00 PM EDT Reads: 1,659