Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo, Microsoft Cloud, Open Source Cloud

Microservices Expo: Article

SOA Transformation and Leveraging Open Source Principles

It's important to recognize that the organizational transformation has to do with cultural transformation

As numerous organizations are planning to embark on their first endeavors in service-oriented architecture (SOA), it is important to recognize that the necessary organizational transformation has as much to do with cultural transformation, as it has to do with open, Internet standards-based design. In fact, the very nature of how business and IT view each other's role and how the enterprise views its relationships with its marketplace partners and customers is being altered. Such cultural change has never come easily and represents a significant organizational dilemma.

At the heart of an enterprise SOA transformation is also the need to "think differently" about how an enterprise acquires and manages IT systems. Albert Einstein might well have been talking about the challenges of SOA rollout when he said, "No problem can be solved by the same level of consciousness that created it." Building service-based systems involves a shift in thinking from large-scale, centrally planned IT systems to smaller, modular development that requires collaboration and consensus building among all of the stakeholders of an end-to-end IT process.

These aspects of SOA transformation alone represent a large impediment to success for most organizations. Some simple questions that can highlight the importance are: How well do your business and IT stakeholders collaborate in the fielding of new business functions? How well do you execute globally distributed software development - not only within your enterprise, but also in the extended enterprise that includes partners, suppliers, and clients? Do you have a successful culture of component-based design and software reuse? Do you consider the elimination of silos within your enterprise to be a major goal? How well aligned are your operations infrastructure and capabilities with the role of IT service provider?

The aforementioned questions lead to a few aspects of open source principles and characteristics that may mitigate or address some of the significant challenges of SOA transformation. Table 1 lists SOA challenges matched with associated open source characteristics. The list is by no means comprehensive, but it is thorough enough to show the potential relationship in applying open source principles to the business problem of implementing SOA.

Given the success of open source software, the overall premise is that at a minimum there is something that can be learned from the open source development model, which enables organizations to become service-oriented enterprises (SOE).

Problem Illustration
The starting point for many SOA transformations is the desire of the enterprise to define an overall enterprise architecture (EA). EA is often employed as the tool of business modernization. In any EA initiative, the first step is usually an evaluation of the current state of affairs by means of assessment. The EA assessment usually reveals a map of redundancies across business processes and systems. These redundancies become the "low hanging fruit" for the first areas to be targeted for migration to enterprise services.

An oversimplified example might be an organization with multiple business systems, all of which require a software process to calculate loan risk or interest rate. Upon such a discovery, a natural inclination might be to fund a project to create a common shared service that could be leveraged by all such business systems. Although this approach is logical and has an obvious supporting ROI, the actual implementation will require more consideration and effort.

The first problem is the cultural bias of the existing business systems owners. Some will seek exemptions to not participate, usually based upon some notion that their requirements are somehow unique (i.e., somehow the financial aspects of the math are different for them). The real underlying reasons can run the gambit of NIH, concerns over support, or a reluctance to develop dependencies on other organizations or departments. These are some of the cultural dynamics that are often at work, which tend undermine efforts to create reusable SOA components.

A second problem is very tangible in the sense that building a shared service for this calculation may not be the right thing to do in all cases. Assume we deploy an enterprise service and institute a governance model where we instruct engineers to invoke this service directly rather than running their own copy of the software on their own systems. Well, that might work for some applications that only need to invoke that service a couple of times a minute, but what about other applications that need to invoke it hundreds of times per second, or are in latency-critical situations, such as rendering a Web page? In this case we might option for building the software process as both an "open standard" by giving it a Web services interface and as "open code" by publishing the source and allowing it to be run locally.

This example hopefully provides an appreciation for the application of open source principles in an attempt to address a complex issue. In addressing the cultural problem, we could adopt a community-based development approach in which business systems owners collaboratively participate in the development of a new solution. Such a shared development model could mitigate perceived risks of support and inflexible organizational dependencies. In the performance-based design problem, having easy access to source code and a supporting community of interest (COI) might enable the best solution to be applied. This would still be in general alignment with the goals of reducing development redundancy and having consistent enterprise architecture.

Some Focus Areas
According to the Yankee Group's "2004 Enterprise Web Service Study," 75 percent of the more than 400 US IT organizations surveyed are planning to invest in SOA approaches over the next 12 months. This seems reasonable because in any large-scale organization, the concept of enterprise shared services that employ component-based design is not new and is easily understood for its cost benefits. Often, "where to begin" is the harder problem and reflects where open source processes might be applied.

Driving Adoption and Utilization
In the end, what makes for a successful Web service is fundamentally a case based on whether or not anyone cares. Success is really a proof point based on actual adoption and utilization levels. Successful open source projects understand this point and invest some significant percentage of effort to support and grow the COI around the project. There is a strong correlation between how open source COIs grow and how to succeed in creating highly successful SOA components.

A case in point is discovery. The question is whether or not discovery via UDDI registry or other means is sufficient by itself to drive a service's adoption rate? This problem is analogous to the push to develop reusable software assets based upon traditional component-based design. Many enterprises implemented so called "component repositories" where they encouraged the developers to register their components after development was complete to make them easy to discover.

The theory was that engineering leads and project managers would occasionally search this library of components and reuse them if possible. Given the amount of serious and sustained attempts by qualified organizations and smart people, the fact seems to remain that software reuse initiatives are rarely as successful as has been hoped. Many of these repositories ended up looking like a kitchen junk drawer.

The implication for SOA is that the same fate can befall SOA service registries. Whether is it reusing code or adopting a service, there are a lot questions that potential users want to have answered before making a commitment. Obvious questions are: What's planned on the roadmap? Who is supporting the service? What are the known bugs and limitations? Who else is using it?

The final decision is really about mitigating the risk of adoption. The consumer might feel more comfortable if, through the interaction with a service's associated COI, they can get a sense for the service, its user base, and possibly even the consumer's own desire to become a contributing member of the project. The probability of adoption and the actual utilization of a service can be enhanced by enabling a discovery registry to provide easy access to the COI that is supporting the Web service.

Application Lifecycle Management
While SOA's impact on culture might be intuitive, what is not so obvious is its critical impact on application lifecycle management (ALM). Existing tools and software development processes may not be well aligned to support the move to SOA. At a macro level, an organization needs to understand that is it essentially migrating from insular LAN-centric development with a captive group of resources to a more open WAN-centric development environment with a diverse ecosystem of stakeholders.

In a recent interview on the importance of ALM, Gartner analyst Theresa Lanowitz commented that "As we start to bring forward this idea of a services-oriented architecture, and as we start to see services exist in a business-to-business environment, the services exist from two separate companies, and those services possibly are federated together in a loosely coupled environment. So we find the idea of the application ecosystem, the idea of a development ecosystem, is crucial as well, because there are going to be two different organizations creating a service that will have to be used together."

Many organizations are not prepared to address the need to support across the firewall software development with smaller agile teams from either a technology or process perspective. Figure 1 shows the ALM perspective of the potential impact of moving from monolithic IT applications to granular Web services.

From an open source project perspective, the right side of Figure 1 represents "business as usual" for many open source projects (i.e., smaller and geographically distributed individuals and teams employing agile development processes, working on some modularize software function or component, and using the Internet as the backbone for a software development platform). Project governance is often consensus driven and the project home becomes a focal point for code, context, and community.

More Stories By Michael Kochanik

Mike Kochanik is vice president of alliances at CollabNet (www.collab.net), a provider of on-demand distributed application lifecycle management solutions for software development. Mike has been a major contributor to the creation of network-centric IT strategies that leverage open source software and community-based development processes at Global 1000 organizations.

Comments (3)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Microservices Articles
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...