Welcome!

Microservices Expo Authors: Liz McMillan, Pat Romanski, Carmen Gonzalez, Elizabeth White, Jason Bloomberg

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Juggling Multiple Data Models with Services

Guidelines for Success

Recently, a client approached me with a quandary.  When designing XML schemas for Web services, how do you balance the desire to use industry standards such as UBL ( Universal Business Language) or CICA ( Context Inspired Component Architecture) to support data interoperability with the unique needs of particular domains and sub-systems within the enterprise?  The client’s service design team is rightfully concerned with the competing interests of internal enterprise standardization, interoperability with external entities, and addressing the unique needs of local domains and process constraints. How can these competing needs be effectively managed when designing the schema for a given service?

The Standard Answer

Naturally, you start by giving the standard answer - “it depends”.  This is an essential and carefully crafted phrase that all consultants are taught to give according to page 17 of How to Win Clients and Influence Budgets. Of course, the standard answer is rarely sufficient, but it is not without merit.  It is certainly true that the decision regarding your data model design depends significantly upon a host of factors:

  • Which use cases are high priority?
  • Which use cases are high impact (to stakeholders and/or to customers)?
  • Which systems and/or processes are mission critical?
  • Are you including industry standard data models in you enterprise out of necessity (i.e. ONLY when you are forced to interact with other entities), or is it part of a broader modernization effort within your organization?
  • Do interoperability challenges or integration challenges currently contribute most significantly to your development and/or maintenance costs?
  • Which interaction scenarios are of higher priority, internal or external service calls?
  • Would aligning more closely to one of the available models lead to a significant reduction in data mapping activities or is the environment simply too fragmented to support this?
  • Is there one particular system or business process that is unique and skewing perceptions regarding our service data models?

While all of these questions (and many more) are important and aid in facilitating a thorough examination of the problem, we rarely have the time to examine each problem from all possible angles.  Consequently, we must look toward guidelines and rules of thumbs.

Guidelines for Managing Multiple Service Schema

1)       Your business processes should only work with a single data model if at all possible.  Business processes are best designed to be data model agnostic and operate off of an internal, process-centric model.  I explained the importance of process-centric data models in a previous post.

2)       Your services should work with as few data models as possible (one model being preferable).  This is just good commonsense.  For each data model that is added to the mix, your development time and long-term maintenance costs increase at a non-linear rate.  The pain will intensify and it will do so rapidly with each new model you add to the mix.

3)       If your services are going to work with multiple models, you should put some sort of taxonomy / categorization scheme in place to distinguish the data models used by services.  For example, services that are outward facing might use a data model accepted more broadly by the industry.  Services used by a particular LOB might use a certain data model, and services used by another LOB might use another.  Another distinction could be infrastructure services vs data services vs application services.  Regardless of the approach, there needs to be some methodology that is objective and governable for when one data model is used vs when another data model is used.

4)       Data model transformation is a necessary evil.  It should be done only when necessary and you should contain the transformation to a designated component.  Transformation activities should be handled by intermediaries (data services, ESB, network appliance, other mediation framework) when possible rather than building it into the internals of a service or process.  This keeps your services clean, provides a nice reusable transformation mechanism, keeps your interoperability more loosely-coupled, and provides for agility and extensibility in the future.

Juggling multiple data models within a service oriented environment is no one’s idea of fun.  When possible, aim for a more comprehensive and strategic analysis of the environment (see the ‘Standard Answer’ outlined above).  When this is not realistic,  try to use the above guidelines and rules of thumb to help you tactically navigate the murky waters of data model incongruity.  Service design isn’t easy, but it doesn’t have to be rocket surgery either.  Best of luck!

More Stories By Kyle Gabhart

Kyle Gabhart is a subject matter expert specializing in strategic planning and tactical delivery of enterprise technology solutions, blending EA, BPM, SOA, Cloud Computing, and other emerging technologies. Kyle currently serves as a director for Web Age Solutions, a premier provider of technology education and mentoring. Since 2001 he has contributed extensively to the IT community as an author, speaker, consultant, and open source contributor.

Microservices Articles
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, will discuss why containers should be paired with new architectural practices such as microservices ra...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...