Welcome!

Microservices Expo Authors: Liz McMillan, Elizabeth White, Pat Romanski, Yeshim Deniz, Zakia Bouachraoui

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Juggling Multiple Data Models with Services

Guidelines for Success

Recently, a client approached me with a quandary.  When designing XML schemas for Web services, how do you balance the desire to use industry standards such as UBL ( Universal Business Language) or CICA ( Context Inspired Component Architecture) to support data interoperability with the unique needs of particular domains and sub-systems within the enterprise?  The client’s service design team is rightfully concerned with the competing interests of internal enterprise standardization, interoperability with external entities, and addressing the unique needs of local domains and process constraints. How can these competing needs be effectively managed when designing the schema for a given service?

The Standard Answer

Naturally, you start by giving the standard answer - “it depends”.  This is an essential and carefully crafted phrase that all consultants are taught to give according to page 17 of How to Win Clients and Influence Budgets. Of course, the standard answer is rarely sufficient, but it is not without merit.  It is certainly true that the decision regarding your data model design depends significantly upon a host of factors:

  • Which use cases are high priority?
  • Which use cases are high impact (to stakeholders and/or to customers)?
  • Which systems and/or processes are mission critical?
  • Are you including industry standard data models in you enterprise out of necessity (i.e. ONLY when you are forced to interact with other entities), or is it part of a broader modernization effort within your organization?
  • Do interoperability challenges or integration challenges currently contribute most significantly to your development and/or maintenance costs?
  • Which interaction scenarios are of higher priority, internal or external service calls?
  • Would aligning more closely to one of the available models lead to a significant reduction in data mapping activities or is the environment simply too fragmented to support this?
  • Is there one particular system or business process that is unique and skewing perceptions regarding our service data models?

While all of these questions (and many more) are important and aid in facilitating a thorough examination of the problem, we rarely have the time to examine each problem from all possible angles.  Consequently, we must look toward guidelines and rules of thumbs.

Guidelines for Managing Multiple Service Schema

1)       Your business processes should only work with a single data model if at all possible.  Business processes are best designed to be data model agnostic and operate off of an internal, process-centric model.  I explained the importance of process-centric data models in a previous post.

2)       Your services should work with as few data models as possible (one model being preferable).  This is just good commonsense.  For each data model that is added to the mix, your development time and long-term maintenance costs increase at a non-linear rate.  The pain will intensify and it will do so rapidly with each new model you add to the mix.

3)       If your services are going to work with multiple models, you should put some sort of taxonomy / categorization scheme in place to distinguish the data models used by services.  For example, services that are outward facing might use a data model accepted more broadly by the industry.  Services used by a particular LOB might use a certain data model, and services used by another LOB might use another.  Another distinction could be infrastructure services vs data services vs application services.  Regardless of the approach, there needs to be some methodology that is objective and governable for when one data model is used vs when another data model is used.

4)       Data model transformation is a necessary evil.  It should be done only when necessary and you should contain the transformation to a designated component.  Transformation activities should be handled by intermediaries (data services, ESB, network appliance, other mediation framework) when possible rather than building it into the internals of a service or process.  This keeps your services clean, provides a nice reusable transformation mechanism, keeps your interoperability more loosely-coupled, and provides for agility and extensibility in the future.

Juggling multiple data models within a service oriented environment is no one’s idea of fun.  When possible, aim for a more comprehensive and strategic analysis of the environment (see the ‘Standard Answer’ outlined above).  When this is not realistic,  try to use the above guidelines and rules of thumb to help you tactically navigate the murky waters of data model incongruity.  Service design isn’t easy, but it doesn’t have to be rocket surgery either.  Best of luck!

More Stories By Kyle Gabhart

Kyle Gabhart is a subject matter expert specializing in strategic planning and tactical delivery of enterprise technology solutions, blending EA, BPM, SOA, Cloud Computing, and other emerging technologies. Kyle currently serves as a director for Web Age Solutions, a premier provider of technology education and mentoring. Since 2001 he has contributed extensively to the IT community as an author, speaker, consultant, and open source contributor.

Microservices Articles
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
"We do one of the best file systems in the world. We learned how to deal with Big Data many years ago and we implemented this knowledge into our software," explained Jakub Ratajczak, Business Development Manager at MooseFS, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Enterprise architects are increasingly adopting multi-cloud strategies as they seek to utilize existing data center assets, leverage the advantages of cloud computing and avoid cloud vendor lock-in. This requires a globally aware traffic management strategy that can monitor infrastructure health across data centers and end-user experience globally, while responding to control changes and system specification at the speed of today’s DevOps teams. In his session at 20th Cloud Expo, Josh Gray, Chie...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...