Welcome!

Microservices Expo Authors: Flint Brenton, Elizabeth White, Stackify Blog, Dana Gardner, Liz McMillan

Related Topics: Microservices Expo

Microservices Expo: Blog Feed Post

Juggling Multiple Data Models with Services

Guidelines for Success

Recently, a client approached me with a quandary.  When designing XML schemas for Web services, how do you balance the desire to use industry standards such as UBL ( Universal Business Language) or CICA ( Context Inspired Component Architecture) to support data interoperability with the unique needs of particular domains and sub-systems within the enterprise?  The client’s service design team is rightfully concerned with the competing interests of internal enterprise standardization, interoperability with external entities, and addressing the unique needs of local domains and process constraints. How can these competing needs be effectively managed when designing the schema for a given service?

The Standard Answer

Naturally, you start by giving the standard answer - “it depends”.  This is an essential and carefully crafted phrase that all consultants are taught to give according to page 17 of How to Win Clients and Influence Budgets. Of course, the standard answer is rarely sufficient, but it is not without merit.  It is certainly true that the decision regarding your data model design depends significantly upon a host of factors:

  • Which use cases are high priority?
  • Which use cases are high impact (to stakeholders and/or to customers)?
  • Which systems and/or processes are mission critical?
  • Are you including industry standard data models in you enterprise out of necessity (i.e. ONLY when you are forced to interact with other entities), or is it part of a broader modernization effort within your organization?
  • Do interoperability challenges or integration challenges currently contribute most significantly to your development and/or maintenance costs?
  • Which interaction scenarios are of higher priority, internal or external service calls?
  • Would aligning more closely to one of the available models lead to a significant reduction in data mapping activities or is the environment simply too fragmented to support this?
  • Is there one particular system or business process that is unique and skewing perceptions regarding our service data models?

While all of these questions (and many more) are important and aid in facilitating a thorough examination of the problem, we rarely have the time to examine each problem from all possible angles.  Consequently, we must look toward guidelines and rules of thumbs.

Guidelines for Managing Multiple Service Schema

1)       Your business processes should only work with a single data model if at all possible.  Business processes are best designed to be data model agnostic and operate off of an internal, process-centric model.  I explained the importance of process-centric data models in a previous post.

2)       Your services should work with as few data models as possible (one model being preferable).  This is just good commonsense.  For each data model that is added to the mix, your development time and long-term maintenance costs increase at a non-linear rate.  The pain will intensify and it will do so rapidly with each new model you add to the mix.

3)       If your services are going to work with multiple models, you should put some sort of taxonomy / categorization scheme in place to distinguish the data models used by services.  For example, services that are outward facing might use a data model accepted more broadly by the industry.  Services used by a particular LOB might use a certain data model, and services used by another LOB might use another.  Another distinction could be infrastructure services vs data services vs application services.  Regardless of the approach, there needs to be some methodology that is objective and governable for when one data model is used vs when another data model is used.

4)       Data model transformation is a necessary evil.  It should be done only when necessary and you should contain the transformation to a designated component.  Transformation activities should be handled by intermediaries (data services, ESB, network appliance, other mediation framework) when possible rather than building it into the internals of a service or process.  This keeps your services clean, provides a nice reusable transformation mechanism, keeps your interoperability more loosely-coupled, and provides for agility and extensibility in the future.

Juggling multiple data models within a service oriented environment is no one’s idea of fun.  When possible, aim for a more comprehensive and strategic analysis of the environment (see the ‘Standard Answer’ outlined above).  When this is not realistic,  try to use the above guidelines and rules of thumb to help you tactically navigate the murky waters of data model incongruity.  Service design isn’t easy, but it doesn’t have to be rocket surgery either.  Best of luck!

More Stories By Kyle Gabhart

Kyle Gabhart is a subject matter expert specializing in strategic planning and tactical delivery of enterprise technology solutions, blending EA, BPM, SOA, Cloud Computing, and other emerging technologies. Kyle currently serves as a director for Web Age Solutions, a premier provider of technology education and mentoring. Since 2001 he has contributed extensively to the IT community as an author, speaker, consultant, and open source contributor.

@MicroservicesExpo Stories
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developer...
JetBlue Airways uses virtual environments to reduce software development costs, centralize performance testing, and create a climate for continuous integration and real-time monitoring of mobile applications. The next BriefingsDirect Voice of the Customer performance engineering case study discussion examines how JetBlue Airways in New York uses virtual environments to reduce software development costs, centralize performance testing, and create a climate for continuous integration and real-tim...
Don’t go chasing waterfall … development, that is. According to a recent post by Madison Moore on Medium featuring insights from several software delivery industry leaders, waterfall is – while still popular – not the best way to win in the marketplace. With methodologies like Agile, DevOps and Continuous Delivery becoming ever more prominent over the past 15 years or so, waterfall is old news. Or, is it? Moore cites a recent study by Gartner: “According to Gartner’s IT Key Metrics Data report, ...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
"This all sounds great. But it's just not realistic." This is what a group of five senior IT executives told me during a workshop I held not long ago. We were working through an exercise on the organizational characteristics necessary to successfully execute a digital transformation, and the group was doing their ‘readout.' The executives loved everything we discussed and agreed that if such an environment existed, it would make transformation much easier. They just didn't believe it was reali...
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
"Opsani helps the enterprise adopt containers, help them move their infrastructure into this modern world of DevOps, accelerate the delivery of new features into production, and really get them going on the container path," explained Ross Schibler, CEO of Opsani, and Peter Nickolov, CTO of Opsani, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
The purpose of this article is draw attention to key SaaS services that are commonly overlooked during contact signing that are essential to ensuring they meet the expectations and requirements of the organization and provide guidance and recommendations for process and controls necessary for achieving quality SaaS contractual agreements.
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Archi...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...