Welcome!

Microservices Expo Authors: Liz McMillan, Elizabeth White, Yeshim Deniz, Pat Romanski, Zakia Bouachraoui

Related Topics: Microservices Expo

Microservices Expo: Article

Best Practices for SOA: Building a Data Services Layer

Choosing best-of-breed data access middleware is an important enabler of SOA

The Importance of Best-of-Breed Data Access in Building a Data Services Layer
Ask just about any business executive about the core value IT provides to the business, and they’re likely to respond that data (in the form of useful information) rather than applications form the lifeblood of their business. In essence they’re correct; information technology is by definition all about information. But actually the two – data and applications – can only be separated in theory, not in fact. Data by itself is often inaccessible and/or unintelligible without the applications that process it, and most applications in turn serve no real business purpose without data.

While establishing a data services layer is essential if you want to fully apply SOA best practices to data, it’s equally important to realize that the foundation of that data services layer consists of data access. Data access, in turn, depends on standards such as ODBC, JDBC, ADO.NET, and SDO. Even if you’re using a persistence layer like one built with the open source Hibernate toolkit, for example, high-quality data access middleware is critical. A slow JDBC driver under Hibernate or a slow ADO.NET provider under Microsoft’s Language Integrated Query (LINQ) will inevitably impede the services. The results can be far reaching. Accessing the data in the various data stores across your organization in the most efficient, flexible manner is a core reason for building data services. It’s therefore critical for building all the services in your SOA implementation.

Performance speed is not the least of the issues that can be affected by data access middleware. And, because the data services layer essentially virtualizes data access, those services can hide a plethora of potential data access pitfalls. Other such pitfalls can include:

  • Scalability issues
  • Database platform, application, and version incompatibilities
  • Differences in how the various data sources handle the details of standard data access operations such as create, read, search, update, and delete
  • Data source security priorities, which might vary from database to database, table to table, or even row to row
  • Data mapping challenges arising from semantic differences among heterogeneous data
  • Problems arising from the mixture of structured and unstructured data — such as file formatting issues
  • Differences in versions of SQL

SOA can actually serve to magnify limitations in scalability, performance, and interoperability imposed by data access because organizations are both more likely to reuse the underlying code and leverage existing data services across many services and composite applications.

Your first step then in resolving these data-related issues is to build shared, centralized data services, which will result in an adaptable and easily maintained SOA implementation. This way data access logic only appears in one place, no matter how many applications consume it. The result is that, instead of scattering data-related services invocation code throughout each business service, centralized data access helps you create an environment that enables a best-of-breed data access solution to address all those data access issues.

This last point – the use of best-of-breed access solutions – can’t be stressed strongly enough. Even many people involved in data management rarely give data access middleware more than a passing thought. One reason for this is that commercial databases all include connectivity drivers bundled with the database software, which are often used by default. The fact that these “free” drivers may be less than optimal for a given IT environment typically doesn’t come up unless and until either a technical or a maintenance issue becomes serious enough to be traced back to the data connectivity layer. For reasons we’ve described, such issues can be compounded in an SOA and can also be extremely difficult to diagnose.

Quite simply, it is ill advised to rely on database-bundled data access middleware for your data services layer in an enterprise SOA. The same holds true for freely available open source drivers. As the very foundation of your data services layer, the type of data access you use deserves serious consideration and careful planning. Dynamic environments where multiple services reuse data access code and new services regularly go into production require that such code meet rigorous requirements.

Your best bet is to go with third-party data access middleware, preferably from a supplier whose core business and expertise consists of data connectivity. Look for specific attributes for your data access middleware, including capabilities for boosting query performance. Such attributes can include connection pooling as well as support for tunable data access performance such as adjusting network packet size. For scalability and high availability, data access middleware should be multithreaded and thread-safe, and offer client load balancing and failover to alternate servers.

It’s also important for data access middleware to support different types and versions of databases as well as all the subtle variations in SQL they support. Such support for heterogeneity should also extend to multiple computing platforms, chipsets, and operating systems. In addition, for the best performance and flexibility, data access middleware should offer wire protocol drivers to avoid the overhead and maintenance issues of off-the-shelf database drivers. Wire protocol drivers obviate the need for database client software and libraries, simplifying installation and administration as well as offering a much more efficient and high-performing operation. Such drivers must support the full panoply of relevant standards, including JDBC, ODBC, ADO.NET, and – over time – SDO.

Finally, you should incorporate data access middleware in a comprehensive IT security strategy that covers both network security and database security, with secure communications and secure code. It should integrate with multiple solutions for authentication and authorization, too.

It’s important to choose best-of-breed data access middleware as a critical building block for any SOA initiative you undertake in your business (see sidebar). Data access is a fundamental building block of SOA, and if you fail to make good choices there, the entire services infrastructure will suffer for it. Fundamentally, regardless of the SOA infrastructure that runs above the data services layer, there’s no question that data access remains a key building block technology for SOA.

More Stories By John Goodson

As vice-president of product operations, John Goodson leads the product strategy, direction, and development efforts at DataDirect Technologies. For more than 10 years, he has worked closely with Sun and Microsoft on the development and evolution of database connectivity standards including J2EE, JDBC, .NET, ODBC, and ADO. His active memberships in various standards committees, including the JDBC Expert Group, have helped Goodson's team develop the most
technically advanced data connectivity technologies. He holds a BS in computer science from Virginia Tech.

More Stories By Jason Bloomberg

Jason Bloomberg is a leading IT industry analyst, Forbes contributor, keynote speaker, and globally recognized expert on multiple disruptive trends in enterprise technology and digital transformation. He is ranked #5 on Onalytica’s list of top Digital Transformation influencers for 2018 and #15 on Jax’s list of top DevOps influencers for 2017, the only person to appear on both lists.

As founder and president of Agile Digital Transformation analyst firm Intellyx, he advises, writes, and speaks on a diverse set of topics, including digital transformation, artificial intelligence, cloud computing, devops, big data/analytics, cybersecurity, blockchain/bitcoin/cryptocurrency, no-code/low-code platforms and tools, organizational transformation, internet of things, enterprise architecture, SD-WAN/SDX, mainframes, hybrid IT, and legacy transformation, among other topics.

Mr. Bloomberg’s articles in Forbes are often viewed by more than 100,000 readers. During his career, he has published over 1,200 articles (over 200 for Forbes alone), spoken at over 400 conferences and webinars, and he has been quoted in the press and blogosphere over 2,000 times.

Mr. Bloomberg is the author or coauthor of four books: The Agile Architecture Revolution (Wiley, 2013), Service Orient or Be Doomed! How Service Orientation Will Change Your Business (Wiley, 2006), XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996). His next book, Agile Digital Transformation, is due within the next year.

At SOA-focused industry analyst firm ZapThink from 2001 to 2013, Mr. Bloomberg created and delivered the Licensed ZapThink Architect (LZA) Service-Oriented Architecture (SOA) course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, which was acquired by Dovel Technologies in 2011.

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting), and several software and web development positions.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Microservices Articles
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
Consumer-driven contracts are an essential part of a mature microservice testing portfolio enabling independent service deployments. In this presentation we'll provide an overview of the tools, patterns and pain points we've seen when implementing contract testing in large development organizations.
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
"We do one of the best file systems in the world. We learned how to deal with Big Data many years ago and we implemented this knowledge into our software," explained Jakub Ratajczak, Business Development Manager at MooseFS, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.