Microservices Expo Authors: Liz McMillan, Zakia Bouachraoui, Elizabeth White, Pat Romanski, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: Article

Best Practices for SOA: Building a Data Services Layer

Choosing best-of-breed data access middleware is an important enabler of SOA

The Importance of Best-of-Breed Data Access in Building a Data Services Layer
Ask just about any business executive about the core value IT provides to the business, and they’re likely to respond that data (in the form of useful information) rather than applications form the lifeblood of their business. In essence they’re correct; information technology is by definition all about information. But actually the two – data and applications – can only be separated in theory, not in fact. Data by itself is often inaccessible and/or unintelligible without the applications that process it, and most applications in turn serve no real business purpose without data.

While establishing a data services layer is essential if you want to fully apply SOA best practices to data, it’s equally important to realize that the foundation of that data services layer consists of data access. Data access, in turn, depends on standards such as ODBC, JDBC, ADO.NET, and SDO. Even if you’re using a persistence layer like one built with the open source Hibernate toolkit, for example, high-quality data access middleware is critical. A slow JDBC driver under Hibernate or a slow ADO.NET provider under Microsoft’s Language Integrated Query (LINQ) will inevitably impede the services. The results can be far reaching. Accessing the data in the various data stores across your organization in the most efficient, flexible manner is a core reason for building data services. It’s therefore critical for building all the services in your SOA implementation.

Performance speed is not the least of the issues that can be affected by data access middleware. And, because the data services layer essentially virtualizes data access, those services can hide a plethora of potential data access pitfalls. Other such pitfalls can include:

  • Scalability issues
  • Database platform, application, and version incompatibilities
  • Differences in how the various data sources handle the details of standard data access operations such as create, read, search, update, and delete
  • Data source security priorities, which might vary from database to database, table to table, or even row to row
  • Data mapping challenges arising from semantic differences among heterogeneous data
  • Problems arising from the mixture of structured and unstructured data — such as file formatting issues
  • Differences in versions of SQL

SOA can actually serve to magnify limitations in scalability, performance, and interoperability imposed by data access because organizations are both more likely to reuse the underlying code and leverage existing data services across many services and composite applications.

Your first step then in resolving these data-related issues is to build shared, centralized data services, which will result in an adaptable and easily maintained SOA implementation. This way data access logic only appears in one place, no matter how many applications consume it. The result is that, instead of scattering data-related services invocation code throughout each business service, centralized data access helps you create an environment that enables a best-of-breed data access solution to address all those data access issues.

This last point – the use of best-of-breed access solutions – can’t be stressed strongly enough. Even many people involved in data management rarely give data access middleware more than a passing thought. One reason for this is that commercial databases all include connectivity drivers bundled with the database software, which are often used by default. The fact that these “free” drivers may be less than optimal for a given IT environment typically doesn’t come up unless and until either a technical or a maintenance issue becomes serious enough to be traced back to the data connectivity layer. For reasons we’ve described, such issues can be compounded in an SOA and can also be extremely difficult to diagnose.

Quite simply, it is ill advised to rely on database-bundled data access middleware for your data services layer in an enterprise SOA. The same holds true for freely available open source drivers. As the very foundation of your data services layer, the type of data access you use deserves serious consideration and careful planning. Dynamic environments where multiple services reuse data access code and new services regularly go into production require that such code meet rigorous requirements.

Your best bet is to go with third-party data access middleware, preferably from a supplier whose core business and expertise consists of data connectivity. Look for specific attributes for your data access middleware, including capabilities for boosting query performance. Such attributes can include connection pooling as well as support for tunable data access performance such as adjusting network packet size. For scalability and high availability, data access middleware should be multithreaded and thread-safe, and offer client load balancing and failover to alternate servers.

It’s also important for data access middleware to support different types and versions of databases as well as all the subtle variations in SQL they support. Such support for heterogeneity should also extend to multiple computing platforms, chipsets, and operating systems. In addition, for the best performance and flexibility, data access middleware should offer wire protocol drivers to avoid the overhead and maintenance issues of off-the-shelf database drivers. Wire protocol drivers obviate the need for database client software and libraries, simplifying installation and administration as well as offering a much more efficient and high-performing operation. Such drivers must support the full panoply of relevant standards, including JDBC, ODBC, ADO.NET, and – over time – SDO.

Finally, you should incorporate data access middleware in a comprehensive IT security strategy that covers both network security and database security, with secure communications and secure code. It should integrate with multiple solutions for authentication and authorization, too.

It’s important to choose best-of-breed data access middleware as a critical building block for any SOA initiative you undertake in your business (see sidebar). Data access is a fundamental building block of SOA, and if you fail to make good choices there, the entire services infrastructure will suffer for it. Fundamentally, regardless of the SOA infrastructure that runs above the data services layer, there’s no question that data access remains a key building block technology for SOA.

More Stories By John Goodson

As vice-president of product operations, John Goodson leads the product strategy, direction, and development efforts at DataDirect Technologies. For more than 10 years, he has worked closely with Sun and Microsoft on the development and evolution of database connectivity standards including J2EE, JDBC, .NET, ODBC, and ADO. His active memberships in various standards committees, including the JDBC Expert Group, have helped Goodson's team develop the most
technically advanced data connectivity technologies. He holds a BS in computer science from Virginia Tech.

More Stories By Jason Bloomberg

Jason Bloomberg is a leading IT industry analyst, Forbes contributor, keynote speaker, and globally recognized expert on multiple disruptive trends in enterprise technology and digital transformation. He is ranked #5 on Onalytica’s list of top Digital Transformation influencers for 2018 and #15 on Jax’s list of top DevOps influencers for 2017, the only person to appear on both lists.

As founder and president of Agile Digital Transformation analyst firm Intellyx, he advises, writes, and speaks on a diverse set of topics, including digital transformation, artificial intelligence, cloud computing, devops, big data/analytics, cybersecurity, blockchain/bitcoin/cryptocurrency, no-code/low-code platforms and tools, organizational transformation, internet of things, enterprise architecture, SD-WAN/SDX, mainframes, hybrid IT, and legacy transformation, among other topics.

Mr. Bloomberg’s articles in Forbes are often viewed by more than 100,000 readers. During his career, he has published over 1,200 articles (over 200 for Forbes alone), spoken at over 400 conferences and webinars, and he has been quoted in the press and blogosphere over 2,000 times.

Mr. Bloomberg is the author or coauthor of four books: The Agile Architecture Revolution (Wiley, 2013), Service Orient or Be Doomed! How Service Orientation Will Change Your Business (Wiley, 2006), XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996). His next book, Agile Digital Transformation, is due within the next year.

At SOA-focused industry analyst firm ZapThink from 2001 to 2013, Mr. Bloomberg created and delivered the Licensed ZapThink Architect (LZA) Service-Oriented Architecture (SOA) course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, which was acquired by Dovel Technologies in 2011.

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting), and several software and web development positions.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Microservices Articles
The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
When building large, cloud-based applications that operate at a high scale, it’s important to maintain a high availability and resilience to failures. In order to do that, you must be tolerant of failures, even in light of failures in other areas of your application. “Fly two mistakes high” is an old adage in the radio control airplane hobby. It means, fly high enough so that if you make a mistake, you can continue flying with room to still make mistakes. In his session at 18th Cloud Expo, Lee A...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addresse...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...