Maybe I'm just a geek, but to me, in our ever growing, massively scaled enterprise computing landscape, there are few technologies that peak my interest like memory-based data grids. It is nothing short of amazing to see an increasing number of enterprises use these solutions in a myriad of ways, all to solve an old dilemma: How can one efficiently and cost-effectively scale data while preserving quality of service characteristics of the data such as performance, availability, consistency, and manageability? In my view, the use cases emerging from these solutions are among the most creative and intriguing that we see in the enterprise computing world today.

At a recent conference, I was chatting with two colleagues that work with customers interested in and implementing some of our IBM data grid technologies. I took away a couple of things from that chat. The first is that the adoption of data grid technologies is definitely on the upward swing. Enterprises are investigating various data grid solutions, and they are putting them to use in continually growing numbers. Some of this growing adoption probably owes to the increasing importance of elastic data tiers in cloud computing architecture. In other cases, enterprises have simply come to a tipping point where the current way they interact with their data is inefficient and costly.

The second takeaway from my chat was not as much about adoption as usage. Both of my colleagues, who I should say work with different users in different industries and geographies, agreed on a common emerging usage scenario. More and more enterprises are reducing their overall number of grids, and they are creating grids with a variety of data served up to numerous applications. From a management standpoint, this certainly makes sense. While effectively partitioning such diverse sets of data may be a bit more of a burden, the administrative efficiency gained by managing fewer grid instantiations probably makes it worthwhile (especially when you consider that many data grid technologies provide quite a bit of partitioning know-how).

In some ways, this natural evolution to fewer, more highly heterogeneous grids seems inevitable. In that sense, it is interesting to think about what this emerging usage pattern may mean to the evolution of data grid use within the enterprise. Specifically, what will enterprises start to do differently in adopting this type of data grid usage?

Now, I'm no data grid expert, and I did not stay at a Holiday Inn Express last night, but that won't keep me from giving my opinion on this matter. As I see it, the move to more centralized data grids will dictate the need for enterprises to more tightly align data grid and enterprise service bus technologies.

As data grid centralization increases, the number of different applications accessing a single grid also increases. It is conceivable, and in fact likely, that applications written in different languages and running on different platforms will need to access and perhaps modify the same data within the grid. The data as it is stored in the grid may not be an ideal format for all of the applications. Certainly, the applications could retrieve the data in some sort of least common denominator form (i.e. XML) and do the appropriate transformations. However, that leaves you with application code that does data transformation work before it actually works on the data. Clearly, this is not ideal. Instead, enterprises may benefit from using an ESB to transform the data returned from the grid to a format more desirable to the requesting application. This moves transformation code out of the application, where I think we all agree it does not belong, to the ESB.

In addition to the data transformation capabilities ESB solutions afford, they also typically provide protocol transformation capabilities. This may come in handy as the number of applications, and presumably protocol access methods, accessing a single grid increases. Instead of requiring a data grid to accommodate untold number of access protocols, the ESB could intercept various different types of request, and then send that request along to the data grid using a supported protocol. While data grid support for commonly used protocols like HTTP may mitigate this need to some degree, I still believe this will afford enterprises and their applications a nice degree of flexibility.

Now, I am not putting my head in the sand about some of the effects of using data grid and ESB technologies in tandem. Chief among the concerns would be the fact that an ESB between applications and the data grid would certainly introduce some degree of latency. In many cases, users leverage grids for extremely fast access to data, so this pattern may not be ideal. However, in other cases the value provided by the ESB/data grid combination may outweigh any latency concerns. So, what do you think? Am I way off base here, or is there value in architectural patterns that combine ESB and memory-based data grid solutions?

• Tangosol Extends Data Grid Access throughout Enterprise
• Delta Air Lines Selects Tangosol Coherence Data Grid
• Tangosol Announces Results of Data Grid Scalability Benchmark Effort With Intel and IBM