|By Rickland Hollar||
|August 19, 2003 12:00 AM EDT||
The Internet makes it possible to deliver information almost instantaneously - anytime, anywhere - and is redefining the traditional boundaries around organizations and their IT systems. The Internet has turned buyers into sellers, sellers into buyers, and set new expectations for how services should be delivered. These expectations raise the bar for applications in terms of their need for interconnectivity and responsiveness. For businesses to remain competitive in this environment - or in the case of government agencies, responsive - they must embrace the idea that speed not only matters, but that it is now a key discriminator. Enterprises able to leverage the Internet's real-time nature and its technologies create competitive advantages that let them reap the benefits of greater efficiency, responsiveness, market share, and profitability. This promise has led businesses to look to interconnect their enterprise resource management (ERM), supply chain management (SCM), and customer relationship management (CRM) systems, both internally and externally; and government agencies to look for better ways to connect their systems with the public, their suppliers, and each other.
The Gartner Group coined the term "zero latency enterprise (ZLE)" to describe organizations that can exchange information with employees, trading partners and customers in near real time). The original focus was on internal systems, but much of what Gartner said applies equally to eliminating latency between internal and external systems. Figure 1 illustrates many of the concepts underpinning ZLE. In a ZLE organization, business events trigger system events that post actions and send responses throughout the enterprise. Bill Gates calls this level of interconnectivity a "digital nervous system." Like the human nervous system, the applications in a ZLE organization interconnect in such a way that they eliminate latency, which is the time gap between when the system receives information at one point and uses it, wherever needed, at others.
Becoming a ZLE
Transforming your organization into a ZLE is a three-phase process. You must understand where latency exists within your current processes and systems and your options for reducing or eliminating it. You must then create an architecture that focuses on minimizing latency along the critical path of as many business processes as possible. Finally, you must translate the architecture into an implementation plan that provides the roadmap for yours becoming a ZLE organization.
The first step in becoming a ZLE organization is identifying the major business processes within your organization that the ZLE architecture must support. One goal at this stage is to establish the architectural boundaries of the effort; will it deal with internal systems, external systems, or both? Another goal is to understand the dynamics of each business process: its tempo, meter, natural pauses and breaks. Understanding these dynamics is critical to recognizing latency and bounding the parameters for fixing it. It is important to remember that what constitutes latency in one process may be completely acceptable in another, even for the same application.
The next step is to decompose each business process into its applications and identify any latency points that exist. You should ask: What applications make up this process? Is latency a problem in either the overall process or one or more of its supporting applications? If so, how much of a problem? How much does its timing need to change? The result of asking and answering these questions is a list of business processes and applications that have latency problems.
Next you need to learn as much as possible about each latency point so that you can later devise techniques for removing, or at least minimizing, the latency. For internal systems, latency stems from several root causes (see Figure 2). Legacy systems are often stovepiped applications that were developed independently, over time, using different technologies. These applications create islands of information and functionality that are by their very nature difficult to integrate and share.
The same data belonging to different applications may be in different formats, follow different data validation and business rules, or be updated through completely different business processes. Interfaces within these older applications tend to be synchronous, tightly coupled, and driven more by the underlying technologies than the business needs they serve. Proprietary drivers, proprietary APIs, and proprietary formats represent only the tip of the iceberg when it comes to tying these systems together. One question you should ask is: What are each application's processing characteristics: batch, on-demand, or continuously running? Some may be batch oriented where you need them to be real time, others may have availability and reliability problems in cases where you need them to be 24x7. These issues frequently reflect age and technology differences that increase the difficulties in creating a coherent architecture.
Latency's causes multiply when you look at connecting internal and external systems. Each external system potentially represents a different set of technology, security, reliability, and manageability characteristics that your architecture must address.
At the end of the first phase, you should have a good understanding of your organization's internal and external business processes and the latency points you need to address within each. You're now ready to lay out the major business processes and applications and begin developing an overall ZLE architecture. It's important that your architecture address four key elements: business process management, data communications and routing, data transformation and formatting, and applications connectivity.
Business process management is, in my opinion, the most important part of the architecture; it's the glue that ties applications together. It should reflect the enterprise's business processes: assembling, sequencing, and orchestrating applications to align them with the business's natural processes and work flows. A workflow manager, a rules engine, and collaborative tools can be critical components at this level. Employees, business partners, and customers should find easy-to-use, intuitive interfaces supporting your core business processes.
Data communications and routing in conjunction with business process management create the central nervous system for the ZLE architecture. Two fundamental architectures, shown in Figure 3, have evolved in this area: hub-and-spoke; and data, or information, bus. The hub-and-spoke architecture uses a central integration engine and message queuing products, such as MQSeries and MSMQ, to integrate across applications. In this architecture, applications deal with one another through the central hub; this is responsible for extracting, transforming, and routing data and coordinating activities throughout the overall system.
The information bus architecture takes a decentralized approach. This architecture implements a common messaging framework, frequently using a publish and subscribe model, for intercommunication. Applications connect to this bus through application adapters and pass messages to one another by placing them onto the bus. The information bus may use either a messaging or workflow manager to assist in routing messages. You can use either architecture internally; the information bus is clearly superior when connecting between internal and external systems.
XML has become the lingua franca for solving the data transformation and formatting problem. It provides a flexible, extensible syntax for expressing both information and its structure in a meaningful format. Legacy applications can apply Extensible Stylesheet Language Transformations (XSLTs) to XML documents to convert information within those documents into whatever format they need. Data transfer and replication tools are also available for extracting, transforming, cleansing, and loading data for those wanting to make minimum modifications to existing applications.
Application integration can occur at many different levels (see Figure 4). A key question is whether there is overlap in the data the applications process or the business rules they enforce. User interface integration integrates applications at the presentation layer. This level of integration is valuable for connecting independent applications into common business processes. Data integration integrates applications at the database level by copying, transferring, or replicating information from one data source to another. This is a good strategy when transfers are timely and business rules are sufficiently compatible. Business logic integration integrates applications' middle tiers, allowing each application to retain its original business rules and logic. This level of integration works best for tying existing, interdependent applications together into more streamlined processes. Component integration integrates applications through their application programming interfaces (APIs), common components, or function calls. Integration at this level may require you to write proxy interfaces for some components; change call interfaces from direct to RPC for others; or adopt a distributed object model such as DCOM, CORBA, or Web services. This integration form is most useful for creating components several applications or processes can share.
A critical part of the application integration analysis is looking closely at each latency point to determine both the level of integration and corrections needed. The first step is to identify the appropriate integration level for each application: presentation, business logic, or data. Simply changing the application's invocation characteristics may be enough to also change its latency characteristics for some applications.
In situations where that is not the case, the next step is to drill down into the application and its interfaces with an eye towards improving the application's performance characteristics. The first, and simplest, corrective measure is to identify and remove any inefficiencies or chokepoints within the application. A second option is to look at overlapping the application's processing with that of others by making it an asynchronous process. Making an application asynchronous is straight-forward; you simply need to add a queue and alerting and rendezvous mechanisms. This can also be a good approach for dealing with reliability and availability problems caused by older systems. A third, and sometimes only, option is to redesign and rewrite the application.
You may need to make several passes through each of the four architectural elements to finalize the ZLE architecture. That isn't unusual. It's important that you come away with an overall architectural strategy, a list of integration points, and an idea of the integration strategies you'll need to address as part of the implementation process, which is the next step. Before proceeding to the implementation phase, it's a good idea to create a set of guiding principles to help in making architectural tradeoffs and selecting products. Questions you should answer include: Are the number or types of products you use of concern? How about the amount of code you write? Is it important to use the same solution for solving the latency problem between both internal and external applications? Is it important to use the same integration solution for integration points at the same level? Do you have large investments in ERM, SCM, or CRM solutions that will drive the implementation? With the answers to these questions in hand, you're now ready to look at options for implementing the architecture.
Web services provides a lightweight, standards-based solution for implementing a ZLE architecture. Web services offers an integration model that brings applications together as loosely coupled components within a larger architectural framework. This standards-based framework closely aligns to the four elements in the ZLE architecture (see Figure 5). Business Process Execution Language for Web Services (BPEL4WS) and WS-Choreography are standards proposals for modeling, defining, orchestrating, and implementing business processes. WS-Transaction and WS-Security supply protocols for implementing atomic and business transactions, and security features such as authentication and encryption that are necessary for tying applications together into new business processes. The Simple Object Access Protocol (SOAP), HTTP, and TCP/IP create the backbone for data communications. WS-Routing, and WS-Referral address the data routing problem.
XML, which is the heart of Web services, provides a standard for data representation. XSLT adds a language for data transformation and formatting. SOAP-RPC contributes a lightweight, standards-based, platform-independent component model for implementing distributed components. In short, Web services provides all the elements necessary to implement whatever ZLE architecture you ultimately develop. With several of the standards still evolving, the issue is that products lag behind standards; that means you have to write more code.
If that is a concern, off-the-shelf enterprise application integration (EAI) products offer a good foundation for moving towards a ZLE organization. EAI products provide message broker and adapter technologies that quickly integrate applications to exchange and share information at the data, business logic, or presentation layers. Most EAI solutions implement either a message broker or bus concept corresponding to the hub-and-spoke and information bus architectures. If you decide this is the best approach for you, choose a product that fits into your overall integration strategy by providing the greatest number of integration adapters corresponding to the integration levels, points, and products you identified as part of your analysis.
EAI and Web services are extremely powerful together - EAI for fine-grained interfaces, Web services for coarse-grained interfaces. Many EAI vendors, such as SeeBeyond, TIBCO, webMethods, and IBM, recognize this synergy and offer products that are in fact a marriage between traditional EAI technologies and Web services. These products give you a best of both worlds option. Ultimately, the question boils down to which strategy works best with your architecture within your organization.
The costs of not becoming a ZLE organization are high; they translate to frustrated customers, disappointed partners, and missed opportunities. The challenges are in understanding critical business processes and developing an architecture that removes the problems creating latency both in the enterprise's internal systems and in their connections to systems be-longing to trading partners and customers. Web services standards, which EAI products are rapidly adapting, lay out the framework you need for implementing this architecture. As more companies adopt them, low cost, standards-based solutions for implementing ZLE applications may finally become a reality.
|Wil Marshman 09/08/03 05:17:13 PM EDT|
We [HP's NonStop Enterprise Division (NED), aka Tandem] picked up on ZLE several years ago and have produced a framework [as a product/solution] to facilitate customers doing what Holler describes. The Real Time Enterprise is an idea whose time has come [I assert that certain data needs to be accessible in real time for an enterprise to function effectively and many of our IT systems are not amenable to the changes required, mostly due to the silos that enterprises have built].
My point is that there are companies who can make the journey that Mr. Hollar describes easier. HP's ZLE implementation is ahead of the game.
One of the ways to increase scalability of services – and applications – is to go “stateless.” The reasons for this are many, but in general by eliminating the mapping between a single client and a single app or service instance you eliminate the need for resources to manage state in the app (overhead) and improve the distributability (I can make up words if I want) of requests across a pool of instances. The latter occurs because sessions don’t need to hang out and consume resources that could ...
Aug. 1, 2015 04:00 PM EDT Reads: 201
You often hear the two titles of "DevOps" and "Immutable Infrastructure" used independently. In his session at DevOps Summit, John Willis, Technical Evangelist for Docker, covered the union between the two topics and why this is important. He provided an overview of Immutable Infrastructure then showed how an Immutable Continuous Delivery pipeline can be applied as a best practice for "DevOps." He ended the session with some interesting case study examples.
Aug. 1, 2015 03:30 PM EDT Reads: 205
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin,...
Aug. 1, 2015 03:00 PM EDT Reads: 505
Approved this February by the Internet Engineering Task Force (IETF), HTTP/2 is the first major update to HTTP since 1999, when HTTP/1.1 was standardized. Designed with performance in mind, one of the biggest goals of HTTP/2 implementation is to decrease latency while maintaining a high-level compatibility with HTTP/1.1. Though not all testing activities will be impacted by the new protocol, it's important for testers to be aware of any changes moving forward.
Aug. 1, 2015 02:15 PM EDT Reads: 178
Aug. 1, 2015 02:00 PM EDT Reads: 300
Alibaba, the world’s largest ecommerce provider, has pumped over a $1 billion into its subsidiary, Aliya, a cloud services provider. This is perhaps one of the biggest moments in the global Cloud Wars that signals the entry of China into the main arena. Here is why this matters. The cloud industry worldwide is being propelled into fast growth by tremendous demand for cloud computing services. Cloud, which is highly scalable and offers low investment and high computational capabilities to end us...
Aug. 1, 2015 01:30 PM EDT Reads: 149
[slides] Storage for Docker Containers By @OnModulus | @DevOpsSummit #DevOps #Docker #Containers #Microservices
Learn how to solve the problem of keeping files in sync between multiple Docker containers. In his session at 16th Cloud Expo, Aaron Brongersma, Senior Infrastructure Engineer at Modulus, discussed using rsync, GlusterFS, EBS and Bit Torrent Sync. He broke down the tools that are needed to help create a seamless user experience. In the end, can we have an environment where we can easily move Docker containers, servers, and volumes without impacting our applications? He shared his results so yo...
Jul. 31, 2015 11:45 PM EDT Reads: 787
Modern DevOps Tool Kit By @Logentries and @NewRelic | @DevOpsSummit #DevOps #Containers #Microservices
Auto-scaling environments, micro-service architectures and globally-distributed teams are just three common examples of why organizations today need automation and interoperability more than ever. But is interoperability something we simply start doing, or does it require a reexamination of our processes? And can we really improve our processes without first making interoperability a requirement for how we choose our tools?
Jul. 31, 2015 11:15 PM EDT Reads: 426
Cloud Migration Management (CMM) refers to the best practices for planning and managing migration of IT systems from a legacy platform to a Cloud Provider through a combination professional services consulting and software tools. A Cloud migration project can be a relatively simple exercise, where applications are migrated ‘as is’, to gain benefits such as elastic capacity and utility pricing, but without making any changes to the application architecture, software development methods or busine...
Jul. 31, 2015 10:00 PM EDT Reads: 1,352
The Internet of Things. Cloud. Big Data. Real-Time Analytics. To those who do not quite understand what these phrases mean (and let’s be honest, that’s likely to be a large portion of the world), words like “IoT” and “Big Data” are just buzzwords. The truth is, the Internet of Things encompasses much more than jargon and predictions of connected devices. According to Parker Trewin, Senior Director of Content and Communications of Aria Systems, “IoT is big news because it ups the ante: Reach out ...
Jul. 31, 2015 07:00 AM EDT Reads: 409
Where the Network Got Invited to the Party By @LMacVittie | @DevOpsSummit #DevOps #Docker #Containers #Microservices
At DevOps Summit NY there’s been a whole lot of talk about not just DevOps, but containers, IoT, and microservices. Sessions focused not just on the cultural shift needed to grow at scale with a DevOps approach, but also made sure to include the network ”plumbing” needed to ensure success as applications decompose into the microservice architectures enabling rapid growth and support for the Internet of (Every)Things.
Jul. 30, 2015 08:15 PM EDT Reads: 1,775
Designing the IT Architecture of the Future with Adrian Cockcroft | @DevOpsSummit #DevOps #Docker #Containers #Microservices
Our guest on the podcast this week is Adrian Cockcroft, Technology Fellow at Battery Ventures. We discuss what makes Docker and Netflix highly successful, especially through their use of well-designed IT architecture and DevOps.
Jul. 30, 2015 08:00 PM EDT Reads: 787
[slides] A New Architecture for the Internet of Things By @JKirklan | @ThingsExpo @RedHatNews #IoT #M2M #InternetOfThings
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Arch...
Jul. 30, 2015 07:30 PM EDT Reads: 1,406
This week, I joined SOASTA as Senior Vice President of Performance Analytics. Given my background in cloud computing and distributed systems operations — you may have read my blogs on CNET or GigaOm — this may surprise you, but I want to explain why this is the perfect time to take on this opportunity with this team. In fact, that’s probably the best way to break this down. To explain why I’d leave the world of infrastructure and code for the world of data and analytics, let’s explore the timing...
Jul. 30, 2015 05:45 PM EDT Reads: 385
Take the Long View with Digital Transformation By @IoT2040 | @ThingsExpo #IoT #M2M #API #Microservices #InternetOfThings
Digital Transformation is the ultimate goal of cloud computing and related initiatives. The phrase is certainly not a precise one, and as subject to hand-waving and distortion as any high-falutin' terminology in the world of information technology. Yet it is an excellent choice of words to describe what enterprise IT—and by extension, organizations in general—should be working to achieve. Digital Transformation means: handling all the data types being found and created in the organizat...
Jul. 30, 2015 05:00 PM EDT Reads: 1,096
[slides] Workloads and Public Cloud at @CloudExpo By @utollwi | @ProfitBricksUSA #DevOps #Containers #Microservices
Public Cloud IaaS started its life in the developer and startup communities and has grown rapidly to a $20B+ industry, but it still pales in comparison to how much is spent worldwide on IT: $3.6 trillion. In fact, there are 8.6 million data centers worldwide, the reality is many small and medium sized business have server closets and colocation footprints filled with servers and storage gear. While on-premise environment virtualization may have peaked at 75%, the Public Cloud has lagged in adop...
Jul. 30, 2015 04:00 PM EDT Reads: 2,216
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
Jul. 30, 2015 03:45 PM EDT Reads: 464
MuleSoft has announced the findings of its 2015 Connectivity Benchmark Report on the adoption and business impact of APIs. The findings suggest traditional businesses are quickly evolving into "composable enterprises" built out of hundreds of connected software services, applications and devices. Most are embracing the Internet of Things (IoT) and microservices technologies like Docker. A majority are integrating wearables, like smart watches, and more than half plan to generate revenue with ...
Jul. 30, 2015 02:30 PM EDT Reads: 117
[session] DevOps State of Mind By @RedHatNews | @DevOpsSummit #DevOps #PaaS #Jenkins #Kubernetes #Docker
Rapid innovation, changing business landscapes, and new IT demands force businesses to make changes quickly. The DevOps approach is a way to increase business agility through collaboration, communication, and integration across different teams in the IT organization. In his session at DevOps Summit, Chris Van Tuin, Chief Technologist for the Western US at Red Hat, will discuss: The acceleration of application delivery for the business with DevOps
Jul. 30, 2015 12:45 PM EDT Reads: 1,125
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Opening Keynote at 16th Cloud Expo, S...
Jul. 30, 2015 12:00 PM EDT Reads: 2,063