|By Michael Poulin||
|May 13, 2006 11:00 AM EDT||
What could be easier than to take your application, wrap it with a Web Service, announce it or register it in the UDDI and get a SOA Service? Even better - take a data warehouse, cover a SQL executing code with a Web Service and expose it to SOA, isn't it simple? This article is for those architects and managers who like such "simplicity." If you believe that a Web Service itself doesn't convert an application into the SOA Service, you might read the article just out of curiosity.
A Service or Not a Service
Discussions about Service Oriented Architecture (SOA) initiated by people working with legacy applications and data storages like Data Warehouse (DW) have gotten a lot of press attention recently. While it's good for SOA's popularity the discussions typically declare a few SOA characteristics and say something like "We have so rich/important/business crucial data and, if we just expose it into a SOA, it will be great opportunity for us." An intention of exposing DW to a SOA sounds suspiciously like the five-year-old movement of exposing all applications to the Internet. Have we forgotten that mistake already? Have we understood the difference between Web applications and Web-enabled applications? Did anybody count the man-centuries spent in IT to convert and modify existing applications to make them really work in the Internet environment?
When I participate in such discussions, I usually ask one question: "To make some ground for DW working in SOA, please tell me what is a service in DW?" I am given two types of answers: either "none" or "We do a lot of data transformations and/or perform sophisticated data aggregation and produce business intelligence reports out of the DW." Well, data manipulation procedures performed in the DW might be considered as services, however, I have not found a definition of DW that would describe a service as a part of it. That is, DW wasn't designed for services.
Anyway, what's so special about a "service"? Why is wrapping access to a DW or an application with a Web Service not enough to obtain a service? What has to be done, if anything, to make such applications work in a SOA? Here I'll try to identify a process that could take place when one prepares a legacy application to support a Service for SOA.
Approach to a Definition of a "Service"
In different spheres of human activity, there may be many definitions of a service. My understanding of service behavior with regard to software components is an action/activity performed by the service provider for the service consumer in accordance to a contract between the provider and the consumer. While contracts may vary, the most flexible type unties/decouples/isolates a consumer from a provider. In the business, as we know, the consumer pays for the service and tends to consider a service provider a servant. This leads to two conclusions:
- a consumer looks for a provider with the contract that meets the consumer's goals.
- a consumer can agree with some of the constraints a contract puts on him if the contract still meets the goals and decouples the consumer from the provider's internal conditions.
Jumping ahead let me say that one SOA service can engage another SOA service doing this transparently to the consumer. If the second service provides the proper data based on an "update schedule", i.e., the data are obsolete for some period of time, the first service tends either to find a substitute service with the correct data for that period of time or switch to a totally different service, without any "schedule problems." In any case, the consumer shouldn't depend on that schedule. Otherwise, all service contracts have to reflect one service specifics, which leads to quite an insufficient architecture and, actually, couples services and consumer together. Described is not a SOA rule, it's a business rule and due to SOA agility principle, SOA promotes the same behavior.
Thus, the service contract can be used as a requirement when one transforms the application into a service. A well-known expression of a contract is a Service Level Agreement (SLA). There may be a single SLA for all consumers or the provider can maintain individual SLAs with every consumer. If we took a typical SLA used for DW, for instance, and a SLA used in SOA, we'd get the starting and ending points for the process we have to implement when exposing a DW to a SOA.
The process includes not only a new connectivity interface but changes in behavior dictated by the Service SLA. For immutable applications, the process assumes building a service façade layer to interact with other services and consumers.
Service Interface in SOA
SOA implies certain requirements in the service interface. In short, the interface has to be:
3) independent from provider implementation and its resources
5) accessible programmatically
7) accessible under control (security)
As you see, there's nothing about the Web in the requirements. Indeed, any particular interface implementation is suitable if it meets the requirements. It may be, for example, CORBA IDL. As we know, many elements of Web Services specifications were derived from CORBA. So, why is a Web Service (WS) as an interface so attractive for SOA but still not enough?
Web Services technology defines the abstraction of service behavior via standard WS Description Language (WSDL). It also provides for the abstraction of data via XML and the metadata definition via XML Schema. WS technology offers a WS registry known as UDDI (Universal Description, Discovery, and Integration) and a set of standards to support security ( WS-Security and related standards), transaction (WS-TXM, WS-Coordination, WS-AtomicTransaction, and WS-BusinessActivity), aggregation and management (WS-BPEL and WS-CDL), and interoperability (WS-I Basic Profile).
Agreement between leading technology vendors on XML and most WS standards makes Web Services really the best candidate for a universal interface for SOA. Some specialists include SOAP (Simple Object Access Protocol) in the basis of SOA while others disagree with it and consider SOAP only one of several possible and convenient protocols for WS binding. SOAP alone simply doesn't provide enough abstraction for a service definition, that's why we need WSDL and we already know a few WS models that don't use SOAP binding.
What's not included in WSDL is the information considered by a consumer when he makes a business decision about the service. That is, you can automatically invoke a WS but you're not supposed to do so blindly, without estimating the inherited risk. It's simple: if you want to cook some French fries, you'd not use a pan without checking if it burns oil. The same relates to WS - the service may be accepted if you know its connectivity interface and its quality characteristics like performance, change control rules, error-handling scenarios, and so on. This information is usually described in the SLA. So a SLA can be treated as a container that includes business knowledge and service interface definition - both to meet the consumer's needs. A lot of legacy systems also work on the basis of SLA, however, those SLA are traditionally provider/application-centric.
Service Level Agreement for SOA Service
Currently, some efforts are made in the industry to define and formalize SLA for WS - these are specifications for Web Service Level Agreements (WSLA) and the Web Service Offerings Language (WSOL). The WSLA provides a structure of definitions of basic SLA elements. While it is a subject for separate article, we note the fact that the WSLA pairs SLA parameters with their metrics. That is, providing for SOA SLA means constant monitoring, analysis, and compliance reporting of actual runtime parameter values that aren't a part of most legacy application SLAs.
Though a SOA SLA isn't standardized yet, it's very important to understand what can be included in it. Table 1 gives an example of a SOA SLA. Some parameters are measurable (as represented in WSLA) and some of them aren't.
There are no mandatory or optional parameters in the SLA because they are all business- and context-specific. As mentioned already, neither Web Service/WSDL nor any other programming interface can address such SLAs. On the other hand, not all services require such detailed SLAs. Its can be periodically refined and extended when more objective information is gathered about the Service. The SLA demonstrates that participation in SOA requires both - the service interface and service behavior.
The only other thing I'd like to note is that it's better for maintenance and further enhancements if all services in your system have a unified SLA, especially when the number of Services is expected to grow over the time. Otherwise, managing the Services quickly gets out of hands. That is, the better you define the Service, the fewer problems you have at runtime and the more consumers you attract to use it.
Thinking in SOA
We can find tons of publications describing how to wrap a legacy application with a Web Service. Many serious works recommend the same solution - the most scalable and flexible way of wrapping is to create a thin layer on top of the legacy application. The "preserve-and-extend" approach has gained a reputation as being the most reliable and cost-effective model for dealing with business-critical legacy apps.
Some vendors propose developing connectors and gateways where the former run in a mainframe and latter outside the mainframe (screen-scraping). Both models act as proxies shielding actual applications and helping to provide SLA. Merrill Lynch, in particular, has created an integration platform using plug-ins with parsers, metadata assemblies, and drivers that runs on the mainframe and exposes legacy apps through Web Services via HTTP and MOM protocols. This platform is even called a Service Oriented Legacy Architecture. However, many experts suggest that transitioning a legacy application to a Service for a SOA isn't that straightforward: simple wrapping just "preserves" the application but doesn't make it a "first-class SOA citizen" without an "extension."
Let me give you two examples. The first one is about a traditional application in the financial industry; it may be considered a base line for the second example. Assume that the task is to calculate the credit risk of swap transactions (a special type of financial transactions). The transactions comprise the transaction data themselves, related financial streams, and cash flow data. These are dynamic elements that usually persist in an operational data store. Other related data about supply feeds and financial clients are static and usually persist in a reference data store.
The traditional application-centric approach of building a credit risk calculation component addresses the access interface(s)-access protocol(s) and defines data location, the data access mechanism (direct SQL, Stored Procedures, Views, security controls, etc.) and the data availability schedule. The team developing the application has to deal with its consumers (dictating application constraints) and constantly negotiate data status and updates with the database maintenance team. It leads to per-application specialized code that has to transform data to meet particular application needs. Any change in the data affects the application and all its consumers (via production re-deployment, at least). Such complex daily management task may be sufficiently executed if the team has enough resources and deals with only a few applications but this is a dream nowadays. A more realistic consequence is a shortage of resources and degraded quality, as well we know. And, there's no room left to support business growth.
A service-oriented approach defines the credit risk calculation engine and the metadata that the engine can work with. That is, the engine knows about different calculation formulas and intermediary data storages, if needed; it also knows how to deal with data if it meets metadata requirements. That's it. The consumer of the calculation service specifies what calculation to perform - for intra-day or end-of-day transactions. The input data is provided by another service, e.g., the Data Access Service (DAS), which also knows where to place calculated results. The calculation engine hasn't a clue where to get data from, what transactions to process, when to do the job; it only cares whether the DAS provides a SLA and acts appropriately if the SLA is violated. Now every development and support team can concentrate on its own business and provide related SLAs.
One of the biggest challenges when developing connected devices is identifying user value and delivering it through successful user experiences. In his session at Internet of @ThingsExpo, Mike Kuniavsky, Principal Scientist, Innovation Services at PARC, described an IoT-specific approach to user experience design that combines approaches from interaction design, industrial design and service design to create experiences that go beyond simple connected gadgets to create lasting, multi-device experiences grounded in people's real needs and desires.
Nov. 27, 2014 08:00 AM EST Reads: 1,078
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
Nov. 27, 2014 07:45 AM EST Reads: 1,388
The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets. According to a recent IDG Research Services Survey this rate of traffic will only grow. What's driving t...
Nov. 27, 2014 07:00 AM EST Reads: 1,376
Connected devices and the Internet of Things are getting significant momentum in 2014. In his session at Internet of @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, examined three key elements that together will drive mass adoption of the IoT before the end of 2015. The first element is the recent advent of robust open source protocols (like AllJoyn and WebRTC) that facilitate M2M communication. The second is broad availability of flexible, cost-effective storage designed to handle the massive surge in back-end data in a world where timely analytics is e...
Nov. 27, 2014 06:45 AM EST Reads: 1,175
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
Nov. 27, 2014 06:45 AM EST Reads: 1,289
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 27, 2014 04:00 AM EST Reads: 1,039
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will want to use their existing identities, but these will have credentials already that are (hopefully) i...
Nov. 27, 2014 04:00 AM EST Reads: 1,087
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles ...
Nov. 26, 2014 11:30 PM EST Reads: 1,077
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges. In his session at @ThingsExpo, Jeff Kaplan, Managing Director of THINKstrategies, will examine why IT must finally fulfill its role in support of its SBUs or face a new round of...
Nov. 26, 2014 09:00 PM EST Reads: 1,096
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...
Nov. 26, 2014 06:00 PM EST Reads: 1,088
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
Nov. 26, 2014 04:00 PM EST Reads: 1,132
P2P RTC will impact the landscape of communications, shifting from traditional telephony style communications models to OTT (Over-The-Top) cloud assisted & PaaS (Platform as a Service) communication services. The P2P shift will impact many areas of our lives, from mobile communication, human interactive web services, RTC and telephony infrastructure, user federation, security and privacy implications, business costs, and scalability. In his session at @ThingsExpo, Robin Raymond, Chief Architect at Hookflash, will walk through the shifting landscape of traditional telephone and voice services ...
Nov. 26, 2014 02:00 PM EST Reads: 1,530
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series data. By focusing on enterprise applications and the data center, he will use OpenTSDB as an example t...
Nov. 25, 2014 09:30 PM EST Reads: 1,344
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Architect for the Internet of Things and Intelligent Systems at Red Hat, described how to revolutioniz...
Nov. 24, 2014 07:00 PM EST Reads: 1,659
Bit6 today issued a challenge to the technology community implementing Web Real Time Communication (WebRTC). To leap beyond WebRTC’s significant limitations and fully leverage its underlying value to accelerate innovation, application developers need to consider the entire communications ecosystem.
Nov. 24, 2014 12:00 PM EST Reads: 1,553
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from hardware to software, or as we like to say, it’s an Internet of many different things. The difference ...
Nov. 24, 2014 11:00 AM EST Reads: 1,685
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
Nov. 24, 2014 09:00 AM EST Reads: 1,711
SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, phone and digital TV services to consumers primarily in rural areas.
Nov. 23, 2014 07:30 PM EST Reads: 1,867
"There is a natural synchronization between the business models, the IoT is there to support ,” explained Brendan O'Brien, Co-founder and Chief Architect of Aria Systems, in this SYS-CON.tv interview at the 15th International Cloud Expo®, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 23, 2014 12:00 PM EST Reads: 1,821
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com), moderated by Ashar Baig, Research Director, Cloud, at Gigaom Research, Nate Gordon, Director of T...
Nov. 23, 2014 07:45 AM EST Reads: 1,853