Welcome!

Microservices Expo Authors: Pat Romanski, Krishnan Manjeri, Elizabeth White, Ian Khan, Yeshim Deniz

Related Topics: Microservices Expo

Microservices Expo: RSS Feed Item

Agile SOA Across the Lifecycle - Part Five: IT and SOA Governance

This is the fifth of a six part series of posts on the Agile SOA life cycle

This is the fifth of a six part series of posts on the Agile SOA life cycle. Here we will at look at IT and SOA Governance. With the introduction of agile, spiral, and scrum development methodologies, the traditional waterfall development approach of testing a near-finished app at the end of many Agile development cycles won't be agile at all, as the elements of the application are constantly changing.  Traditional models of IT governance will also not work. To aggravate testing, the service-oriented architecture (SOA) design pattern is used to make IT  more responsive to changes requested by business. New process tooling has been introduced to specifically assist in the cataloging of service assets, and organization of policies governing SOA. This new set of tooling created to support SOA revolves around governance platforms like HP Systinet / S2, SAG Centrasite, SOA Software, TIBCO ActiveMatrix and Oracle Fusion. These SOA Governance tools manage the collection and cataloging of metadata about services, and organize the interdependencies they have with each other, along with the documentation of SOA Policies to define how combined services should meet business requirements.  Within the Governance toolset, SOA Registry/Repositories provide a new platform for LISA to automate testing and validation.

The traditional system requirement and functional testing will still occur, but in a SOA there are more opportunities to automate the validation and enforcement of policies. The current thinking around SOA Governance is largely siloed and specialized around design-time WSDL validation, runtime performance, and security policy enforcement.  As SOA Governance matures to support robust and widely diverse SOA initiatives, this thinking must expand dramatically. One of the greatest values a SOA Governance platform must provide is the proof that policies written in human or business terms are in fact implemented in the system, on a continuous basis. Validation with our LISA solution takes the form of positive and negative tests on the use of these policies, which are executed continuously at change and runtime, as services and their underlying technologies and data are constantly changing and evolving. The validation of the LISA test becomes a demonstrable artifact for exposing and enforcing SOA Policy, within the changing workflow of the SOA Governance framework. Example Scenario of Validating SOA Policies Let's look at a sample deployment, a series of web service definitions, as WSDL files, are loaded into the UDDI registry.  The relationships of the services to each other, and the documentation of how they can be tied together to create a business process is documented.  For a new employee introduction, there may be a series of services from the human resources package to identify the employee's location and managers, there are services that provision IT resources like computer and e-mail, and services to a external partner for insurance and benefits enrollment.  These services would be documented via their WSDLs and stored in the UDDI registry.  A HR-to-payroll business process will be defined utilizing these assets. This all seems simple as long as each participant in the business process provides a well documented, functioning, reliable, and secure service.  The policies around the structural, behavior, and performance of these individual services are documented the governance tool.  For example, the business analyst will write policies in a tool like CentraSite ActiveSOA, describing the behavior of the services.  The IT service will accept a new employee's name and id number and return an e-mail address, domain login, and default password.  The security service will accept the manager and role and provision the appropriate access control to systems for the new employee. The security manager will specify that the services must only run if encrypted and signed data is sent to them.  The signature must be validated against the companies master certificate provider to prove that the request is from only authorized HR personnel.  Lastly, the IT operations team will provide policies around how many transactions per second (TPS) the service should handle, and the maximum response time the service will take, so the component will be a good participant in a decomposed transaction. That's far too many steps to carry out if it's a strictly manual validation process. The execution of the tests needs to live within the SOA Governance process. Testing, Validation and Policy Enforcement SolutionsGiven all of the processes enterprises must employ to manage the design, construction, integration and management of the IT environment, it makes sense to have a common, reusable way to apply validation and enforcement across these processes. SOA Validation -becomes the long arm of the law- with reusable and rich testing that merges into the workflow of each of these processes, and the tools that support them. The Validation arm shown above (with our LISA as the "long arm") provides capabilities to run test cases to verify different rules and policies defined in leading IT process tools. The policing action is accomplished by tying automatic running of tests to the expected behaviors and policies in the workflow of the process tool.  The invocation of  tests from a governance platform to ensure SOA policy is similar to the policing action done for test management solutions, the biggest difference is the context and stage in the lifecycle of the component that needs to be verified.  In a traditional test of a waterfall development there is a specific point in time in which the system is tested and deemed ready for production.  In today's services-based applications, there is no longer one point in time in which we can identify a test run to ensure quality of the application. In a SOA lifecycle, we think of design time, run time, and change time as the three stages of a service or business process being constructed within these loosely coupled systems.  In the example above some of the services are interactions with packaged applications (HR system), some from home grown systems (IT user and security provisioning), and other from external partners (benefits).  The development and release cycles of these three areas will be different and will not be coordinated for a single big bang release.  One advantage of SOA is the ability to decouple systems and utilize services created by different parties instead of building everything yourself. At design time, a test will be run to make sure that WSDLs are WS-I compliant, that they follow an RPC or doc-literal call structure, and that a customer-specific XML tag is used for service identification.  Based the on passing of a compliance test, the WSDL will then be made available in the Repository for developers to code the functionality behind it.  Once the developers have created the necessary code to implement the service and its underlying technology components, the behavior and security of the service needs to be validated.  The only way to verify if the code behind a service is working properly, since there are no hard-coded responses in the real world, and that security is respected, is to invoke the service, then verify that the underlying systems of record are updated properly. Again, a test case is used to invoke and verify that the business requirements and the behavioral and security policies have been implemented. If the candidate Service can pass these Policy tests, it can now be published in the registry for a consumer to use.  If these tests fail, the service should not be promoted for availability to consumers in the registry. As a best practice, the individual services should undergo load testing once they are made available. Far less risk is introduced if we test for load while the components are being built, when something can be done to improve performance issues, rather than waiting until the system is deployed, when issues are much more costly to repair. Progressive development shops start their load testing once the service functionality has been verified, so that developers can make code changes, and optimize logic and data access before consumers start utilizing the service. In run time, the service is now promoted to the Registry, along with a test that demonstrates its functionality, and made available for consumers to leverage as part of their own application workflows. These consumers also bear responsibility for validating that their intended use of the service is actually supportable. The best way to do this is to create an executable test asset to validate that the structural, behavioral and performance aspects of the overall consumer's workflow are supported when the service is leveraged. This will pay big dividends when the test is run continuously as part of the governance process. The final stage is change time. This is when the true advantages of SOA and agile development come into play. If a new service needs to be added, or a modification to a existing service needs to be made to respond to changing business requirements, the policies for the service must be re-verified.  Not only do we want to update any behavioral policy with the new expected behavior, but the regression of existing policies must occur. Put yourself in the place of the consumer of a service. How can you trust that changes will not create unintended consequences (e.g. failures) for a service you depend upon, and break your existing business processes?  Each consumer needs to place their expected behavior as a policy in the registry, with an accompanying test, so that it can be automatically validated before that change is made.  The automated structural and behavior policies in the test represent the consumer???s rights ??? and their responsibility for using the services as defined. When these tests run automatically as part of SOA Governance, trust between service consumers and producers is achieved. For more on SOA governance see Joe McKenrick's posting of the full transcript of a recent eBizQ SOA governance panel that John Michelsen participated in. In our next post, we will provide our conclusions. You can download this complete series as the "Agile SOA Across the Lifecycle" at our ITKO LISA resources page. 

Read the original blog entry...

@MicroservicesExpo Stories
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
With online viewership and sales growing rapidly, enterprises are interested in understanding how they analyze performance to positively impact business metrics. Deeper insight into the user experience is needed to understand why conversions are dropping and/or bounce rates are increasing or, preferably, to understand what has been helping these metrics improve. The digital performance management industry has evolved as application performance management companies have broadened their scope beyo...
Video experiences should be unique and exciting! But that doesn’t mean you need to patch all the pieces yourself. Users demand rich and engaging experiences and new ways to connect with you. But creating robust video applications at scale can be complicated, time-consuming and expensive. In his session at @ThingsExpo, Zohar Babin, Vice President of Platform, Ecosystem and Community at Kaltura, will discuss how VPaaS enables you to move fast, creating scalable video experiences that reach your ...
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lea...
Just over a week ago I received a long and loud sustained applause for a presentation I delivered at this year’s Cloud Expo in Santa Clara. I was extremely pleased with the turnout and had some very good conversations with many of the attendees. Over the next few days I had many more meaningful conversations and was not only happy with the results but also learned a few new things. Here is everything I learned in those three days distilled into three short points.
24Notion is full-service global creative digital marketing, technology and lifestyle agency that combines strategic ideas with customized tactical execution. With a broad understand of the art of traditional marketing, new media, communications and social influence, 24Notion uniquely understands how to connect your brand strategy with the right consumer. 24Notion ranked #12 on Corporate Social Responsibility - Book of List.
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
It's been a busy time for tech's ongoing infatuation with containers. Amazon just announced EC2 Container Registry to simply container management. The new Azure container service taps into Microsoft's partnership with Docker and Mesosphere. You know when there's a standard for containers on the table there's money on the table, too. Everyone is talking containers because they reduce a ton of development-related challenges and make it much easier to move across production and testing environm...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Apache Hadoop is a key technology for gaining business insights from your Big Data, but the penetration into enterprises is shockingly low. In fact, Apache Hadoop and Big Data proponents recognize that this technology has not yet achieved its game-changing business potential. In his session at 19th Cloud Expo, John Mertic, director of program management for ODPi at The Linux Foundation, will explain why this is, how we can work together as an open data community to increase adoption, and the i...
About a year ago we tuned into “the need for speed” and how a concept like "serverless computing” was increasingly catering to this. We are now a year further and the term “serverless” is taking on unexpected proportions. With some even seeing it as the successor to cloud in general or at least as a successor to the clouds’ poorer cousin in terms of revenue, hype and adoption: PaaS. The question we need to ask is whether this constitutes an example of Hype Hopping: to effortlessly pivot to the ...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
Large enterprises today are juggling an enormous variety of network equipment. Business users are asking for specific network throughput guarantees when it comes to their critical applications, legal departments require compliance with mandated regulatory frameworks, and operations are asked to do more with shrinking budgets. All these requirements do not easily align with existing network architectures; hence, network operators are continuously faced with a slew of granular parameter change req...
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lo...
Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. In his session at @DevOpsSummit at 18th Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, showed how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningful f...
Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS - software, platform, and infrastructure as a service.