Welcome!

Microservices Expo Authors: Pat Romanski, Elizabeth White, Steve Wilson, Stackify Blog, Derek Weeks

Related Topics: Microservices Expo

Microservices Expo: RSS Feed Item

Agile SOA Across the Lifecycle - Part Five: IT and SOA Governance

This is the fifth of a six part series of posts on the Agile SOA life cycle

This is the fifth of a six part series of posts on the Agile SOA life cycle. Here we will at look at IT and SOA Governance. With the introduction of agile, spiral, and scrum development methodologies, the traditional waterfall development approach of testing a near-finished app at the end of many Agile development cycles won't be agile at all, as the elements of the application are constantly changing.  Traditional models of IT governance will also not work. To aggravate testing, the service-oriented architecture (SOA) design pattern is used to make IT  more responsive to changes requested by business. New process tooling has been introduced to specifically assist in the cataloging of service assets, and organization of policies governing SOA. This new set of tooling created to support SOA revolves around governance platforms like HP Systinet / S2, SAG Centrasite, SOA Software, TIBCO ActiveMatrix and Oracle Fusion. These SOA Governance tools manage the collection and cataloging of metadata about services, and organize the interdependencies they have with each other, along with the documentation of SOA Policies to define how combined services should meet business requirements.  Within the Governance toolset, SOA Registry/Repositories provide a new platform for LISA to automate testing and validation.

The traditional system requirement and functional testing will still occur, but in a SOA there are more opportunities to automate the validation and enforcement of policies. The current thinking around SOA Governance is largely siloed and specialized around design-time WSDL validation, runtime performance, and security policy enforcement.  As SOA Governance matures to support robust and widely diverse SOA initiatives, this thinking must expand dramatically. One of the greatest values a SOA Governance platform must provide is the proof that policies written in human or business terms are in fact implemented in the system, on a continuous basis. Validation with our LISA solution takes the form of positive and negative tests on the use of these policies, which are executed continuously at change and runtime, as services and their underlying technologies and data are constantly changing and evolving. The validation of the LISA test becomes a demonstrable artifact for exposing and enforcing SOA Policy, within the changing workflow of the SOA Governance framework. Example Scenario of Validating SOA Policies Let's look at a sample deployment, a series of web service definitions, as WSDL files, are loaded into the UDDI registry.  The relationships of the services to each other, and the documentation of how they can be tied together to create a business process is documented.  For a new employee introduction, there may be a series of services from the human resources package to identify the employee's location and managers, there are services that provision IT resources like computer and e-mail, and services to a external partner for insurance and benefits enrollment.  These services would be documented via their WSDLs and stored in the UDDI registry.  A HR-to-payroll business process will be defined utilizing these assets. This all seems simple as long as each participant in the business process provides a well documented, functioning, reliable, and secure service.  The policies around the structural, behavior, and performance of these individual services are documented the governance tool.  For example, the business analyst will write policies in a tool like CentraSite ActiveSOA, describing the behavior of the services.  The IT service will accept a new employee's name and id number and return an e-mail address, domain login, and default password.  The security service will accept the manager and role and provision the appropriate access control to systems for the new employee. The security manager will specify that the services must only run if encrypted and signed data is sent to them.  The signature must be validated against the companies master certificate provider to prove that the request is from only authorized HR personnel.  Lastly, the IT operations team will provide policies around how many transactions per second (TPS) the service should handle, and the maximum response time the service will take, so the component will be a good participant in a decomposed transaction. That's far too many steps to carry out if it's a strictly manual validation process. The execution of the tests needs to live within the SOA Governance process. Testing, Validation and Policy Enforcement SolutionsGiven all of the processes enterprises must employ to manage the design, construction, integration and management of the IT environment, it makes sense to have a common, reusable way to apply validation and enforcement across these processes. SOA Validation -becomes the long arm of the law- with reusable and rich testing that merges into the workflow of each of these processes, and the tools that support them. The Validation arm shown above (with our LISA as the "long arm") provides capabilities to run test cases to verify different rules and policies defined in leading IT process tools. The policing action is accomplished by tying automatic running of tests to the expected behaviors and policies in the workflow of the process tool.  The invocation of  tests from a governance platform to ensure SOA policy is similar to the policing action done for test management solutions, the biggest difference is the context and stage in the lifecycle of the component that needs to be verified.  In a traditional test of a waterfall development there is a specific point in time in which the system is tested and deemed ready for production.  In today's services-based applications, there is no longer one point in time in which we can identify a test run to ensure quality of the application. In a SOA lifecycle, we think of design time, run time, and change time as the three stages of a service or business process being constructed within these loosely coupled systems.  In the example above some of the services are interactions with packaged applications (HR system), some from home grown systems (IT user and security provisioning), and other from external partners (benefits).  The development and release cycles of these three areas will be different and will not be coordinated for a single big bang release.  One advantage of SOA is the ability to decouple systems and utilize services created by different parties instead of building everything yourself. At design time, a test will be run to make sure that WSDLs are WS-I compliant, that they follow an RPC or doc-literal call structure, and that a customer-specific XML tag is used for service identification.  Based the on passing of a compliance test, the WSDL will then be made available in the Repository for developers to code the functionality behind it.  Once the developers have created the necessary code to implement the service and its underlying technology components, the behavior and security of the service needs to be validated.  The only way to verify if the code behind a service is working properly, since there are no hard-coded responses in the real world, and that security is respected, is to invoke the service, then verify that the underlying systems of record are updated properly. Again, a test case is used to invoke and verify that the business requirements and the behavioral and security policies have been implemented. If the candidate Service can pass these Policy tests, it can now be published in the registry for a consumer to use.  If these tests fail, the service should not be promoted for availability to consumers in the registry. As a best practice, the individual services should undergo load testing once they are made available. Far less risk is introduced if we test for load while the components are being built, when something can be done to improve performance issues, rather than waiting until the system is deployed, when issues are much more costly to repair. Progressive development shops start their load testing once the service functionality has been verified, so that developers can make code changes, and optimize logic and data access before consumers start utilizing the service. In run time, the service is now promoted to the Registry, along with a test that demonstrates its functionality, and made available for consumers to leverage as part of their own application workflows. These consumers also bear responsibility for validating that their intended use of the service is actually supportable. The best way to do this is to create an executable test asset to validate that the structural, behavioral and performance aspects of the overall consumer's workflow are supported when the service is leveraged. This will pay big dividends when the test is run continuously as part of the governance process. The final stage is change time. This is when the true advantages of SOA and agile development come into play. If a new service needs to be added, or a modification to a existing service needs to be made to respond to changing business requirements, the policies for the service must be re-verified.  Not only do we want to update any behavioral policy with the new expected behavior, but the regression of existing policies must occur. Put yourself in the place of the consumer of a service. How can you trust that changes will not create unintended consequences (e.g. failures) for a service you depend upon, and break your existing business processes?  Each consumer needs to place their expected behavior as a policy in the registry, with an accompanying test, so that it can be automatically validated before that change is made.  The automated structural and behavior policies in the test represent the consumer???s rights ??? and their responsibility for using the services as defined. When these tests run automatically as part of SOA Governance, trust between service consumers and producers is achieved. For more on SOA governance see Joe McKenrick's posting of the full transcript of a recent eBizQ SOA governance panel that John Michelsen participated in. In our next post, we will provide our conclusions. You can download this complete series as the "Agile SOA Across the Lifecycle" at our ITKO LISA resources page. 

Read the original blog entry...

@MicroservicesExpo Stories
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
When you focus on a journey from up-close, you look at your own technical and cultural history and how you changed it for the benefit of the customer. This was our starting point: too many integration issues, 13 SWP days and very long cycles. It was evident that in this fast-paced industry we could no longer afford this reality. We needed something that would take us beyond reducing the development lifecycles, CI and Agile methodologies. We made a fundamental difference, even changed our culture...
As many know, the first generation of Cloud Management Platform (CMP) solutions were designed for managing virtual infrastructure (IaaS) and traditional applications. But that’s no longer enough to satisfy evolving and complex business requirements. In his session at 21st Cloud Expo, Scott Davis, Embotics CTO, will explore how next-generation CMPs ensure organizations can manage cloud-native and microservice-based application architectures, while also facilitating agile DevOps methodology. He wi...
These days, change is the only constant. In order to adapt and thrive in an ever-advancing and sometimes chaotic workforce, companies must leverage intelligent tools to streamline operations. While we're only at the dawn of machine intelligence, using a workflow manager will benefit your company in both the short and long term. Think: reduced errors, improved efficiency and more empowered employees-and that's just the start. Here are five other reasons workflow automation is leading a revolution...
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developer...
We have Continuous Integration and we have Continuous Deployment, but what’s continuous across all of what we do is people. Even when tasks are automated, someone wrote the automation. So, Jayne Groll evangelizes about Continuous Everyone. Jayne is the CEO of the DevOps Institute and the author of Agile Service Management Guide. She talked about Continuous Everyone at the 2016 All Day DevOps conference. She describes it as "about people, culture, and collaboration mapped into your value streams....
Cloud adoption is often driven by a desire to increase efficiency, boost agility and save money. All too often, however, the reality involves unpredictable cost spikes and lack of oversight due to resource limitations. In his session at 20th Cloud Expo, Joe Kinsella, CTO and Founder of CloudHealth Technologies, tackled the question: “How do you build a fully optimized cloud?” He will examine: Why TCO is critical to achieving cloud success – and why attendees should be thinking holistically ab...
Docker is on a roll. In the last few years, this container management service has become immensely popular in development, especially given the great fit with agile-based projects and continuous delivery. In this article, I want to take a brief look at how you can use Docker to accelerate and streamline the software development lifecycle (SDLC) process.
We define Hybrid IT as a management approach in which organizations create a workload-centric and value-driven integrated technology stack that may include legacy infrastructure, web-scale architectures, private cloud implementations along with public cloud platforms ranging from Infrastructure-as-a-Service to Software-as-a-Service.
Did you know that you can develop for mainframes in Java? Or that the testing and deployment can be automated across mobile to mainframe? In his session and demo at @DevOpsSummit at 21st Cloud Expo, Dana Boudreau, a Senior Director at CA Technologies, will discuss how increasingly teams are developing with agile methodologies, using modern development environments, and automating testing and deployments, mobile to mainframe.
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single-threaded, you can effectively identify hot spots in your serverless code. In his session at @DevOpsSummit at 21st Cloud Expo, Dave Martin, Product owner at CA Technologies, will give a live demonstration and code walkthrough, showing how ...
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
There are several reasons why businesses migrate their operations to the cloud. Scalability and price are among the most important factors determining this transition. Unlike legacy systems, cloud based businesses can scale on demand. The database and applications in the cloud are not rendered simply from one server located in your headquarters, but is instead distributed across several servers across the world. Such CDNs also bring about greater control in times of uncertainty. A database hack ...
API Security is complex! Vendors like Forum Systems, IBM, CA and Axway have invested almost 2 decades of engineering effort and significant capital in building API Security stacks to lockdown APIs. The API Security stack diagram shown below is a building block for rapidly locking down APIs. The four fundamental pillars of API Security - SSL, Identity, Content Validation and deployment architecture - are discussed in detail below.
With Cloud Foundry you can easily deploy and use apps utilizing websocket technology, but not everybody realizes that scaling them out is not that trivial. In his session at 21st Cloud Expo, Roman Swoszowski, CTO and VP, Cloud Foundry Services, at Grape Up, will show you an example of how to deal with this issue. He will demonstrate a cloud-native Spring Boot app running in Cloud Foundry and communicating with clients over websocket protocol that can be easily scaled horizontally and coordinate...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
IT organizations are moving to the cloud in hopes to approve efficiency, increase agility and save money. Migrating workloads might seem like a simple task, but what many businesses don’t realize is that application migration criteria differs across organizations, making it difficult for architects to arrive at an accurate TCO number. In his session at 21st Cloud Expo, Joe Kinsella, CTO of CloudHealth Technologies, will offer a systematic approach to understanding the TCO of a cloud application...
Most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes a lot of work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reduction in cost ...