Welcome!

Microservices Expo Authors: Elizabeth White, Liz McMillan, Carmen Gonzalez, Jyoti Bansal, Pat Romanski

Related Topics: Microservices Expo

Microservices Expo: Article

SOA World Product Review — Testing...1...2...3...Testing

A review of SOAtest 5.0

Few topics evoke more groans and eye rolling from software engineers and Web developers than the dreaded "TESTING." Testing falls into the same category as documentation, refactoring code, dusting, and visiting the dentist. Put it off until the last minute, do as little as possible, do it quickly, and move on to something else. I must confess that I have the same visceral reaction to the thought of 'testing' as others do. Consequently, I approached the prospect of reviewing a testing tool with the loathing of visiting the dentist. I was very relieved to discover that Parasoft's SOAtest 5.0 took a lot of the pain, frustration, and busy work out of the testing experience.

Parasoft's SOAtest 5.0 is a comprehensive testing and analysis tool suite tailored to the unique testing and validation needs of Service Oriented Architectures. It supports functional testing, scenario-based testing, stress testing, client testing with a mock service, and a whole range of validation capabilities (XML Schema, WSDL, WS-Security, BPEL, etc.). SOAtest 5.0 then further supports the creation and automated execution of regression test suites. It supports a broad range of SOA specifications and standards, and is designed from the ground up to support the dynamic and evolving nature of service-oriented systems.

Getting Started with SOAtest 5.0
Installing SOAtest was a breeze. The process is as follows:

  • Run the setup executable from media or by downloading it from www.parasoft.com,
  • Follow the on-screen prompts.
  • Decide if you want to set up SOAtest as a Windows service.

Once the software is installed, you're ready to begin using SOAtest. The first time that you launch the software, you'll be prompted either to input your individual license information or point to an available license server.

After providing licensing information, you will be confronted with an option to create a New Project, Open an Existing Project, or access the SOAtest Tutorial. The tutorial is well-written and provides a nice tour of the tool's major features.

Creating Test Cases
Tests can either be created individually or as a part of a larger test suite. The tool seems to drive you toward creating test suites rather than individual tests, which is nice for reuse, organization, and best of all - regression testing a collection of tests. SOAtest supports the creation of test suites from a wide range of sources including:

  • Web Services Description Language (WSDL) files
  • Business Process Execution Language (BPEL) files
  • Universal Description, Discovery & Integration (UDDI) registry end-points
  • Web Services Inspection Language (WSIL) files
  • BEA Aqualogic Enterprise Repository end-points

The test suite creation wizard will ask you to designate a file, URL, or end-point to query and also ask you to designate what tests you want to create. Upon completing the wizard, you'll have a whole set of tests automatically generated and ready-to-run as is or customized prior to execution.

Functional Testing
I was very impressed with the functional verification testing capabilities. SOAtest supports the following functional verification features:

  • Check schema validity against a WSDL
  • XML-aware diff engine that flags only true XML structure changes
  • Surgical inclusion/exclusion of XML message elements in the verification process via XPATH (see Figure 1)
  • A graphical rules engine for managing assertions

Every one of these features is enabled through simple intuitive menu options, dropdown lists, check boxes, and XML tree structures. The interface is simple to navigate and although there's a wealth of options, they're organized to avoid the feeling of being overwhelmed.

Executing one or more functional tests multiple times based on an external data set (database or spreadsheet) was another feature that I sought out. SOAtest came through in this area as well supporting the ability to point at a data source (CSV, Excel, relational databases, etc.) and run through a battery of tests that pull values out of the data source and send service requests containing the extracted data values. This lets you define a single test case or suite of tests and then automatically test the full range of data values that the test case needs to support without creating additional tests for each value.

Testing with Mock Services
In a truly test-driven environment, I'd create service interfaces and corresponding suites of test cases before any implementation code is developed. Although this is good in theory, without a supporting toolset, it's not very realistic. SOAtest provides all the capabilities needed to support this kind of development model. When creating a new project/test suite, simply point the tool at a WSDL file and check the "Generate Server Stubs" radio button. The server stubs will run on SOAtest's embedded Tomcat server and let you create and run testing scenarios before you've written a single line of implementation code. This way, when you're ready to implement the service interfaces, you have a defined a suite of tests for verification. This helps to control the scope (when you meet all the tests, stop working) and provides one or more test suites that can be incorporated into automatic regression testing (see below) to ensure that functionality isn't compromised as the project progresses.

Of all the features that I've worked with, I found the service mock-up process the most cumbersome. The other aspects of SOAtest were intuitive and easy-to-work with, whereas I had to wrestle a while to get the mock service capability working on anything other than the tutorial walk-through. In the end, the functionality of this capability was excellent, albeit a bit difficult to initially configure.

Regression Testing
Running your tests once is nice. Running your tests regressively is better. Running your tests regressively and automatically at night is divine. SOAtest supports all three scenarios. Any of the test suites that you define in a SOAtest project can be converted to regression test suites. Furthermore, using XPATH you can selectively indicate which portions of the test cases may change from test to test and which values should never change. Once you have a set of regression tests that you're happy with, SOAtest provides a command-line mechanism to kick off your test suites automatically. Thus in an agile, continuous integration environment you can run regression testing at night, at lunch, or every hour on the hour to ensure that you find bugs early and often.

Reporting
SOAtest provides several reporting features (auto-generated reports for nightly regression tests, on-demand, detailed summary reports for test suites, and WS-I interoperability reports). I found the quality and readability of the reports developed by SOAtest to be quite good. Figure 2 provides a snapshot of part of the SOAtest detailed report. The WS-I interoperability reports were pretty low quality, difficult to navigate, and provided information overload. In fairness to SOAtest, those reports are copyrighted by WS-I and seem to be auto-generated by one or more WS-I tools. Consequently, I'm not sure that Parasoft has any control over the quality of these reports. Nonetheless, I would have liked a WS-I conformance report of the same quality and user-friendliness as the native SOAtest reports.

Advanced Testing Features
The palette of testing and analysis features in SOAtest is extensive. In the interest of not filling up this entire magazine with feature descriptions, I'll list several of the compelling features that I've not covered already:
- Scenario-based testing where subsequent test cases depend on data returned from previous test cases
- Stress/load testing your SOA and specifying Quality of Service parameters
- Testing non-XML services (JMS, MQ, TIBCO, EJB, REST, Binary, Text, etc.)
- Validate SOAP security using WS-Security (encryption, digital signatures, and authentication)
- Asynchronous service testing
- Custom scripting with Python, JavaScript, or Java (Figure 3)

Drawbacks
As I mentioned earlier, the mock service feature is a bit awkward to work with initially. Also, the WS-I conformance report was not up to the same quality standards as the native SOAtest reports. With SOA's strong integration and interoperability play, there are a lot of enterprises that have Java and other services that all need to be involved in the same business process and testing scenario.

Summary
In spite of my general dislike for software analysis and testing, I found Parasoft's SOAtest 5.0 to be a well-designed tool that took a lot of the pain and work out of testing and validating a SOA. The tool isn't perfect, but it is easily one of the best SOA testing tools that I've ever worked with.

Product Snapshot

Target Audience: SOA architects, developers, QA/testers, and analysts

Level: Beginner to Advanced

Pros: Powerful, intuitive UI, robust testing and analysis tools

Cons: The mock service feature is a bit awkward, only supports Java clients for now

Testing environment: Dell Inspiron 640m, 1.6GHz Intel Core-Duo, 2GB RAM, Windows XP Pro with SP2

Platforms: Windows 2000/XP, Linux, Solaris

More Stories By Kyle Gabhart

Kyle Gabhart is a subject matter expert specializing in strategic planning and tactical delivery of enterprise technology solutions, blending EA, BPM, SOA, Cloud Computing, and other emerging technologies. Kyle currently serves as a director for Web Age Solutions, a premier provider of technology education and mentoring. Since 2001 he has contributed extensively to the IT community as an author, speaker, consultant, and open source contributor.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
SOA News 04/04/07 04:11:45 PM EDT

Few topics evoke more groans and eye rolling from software engineers and Web developers than the dreaded 'TESTING.' Testing falls into the same category as documentation, refactoring code, dusting, and visiting the dentist. Put it off until the last minute, do as little as possible, do it quickly, and move on to something else. I must confess that I have the same visceral reaction to the thought of 'testing' as others do. Consequently, I approached the prospect of reviewing a testing tool with the loathing of visiting the dentist. I was very relieved to discover that Parasoft's SOAtest 5.0 took a lot of the pain, frustration, and busy work out of the testing experience.

@MicroservicesExpo Stories
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
SYS-CON Events announced today that Outlyer, a monitoring service for DevOps and operations teams, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outlyer is a monitoring service for DevOps and Operations teams running Cloud, SaaS, Microservices and IoT deployments. Designed for today's dynamic environments that need beyond cloud-scale monitoring, we make monitoring effortless so you...
Cloud Expo, Inc. has announced today that Andi Mann and Aruna Ravichandran have been named Co-Chairs of @DevOpsSummit at Cloud Expo 2017. The @DevOpsSummit at Cloud Expo New York will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and @DevOpsSummit at Cloud Expo Silicon Valley will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
DevOps and microservices are permeating software engineering teams broadly, whether these teams are in pure software shops but happen to run a business, such Uber and Airbnb, or in companies that rely heavily on software to run more traditional business, such as financial firms or high-end manufacturers. Microservices and DevOps have created software development and therefore business speed and agility benefits, but they have also created problems; specifically, they have created software securi...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, Cloud Expo and @ThingsExpo are two of the most important technology events of the year. Since its launch over eight years ago, Cloud Expo and @ThingsExpo have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, I provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading the...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
In their general session at 16th Cloud Expo, Michael Piccininni, Global Account Manager - Cloud SP at EMC Corporation, and Mike Dietze, Regional Director at Windstream Hosted Solutions, reviewed next generation cloud services, including the Windstream-EMC Tier Storage solutions, and discussed how to increase efficiencies, improve service delivery and enhance corporate cloud solution development. Michael Piccininni is Global Account Manager – Cloud SP at EMC Corporation. He has been engaged in t...
TechTarget storage websites are the best online information resource for news, tips and expert advice for the storage, backup and disaster recovery markets. By creating abundant, high-quality editorial content across more than 140 highly targeted technology-specific websites, TechTarget attracts and nurtures communities of technology buyers researching their companies' information technology needs. By understanding these buyers' content consumption behaviors, TechTarget creates the purchase inte...
Software development is a moving target. You have to keep your eye on trends in the tech space that haven’t even happened yet just to stay current. Consider what’s happened with augmented reality (AR) in this year alone. If you said you were working on an AR app in 2015, you might have gotten a lot of blank stares or jokes about Google Glass. Then Pokémon GO happened. Like AR, the trends listed below have been building steam for some time, but they’ll be taking off in surprising new directions b...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound e...
"We're bringing out a new application monitoring system to the DevOps space. It manages large enterprise applications that are distributed throughout a node in many enterprises and we manage them as one collective," explained Kevin Barnes, President of eCube Systems, in this SYS-CON.tv interview at DevOps at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
Docker containers have brought great opportunities to shorten the deployment process through continuous integration and the delivery of applications and microservices. This applies equally to enterprise data centers as well as the cloud. In his session at 20th Cloud Expo, Jari Kolehmainen, founder and CTO of Kontena, will discuss solutions and benefits of a deeply integrated deployment pipeline using technologies such as container management platforms, Docker containers, and the drone.io Cl tool...
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, John Jelinek IV, a web developer at Linux Academy, will discuss why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers...
DevOps has often been described in terms of CAMS: Culture, Automation, Measuring, Sharing. While we’ve seen a lot of focus on the “A” and even on the “M”, there are very few examples of why the “C" is equally important in the DevOps equation. In her session at @DevOps Summit, Lori MacVittie, of F5 Networks, explored HTTP/1 and HTTP/2 along with Microservices to illustrate why a collaborative culture between Dev, Ops, and the Network is critical to ensuring success.
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Both SaaS vendors and SaaS buyers are going “all-in” to hyperscale IaaS platforms such as AWS, which is disrupting the SaaS value proposition. Why should the enterprise SaaS consumer pay for the SaaS service if their data is resident in adjacent AWS S3 buckets? If both SaaS sellers and buyers are using the same cloud tools, automation and pay-per-transaction model offered by IaaS platforms, then why not host the “shrink-wrapped” software in the customers’ cloud? Further, serverless computing, cl...
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like? In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists l...
When building DevOps or continuous delivery practices you can learn a great deal from others. What choices did they make, what practices did they put in place, and how did they connect the dots? At Sonatype, we pulled together a set of 21 reference architectures for folks building continuous delivery and DevOps practices using Docker. Why? After 3,000 DevOps professionals attended our webinar on "Continuous Integration using Docker" discussing just one reference architecture example, we recogn...