|By Andreas Grabner||
|October 28, 2011 11:15 AM EDT||
The term Business Transactions and Business Transaction Management is widely used in the industry but it's not always well understood what we really mean by it. The BTM Industry Portal provides some good articles on this topic and is definitely recommended to check out. The general goal is to answer business-relevant questions that business owners have for application owners: "How much revenue is generated by a certain products?", "What are my conversion and bounce rates and what impacts them?" or "Do we meet our SLAs to our premium account users?"
Challenge 1: Contextual Information Is More than just the URL
In order to answer these questions we need information captured from the underlying technical transactions that get executed by your applications when users interact with your services/web site. Knowing the accessed URL, its average response time and then mapping it to a Business Transaction is the simplest form of Business Transaction Management - but doesn't work in most cases because modern applications don't pass the whole business transaction context in the URL.
Business Context information such as the username, product details or cash information usually comes from method arguments, the user session on the application server or from service calls that are made along the processed transaction.
Challenge 2: Business Context Is Somewhere Along a Distributed Transactions
Modern applications are no longer monolithic. The challenge with that is that transactions are distributed, they take different paths, and data we need for our business context (username, product information, cash information...) is often available on different tiers. This requires us to trace every single transaction across all tiers in order to collect this data for a single user transaction:
Every transaction is different, it involves different services, crosses multiple tiers and we need to capture business information along the full end-to-end transaction
Challenge 3: Understanding Your Users Means: All Users, All Actions, All the Time
Knowing that a certain transaction of a user failed, including all contextual information, is great. In modern applications users have many ways to reach a certain point in the application, e.g.: checking out. So the questions are: How did the user get to this point? What were the actions prior to the problem? Is one of the previous actions responsible for the problem we see?
In order to answer these questions we need access to all transactions of every single user. This allows us to understand:
- How our users interact with the system
- How users reach our site (identify Landing Pages)
- Where users drop off the site (important for bounce rate and bounce page evaluation)
- Which paths tend to lead to errors
It also supports a critical use case in Business Transaction Management which is User Complaint Resolution, e.g., when user "Maria" calls the support line we can look up all her transactions, follow her steps and identify where things went wrong:
Knowing every action of every user allows us to better understand our users, follow their click path and speed up individual problem resolutions
Why Continue Reading?
In this article I provide more examples on Business Transaction Management and focus on the challenges I just explained:
- It's more than just URLs we need to analyze as modern web applications are more complex
- We live in a distributed transactional world where business context data can come from every involved tier
- We need to focus on every user and every action to understand our users
To make it easier to understand I also bring examples from PGAC and other real-life BTM implementations.
An Example of Business Transactions
Let's assume we have a website for a travel agency. The interesting business data for this type of application is
- What destinations are people searching for as I want to make sure I offer trips to those regions and people find what they are looking for
- How many trips are actually being sold?
- How much money do we make?
- How many people that search actually buy (convert)? Are there certain trips that have a better conversion rate?
From a technical perspective we can monitor every single technical transaction - that means we can look at every single Web Request that is processed by our application server. If we do that, we basically look at a huge set of technical transactions. Some of these transactions represent those that handle search requests. Some of them handle the shopping cart or check-out procedure. The first goal for us, however, is to identify the individual business transactions such as "Search":
Among all technical transactions we want to identify those that perform a certain Business Transaction, e.g., Search, Checkout, Logon...
Now not every search is the same. In our case the search keyword or trip destination is interesting so we want to split the search transactions by the destination used in the search criteria. That allows us to see how many search requests are executed by destination and also how long these search requests take to execute. Assuming we have to query external services to fetch the latest trip offerings, it could be very interesting to see how fast or slow searches for specific destinations are or whether the number of search results impacts query time and therefore user experience.
Splitting the searches by destination allows us to see how many searches are actually executed and whether there are any performance implications on specific destinations
The next interesting aspect is of course the actual business we do or don't do. If our website serves different markets (US, EMEA, ASIA...) it's interesting to see revenue by market. The next illustration shows that we can focus on the actual Purchase Business Transactions. On these we evaluate generated revenue, the market that generated the revenue and whether there is a performance impact on business by looking at Response Time as well.
Looking at those Business Transactions that actually handle the purchase allows us to see how much money we actually make split by markets we serve
Contextual Information for Every Transaction
In order for all of this to actually work we need more than just capturing the URL and Response Time. A lot of times the type of context information we need is in the user session, in the HTTP POST Body or some parameter on a method call or SQL Bind Value. The next illustration shows us what technical information we are talking about in order to map these values to Business Transactions.
Capturing context information for every single technical transaction is mandatory for business transaction management
As we live in a distributed transactional world we need the full end-to-end transaction and on that transaction the ability to capture this type of technical but business relevant data.
Every Action of Every Visit to Identify Landing Pages, Conversion and Bounce Rates
Some of the most interesting metrics for Business Owners are Conversion Rates, Bounce Rates, User Satisfaction and how well Landing Pages do. In order to get these metrics we need to identify every individual user (typically called Visitor), all actions (individual technical transactions/requests) for the user and the information on whether the user had a good or bad experience (we factor performance and functional problems into that). With that information we can:
- Identify the Landing Page of a Visitor: that's the first page that was requested by an individual Visitor. The Landing Page tells you how people get to your website and whether promotions or ads/banners work
- Identify the Bounce Rates and Bounce Pages: a Visitor bounces of your site if only one page was visited. We want to know which pages typically let people bounce off the page so we can optimize them. If you spend a lot of money on marketing campaigns but due to bad performance or a problem on these landing pages people bounce off its money that is wasted
- Identify click paths visitors take until they actually convert (buy a product) or where along this path they actually leave: Visitors usually need to click through a series of pages before they click on the "Confirm" button. If they do these Visitors convert. If they don't and leave your site somewhere along the way we want to know where and why they are leaving in order to optimize these pages and therefore increase conversion rates
- Identify how satisfied our end users are based by looking at response times and any functional problems along their execution path: Visitors that experience long page load times or run into any type of functional problem are more likely to be frustrated and leave the site before converting. Therefore we want to track this and identify which actions result in a bad user experience.
Landing Pages Impact Whether Users Make It Across the First Site Impression
Knowing the first action of every visit lets us identify our Landing Pages. Visitors that don't make it further than the landing page bounce off our site right away. This is the first problem we want to address as these people are never going to generate any business. The next screenshot shows how we want to analyze landing pages. It is interesting to see which landing pages we have, which are frequented more often, what's the bounce rate of these landing pages and is there any correlation to performance (e.g., higher bounce rate on pages that take longer).
Landing Page Report shows Bounce Rates, Access Count and compares Load Time to the industry standard
Bounce and Conversion Rates, User Satisfaction and Activity Are Important Metrics for Business Owners
Having every action of every visitor available, knowing whether visitors only visit the landing page and bounce off or whether they also make it to the checkout page allows us to calculate Bounce and Conversion Rates. Also looking at response times, request count and errors allows us to look at Visitor Satisfaction and usage. The following illustration shows all these metrics a Business Owner is interest in.
Dashboard for Business Owner to monitor those metrics that impact Business
Understanding Which Path Visitors Take Allows You to Improve Conversion and Lower Bounce Rates
The underlying data of all these metrics we just discussed are the individual actions of every visitor. Only with that information can we identify why certain Visitors convert and others don't, which will help us to improve Conversion Rate and lower the Bounce Rate. The following screenshot shows a detailed view of visits. Having all this data allows us to see which visitors actually bounce off of the site after just looking at their landing page, which users actually convert and what path users take when interacting with our web site.
Having every Visitor and all its Actions allows us to manage and optimize business transactions
Speed up User Complaint Resolution with All Actions Available
As already explained in the introduction section of this blog, one important use case is speeding up the resolution process of individual users. If you have a call center and one of our users complain about a problem, it is best to see what this particular user did to get to the explained error. Having all user actions available and knowing the actual user along the captured transactions allows us to look up only those transactions for that particular user and with that really seeing what happened.
When user Maria calls in we can look up all the actions from this user and see exactly what error occurred
Deep Technical Information for Fast Problem Resolution
Besides using technical context information as input for business transactions (e.g., username, search keywords, cash amount ...) we also need very deep technical information in scenarios where we need to troubleshoot problems. If visitors bounce of the page because of slow response time or because of an error we want to identify and fix this problem right away. In order to do that our engineers need access to the technical data captured for those users that ran into a particular problem. The following screenshot shows a user that experienced a problem on one of the pages which is great information for the Business Owner as he can proactively contact this user. The engineer can now go deeper and access the underlying captured technical information including transactional traces that show the encountered problem such as a failed Web Service Call, a long running database statement or an unhandled exception.
From Business Impact to Technical Problem Description: Technical context information helps to solve problems faster
To Sum Up: What We Need for Business Transaction Management
There are several key elements we need to perform the type of Business Transaction Management explained.
- All Visitors, All Actions, All the Time
- First Action is Landing Page
- Last Action is Bounce Page
- Helps us to understand the Click Path through the site, where people bounce off, which path people take that convert
- Knowing the Click Paths allows us to improve Conversion Rate and lower Bounce Rates
- Looking up the actions of a complaining users speeds up problem resolution
- Technical Context Information on a Distributed Transaction
- URLs alone are not enough as the Business Transaction itself is not always reflected in URL or URL Parameters
- Need to capture Business Context Information from HTTP Session, Method Arguments, Web Service Calls, SQL Statements, ...
- This information comes from multiple tiers that participate in a distributed transaction
- Technical context information speeds up problem resolution
Out-of-the-Box Business Transactions for Standard Applications
From an implementation perspective the question that always comes up is: Do I need to configure all my business transactions manually? The answer is: not all, but for most applications we have seen that it's necessary as business context information is somewhere buried along the technical transaction and is not necessarily part of the URL. Identifying Business Transactions based on the URL or Web Services that get called is of course a valid start and is something that most Business Transaction Management Solutions provide, and it actually works quite well for standard applications that use standard Web Frameworks. For more complex or customized applications it is a different story.
Business Transactions by URL
The easiest way to identify Business Transactions is by URL, assuming your application uses URLs that tell you something about the related business transactions. If your application uses URLs like the following list, you can easily map these URLs to Business Transactions:
- /home maps to "Home"
- /search maps to "Search"
- /user/cart maps to "Cart"
- /search?q=shoes still maps to "Search" but it would be great to actually see the Search by Keyword
The following screenshot shows the automatically identified Business Transactions based on URLs in dynaTrace. We automatically get the information if there are any errors on these transactions, what the response time is or how much time is spent in the database.
Business Transactions by Web Request URL works well for standard web applications meaningful URLs that can easily be mapped
Business Transactions by Service/Servlet
Another often-seen scenario is Business Transactions based on Servlet Names or Web Service calls that are executed by the technical transaction. This is most often very interesting as you want to know how your calls to the Search or CreditCard Web Service are doing. The name of the invoked web service method is often very descriptive and can therefore be used for automatic business transactions. Here is an example:
Business Transactions by Web Service Method name works well as method names are often very descriptive
Business Transactions by Page Titles
Better than URLs are very often Page Titles - that's the actual Title of the Pages users visit. The following shows us business transactions per page title including information on whether problems are to be found in the browser (Browser Errors), Client, Network or Server allowing a first quick root cause analysis.
Business Transaction by Page Title helps us to understand End User Experience
Customized Business Transactions for Non-standard Applications
A lot of applications our customers are using don't use standard web frameworks where URLs tell them everything they need to identify their business transactions. Here are some examples:
- Web 2.0 applications use a single service URL. The actual executed business transaction name can only be captured from a method argument of the dispatcher method on the backend
- Enforcing SLAs by account types (Free, Premium, Elite Members). The account type of a user doesn't come via the URL but is enumerated with an authentication call on the backend
- Search options are passed via URL using internal IDs. The "human readable" name of this option is queried from the database.
- The Booking Business Transaction is only valid if multiple conditions along the technical transaction are fulfilled, e.g, CreditCardCheck OK and Booking was forwarded to Delivery.
Let's have a look of some examples on how we use Customized Business Transactions in our own environment
Requests by User Type
On our Community Portal we have different User Types that access our pages. It includes employees, customers, partners, etc.; when a user is logged in we only get the actual user type from an authentication call that is made to the backend JIRA Authentication Service. We can capture the return value of the getGroup service call and use this to define a Business Transaction that splits all authenticated transactions by this User Type allowing me to see which types of users are actually consuming content on our Community Portal.
Using a method return value allows me to analyze activity per user type
Search Conversion Rates
We have a custom search feature on our community portal. In order to ensure that people find content based on their keywords I need to understand two things:
- What keywords are used and
- Which keywords result in a successful click to a search result and which ones don't - that helps me to optimize the search index.
The following screenshot shows the two Business Transactions I created. The first splits the search requests based on the keyword which is passed as HTTP POST Parameter on an AJAX call. The second looks at clicks to content pages and shows those that actually came from a previous search result. For that I use the referrer header (I know the user came from a search result page) and I use the last used search keyword (part of the user session).
Using HTTP POST Parameter, Referrer Header and HTTP Session Information to identify Search Keywords and the Conversion Rate to actual Result Clicks
These were just two examples on how we internally use Business Transactions. In the described use cases it's not possible to just look at a URL, a Servlet or Web Service Name to identify the actual Business Transaction. In these scenarios and scenarios that we always see with our customers it is required to capture information from within the technical transaction and then define these Business Transactions based on the captured context data.
A Practical Example: How PGAC uses Business Transactions
As a last example on Business Transaction Management in real life I want to highlight some of what was shown during last week's Webinar by Glen Taylor, Web Service Architect, from PGAC. PGAC runs TheGeneral.com and PGAC.com. Their web application doesn't have URLs that tells them whether the user is currently asking for an insurance quote or whether they are in the process of verifying their credit card. The information about the actual business transaction comes from the Instance Class of an object passed to their ProcessFactory. dynaTrace captures the object passed as argument and its actual class. With that information they are able to split their technical transactions into Business Transactions.
PGAC uses the instance name of a class to define their Business Transactions
If you are interested in their full story check out the recorded webinar. It will be available for download on the dynaTrace recorded webinar page.
More on Business Transaction Management
If you are already a dynaTrace user you should check out the material we have on our dynaTrace Community Portal: Business Transaction in Core Concepts and Business Transactions in Production
The IT industry is undergoing a significant evolution to keep up with cloud application demand. We see this happening as a mindset shift, from traditional IT teams to more well-rounded, cloud-focused job roles. The IT industry has become so cloud-minded that Gartner predicts that by 2020, this cloud shift will impact more than $1 trillion of global IT spending. This shift, however, has left some IT professionals feeling a little anxious about what lies ahead. The good news is that cloud computin...
Mar. 30, 2017 08:30 AM EDT Reads: 1,687
Lots of cloud technology predictions and analysis are still dealing with future spending and planning, but there are plenty of real-world cloud use cases and implementations happening now. One approach, taken by stalwart GE, is to use SaaS applications for non-differentiated uses. For them, that means moving functions like HR, finance, taxes and scheduling to SaaS, while spending their software development time and resources on the core apps that make GE better, such as inventory, planning and s...
Mar. 30, 2017 06:30 AM EDT Reads: 1,033
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like? In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists l...
Mar. 30, 2017 05:00 AM EDT Reads: 10,115
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service.
Mar. 30, 2017 04:00 AM EDT Reads: 3,339
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Mar. 30, 2017 01:30 AM EDT Reads: 2,675
The rise of containers and microservices has skyrocketed the rate at which new applications are moved into production environments today. While developers have been deploying containers to speed up the development processes for some time, there still remain challenges with running microservices efficiently. Most existing IT monitoring tools don’t actually maintain visibility into the containers that make up microservices. As those container applications move into production, some IT operations t...
Mar. 30, 2017 01:30 AM EDT Reads: 3,243
DevOps tends to focus on the relationship between Dev and Ops, putting an emphasis on the ops and application infrastructure. But that’s changing with microservices architectures. In her session at DevOps Summit, Lori MacVittie, Evangelist for F5 Networks, will focus on how microservices are changing the underlying architectures needed to scale, secure and deliver applications based on highly distributed (micro) services and why that means an expansion into “the network” for DevOps.
Mar. 30, 2017 01:00 AM EDT Reads: 8,389
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
Mar. 29, 2017 04:00 PM EDT Reads: 3,305
As Enterprise business moves from Monoliths to Microservices, adoption and successful implementations of Microservices become more evident. The goal of Microservices is to improve software delivery speed and increase system safety as scale increases. Documenting hurdles and problems for the use of Microservices will help consultants, architects and specialists to avoid repeating the same mistakes and learn how and when to use (or not use) Microservices at the enterprise level. The circumstance w...
Mar. 29, 2017 03:00 PM EDT Reads: 4,534
SYS-CON Events announced today that Auditwerx will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Auditwerx specializes in SOC 1, SOC 2, and SOC 3 attestation services throughout the U.S. and Canada. As a division of Carr, Riggs & Ingram (CRI), one of the top 20 largest CPA firms nationally, you can expect the resources, skills, and experience of a much larger firm combined with the accessibility and attent...
Mar. 29, 2017 02:30 PM EDT Reads: 898
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
Mar. 29, 2017 02:15 PM EDT Reads: 3,399
Everyone wants to use containers, but monitoring containers is hard. New ephemeral architecture introduces new challenges in how monitoring tools need to monitor and visualize containers, so your team can make sense of everything. In his session at @DevOpsSummit, David Gildeh, co-founder and CEO of Outlyer, will go through the challenges and show there is light at the end of the tunnel if you use the right tools and understand what you need to be monitoring to successfully use containers in your...
Mar. 29, 2017 01:45 PM EDT Reads: 2,011
What if you could build a web application that could support true web-scale traffic without having to ever provision or manage a single server? Sounds magical, and it is! In his session at 20th Cloud Expo, Chris Munns, Senior Developer Advocate for Serverless Applications at Amazon Web Services, will show how to build a serverless website that scales automatically using services like AWS Lambda, Amazon API Gateway, and Amazon S3. We will review several frameworks that can help you build serverle...
Mar. 29, 2017 01:30 PM EDT Reads: 2,339
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
Mar. 29, 2017 12:15 PM EDT Reads: 8,040
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
Mar. 29, 2017 10:45 AM EDT Reads: 10,587
By now, every company in the world is on the lookout for the digital disruption that will threaten their existence. In study after study, executives believe that technology has either already disrupted their industry, is in the process of disrupting it or will disrupt it in the near future. As a result, every organization is taking steps to prepare for or mitigate unforeseen disruptions. Yet in almost every industry, the disruption trend continues unabated.
Mar. 29, 2017 09:00 AM EDT Reads: 991
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Mar. 29, 2017 08:00 AM EDT Reads: 7,636
While DevOps most critically and famously fosters collaboration, communication, and integration through cultural change, culture is more of an output than an input. In order to actively drive cultural evolution, organizations must make substantial organizational and process changes, and adopt new technologies, to encourage a DevOps culture. Moderated by Andi Mann, panelists discussed how to balance these three pillars of DevOps, where to focus attention (and resources), where organizations might...
Mar. 29, 2017 06:30 AM EDT Reads: 6,319
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" ...
Mar. 29, 2017 06:00 AM EDT Reads: 9,174
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, will discuss how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He will discuss how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
Mar. 29, 2017 06:00 AM EDT Reads: 2,930