Welcome!

Microservices Expo Authors: Pat Romanski, Elizabeth White, Stackify Blog, Liz McMillan, Yeshim Deniz

Blog Feed Post

Getting the maximum performance of your Java processes

therore-concurrent provides self-tuning thread-pools helping you to make the most of your system.

Recently, I have been working in the optimization of an OLTP system. The software has a SEDA architecture (Staged Event Driven Architecture) with lots of threads doing little works. I had to fight with the hard task of adjusting a hundred of parameters. Each of those parameters affected some others and so on.

For example if the number of concurrent database connections is set too low, it would cause a contention in getting connections. On the contrary, if that number is set too high, it could cause a lock-contention in the database when the threads want to access to some shared resources (index, row, block, etc.)

Even more, not always the processing of an event requires the same type of resources. A sudden change in the type of events that are being treated, can turn an optimal configuration into a suboptimal.

One of the most significant parameters is the number of threads assigned for each component. It is difficult to choose a good value if you don’t know how much the threads use each type of resource and how much are they coupled between each other.

Usually certain tasks have a higher priority and should be processed as soon as possible. This further complicates the choice of the configuration. Enforcing priorities and maximizing throughputs are opposite goals therefore it is necessary to define the scope of both.

In my experience a huge configurability can work against you. In a medium/big SOA system with a lot of service communications and complex workload profiles that even change over time, is almost impossible to get the optimal value for each of those parameters. Because of that I found interesting to develop a library that might be able to adapt quickly at runtime in order to make the most of the system.

Self-tuning thread-pool

Nowadays creating threads manually is not very common. Instead of that, thread-pools are frequently used. A thread-pool manages the creation and allocation of threads. JDK comes with some interesting and useful classes for managing threads. I list two of the most important:

  • ThreadPoolExecutor is a very flexible and configurable thread-pool that supports customization of queue size, minimum and maximum pool size, keep-alive time, etc.
  • Executors is a convenient class that creates thread-pools for the most usual cases.

I have developed the library therore-concurrent that takes advantage of those classes and extends some functionalities. The library contains analogous to the above classes.

  • SelfTuningExecutorService is a thread-pool that implements a mechanism for searching a good value for the pool size. The algorithm tries to maximize the throughput respecting the thread-pool priorities.
  • SelfTuningExecutors acts as the factory of SelfTuningExecutorService. It is recommended to use it as a singleton.

The following charts show how quickly SelfTuningExecutorService finds the optimal value.

selftuning_poolsize_executions_chart

Using SelfTuningExecutors directly

  • Add the dependency to the pom
  • <dependency>
        <groupId>net.therore</groupId>
        <artifactId>therore-concurrent</artifactId>
        <version>1.1.0</version>
    </dependency>
    

  • The following snippet shows how can it be used.
  • SelfTuningExecutors executors = SelfTuningExecutors.defaultSelfTuningExecutors();
    ExecutorServicce service = executors.newSelfTuningExecutor("executor-for-test", corePoolSize, initPoolSize
           , maximumPoolSize, priority, queueSize);
    service.execute(task);
    

The only new parameters are initPoolSize and priority.

  • initPoolSize is the initial amount of threads assigned to the pool.
  • priority is a positive number that works for SelfTuningExecutorService to limit the number of threads of this pool regarding others.

Integrating SelfTuningExecutors with Quartz Scheduler

Quartz-Scheduler has his own thread-pool interface and its name is “ThreadPool” (not surprise). The class SelfTuningThreadPool that is in the artifact therore-concurrent-quartz implements such interface. Integrating it is very easy, follow these steps:

  • Add the dependency to the pom
  • <dependency>
        <groupId>net.therore</groupId>
        <artifactId>therore-concurrent-quartz</artifactId>
        <version>1.1.0</version>
    </dependency>
    

  • Change the configuration properties of quartz
  • # org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
    # org.quartz.threadPool.threadCount = 1
    # org.quartz.threadPool.threadPriority = 5
    org.quartz.threadPool.class = net.therore.concurrent.quartz.SelfTuningThreadPool
    org.quartz.threadPool.corePoolSize = 1
    org.quartz.threadPool.initPoolSize = 1
    org.quartz.threadPool.maximumPoolSize = 100
    org.quartz.threadPool.priority = 5
    org.quartz.threadPool.queueSize = 2
    

Integrating SelfTuningExecutors with Apache Camel

I love Apache Camel. It offers a lot of components supporting integration with different technologies. But if none of them actually help you yet, it’s pretty easy to make your own component.

Camel’s team has thought very well the threading model. They use the concept (and interface) of ThreadPoolProfile which is a kind of thread-pool-template that you can use to instantiate several pools with the same configuration. If that is not enough, you can program your own implementation of ExecutorServiceManager, the Camel’s thread-pool provider. Simplifying, think about it like the Executors class of the JDK.

I’ve just done that, SelfTunigExecutorServiceManager is the name of my own implementation of ExecutorServiceManager. It is located in other maven module therore-concurrent-camel. I’ll explain how to use it.

  • Add the dependency to the pom
  • <dependency>
        <groupId>net.therore</groupId>
        <artifactId>therore-concurrent-camel</artifactId>
        <version>1.1.0</version>
    </dependency>
    

  • The following snippet contains two connected routes with SEDA component and SelfTunigExecutorServiceManager
  • SelfTunigExecutorServiceManager executorManager = new SelfTunigExecutorServiceManager(context);
    context.setExecutorServiceManager(executorManager);
    ThreadPoolProfile profile = new ThreadPoolProfile();
    profile.setId("self-tuning-profile");
    profile.setMaxPoolSize(100);
    profile.setMaxQueueSize(100);
    profile.setDefaultProfile(true);
    executorManager.setDefaultThreadPoolProfile(profile);        
    
    final String sedaEndpointUri = "seda:myseda?blockWhenFull=true&size=1";
    context.addRoutes(new RouteBuilder() {
       @Override
       public void configure() throws Exception {
           from("direct:in")
           .to(sedaEndpointUri);
       }
    });
    context.addRoutes(new RouteBuilder() {
       @Override
       public void configure() throws Exception {
           from(sedaEndpointUri)
           .threads(1, 100)
           .to("bean:mybean");
       }
    });
    
    ProducerTemplate template = context.createProducerTemplate();
    context.start();
    for (int i=0; i<ITERATIONS; i++) {
       template.sendBody("direct:in", "dummy string");
    }
    

Summary

I have figured out that there are many elements that might turn into selftuning ones. I chose ThreadPool because from my point of view is one of the most important, used and easy to test element.

Moreover, most of the modern libraries and frameworks feature different ways to extend their factories, providers and templates. All of that aims to develop general purpose classes and integrate them with lots of frameworks.

Read the original blog entry...

More Stories By Alfredo Diaz

Alfredo Diaz is a Java EE Architect with over 10 years of experience. He is an expert in SOA, real-time processing, scalability and HA. He is an Agile enthusiast.

@MicroservicesExpo Stories
Regardless of what business you’re in, it’s increasingly a software-driven business. Consumers’ rising expectations for connected digital and physical experiences are driving what some are calling the "Customer Experience Challenge.” In his session at @DevOpsSummit at 20th Cloud Expo, Marco Morales, Director of Global Solutions at CollabNet, will discuss how organizations are increasingly adopting a discipline of Value Stream Mapping to ensure that the software they are producing is poised to o...
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
Cloud promises the agility required by today’s digital businesses. As organizations adopt cloud based infrastructures and services, their IT resources become increasingly dynamic and hybrid in nature. Managing these require modern IT operations and tools. In his session at 20th Cloud Expo, Raj Sundaram, Senior Principal Product Manager at CA Technologies, will discuss how to modernize your IT operations in order to proactively manage your hybrid cloud and IT environments. He will be sharing bes...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value...
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
This talk centers around how to automate best practices in a multi-/hybrid-cloud world based on our work with customers like GE, Discovery Communications and Fannie Mae. Today’s enterprises are reaping the benefits of cloud computing, but also discovering many risks and challenges. In the age of DevOps and the decentralization of IT, it’s easy to over-provision resources, forget that instances are running, or unintentionally expose vulnerabilities.
It has never been a better time to be a developer! Thanks to cloud computing, deploying our applications is much easier than it used to be. How we deploy our apps continues to evolve thanks to cloud hosting, Platform-as-a-Service (PaaS), and now Function-as-a-Service. FaaS is the concept of serverless computing via serverless architectures. Software developers can leverage this to deploy an individual "function", action, or piece of business logic. They are expected to start within milliseconds...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
There are two main reasons for infrastructure automation. First, system administrators, IT professionals and DevOps engineers need to automate as many routine tasks as possible. That’s why we build tools at Stackify to help developers automate processes like application performance management, error monitoring, and log management; automation means you have more time for mission-critical tasks. Second, automation makes the management of complex, diverse environments possible and allows rapid scal...
One of the biggest challenges with adopting a DevOps mentality is: new applications are easily adapted to cloud-native, microservice-based, or containerized architectures - they can be built for them - but old applications need complex refactoring. On the other hand, these new technologies can require relearning or adapting new, oftentimes more complex, methodologies and tools to be ready for production. In his general session at @DevOpsSummit at 20th Cloud Expo, Chris Brown, Solutions Marketi...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
SYS-CON Events announced today that HTBase will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. HTBase (Gartner 2016 Cool Vendor) delivers a Composable IT infrastructure solution architected for agility and increased efficiency. It turns compute, storage, and fabric into fluid pools of resources that are easily composed and re-composed to meet each application’s needs. With HTBase, companies can quickly prov...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, Cloud Expo and @ThingsExpo are two of the most important technology events of the year. Since its launch over eight years ago, Cloud Expo and @ThingsExpo have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, I provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading the...
The purpose of this article is draw attention to key SaaS services that are commonly overlooked during contact signing that are essential to ensuring they meet the expectations and requirements of the organization and provide guidance and recommendations for process and controls necessary for achieving quality SaaS contractual agreements.
SYS-CON Events announced today that OpsGenie will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2012, OpsGenie is an alerting and on-call management solution for dev and ops teams. OpsGenie provides the tools needed to design actionable alerts, manage on-call schedules and escalations, and ensure that the right people are notified at the right time, using multiple notification methods.