Welcome!

Microservices Expo Authors: Pat Romanski, Elizabeth White, Kevin Jackson, Yeshim Deniz, Jason Bloomberg

Related Topics: Microservices Expo, Java IoT, Agile Computing

Microservices Expo: Article

Agile - Waterfall: Global and Local Optimizations

Is NP=P and why should we really care?

Did you notice that when something happens to you, it seems to occur to other people as well? For example when you have toddlers, suddenly you see toddlers everywhere - and naturally they are all misbehaving compared to yours J, or when you're planning your wedding, all around you, people are planning theirs, and white dresses become scarce. Lately I had this feeling myself; I was reading a David Baldacci thriller, when half way through, he introduces NP problems and the consequences of finding that P=NP. In addition, I have lately been reading many articles about Critical chain project management. I feel compelled to contribute and provide my take and insights.

There are two kinds of problems, the easy ones which we solve at school, and the hard ones. We might remember the method to solve a quadratic equation by using the x = [-b/2 ± √((b/2)^2 - ac)] /a formula. We are less sure on advising a salesperson how to plan a sales trip to 20 cities in which he can visit every city once and has a limited budget (travelling salesperson problem). When it comes to packing for the holidays we are truly baffled. How are we supposed to fit all the stuff into one checked-in luggage? After continuous deliberation, we land in the exotic resort only to find our bathing suit missing...

Unfortunately and also luckily (more on the luck aspect - later) most of the problems around us are unsolvable, or should I say - not computable in an optimal manner in a reasonable amount of time.

How is this related to project management?

The difficulties we are experiencing managing projects successfully in a resource constrained environment stem from the fact that the Resource Constrained Multi-project Scheduling Problem is very difficult to solve in an optimal manner. Moreover it can't be optimized at all. Multi Project Resource Constrained Scheduling (acronym MPRCS) is known as a NP complete (or NP hard - until this day I am not sure if I understand the exact difference) complexity type problem. In a nutshell, NP problems are very complex mathematical problems. Actually, as mentioned above, most of the problems that we are facing are of NP complexity. For example, those of you who like to travel to Alaska and need to choose equipment from a long list of possibilities, with capacity constraints, are facing an NP problem (knapsack problem).

The theory claims that if we solve one NP problem we can solve all of them. As a byproduct of the solution we would receive a noble prize, render the entire world of encryption useless, and probably get kidnapped by the NSA and never heard of again.  So far this hasn't been the case and NP problems are not computable in an optimal manner in a reasonable amount of time. Hence - luckily for us - long encryption keys are at present undecipherable in a reasonable time, which enables secure global communication networks.  Conspiracy theorists maintain that governments have already developed computational abilities that can solve NP problems - rendering encryption as we know it useless. Dan Brown in his bestseller book Digital Fortress and David Baldacci in his book Simple Genius venture down this path.

Back to project management. Since scheduling is a NP problem, we can't optimally solve a multi project resource constrained system.

Wait just a minute - am I saying that there aren't computerized tools to optimally schedule a bunch of related projects who share resources??? Truth be told, there are tools offering nearly optimal solutions and granted, while they do not provide an optimal solution they calculate, using complex algorithms, a good enough solution. The time required for providing this solution can be very long, depending on the complexity of the problem. There are environments which benefit from these sophisticated optimization tools. Consider for example a medium-size factory with 100 machines and several thousand components. Running the scheduling software for the defined production floor is lengthy and can take up to 3 days. The specific output of the software is a production plan for the explicit setting. The drawback is that once implemented, any unforeseen change in production requires: either rerunning the optimization software which doesn't make sense duration wise, or applying quick fixes which render the nearly optimal production plan obsolete. If ever you watched an operating factory, you have seen the chaotic nature and constant changes with which it is plagued.

There are industries in which nearly optimal scheduling is vital and as the environment is complex, they must invest in a sophisticated software optimization tool (do remember that the solution is nearly optimal). One example is the airline industry. They can't manually allocate numerous aircrafts and particular crews across several hundred destinations worldwide. When Lufthansa allocates specific aircrafts, along with the constraint of having each aircraft operated by a predefined crew, it makes sense to purchase an advanced and pricey optimization tool. Indeed, once a month the scheduling team runs the optimization application which serves as the basis for aircraft and crew allocation, ticket pricing, and routing worldwide. However if one of the pilots is having the shrimp in coconut sauce for dinner and suffers from food poisoning the next day, entire segments of the optimized schedule collapse and emergency escalation protocols are initiated. The central scheduling team in Frankfurt must find a quick fix to the shrimp ordeal.

Ok - for most of our projects it doesn't make sense to invest so much in building the infrastructure to solve, in a near optimal manner, the multi project resource constrained problem.  Especially as the plan is useless once changes occur. All it takes is for a critical resource to be sick one day, for the entire plan to go bust. What is then the alternative? Heuristics - or as we better know them - rules of thumb. These enable local and semi global optimizations. The two famous heuristics to resource constrained project scheduling are: Agile and critical chain. One handles the local optimization and the other semi global optimization. While Goldratt who wrote about critical chain project management knew about heuristics and global optimization, Agile proponents don't really think of it as a local optimization mechanism, but we will leave it for future articles.

More on critical chain, and Goldratt's approach to solving the complexity of resource constrained scheduling, some other time.

A FREE voice over recording of this article is found here.

You can check out my book: The Agile PMO - leading the effective, value-driven, project management office. - - where I further discuss these concepts:

The Agile PMO

More Stories By Michael Nir

Michael Nir - President of Sapir Consulting - (M.Sc. Engineering) has been providing operational, organizational and management consulting and training for over 15 years. He is passionate about Gestalt theory and practice, which complements his engineering background and contributes to his understanding of individual and team dynamics in business. Michael authored 8 Bestsellers in the fields of Influencing, Agile, Teams, Leadership and others. Michael's experience includes significant expertise in the telecoms, hi-tech, software development, R&D environments and petrochemical & infrastructure industries. He develops creative and innovative solutions in project and product management, process improvement, leadership, and team building programs. Michael's professional background is analytical and technical; however, he has a keen interest in human interactions and behaviors. He holds two engineering degrees from the prestigious Technion Institute of Technology: a Bachelor of civil engineering and Masters of Industrial engineering. He has balanced his technical side with the extensive study and practice of Gestalt Therapy and "Instrumental Enrichment," a philosophy of mediated learning. In his consulting and training engagements, Michael combines both the analytical and technical world with his focus on people, delivering unique and meaningful solutions, and addressing whole systems.

@MicroservicesExpo Stories
The reality of data ubiquity is here—data is buried in operational statistics, machine logs, stacks of overflowing tickets and customer details, among other things. How can any user get valuable information amid this rapid influx of data? Imagine a situation where your firm’s revenue takes a hit owing to an unexpected failure in some business process. It would be a nightmare for IT admins to sift through the interminable piles of data to deduce exactly why and where the problem occurred. To sav...
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
"We are a monitoring company. We work with Salesforce, BBC, and quite a few other big logos. We basically provide monitoring for them, structure for their cloud services and we fit into the DevOps world" explained David Gildeh, Co-founder and CEO of Outlyer, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Cloud Expo, Inc. has announced today that Andi Mann and Aruna Ravichandran have been named Co-Chairs of @DevOpsSummit at Cloud Expo Silicon Valley which will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. "DevOps is at the intersection of technology and business-optimizing tools, organizations and processes to bring measurable improvements in productivity and profitability," said Aruna Ravichandran, vice president, DevOps product and solutions marketing...
There's a lot to gain from cloud computing, but success requires a thoughtful and enterprise focused approach. Cloud computing decouples data and information from the infrastructure on which it lies. A process that is a LOT more involved than dragging some folders from your desktop to a shared drive. Cloud computing as a mission transformation activity, not a technological one. As an organization moves from local information hosting to the cloud, one of the most important challenges is addressi...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
In the decade following his article, cloud computing further cemented Carr’s perspective. Compute, storage, and network resources have become simple utilities, available at the proverbial turn of the faucet. The value they provide is immense, but the cloud playing field is amazingly level. Carr’s quote above presaged the cloud to a T. Today, however, we’re in the digital era. Mark Andreesen’s ‘software is eating the world’ prognostication is coming to pass, as enterprises realize they must be...
Both SaaS vendors and SaaS buyers are going “all-in” to hyperscale IaaS platforms such as AWS, which is disrupting the SaaS value proposition. Why should the enterprise SaaS consumer pay for the SaaS service if their data is resident in adjacent AWS S3 buckets? If both SaaS sellers and buyers are using the same cloud tools, automation and pay-per-transaction model offered by IaaS platforms, then why not host the “shrink-wrapped” software in the customers’ cloud? Further, serverless computing, cl...
Hybrid IT is today’s reality, and while its implementation may seem daunting at times, more and more organizations are migrating to the cloud. In fact, according to SolarWinds 2017 IT Trends Index: Portrait of a Hybrid IT Organization 95 percent of organizations have migrated crucial applications to the cloud in the past year. As such, it’s in every IT professional’s best interest to know what to expect.
A common misconception about the cloud is that one size fits all. Companies expecting to run all of their operations using one cloud solution or service must realize that doing so is akin to forcing the totality of their business functionality into a straightjacket. Unlocking the full potential of the cloud means embracing the multi-cloud future where businesses use their own cloud, and/or clouds from different vendors, to support separate functions or product groups. There is no single cloud so...
The taxi industry never saw Uber coming. Startups are a threat to incumbents like never before, and a major enabler for startups is that they are instantly “cloud ready.” If innovation moves at the pace of IT, then your company is in trouble. Why? Because your data center will not keep up with frenetic pace AWS, Microsoft and Google are rolling out new capabilities. In his session at 20th Cloud Expo, Don Browning, VP of Cloud Architecture at Turner, posited that disruption is inevitable for comp...
Companies have always been concerned that traditional enterprise software is slow and complex to install, often disrupting critical and time-sensitive operations during roll-out. With the growing need to integrate new digital technologies into the enterprise to transform business processes, this concern has become even more pressing. A 2016 Panorama Consulting Solutions study revealed that enterprise resource planning (ERP) projects took an average of 21 months to install, with 57 percent of th...
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, Doug Vanderweide, an instructor at Linux Academy, discussed why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers wit...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists examined how DevOps helps to meet the de...
"When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Colocation is a central pillar of modern enterprise infrastructure planning because it provides greater control, insight, and performance than managed platforms. In spite of the inexorable rise of the cloud, most businesses with extensive IT hardware requirements choose to host their infrastructure in colocation data centers. According to a recent IDC survey, more than half of the businesses questioned use colocation services, and the number is even higher among established businesses and busine...
For most organizations, the move to hybrid cloud is now a question of when, not if. Fully 82% of enterprises plan to have a hybrid cloud strategy this year, according to Infoholic Research. The worldwide hybrid cloud computing market is expected to grow about 34% annually over the next five years, reaching $241.13 billion by 2022. Companies are embracing hybrid cloud because of the many advantages it offers compared to relying on a single provider for all of their cloud needs. Hybrid offers bala...
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.