Welcome!

SOA & WOA Authors: Carmen Gonzalez, Lori MacVittie, Trevor Parsons, Jason Bloomberg, Keith Cawley

Related Topics: SOA & WOA, Java, XML, .NET, AJAX & REA, Web 2.0

SOA & WOA: Article

The Odd Couple: Marrying Agile and Waterfall

Interfacing between Linear Waterfall and Agile Approaches

This article depicts the best practice approach for integrating Agile approaches and specifically Scrum development with traditional overarching linear approaches, specifically waterfall methodology. The agile PMO, properly defined, can be positioned to secure Agile-Scrum benefits while maintaining the necessary overarching control.

The challenge
Over the last two decades, various Agile approaches have been introduced and practiced. Of these, in last 5 to 7 years, Scrum has gained the most popularity resulting from a combination of simplicity, ease of use, and effective public relations. Scrum success in software development organizations has been a powerful driver for roll outs across products, industries and businesses. As described, this was exacerbated by a focused marketing effort from Scrum evangelists. Unfortunately, most of these organizations were not structured in a way that supports the Agile-Scrum approach and methods. Even more so, scrum in its raw and pure form is not suitable for the majority of organizations.

The first wave of failed Agile-Scrum implementations brought about an even stricter admonition, based on an unwavering belief from Scrum zealots. The main assertion has been that failed implementations are a result of partial embracing of the true scrum spirit and the full benefits of the approach can only be attained if the entire organization is reengineered. This fanatical attitude left many project teams across organizations big and small, struggling with their already idealistic implementations. Some have been figuring out on their own, how to combine the contemporary and traditional worlds. Other teams have completely abandoned the Agile-Scrum concepts reverting back to the traditional linear waterfall approach and method. Yet other teams, ridden with guilt, manage Scrum by name only, and hiss vehemently at any project management proponent who is unfortunate enough to advise on re-embracing Agile in a more cognizant approach.

The concepts which are presented and embodied in Agile-Scrum are too good to be ignored; however implementing it outside pure software development requires adaptation.

Complex scenarios for Agile
The main hurdle in achieving the benefits of Agile- Scrum outside software development is integrating it with existing control mechanisms. These control mechanisms are stipulated due to various organizational prerequisites and are normally actualized by using the Linear Waterfall approach and methodology. Four of these typical organizational prerequisites are depicted below:

  • Big global corporates: in these hierarchical matrix organizations, top down portfolio control is the rule of the day. The free spirited agile approach has a tough time adjusting to the rigorous controls. Specifically the inherent Agile plan-free concepts, make Agile-Scrum difficult for the organization to swallow.
  • Highly regulated industries: some industries are required by compliance and governance bodies to have a strict binding control mechanism. These can be for example medical equipment, aircraft, and pharmaceuticals research and product development business units. While individual teams might operate Agile-Scrum, the development process must follow a rigid Linear Waterfall approach method for traceability and governance.
  • Complex predefined products: some integrated products which include both hardware, software, imbedded and more are developed as a contract with an end customer under a predefined set of requirements. In these cases the degree of requirement flexibility is small, though larger than what is anticipated initially. The concept of a fully flexible backlog of Agile-Scrum suffers considerably in these cases.
  • Generic IT departments: much of the daily and weekly activities in maintenance driven IT departments is ad hoc. Changes to the daily schedules are numerous and immediate. Constant interferences in the teams work are the norm. The concept of time boxing and no interference is difficult to maintain in these situations.

Naturally - many times the four discrete categories detailed above, actually mix; so it is common to find a complex product in a global big corporate which is required to comply with firm regulation.

Based on practical experience, the recommended method to tackle these scenarios and others is by empowering the Agile PMO to act as an enabler, driver and translator between the emerging Agile-Scrum teams and the Linear Waterfall elements.

Refer to the table below for specific guidelines

The Agile PMO - leading the hybrid organization - guidelines

Scenario

Challenge

Possible solution

Comments

Insights

Big global corporates

Strict controls manifested in Linear Waterfall

The Agile PMO is the buffer between Agile-Scrum teams and the Linear view

Burn down charts are translated to phases for control;

Requirement traceability done by PMO architect;

Agile PMO maintains the dictionary between sprint planning, execution and the phase gating mechanism

Product owners can be part of the Agile PMO;

Project initiating and closing managed by the PMO

Scenario

Challenge

Possible solution

Comments

Insights

Highly regulated industries

Strict compliance and paper trail requirement including product risk analysis

The Agile PMO is also resourced by administrative staff to ensure compliance with regulations

Product risk is managed on a lifecycle view with members of the Scrum-Agile team;

Backlog populated by Non-functional yet critical requirement and owned by the Agile PMO

Agile PMO staff maintains traceability of these requirements.

Necessary documentation is part of the backlog

The added administrative effort handled by the PMO is compensated by the increased velocity of the Agile teams.

Administrative PMO staff can also be non-functional product owners to ensure compliance aspects

Scenario

Challenge

Possible solution

Comments

Insights

Complex predefined products

Limited flexibility in product scope tends to deteriorate  Agile implementations to Agile by name only; Also, hardware elements of product can't be performed in an Agile approach

The Agile PMO owns the backlog interfacing with the various components of product development - managing a hybrid Agile-Linear project

This is probably the most difficult and tricky scenario to tackle;

It requires technical as well as leadership propensity and know-how.

Experience shows that by investigating creatively - Agile concepts can be implemented in rigid hardware development environments

Also - rigid product requirements still allow usually 20% flexibility

the most value added can be reaped in this scenario by developing a customized mixed approach;

Agile stage deliveries can be used to increase flexibility.

Concepts of incremental deliveries may sometimes not be achievable in all product aspects

Scenario

Challenge

Possible solution

Comments

Insights

Generic IT departments

Constant changes to team's work, inability to see the big picture due to ad-hoc work interfering;

missing a true product owner

The Agile PMO substitutes the product owner role in acting as a buffer to oncoming requests also protecting effort to reasonable levels

Many disheartened IT departments have become bitter when trying to use Agile to their development and ongoing work; the result has been fatigue laden team, viewing Agile as a vicious manipulation to increase output without genuine management support; more than a single project management approach can be practiced

Noticeably, Kanban works better for these environments;

Time boxing still makes sense, however a certain predefined buffer for ad-hoc work should be built into each sprint; Sprint durations should be flexible

 

 

Important best practices to remember that go hand in hand with the Agile PMO:

  • Implementing Agile-Scrum as a restricting admonition is exploiting the adaptive nature of Agile
  • There is no one right way - no one size fits them all;
  • There is no silver bullet - you can create what works for you;
  • Being agile and adaptive also allows being flexible with how one uses the methods, process and Methodology;
  • Time boxing is great as long as you are flexible to change the durations of the time box if necessary;
  • Sometimes the client isn't directly available, in these cases marketing and product management are a proper alternate;
  • Arbitrary rules don't complete projects, people do! Empower your team and yourself to choose the appropriate mix of approaches that enable product delivery.

Summary
In my book about the Agile PMO I describe how PMOs can succeed only if they create and enhance value through smart portfolio selection (more about that in a future white paper). With the emerging of Agile approaches and specifically Scrum methods new opportunities have become apparent. Integrating them into an existing control structure - typically presented by a waterfall lifecycle - can be frustrating. We have defined a new key player - the Agile PMO which can be positioned to create a transformation / translation layer between the approached and methods, contributing to higher success levels of these integrations.

 

The Agile PMO Michael Nir

More Stories By Michael Nir

Michael Nir - President of Sapir Consulting - (M.Sc. Engineering) has been providing operational, organizational and management consulting and training for over 15 years. He is passionate about Gestalt theory and practice, which complements his engineering background and contributes to his understanding of individual and team dynamics in business. Michael authored 8 Bestsellers in the fields of Influencing, Agile, Teams, Leadership and others. Michael's experience includes significant expertise in the telecoms, hi-tech, software development, R&D environments and petrochemical & infrastructure industries. He develops creative and innovative solutions in project and product management, process improvement, leadership, and team building programs. Michael's professional background is analytical and technical; however, he has a keen interest in human interactions and behaviors. He holds two engineering degrees from the prestigious Technion Institute of Technology: a Bachelor of civil engineering and Masters of Industrial engineering. He has balanced his technical side with the extensive study and practice of Gestalt Therapy and "Instrumental Enrichment," a philosophy of mediated learning. In his consulting and training engagements, Michael combines both the analytical and technical world with his focus on people, delivering unique and meaningful solutions, and addressing whole systems.

@ThingsExpo Stories
Noted IoT expert and researcher Joseph di Paolantonio (pictured below) has joined the @ThingsExpo faculty. Joseph, who describes himself as an “Independent Thinker” from DataArchon, will speak on the topic of “Smart Grids & Managing Big Utilities.” Over his career, Joseph di Paolantonio has worked in the energy, renewables, aerospace, telecommunications, and information technology industries. His expertise is in data analysis, system engineering, Bayesian statistics, data warehouses, business intelligence, data mining, predictive methods, and very large databases (VLDB). Prior to DataArchon, he served as a VP and Principal Analyst with Constellation Group. He is a member of the Boulder (Colo.) Brain Trust, an organization with a mission “to benefit the Business Intelligence and data management industry by providing pro bono exchange of information between vendors and independent analysts on new trends and technologies and to provide vendors with constructive feedback on their of...
All major researchers estimate there will be tens of billions devices – computers, smartphones, tablets, and sensors – connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be!
Software AG helps organizations transform into Digital Enterprises, so they can differentiate from competitors and better engage customers, partners and employees. Using the Software AG Suite, companies can close the gap between business and IT to create digital systems of differentiation that drive front-line agility. We offer four on-ramps to the Digital Enterprise: alignment through collaborative process analysis; transformation through portfolio management; agility through process automation and integration; and visibility through intelligent business operations and big data.
There will be 50 billion Internet connected devices by 2020. Today, every manufacturer has a propriety protocol and an app. How do we securely integrate these "things" into our lives and businesses in a way that we can easily control and manage? Even better, how do we integrate these "things" so that they control and manage each other so our lives become more convenient or our businesses become more profitable and/or safe? We have heard that the best interface is no interface. In his session at Internet of @ThingsExpo, Chris Matthieu, Co-Founder & CTO at Octoblu, Inc., will discuss how these devices generate enough data to learn our behaviors and simplify/improve our lives. What if we could connect everything to everything? I'm not only talking about connecting things to things but also systems, cloud services, and people. Add in a little machine learning and artificial intelligence and now we have something interesting...
Last week, while in San Francisco, I used the Uber app and service four times. All four experiences were great, although one of the drivers stopped for 30 seconds and then left as I was walking up to the car. He must have realized I was a blogger. None the less, the next car was just a minute away and I suffered no pain. In this article, my colleague, Ved Sen, Global Head, Advisory Services Social, Mobile and Sensors at Cognizant shares his experiences and insights.
We are reaching the end of the beginning with WebRTC and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will want to use their existing identities, but these will have credentials already that are (hopefully) irreversibly encoded. In his session at Internet of @ThingsExpo, Peter Dunkley, Technical Director at Acision, will look at how this identity problem can be solved and discuss ways to use existing web identities for real-time communication.
Can call centers hang up the phones for good? Intuitive Solutions did. WebRTC enabled this contact center provider to eliminate antiquated telephony and desktop phone infrastructure with a pure web-based solution, allowing them to expand beyond brick-and-mortar confines to a home-based agent model. It also ensured scalability and better service for customers, including MUY! Companies, one of the country's largest franchise restaurant companies with 232 Pizza Hut locations. This is one example of WebRTC adoption today, but the potential is limitless when powered by IoT. Attendees will learn real-world benefits of WebRTC and explore future possibilities, as WebRTC and IoT intersect to improve customer service.
From telemedicine to smart cars, digital homes and industrial monitoring, the explosive growth of IoT has created exciting new business opportunities for real time calls and messaging. In his session at Internet of @ThingsExpo, Ivelin Ivanov, CEO and Co-Founder of Telestax, will share some of the new revenue sources that IoT created for Restcomm – the open source telephony platform from Telestax. Ivelin Ivanov is a technology entrepreneur who founded Mobicents, an Open Source VoIP Platform, to help create, deploy, and manage applications integrating voice, video and data. He is the co-founder of TeleStax, an Open Source Cloud Communications company that helps the shift from legacy IN/SS7 telco networks to IP-based cloud comms. An early investor in multiple start-ups, he still finds time to code for his companies and contribute to open source projects.
The Internet of Things (IoT) promises to create new business models as significant as those that were inspired by the Internet and the smartphone 20 and 10 years ago. What business, social and practical implications will this phenomenon bring? That's the subject of "Monetizing the Internet of Things: Perspectives from the Front Lines," an e-book released today and available free of charge from Aria Systems, the leading innovator in recurring revenue management.
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges.
There’s Big Data, then there’s really Big Data from the Internet of Things. IoT is evolving to include many data possibilities like new types of event, log and network data. The volumes are enormous, generating tens of billions of logs per day, which raise data challenges. Early IoT deployments are relying heavily on both the cloud and managed service providers to navigate these challenges. In her session at 6th Big Data Expo®, Hannah Smalltree, Director at Treasure Data, to discuss how IoT, Big Data and deployments are processing massive data volumes from wearables, utilities and other machines.
P2P RTC will impact the landscape of communications, shifting from traditional telephony style communications models to OTT (Over-The-Top) cloud assisted & PaaS (Platform as a Service) communication services. The P2P shift will impact many areas of our lives, from mobile communication, human interactive web services, RTC and telephony infrastructure, user federation, security and privacy implications, business costs, and scalability. In his session at Internet of @ThingsExpo, Erik Lagerway, Co-founder of Hookflash, will walk through the shifting landscape of traditional telephone and voice services to the modern P2P RTC era of OTT cloud assisted services.
While great strides have been made relative to the video aspects of remote collaboration, audio technology has basically stagnated. Typically all audio is mixed to a single monaural stream and emanates from a single point, such as a speakerphone or a speaker associated with a video monitor. This leads to confusion and lack of understanding among participants especially regarding who is actually speaking. Spatial teleconferencing introduces the concept of acoustic spatial separation between conference participants in three dimensional space. This has been shown to significantly improve comprehension and conference efficiency.
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, will discuss single-value, geo-spatial, and log time series data. By focusing on enterprise applications and the data center, he will use OpenTSDB as an example to explain some of these concepts including when to use different storage models.
SYS-CON Events announced today that Gridstore™, the leader in software-defined storage (SDS) purpose-built for Windows Servers and Hyper-V, will exhibit at SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Gridstore™ is the leader in software-defined storage purpose built for virtualization that is designed to accelerate applications in virtualized environments. Using its patented Server-Side Virtual Controller™ Technology (SVCT) to eliminate the I/O blender effect and accelerate applications Gridstore delivers vmOptimized™ Storage that self-optimizes to each application or VM across both virtual and physical environments. Leveraging a grid architecture, Gridstore delivers the first end-to-end storage QoS to ensure the most important App or VM performance is never compromised. The storage grid, that uses Gridstore’s performance optimized nodes or capacity optimized nodes, starts with as few a...
The Transparent Cloud-computing Consortium (abbreviation: T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data processing High speed and high quality networks, and dramatic improvements in computer processing capabilities, have greatly changed the nature of applications and made the storing and processing of data on the network commonplace. These technological reforms have not only changed computers and smartphones, but are also changing the data processing model for all information devices. In particular, in the area known as M2M (Machine-To-Machine), there are great expectations that information with a new type of value can be produced using a variety of devices and sensors saving/sharing data via the network and through large-scale cloud-type data processing. This consortium believes that attaching a huge number of devic...
Innodisk is a service-driven provider of industrial embedded flash and DRAM storage products and technologies, with a focus on the enterprise, industrial, aerospace, and defense industries. Innodisk is dedicated to serving their customers and business partners. Quality is vitally important when it comes to industrial embedded flash and DRAM storage products. That’s why Innodisk manufactures all of their products in their own purpose-built memory production facility. In fact, they designed and built their production center to maximize manufacturing efficiency and guarantee the highest quality of our products.
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. Over the summer Gartner released its much anticipated annual Hype Cycle report and the big news is that Internet of Things has now replaced Big Data as the most hyped technology. Indeed, we're hearing more and more about this fascinating new technological paradigm. Every other IT news item seems to be about IoT and its implications on the future of digital business.
Can call centers hang up the phones for good? Intuitive Solutions did. WebRTC enabled this contact center provider to eliminate antiquated telephony and desktop phone infrastructure with a pure web-based solution, allowing them to expand beyond brick-and-mortar confines to a home-based agent model. Download Slide Deck: ▸ Here
BSQUARE is a global leader of embedded software solutions. We enable smart connected systems at the device level and beyond that millions use every day and provide actionable data solutions for the growing Internet of Things (IoT) market. We empower our world-class customers with our products, services and solutions to achieve innovation and success. For more information, visit www.bsquare.com.