Microservices Expo Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, Zakia Bouachraoui

Related Topics: @DevOpsSummit, Microservices Expo, Containers Expo Blog, @CloudExpo

@DevOpsSummit: Blog Post

DevOps Insights into Conway’s Law By @TheEbizWizard | @DevOpsSummit #DevOps #Microservices

Do organizational structures determine system design, or vice versa?

Both digital transformation and DevOps are organizational and cultural transformations more so than they are technology changes – although in both cases, technology plays a large part in driving the organizational change necessary to achieve the business value for either effort. Just how the organizational changes and technology changes work together, however, is a difficult question.

Fortunately for us, this question is an old one – 47 years old, in fact. In a 1968 paper, computer scientist Melvin Conway wrote, “Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization’s communication structure.”

Conway produced no evidence of this statement, and his paper was initially rejected as a result – but to this day, we refer to this statement as Conway’s Law.

Law, however, is decidedly an overstatement. Observation is actually more accurate. In fact, the Wikipedia page for Conway’s Law states that “Although sometimes construed as humorous, Conway’s law was intended as a valid sociological observation.”

Whether the statement be humorous observation or law, however, today Conway’s Law plays a central role in our efforts to break down organizational silos in order to improve business velocity and better meet the needs of customers – the purported goals of devops and digital transformation, respectively.

For Conway’s Law to be a useful tool, however, we need a better causal story. Can changing our organizational structures impact our technology? Or more importantly, how does technology impact our organizational structures? And why?

A Brief History of Conway’s Law
Melvin Conway
was an early computer scientist whose most interesting accomplishment other than the law itself may have been his part in creating the compiler for the original 1984 version of Mac Pascal.

His eponymous law, however, apparently languished until sometime before 1996 or perhaps even earlier than 1991, when open source guru Eric Raymond included Conway’s Law in his Jargon File. He restated it with the snarky example, “If you have four groups working on a compiler, you’ll get a 4-pass compiler.”

Our story then gets more interesting when the Harvard Business School attempted to put some meat on the bones in a 2008 study. They use an objective measure of the modularity of software to find evidence for Conway’s Law. They conclude:

  • A product’s architecture tends to mirror the structure of the organization within which it is developed.
  • New organizational arrangements can have a distinct impact on the nature of the resulting design, and hence may affect product performance in unintended ways.

It’s important to note two important updates to the context of Conway’s Law by the time of the 2008 study: an organization’s “communication structure” becomes the structure of the organization itself, while “system design” becomes “product architecture” – with the constraint to product architecture being the scope of the study, rather than any sort of judgment on the scope of Conway’s Law itself.

What the 2008 study is essentially lacking, furthermore, is a discussion of the underlying causal principles behind the law. Instead, it is more or less taken for granted that siloed technology teams will create modular systems with the modularity aligning to the team structure simply because that’s the way people behave when they’re in teams.

The reason this question of causality is so important is because it goes to the heart of how we might use Conway’s Law to actually improve things. In particular, will changing organizational structures enable us to build better software?

Jonny Leroy and Matt Simons of ThoughtWorks explored this question when they coined the term “Inverse Conway Maneuver” in an article in the 2010 issue of the Cutter IT Journal. They state:

Conway’s Law … can be summarized as “Dysfunctional organizations tend to create dysfunctional applications.” To paraphrase Einstein, you can’t fix a problem from within the same mindset that created it, so it is often worth investigating whether restructuring your organization or team would prevent the new application from displaying all the same structural dysfunctions as the original. In what could be termed an “inverse Conway maneuver,” you may want to begin by breaking down silos that constrain the team’s ability to collaborate effectively.

The ThoughtWorks article takes the context of Conway’s Law from observational to normative: not satisfied simply to reflect on the way things work, they take the important step of opining on how things should – and should not work. They also intentionally focus on the negative: rather than simply describing how to build good software, they note that Conway’s Law is about building dysfunctional software.

Reversing the Inverse Conway Maneuver
It’s no surprise that ThoughtWorks’ focus is on changing organizational structures in order to build better software – after all, ThoughtWorks is a software development organization, and most modern software development thinking, including both the Agile and Lean movements, focuses on organizational change in furtherance of better software.

For digital transformation efforts, however, we must reverse this discussion. Technology change is driving changing customer preferences and behavior, which in turn are driving organizational change across increasingly software-driven enterprises.

The causality question behind Conway’s Law, therefore, is less about how changing software organizations can lead to better software, but rather how companies can best leverage changing technology in order to transform their organizations.

Hints at how to answer this question surprisingly come from the world of devops – surprising because the focus of devops is ostensibly on building and deploying better software more quickly. Be that as it may, there’s no question that technology change is a primary facilitator and driving force for the devops cultural and organizational shifts.

Connecting the Dots to Conway’s Law
If we didn’t have the cloud computing example of fully automated deployment and operational environments, and if we didn’t have today’s dramatic innovations in continuous development, continuous integration, and continuous delivery tooling, then devops would never have left the whiteboard stage. There’s no question that the devops story is a tale of technology-driven organizational change.

The DevOps technology landscape, however, doesn’t have an end-to-end, seamless technology story. On the contrary, this landscape is cluttered with dozens of various tools, many open source, all of which are in various stages of maturity. Therefore, it’s not readily apparent how to apply Conway’s Law, since we’re trying to leverage diverse toolchains of tools and technologies in order to help evolve our organizational structures.

In fact, Conway’s Law describes how such a diverse tooling marketplace came to be in the first place, as open source teams are generally quite modular, and thus will produce modular software.

Once we’ve solved the organizational challenges of digital and DevOps, breaking down silos in order to deliver customer value at velocity, then we can expect Conway’s Law to kick in again, and predict the rise of end-to-end software solutions as the result of horizontally self-organized teams.

It’s the middle piece, however, we’re struggling with now: how do we leverage a diverse set of disparate technologies to facilitate the cross-cutting reorganization of our businesses, contrary to the observation of Conway’s Law? And do we really think going against Conway’s Law will actually work?

Not to fear. In fact, the technologies that underpin DevOps aren’t as diverse and disparate as the application of Conway’s Law might suggest. True, the creators of all the tools in our DevOps or digital tool belts are working mostly independently of each other, a pattern of behavior which in the past has led to incompatible software mishmashes.

Today, however, we’re seeing the rise of what we might call crowdsourced architecture, as all of these teams work within the context of mature communication protocols, RESTful interfaces, and other emerging architectural trends like containerization and microservices.

As a result, mostly independent doesn’t mean completely independent. Instead, we have the loosely connected communication structure that assures us that Conway’s Law is still alive and well. What’s changed is the context for this notion of an organizational communication structure.

Conway was referring to the communication structures of companies or software development organizations or perhaps individual software development teams. Today, however, we’re talking about the broader technology community itself.

The Intellyx Take
There was no way Melvin Conway could have observed such crowdsourced architectural maturity back in 1968, because the telephone and snail mail-based communication structures of the day weren’t conducive to crowdsourcing anything. In contrast, today we have a plethora of tools and processes for facilitating communication and collaboration across traditional projects, teams, and open source efforts.

The end result is essentially a two-level application of Conway’s Law: a collaborative extended community of technologists that creates not simply a collection of disparate tools but rather chainable tools that leverage crowdsourced architectural principles to facilitate a level of coordination and interactivity we’ve never seen before.

This coordinated technology environment, in turn, facilitates the reorganization within companies, as they now have the tools they need to break down organizational silos, and people within those companies self-organize along horizontal lines, connecting customer experience to back-office software development and operations.

Conway’s Law, therefore, does work both ways. Organizational structures impact system design, and system architectures impact organizational structures as well. In the final analysis, however, Conway’s Law remains stubbornly observational. The underlying causal story – why such observations are remarkably universal – remains to be told. Stay tuned!

Intellyx advises companies on their digital transformation initiatives and helps vendors communicate their agility stories. As of the time of writing, none of the organizations mentioned in this article are Intellyx customers. Image credit: Melvin Conway.

More Stories By Jason Bloomberg

Jason Bloomberg is a leading IT industry analyst, Forbes contributor, keynote speaker, and globally recognized expert on multiple disruptive trends in enterprise technology and digital transformation. He is ranked #5 on Onalytica’s list of top Digital Transformation influencers for 2018 and #15 on Jax’s list of top DevOps influencers for 2017, the only person to appear on both lists.

As founder and president of Agile Digital Transformation analyst firm Intellyx, he advises, writes, and speaks on a diverse set of topics, including digital transformation, artificial intelligence, cloud computing, devops, big data/analytics, cybersecurity, blockchain/bitcoin/cryptocurrency, no-code/low-code platforms and tools, organizational transformation, internet of things, enterprise architecture, SD-WAN/SDX, mainframes, hybrid IT, and legacy transformation, among other topics.

Mr. Bloomberg’s articles in Forbes are often viewed by more than 100,000 readers. During his career, he has published over 1,200 articles (over 200 for Forbes alone), spoken at over 400 conferences and webinars, and he has been quoted in the press and blogosphere over 2,000 times.

Mr. Bloomberg is the author or coauthor of four books: The Agile Architecture Revolution (Wiley, 2013), Service Orient or Be Doomed! How Service Orientation Will Change Your Business (Wiley, 2006), XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996). His next book, Agile Digital Transformation, is due within the next year.

At SOA-focused industry analyst firm ZapThink from 2001 to 2013, Mr. Bloomberg created and delivered the Licensed ZapThink Architect (LZA) Service-Oriented Architecture (SOA) course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, which was acquired by Dovel Technologies in 2011.

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting), and several software and web development positions.

Microservices Articles
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addresse...
Adding public cloud resources to an existing application can be a daunting process. The tools that you currently use to manage the software and hardware outside the cloud aren’t always the best tools to efficiently grow into the cloud. All of the major configuration management tools have cloud orchestration plugins that can be leveraged, but there are also cloud-native tools that can dramatically improve the efficiency of managing your application lifecycle. In his session at 18th Cloud Expo, ...
The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.