Microservices Expo Authors: Automic Blog, Elizabeth White, Dalibor Siroky, XebiaLabs Blog, John Katrick

Related Topics: @CloudExpo, Microservices Expo, @DevOpsSummit

@CloudExpo: Blog Post

Enterprise Architecture: Ripe for Digital Disruption | @CloudExpo #Microservices

The secret to being an agile architect? Not architecting. The secret to managing an agile organization? Not managing.

Ever since I published The Agile Architecture Revolution, people have been confused by what I mean by Agile Architecture. The crux of the confusion: the difference between architecting a software system and architecting a human/software system.

If our goal of following Agile is to build good software, the theory goes, then we should ask ourselves what kind of architecture our software requires, and by definition, such architecture is Agile Architecture. To this day, if you Google "Agile Architecture," you're likely to uncover discussions that presuppose that definition - unless, of course, your search turns up something I've written.

When I use the phrase Agile Architecture, in contrast, I'm talking about a style of Enterprise Architecture whose primary goal is to make our organizations more agile - in other words, better able to deal with change, and to leverage change for competitive advantage.

To accomplish that enterprisewide goal, we must architect the organization itself - and what is an enterprise but a human/software system?

Emergence and Architecting the Enterprise
The key to Agile Architecture is emergence. In fact, business agility is the emergent property we seek from the Complex Adaptive System (CAS) we call the enterprise. (See my recent Cortex newsletter for a discussion of emergence as it relates to architecture).

Agile Architecture is a set of intentional acts we as individuals should take in order to get our enterprises to exhibit this most important of emergent properties. The question of the day, therefore, is what are these intentional acts? How do we actually go about architecting an enterprise to be agile?

At this point many of the enterprise architects reading this will want to argue over whether the Agile Architecture I'm discussing is actually Enterprise Architecture (EA). Frankly, I don't give a damn what you call it.

Arguing over what is or is not EA - or even worse, what EA should or should not be - is a complete waste of time, and happens to be one of the reasons executives wonder why they're spending so much money on EA in the first place.

For the sake of argument, therefore, let's just say that Agile Architecture is a reinvention of EA, which you can call EA if you want. But whatever you call it, it's essential to understand the difference between architecting a software system and architecting a human/software system, in particular at the enterprise level.

City Planning: The Wrong Metaphor for EA
To make this distinction, let's take a common metaphor for EA - the metaphor of city planning. Cities are made up of city blocks connected by streets, and within each block are buildings that contain homes, offices, etc. Those homes and offices are analogs for various software systems and applications. We might consider the blocks to represent the systems in a particular department or line of business.

City planners deal with city-wide issues like traffic, utilities, and the like, just as EAs should deal with enterprisewide issues like business/IT alignment, efficient business processes, etc. The tools planners use to influence their cities, including zoning regulations, public works investments, etc., are analogs for the tools of the traditional EA, namely the various artifacts and governance policies that are the EA's stock in trade. So, is city planning a useful metaphor for EA?

EAs who appreciate the city planning metaphor will point out that there are plenty of people in the city to be sure, and many of their activities influence or otherwise deal with the citizens in their enterprise. They will also rightly claim that city planning focuses on how cities deal with change, rather than how to assemble a static system like a model railroad layout.

But EA-as-city-planning is not Agile Architecture. In fact, it's just the opposite. The more planned a city is, the less agile it becomes. Why? Because city planning allows for change but not for emergence. The question we should be asking instead is: how do we produce the results we want from an unplanned city?

If we take the complicated problems we have today and seek to instill some sense of order and planning in order to achieve a particular final state, we're heading in the wrong direction. Even if we were able to accomplish this Sisyphean task, we'd be no more agile than when we started.

Self-Organization: The Most Important Tool in the Agile Architect's Tool Belt
If instilling order and planning is the wrong approach to EA, then clearly we must rethink our entire notion of EA. Once again, we can find the answer in complex systems theory, and the principle of self-organization.

My earlier Cortex also discussed the importance of self-organizing teams to achieving desirable emergent properties, with the important caveat that emergence won't appear at the two-pizza team size favored by Agile-centric organizations. Nevertheless, self-organization is the key to emergence, just not at the two-pizza level.

In fact, in the context of the organizations in which they participate, the behavior of individual humans is never emergent. If we focus on influencing individual human behavior, therefore, we're focusing on the wrong thing.

For example, if we can craft a test for the behavior we think we want and select for people who can pass the test, then we are selecting for non-emergent behavior. The better we get at selecting people who pass the test, the less agile our organization becomes.

Because every human being acts autonomously and is thus inherently unpredictable, the emergent properties of human/software systems are what CAS theorists refer to as strongly emergent, because you can't derive the emergent behavior by more careful analysis or control over subsystem behavior.

For this reason the iteration central to applying Agile Architecture is absolutely essential, because you can influence (but not control) the emergent behavior by iterating the initial conditions or other constraints that lead to effective self-organizing teams.

In fact, there's no way to know for sure ahead of time if some policy or process we might put in place to aid our self-organizing teams will actually result in better agility overall. Instead, we must try different things, see what emergent properties result, and feed back that information to improve our policies and processes.

The better and faster our organizations can gather the necessary business insight, feed it back to the decision making processes, and make the decisions that will drive business agility, the more agile our enterprises become.

The Intellyx Take: Is It Architecture?
In my opinion, the iteration of constraints and initial conditions that drive and influence self-organization within the enterprise is the actual role of an architect who is architecting emergent behavior - in particular, business agility.

You may call such activities something else - management practice or some such - and to be sure, we must reinvent management practice along the same lines as EA. But whatever we call it, there needs to be an understanding that creating the conditions that lead to effective self-organizing teams is itself an architectural activity, an activity separate from the architectural activities such teams undertake when their goal is to implement a software system.

Furthermore, self-organization at the team level is insufficient. Emergent patterns never appear at the team level, after all. We must also architect self-organization across teams, remembering all the while that the people within the teams are making their decisions about how they should behave and interact.

Managers cannot manage this self-organization from outside the self-organizing teams - either at the team level or across teams. The reason for this impossibility is brutally obvious once you see it: managing a team from outside is part of organizing that team - and if an external party takes that role, then the team is no longer self-organized.

If you're a manager and you think you'll be out of a job as a result, not to worry. Managers can still be on the teams as participants. Even outside the teams, executives have three important roles: communicate the strategic goals of the organization, delineate the constraints, and get out of the way.

The secret to being an agile architect? Not architecting. The secret to managing an agile organization? Not managing. At least, in any traditional sense of architecting or managing.

The good news is that many organizations are already well on their way to implementing this vision of emergent business agility - enough of them, in fact, that the ones who aren't with the program are increasingly at a competitive disadvantage.

This shift, in fact, is at the heart of digital disruption. Agile Architecture is the secret to weathering the storm. Disrupt or be disrupted - your choice.

Intellyx advises companies on their digital transformation initiatives and helps vendors communicate their agility stories. As of the time of writing, none of the organizations mentioned in this article are Intellyx customers. Image credit: Leeann Cafferata.

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@MicroservicesExpo Stories
The benefits of automation are well documented; it increases productivity, cuts cost and minimizes errors. It eliminates repetitive manual tasks, freeing us up to be more innovative. By that logic, surely, we should automate everything possible, right? So, is attempting to automate everything a sensible - even feasible - goal? In a word: no. Consider this your short guide as to what to automate and what not to automate.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
The nature of test environments is inherently temporary—you set up an environment, run through an automated test suite, and then tear down the environment. If you can reduce the cycle time for this process down to hours or minutes, then you may be able to cut your test environment budgets considerably. The impact of cloud adoption on test environments is a valuable advancement in both cost savings and agility. The on-demand model takes advantage of public cloud APIs requiring only payment for t...
It’s “time to move on from DevOps and continuous delivery.” This was the provocative title of a recent article in ZDNet, in which Kelsey Hightower, staff developer advocate at Google Cloud Platform, suggested that “software shops should have put these concepts into action years ago.” Reading articles like this or listening to talks at most DevOps conferences might make you think that we’re entering a post-DevOps world. But vast numbers of organizations still struggle to start and drive transfo...
Many enterprise and government IT organizations are realizing the benefits of cloud computing by extending IT delivery and management processes across private and public cloud services. But they are often challenged with balancing the need for centralized cloud governance without stifling user-driven innovation. This strategy requires an approach that fundamentally reshapes how IT is delivered today, shifting the focus from infrastructure to services aggregation, and mixing and matching the bes...
Cavirin Systems has just announced C2, a SaaS offering designed to bring continuous security assessment and remediation to hybrid environments, containers, and data centers. Cavirin C2 is deployed within Amazon Web Services (AWS) and features a flexible licensing model for easy scalability and clear pay-as-you-go pricing. Although native to AWS, it also supports assessment and remediation of virtual or container instances within Microsoft Azure, Google Cloud Platform (GCP), or on-premise. By dr...
"Codigm is based on the cloud and we are here to explore marketing opportunities in America. Our mission is to make an ecosystem of the SW environment that anyone can understand, learn, teach, and develop the SW on the cloud," explained Sung Tae Ryu, CEO of Codigm, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
High-velocity engineering teams are applying not only continuous delivery processes, but also lessons in experimentation from established leaders like Amazon, Netflix, and Facebook. These companies have made experimentation a foundation for their release processes, allowing them to try out major feature releases and redesigns within smaller groups before making them broadly available. In his session at 21st Cloud Expo, Brian Lucas, Senior Staff Engineer at Optimizely, discussed how by using ne...
"CA has been doing a lot of things in the area of DevOps. Now we have a complete set of tool sets in order to enable customers to go all the way from planning to development to testing down to release into the operations," explained Aruna Ravichandran, Vice President of Global Marketing and Strategy at CA Technologies, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Let's do a visualization exercise. Imagine it's December 31, 2018, and you're ringing in the New Year with your friends and family. You think back on everything that you accomplished in the last year: your company's revenue is through the roof thanks to the success of your product, and you were promoted to Lead Developer. 2019 is poised to be an even bigger year for your company because you have the tools and insight to scale as quickly as demand requires. You're a happy human, and it's not just...
We just came off of a review of a product that handles both containers and virtual machines in the same interface. Under the covers, implementation of containers defaults to LXC, though recently Docker support was added. When reading online, or searching for information, increasingly we see “Container Management” products listed as competitors to Docker, when in reality things like Rocket, LXC/LXD, and Virtualization are Dockers competitors. After doing some looking around, we have decided tha...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
identify the sources of event storms and performance anomalies will require automated, real-time root-cause analysis. I think Enterprise Management Associates said it well: “The data and metrics collected at instrumentation points across the application ecosystem are essential to performance monitoring and root cause analysis. However, analytics capable of transforming data and metrics into an application-focused report or dashboards are what separates actual application monitoring from relat...
"Opsani helps the enterprise adopt containers, help them move their infrastructure into this modern world of DevOps, accelerate the delivery of new features into production, and really get them going on the container path," explained Ross Schibler, CEO of Opsani, and Peter Nickolov, CTO of Opsani, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
DevOps teams have more on their plate than ever. As infrastructure needs grow, so does the time required to ensure that everything's running smoothly. This makes automation crucial - especially in the server and network monitoring world. Server monitoring tools can save teams time by automating server management and providing real-time performance updates. As budgets reset for the New Year, there is no better time to implement a new server monitoring tool (or re-evaluate your current solution)....
While we understand Agile as a means to accelerate innovation, manage uncertainty and cope with ambiguity, many are inclined to think that it conflicts with the objectives of traditional engineering projects, such as building a highway, skyscraper or power plant. These are plan-driven and predictive projects that seek to avoid any uncertainty. This type of thinking, however, is short-sighted. Agile approaches are valuable in controlling uncertainty because they constrain the complexity that ste...
"This all sounds great. But it's just not realistic." This is what a group of five senior IT executives told me during a workshop I held not long ago. We were working through an exercise on the organizational characteristics necessary to successfully execute a digital transformation, and the group was doing their ‘readout.' The executives loved everything we discussed and agreed that if such an environment existed, it would make transformation much easier. They just didn't believe it was reali...
"We're developing a software that is based on the cloud environment and we are providing those services to corporations and the general public," explained Seungmin Kim, CEO/CTO of SM Systems Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
The cloud revolution in enterprises has very clearly crossed the phase of proof-of-concepts into a truly mainstream adoption. One of most popular enterprise-wide initiatives currently going on are “cloud migration” programs of some kind or another. Finding business value for these programs is not hard to fathom – they include hyperelasticity in infrastructure consumption, subscription based models, and agility derived from rapid speed of deployment of applications. These factors will continue to...