Welcome!

Microservices Expo Authors: Derek Weeks, Mehdi Daoudi, Pat Romanski, Liz McMillan, Elizabeth White

Related Topics: @CloudExpo, Containers Expo Blog, SDN Journal, @DevOpsSummit

@CloudExpo: Article

What ‘Software-Defined’ Really Means | @CloudExpo #AI #SDN #SDX #DevOps

It’s time to bring some clarity into the big picture of SD – what it is, and perhaps even more importantly, what it is not.

The visual model to declarative metadata representation to immutable deployment vision is in essence what SD is all about.

The secret to making this approach practical, and thus the key to understanding why SD approaches have become so prevalent, is the word immutable.

Once we get an SD approach right, we no longer have to touch the deployed technology whatsoever. Instead, to make a change, update the model and redeploy.

In a recent Cortex, I bemoaned the fact that as buzzwords go, Digital Transformation is excessively vague. There is yet another buzzword of our times that is suffering the same fate: Software-Defined.

Rare though buzz-adjectives may be among the pantheon of buzz-nouns and the occasional buzz-verb, Software-Defined (SD) has become remarkably pervasive. In fact, it ties together many different, quite disparate concepts into what has become a vague mishmash.

It's time to bring some clarity into the big picture of SD - what it is, and perhaps even more importantly, what it is not.

The Many Uses of Software-Defined
The most concrete use of the SD adjective is perhaps in the phrase Software-Defined Networking (SDN). SDN separates network equipment's control plane (where routing instructions and other metadata go) from the data plane (where the data being routed go), and then shifts the entire control plane to centralized software.

The network, however, is only the beginning. We have SD infrastructure (SDI), SD data centers (SDDCs), SD wide-area networking (SD-WAN), and more. Each of these approaches follows the lead of SDN, shifting control of various pieces of hardware (or virtualized hardware) to centralized, software-based management and configuration applications.

SDI (which includes SDN), in fact, is at the core of cloud computing. Clearly, there's no way to scale a cloud data center if people had to run from server to server making changes.

Furthermore, Network Functions Virtualization (NFV) from the telco world also falls under the SD banner. With NFV, telco service providers shift all control to software, so that the underlying hardware is entirely generic. No more dedicated switches, routers, and specialized telco gear - all the hardware consist of generic, white-label boxes.

Software-Defined: Beyond the Network
While the network-centric context of SD in corporate networks, cloud data centers, and telco infrastructure forms the home base of the SD movement, SDI is also an essential enabler of continuous integration and continuous delivery (CI/CD), core elements of DevOps.

In order to achieve the velocity that CI/CD promise, the ops part of the story must be SD. Instead of ops people managing servers individually, the DevOps team must be able to deploy and manage software automatically via centralized software control. In other words, the immutable infrastructure principle behind DevOps is nothing more than SDI.

In fact, now that virtualization has matured, all the infrastructure from hypervisors down to bare metal is SD.

At the application level, however, the SD story gets more complicated.

Using software to automate the tasks involved in deploying software is nothing new. Developers have been using runbooks for years - scripts that tell various parts of the environment to execute a series of tasks in a particular sequence.

As DevOps has matured, the notion of the mundane runbook has taken on new life, as DevOps vendors automate increasingly broad swaths of the software development lifecycle (SDLC) with ‘recipes' or other scripting approaches.

As applications and the environments they run in get more complicated, however, the world of DevOps automation finds itself in a Catch-22: the automation scripts or recipes themselves become increasingly complex software applications in their own right, and thus must go through an SDLC of their own, with all the testing and governance that go along with it.

As a result, we're back to square one, manually creating, managing, deploying, and versioning software.

Does Software-Defined Mean Declarative?
To address this Catch-22, some DevOps tools take a declarative approach. Instead of scripting the environment step by step, the declarative approach enables the user to describe the desired behavior, and then the tool interprets such a description and takes the necessary actions to implement such behavior out of sight of the user.

In fact, in many contexts, when most vendors say SD, they really mean that they take a declarative approach, separating configuration from the underlying implementation. There's more to SD behavior than simply following a declarative approach, however.

For example, HTML (and markup languages in general) are declarative. And while we could certainly hand-code a web page by pecking out HTML, we're far more likely to use a visual tool for that purpose.

When we build a web site using such a tool, we're essentially working with models. The model is a visual, configurable representation of the page that the tool can convert into HTML for browsers to render into the page itself for users to view.

In this example, therefore, we have three different ways of thinking about the page: as a visual model, independent of any particular technology implementation of the page; as the HTML markup for that page; and as the action of the browser itself, an application purpose-built to render HTML into visual pages.

Architects and other shrewd readers will recognize the pattern above as being an instance of Model-Driven Architecture (MDA), or its common implementation, Model-Driven Development (MDD).

Does Software-Defined Mean Model-Driven?
MDA is an Object Management Group (OMG) standard
for creating metamodels that represent platform-independent models (our visual model, above) and platform-specific models (the HTML markup in the example), as well as an abstracted approach for turning the former into the latter.

Models, especially visual ones, are in broad use today, but MDA and MDD's best days are behind them. The reason: they didn't deal as well with change as MDA's creators had hoped.

In the MDD world, a developer might build a (platform-independent) model of an application in a model-driven tool and then push a button and out would pop the (platform-specific) source code that represented the working application.

However, if developers wanted to subsequently make a change, they would either need to change the model and regenerate and redeploy all the code (an onerous and time-consuming task), or tweak the auto-generated code itself, thus making it inconsistent with the model.

Round-trip tooling that would take tweaked code and automatically update the model - the holy grail of MDD - has proven impractical.

If we combine some of the principles from MDD with the declarative approach, however, we finally see some light at the end of the tunnel. Instead of the code-generating context of MDA reminiscent of CASE tools of yore, the platform-specific representation for a declarative model consists of a metadata representation of a configuration.

In practice, tools that take this approach create such metadata representations in JSON, XML, or a domain-specific language appropriate to the task at hand. Developers occasionally have reason to view such metadata, but rarely if ever have call to monkey with it directly.

Instead, users - who need not be developers - simply make changes in the model, typically via direct interaction with icons or other visual elements, or by selecting appropriate configurations. The underlying platform takes care of the rest.

The Intellyx Take
The round-trip code-generation vision of MDD proved unworkable, but the visual model to declarative metadata representation to immutable deployment vision is in essence what SD is all about.

The secret to making this approach practical, and thus the key to understanding why SD approaches have become so prevalent, is the word immutable.

Once we get an SD approach right, we no longer have to touch the deployed technology whatsoever. Instead, to make a change, update the model and redeploy.

The most important takeaway from this Cortex: this core SD pattern is fully generalizable. It works with networks, data centers, DevOps-based deployments, and as I'll cover in part two, it's also at the core of the Low-Code/No-Code movement.

It's no wonder, therefore, that Software-Defined Everything (SDX) is rising to the top of the buzzword heap - but SDX is no mere buzzword. It describes the central technological principles behind Agile Digital Transformation.

Copyright © Intellyx LLC. Intellyx publishes the Agile Digital Transformation Roadmap poster, advises companies on their digital transformation initiatives, and helps vendors communicate their agility stories. As of the time of writing, none of the organizations mentioned in this article are Intellyx customers. Image credit: Tim Adams.

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

@MicroservicesExpo Stories
We have Continuous Integration and we have Continuous Deployment, but what’s continuous across all of what we do is people. Even when tasks are automated, someone wrote the automation. So, Jayne Groll evangelizes about Continuous Everyone. Jayne is the CEO of the DevOps Institute and the author of Agile Service Management Guide. She talked about Continuous Everyone at the 2016 All Day DevOps conference. She describes it as "about people, culture, and collaboration mapped into your value streams....
In our first installment of this blog series, we went over the different types of applications migrated to the cloud and the benefits IT organizations hope to achieve by moving applications to the cloud. Unfortunately, IT can’t just press a button or even whip up a few lines of code to move applications to the cloud. Like any strategic move by IT, a cloud migration requires advanced planning.
Did you know that you can develop for mainframes in Java? Or that the testing and deployment can be automated across mobile to mainframe? In his session and demo at @DevOpsSummit at 21st Cloud Expo, Dana Boudreau, a Senior Director at CA Technologies, will discuss how increasingly teams are developing with agile methodologies, using modern development environments, and automating testing and deployments, mobile to mainframe.
“Why didn’t testing catch this” must become “How did this make it to testing?” Traditional quality teams are the crutch and excuse keeping organizations from making the necessary investment in people, process, and technology to accelerate test automation. Just like societies that did not build waterways because the labor to keep carrying the water was so cheap, we have created disincentives to automate. In her session at @DevOpsSummit at 20th Cloud Expo, Anne Hungate, President of Daring System...
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
Most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes a lot of work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reduction in cost ...
Docker is on a roll. In the last few years, this container management service has become immensely popular in development, especially given the great fit with agile-based projects and continuous delivery. In this article, I want to take a brief look at how you can use Docker to accelerate and streamline the software development lifecycle (SDLC) process.
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
Cloud adoption is often driven by a desire to increase efficiency, boost agility and save money. All too often, however, the reality involves unpredictable cost spikes and lack of oversight due to resource limitations. In his session at 20th Cloud Expo, Joe Kinsella, CTO and Founder of CloudHealth Technologies, tackled the question: “How do you build a fully optimized cloud?” He will examine: Why TCO is critical to achieving cloud success – and why attendees should be thinking holistically ab...
DevOps is good for organizations. According to the soon to be released State of DevOps Report high-performing IT organizations are 2X more likely to exceed profitability, market share, and productivity goals. But how do they do it? How do they use DevOps to drive value and differentiate their companies? We recently sat down with Nicole Forsgren, CEO and Chief Scientist at DORA (DevOps Research and Assessment) and lead investigator for the State of DevOps Report, to discuss the role of measure...
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single-threaded, you can effectively identify hot spots in your serverless code. In his session at @DevOpsSummit at 21st Cloud Expo, Dave Martin, Product owner at CA Technologies, will give a live demonstration and code walkthrough, showing how ...
If you are part of the cloud development community, you certainly know about “serverless computing”, almost a misnomer. Because it implies there are no servers which is untrue. However the servers are hidden from the developers. This model eliminates operational complexity and increases developer productivity. We came from monolithic computing to client-server to services to microservices to serverless model. In other words, our systems have slowly “dissolved” from monolithic to function-by-func...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
IT organizations are moving to the cloud in hopes to approve efficiency, increase agility and save money. Migrating workloads might seem like a simple task, but what many businesses don’t realize is that application migration criteria differs across organizations, making it difficult for architects to arrive at an accurate TCO number. In his session at 21st Cloud Expo, Joe Kinsella, CTO of CloudHealth Technologies, will offer a systematic approach to understanding the TCO of a cloud application...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
API Security has finally entered our security zeitgeist. OWASP Top 10 2017 - RC1 recognized API Security as a first class citizen by adding it as number 10, or A-10 on its list of web application vulnerabilities. We believe this is just the start. The attack surface area offered by API is orders or magnitude larger than any other attack surface area. Consider the fact the APIs expose cloud services, internal databases, application and even legacy mainframes over the internet. What could go wrong...
With Cloud Foundry you can easily deploy and use apps utilizing websocket technology, but not everybody realizes that scaling them out is not that trivial. In his session at 21st Cloud Expo, Roman Swoszowski, CTO and VP, Cloud Foundry Services, at Grape Up, will show you an example of how to deal with this issue. He will demonstrate a cloud-native Spring Boot app running in Cloud Foundry and communicating with clients over websocket protocol that can be easily scaled horizontally and coordinate...
In his session at 20th Cloud Expo, Chris Carter, CEO of Approyo, discussed the basic set up and solution for an SAP solution in the cloud and what it means to the viability of your company. Chris Carter is CEO of Approyo. He works with business around the globe, to assist them in their journey to the usage of Big Data in the forms of Hadoop (Cloudera and Hortonwork's) and SAP HANA. At Approyo, we support firms who are looking for knowledge to grow through current business process, where even 1%...
The goal of Continuous Testing is to shift testing left to find defects earlier and release software faster. This can be achieved by integrating a set of open source functional and performance testing tools in the early stages of your software delivery lifecycle. There is one process that binds all application delivery stages together into one well-orchestrated machine: Continuous Testing. Continuous Testing is the conveyer belt between the Software Factory and production stages. Artifacts are m...
From manual human effort the world is slowly paving its way to a new space where most process are getting replaced with tools and systems to improve efficiency and bring down operational costs. Automation is the next big thing and low code platforms are fueling it in a significant way. The Automation era is here. We are in the fast pace of replacing manual human efforts with machines and processes. In the world of Information Technology too, we are linking disparate systems, softwares and tool...