Welcome!

Microservices Expo Authors: Flint Brenton, Liz McMillan, Pat Romanski, Elizabeth White, Charles Araujo

Related Topics: @DevOpsSummit, Microservices Expo, Linux Containers

@DevOpsSummit: Article

Docker Containers and #Microservices | @DevOpsSummit #DevOps #Docker

How Docker has transformed continuous delivery and how Linux and Windows can further accelerate our digital future

A History of Docker Containers and the Birth of Microservices
by Scott Willson

From the conception of Docker containers to the unfolding microservices revolution we see today, here is a brief history of what I like to call 'containerology'.

In 2013, we were solidly in the monolithic application era. I had noticed that a growing amount of effort was going into deploying and configuring applications. As applications had grown in complexity and interdependency over the years, the effort to install and configure them was becoming significant. But the road did not end with a single deployment, no, the installation and configuration work was repeated over and over again, not only for each software release but for each and every environment that we promote applications to until finally being deposited into production, where this we repeat this exercise one last time.

What struck me in 2013 was that these monolithic apps were overwhelmingly being deployed inside virtual machines (VMs). Whether the targeted environment was for Development, QA or Production, VMs were the deployment endpoint that hosted the applications.

Promoting the VM image
At that time, I thought that it would save considerable time and effort if the VM image is directly promoted instead of the myriad number application artifacts. Think of it; I told people. IT personnel need only perform update and configuration tasks once, then after the application execution proves stable, the VM image can then be promoted up the delivery pipeline as a ready-to-run container. Conceptually, all that was needed was for an IT professional to make a few network changes to the VM each step along the way, and then swap out the older versioned VM image for the new image.

It sounded simple enough. As is often the case, reality turned out to be more difficult. The problem was that VM images were too big to be considered conveniently deployable artifacts, and there were more changes needed to the VMs than simple network settings, such as infrastructure, security and storage properties.

Though using a VM image as a transportable application container wasn't feasible at the time, I fell in love with the idea of being able to promote an immutable package that was tested, verified and warranted, rather than deploying numerous files that required various configuration changes. What I didn't realize at the time was that Linux kernel partitioning would provide the foundation for fulfilling my vision.

Docker is born
The Linux kernel had been undergoing changes along these lines since 2006 but had matured enough for a company called, dotCloud to create the Docker project in 2010. Docker wasn't just providing a framework for spinning up and spinning down Linux kernel partitions, or containers; they were focused on using these containers as stateless app-in-a-box hosts for single applications. Docker containers were set to change fundamentally the way applications are architected, developed and delivered.

In 2013 Docker Inc. was born, and in 2016, the Docker Data Center and Docker Cloud have come online. Docker provides an abstraction layer to Linux containers, which guarantees the runtime environment exposed to an application will be identical no matter where the container is hosted and runs as long as the container is running in a Docker host. The Docker image is now the immutable package that can be promoted up through the Continuous Delivery pipeline and can safely enable Continuous Deployment.

Container-Map.jpg

Since containers are an isolated process space running inside the Linux OS (figure 1), their "boot time" is measured in seconds if not milliseconds. Swapping in and out new images for old ones happens virtually instantaneously, and Docker images are small enough to reside within versioned repositories meaning rolling back a failed deployment is easy and nearly instantaneous. If an error is detected post-deployment, simply swap out the current image with the previous version.

Microsoft joins the ‘containerology' party
Microsoft also recognized the benefit of containerology - the architecting, developing, hosting, and deploying of containerized applications. In 2015, they announced that Windows too will offer container technology (Figure 2).

Windows containers come in two runtime flavors, Windows Server Core and Nano Server. Windows containers also provide two different types of isolation. A Windows Server Container is like its Linux counterpart, in that it is an isolated process space running inside the Windows OS. Additionally, like Linux, all containers share the same kernel. However, Microsoft offers a second, more secure, version of a container called a Hyper-V container. In a Hyper-V container, the kernel the application interacts with is a virtual kernel, and not the OS's actual kernel. In other words, Hyper-V containers are completely isolated from one another even down to the kernel level.

Windows-Service-Docker-Engine.jpg

Not only has Microsoft jumped on the container bandwagon, but they also shared the vision of Docker's application focused model for containers. Microsoft partnered with Docker, and as a result, one can run Linux or Windows containers with Docker. Being able to run applications in either Linux or Windows hosted containers will provide companies flexibility and reduce any refactoring costs associated with rewriting, tweaking or re-architecting existing applications.

The bold new world that containerology will take us to is that of microservices. In my opinion, microservices (specifically as enabled by Docker) represent the first feasible step towards mechanized or industrialized applications. In the mechanical engineering world, complex systems were built buying off the shelf components and widgets. In contrast, the software world was accustomed to fabricating every part needed to built complex applications.

Microsoft and the Object Management Group attempted to address this problem by defining COM and CORBA respectively, however, these had their challenges and neither standard ever fully realized a universal market of reusable components that any developer could assemble to build any application on any platform. I am not going to go into SOA or SOAP in this article, but suffice it to say, the software industry has tried and failed to deliver anything that approached the standardization, and standardized tooling of the manufacturing sector.

How microservices can revolutionize app development
Microservices can change that. Each microservice offers a single focused application that performs a particular function. A Docker container provides an immutable, portable and stateless package for microservices. Docker container images can be shipped, shared and versioned as well as be used as a foundation for building new containers. Docker Hub provides ready to use images that can be downloaded and assembled into more complex applications.

Need the software equivalent of an actuator, a cog, wheel or gear? As of today, and going forward one will be able to download desired "prefab" components rather than having to build each and every widget, component or interface from scratch. Docker has addressed security concerns with this level of sharing and reuse with their Content Trust. Docker Content Trust makes it possible to verify the publisher of Docker images and guarantees the contents of the image.

We are heading into yet another technology transformation which is both exciting and challenging - it always is. The word 'disruptive' has come into vogue of late, but the when hasn't the software industry been disruptive? Look what the invention of the spreadsheet did to floors of accounting departments.

Word processors, ERP systems, RDBMSs, smart phones, the Internet. The list goes on, and will continue to go on - change is the norm in the world of technology and especially software. I share Docker's vision, a world of downloadable, reusable and adaptable components that can be used to assemble sophisticated or complex applications. I hope that the need to continually reinvent the wheel will become more of an exception than the rule in the future.

More Stories By Automic Blog

Automic, a leader in business automation, helps enterprises drive competitive advantage by automating their IT factory - from on-premise to the Cloud, Big Data and the Internet of Things.

With offices across North America, Europe and Asia-Pacific, Automic powers over 2,600 customers including Bosch, PSA, BT, Carphone Warehouse, Deutsche Post, Societe Generale, TUI and Swisscom. The company is privately held by EQT. More information can be found at www.automic.com.

@MicroservicesExpo Stories
Don’t go chasing waterfall … development, that is. According to a recent post by Madison Moore on Medium featuring insights from several software delivery industry leaders, waterfall is – while still popular – not the best way to win in the marketplace. With methodologies like Agile, DevOps and Continuous Delivery becoming ever more prominent over the past 15 years or so, waterfall is old news. Or, is it? Moore cites a recent study by Gartner: “According to Gartner’s IT Key Metrics Data report, ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual ...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
"This all sounds great. But it's just not realistic." This is what a group of five senior IT executives told me during a workshop I held not long ago. We were working through an exercise on the organizational characteristics necessary to successfully execute a digital transformation, and the group was doing their ‘readout.' The executives loved everything we discussed and agreed that if such an environment existed, it would make transformation much easier. They just didn't believe it was reali...
Without lifecycle traceability and visibility across the tool chain, stakeholders from Planning-to-Ops have limited insight and answers to who, what, when, why and how across the DevOps lifecycle. This impacts the ability to deliver high quality software at the needed velocity to drive positive business outcomes. In his general session at @DevOpsSummit at 19th Cloud Expo, Eric Robertson, General Manager at CollabNet, will discuss how customers are able to achieve a level of transparency that e...
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
We all know that end users experience the Internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices – not doing so will be a path to eventual b...
"Opsani helps the enterprise adopt containers, help them move their infrastructure into this modern world of DevOps, accelerate the delivery of new features into production, and really get them going on the container path," explained Ross Schibler, CEO of Opsani, and Peter Nickolov, CTO of Opsani, in this SYS-CON.tv interview at DevOps Summit at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-side - in isolated containers - resulting in better compute density. It's something that many developer...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
"I focus on what we are calling CAST Highlight, which is our SaaS application portfolio analysis tool. It is an extremely lightweight tool that can integrate with pretty much any build process right now," explained Andrew Siegmund, Application Migration Specialist for CAST, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"We view the cloud not as a specific technology but as a way of doing business and that way of doing business is transforming the way software, infrastructure and services are being delivered to business," explained Matthew Rosen, CEO and Director at Fusion, in this SYS-CON.tv interview at 18th Cloud Expo (http://www.CloudComputingExpo.com), held June 7-9 at the Javits Center in New York City, NY.
You often hear the two titles of "DevOps" and "Immutable Infrastructure" used independently. In his session at DevOps Summit, John Willis, Technical Evangelist for Docker, covered the union between the two topics and why this is important. He provided an overview of Immutable Infrastructure then showed how an Immutable Continuous Delivery pipeline can be applied as a best practice for "DevOps." He ended the session with some interesting case study examples.
In his session at Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to maximize project result...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Archi...