|By Ali Hussain||
|April 18, 2014 12:00 PM EDT||
Setting up a deployment process on the cloud means a variety of choices. Most likely you're prepared to make some tradeoffs. But getting a view across these potential tradeoffs can be difficult. Here are six popular deployments and advice for making the best choice for your organization's needs.
Let's assume you want a deployment for a small startup with fewer than 20 developers, each needing to host a web app that's gaining traction and for which rapid growth is expected. Its requirements are as follows:
- Autoscaling support to handle expected surges in demand
- Maximizing developer efficiency by automating tedious tasks and improving dev flow
- Encouraging mature processes for building a stable foundation as the codebase grows
- Maintaining flexibility and agility to handle hotfixes of a relatively immature codebase
- Counting on a few sources to fail, because any of them can cause deployment failure - imagine GitHub failing or a required plugin becoming unavailable
Narrowing the focus a bit more, let's assume the codebase is using Ruby on Rails, as is often the case. We'll examine various deployment choices in detail, walk through a thorough analysis and then provide recommendations for anyone that fits our sample client profile.
1. The Plain Vanilla AMI Method
Amazon OpsWorks: This proven deployment is a well-tested Amazon OpsWorks Standard recommendation. Each time a new node comes up fresh, it requires running all Chef recipes. To automate this process, Cloud-init is used to run scripts for handling code and environment updates that occur when running nodes.
Pros: This approach requires no AMI management. The process is straightforward, self-documenting and brings up a clean environment every time. Updates and patches are applied very quickly.
Cons: Bringing up new instances is extremely slow, there are many moving parts, and there's a high risk of failure.
Bottom Line: While this is a clean solution, the frequent-failure rate and amount of time needed for bringup makes the Plain Vanilla AMI impractical for a use case with autoscaling.
2. The Bake-Everything AMI Method
This deployment option is proven to work at Amazon Video and Netflix. It runs all Chef recipes once, fetches the codebase and then bakes and uses the AMI. Each change requires a new AMI and an ASG replacement within the ELB, including code and environment changes.
Keep in mind that the environment and configuration management parts of the deployment still need automation using tools like Chef and Puppet. Lack of automation can otherwise make AMI management a nightmare, as one tends to lose track of how the environment actually looks within the AMI.
Pros: Provides the fastest bringup, requires no installation, and includes the fewest moving parts, so error rates are very low.
Cons: Each code deployment requires baking a new AMI. This requires a lot of effort to ensure that the process is as fast as possible in order to avoid developer bottlenecks. This setup also makes it harder to deploy hotfixes.
Bottom Line: This is generally a best practice, but requires a certain level of codebase maturity and a high level of infrastructure sophistication. For example, Netflix has spent a lot of time speeding up the process of baking AMIs by using their Aminator project.
3. A Hybrid Method Using Chef to Handle Complete Deployment
This method strikes a balance between the Plain Vanilla AMI and the Bake-Everything AMI. An AMI is baked using Chef for configuration and environment, but one can't check the codebase or deploy the app. Chef does those once the node is brought up.
Pros: Since all packages are pre-installed, this method is significantly faster than using a Plain Vanilla AMI. Also, since the code is pulled once a node is commissioned, the ability to provide hotfixes is improved.
Cons: Because we're relying on Chef in production, there's a dependency on the repository, and pulling from the repository may fail.
Bottom Line: We consider this to be a medium-risk implementation due to its reliance on Chef.
4. A Hybrid Method Using Capistrano to Handle Code Deployment
This is similar to the hybrid Chef deployment approach, but with code deployed through Capistrano. Capistrano is a mature platform for deploying Rails code that includes several features and fail-safe mechanisms that make it better than Chef. In particular, if pull from the repository fails, Capistrano deploys an older revision from its backups.
Pros: The same as for the Chef hybrid, except that Capistrano is more mature than Chef, especially in handling repository failures.
Cons: It requires two tools instead of one, which increases management overhead even though they're tied together. In addition, the gap between environment and code is wider, and managing the tools separately is difficult.
Bottom Line: Capistrano is a better Rails solution for code deployment than Chef, and the ability to apply fixes quickly may make it the best solution.
5. The AMI-Bake and CRON-Based Chef-Client Method
This deployment method resembles that of the hybrids. However, it provisions features allow auto-propagation of changes because each AMI runs chef-client every N minutes. New AMIs are baked only for major changes. It can provide continuous deployment, but continuous deployment is an aggressive tactic that requires excellent continuous integration on the back end.
Pros: Allows continuous code deployment.
Cons: It's prone to errors if Continuous Integration is not stable. In addition, Chef re-bootstraps aren't reliable and may fail.
Bottom Line: Not recommended unless CI is solid.
6. The Cloud-Init and Docker Method
All indications are that Docker is the best choice for this use case. It comes closer to a bake-everything solution while getting around bake-everything's biggest drawbacks. It allows AMIs to be baked once and rarely changes after that. Both the environment and the app code are contained inside an LXC container, with each AMI consisting of one container. Upon code deployment, a new container is simply pushed, which provides deployment-process flexibility.
Pros: Docker containers provide a history with which one can compare containers, helps with issues of undocumented steps in image creation. Code and environment are tied together. The repository structure of containers leads to faster deployment than does which baking a new AMI. Docker also helps to create a local environment similar to the production environment.
Cons: Docker is still in early phases of development and suffers from some growing pains, including a few bugs, a limited tools ecosystem, some app compatibility issues and a limited feature set.
Bottom Line: If you adopt this approach, you'll be doing considerable trailblazing. There's little information available, so comparing notes with other pioneers will be helpful.
While there are many options for deploying Ruby on Rails in AWS environments, there isn't a single best solution. Taking the time to review the options and tradeoffs can save headaches along the way. Talk to peers and experienced consultants about their experiences before making the final decisions.
What are your comments in regard to using these deployments?
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
Sep. 26, 2016 01:00 AM EDT Reads: 1,949
Analysis of 25,000 applications reveals 6.8% of packages/components used included known defects. Organizations standardizing on components between 2 - 3 years of age can decrease defect rates substantially. Open source and third-party packages/components live at the heart of high velocity software development organizations. Today, an average of 106 packages/components comprise 80 - 90% of a modern application, yet few organizations have visibility into what components are used where.
Sep. 25, 2016 11:45 PM EDT Reads: 503
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
Sep. 25, 2016 10:45 PM EDT Reads: 4,408
If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
Sep. 25, 2016 10:45 PM EDT Reads: 1,581
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management solutions, helping companies worldwide activate their data to drive more value and business insight and to transform moder...
Sep. 25, 2016 09:30 PM EDT Reads: 2,494
Throughout history, various leaders have risen up and tried to unify the world by conquest. Fortunately, none of their plans have succeeded. The world goes on just fine with each country ruling itself; no single ruler is necessary. That’s how it is with the container platform ecosystem, as well. There’s no need for one all-powerful, all-encompassing container platform. Think about any other technology sector out there – there are always multiple solutions in every space. The same goes for conta...
Sep. 25, 2016 09:00 PM EDT Reads: 955
Let's recap what we learned from the previous chapters in the series: episode 1 and episode 2. We learned that a good rollback mechanism cannot be designed without having an intimate knowledge of the application architecture, the nature of your components and their dependencies. Now that we know what we have to restore and in which order, the question is how?
Sep. 25, 2016 07:00 PM EDT Reads: 1,202
About a year ago we tuned into “the need for speed” and how a concept like "serverless computing” was increasingly catering to this. We are now a year further and the term “serverless” is taking on unexpected proportions. With some even seeing it as the successor to cloud in general or at least as a successor to the clouds’ poorer cousin in terms of revenue, hype and adoption: PaaS. The question we need to ask is whether this constitutes an example of Hype Hopping: to effortlessly pivot to the ...
Sep. 25, 2016 05:45 PM EDT Reads: 851
SYS-CON Events announced today the Kubernetes and Google Container Engine Workshop, being held November 3, 2016, in conjunction with @DevOpsSummit at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA. This workshop led by Sebastian Scheele introduces participants to Kubernetes and Google Container Engine (GKE). Through a combination of instructor-led presentations, demonstrations, and hands-on labs, students learn the key concepts and practices for deploying and maintainin...
Sep. 25, 2016 02:00 PM EDT Reads: 2,627
Enterprise IT has been in the era of Hybrid Cloud for some time now. But it seems most conversations about Hybrid are focused on integrating AWS, Microsoft Azure, or Google ECM into existing on-premises systems. Where is all the Private Cloud? What do technology providers need to do to make their offerings more compelling? How should enterprise IT executives and buyers define their focus, needs, and roadmap, and communicate that clearly to the providers?
Sep. 25, 2016 02:00 PM EDT Reads: 1,524
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
Sep. 25, 2016 12:45 PM EDT Reads: 2,482
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Sep. 25, 2016 12:15 PM EDT Reads: 3,395
The many IoT deployments around the world are busy integrating smart devices and sensors into their enterprise IT infrastructures. Yet all of this technology – and there are an amazing number of choices – is of no use without the software to gather, communicate, and analyze the new data flows. Without software, there is no IT. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the protocols that communicate data and the emerging data analy...
Sep. 25, 2016 11:00 AM EDT Reads: 1,580
All clouds are not equal. To succeed in a DevOps context, organizations should plan to develop/deploy apps across a choice of on-premise and public clouds simultaneously depending on the business needs. This is where the concept of the Lean Cloud comes in - resting on the idea that you often need to relocate your app modules over their life cycles for both innovation and operational efficiency in the cloud. In his session at @DevOpsSummit at19th Cloud Expo, Valentin (Val) Bercovici, CTO of So...
Sep. 25, 2016 10:00 AM EDT Reads: 1,359
Video experiences should be unique and exciting! But that doesn’t mean you need to patch all the pieces yourself. Users demand rich and engaging experiences and new ways to connect with you. But creating robust video applications at scale can be complicated, time-consuming and expensive. In his session at @ThingsExpo, Zohar Babin, Vice President of Platform, Ecosystem and Community at Kaltura, will discuss how VPaaS enables you to move fast, creating scalable video experiences that reach your...
Sep. 25, 2016 10:00 AM EDT Reads: 958
SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
Sep. 25, 2016 06:30 AM EDT Reads: 2,839
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lo...
Sep. 25, 2016 04:30 AM EDT Reads: 927
As applications are promoted from the development environment to the CI or the QA environment and then into the production environment, it is very common for the configuration settings to be changed as the code is promoted. For example, the settings for the database connection pools are typically lower in development environment than the QA/Load Testing environment. The primary reason for the existence of the configuration setting differences is to enhance application performance. However, occas...
Sep. 25, 2016 03:45 AM EDT Reads: 716
When scaling agile / Scrum, we invariable run into the alignment vs autonomy problem. In short: you cannot have autonomous self directing teams if they have no clue in what direction they should go, or even shorter: Alignment breeds autonomy. But how do we create alignment? and what tools can we use to quickly evaluate if what we want to do is part of the mission or better left out? Niel Nickolaisen created the Purpose Alignment model and I use it with innovation labs in large enterprises to de...
Sep. 25, 2016 01:15 AM EDT Reads: 1,242
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
Sep. 25, 2016 12:45 AM EDT Reads: 1,069