Welcome!

SOA & WOA Authors: Carmen Gonzalez, Andrew Phillips, Sergio Varga, Liz McMillan, Elizabeth White

Related Topics: Cloud Expo, SOA & WOA, Virtualization

Cloud Expo: Article

Do-It-Yourself Cloud Computing Management – Is It Worth It?

Do all the math, not just some of it

Does your IT or devops team run a lot of do-it-yourself projects? DIY is tempting, isn't it? You have smart engineers, easy access to storage and computing power, and a mandate this year to start moving some of (or more of) your applications to the cloud. Why not do it yourself?

I'm not saying that you shouldn't try DIY cloud computing management, but if you're going that route, you should know what you're in for. In particular, you should keep a sharp lookout for the point of no (easy) return.

The Five Cs of DIY
First of all, what compels people to try to roll their own end-to-end cloud management strategy? There are five arguments we often hear for DIY (all of which begin with "C"), depicted in Figure 1.

  1. Control: When you do it yourself, you're able to control everything, right, wrong or indifferent. If there are problems, you can handle them internally. If your business requires custom scripts or exporting usage data into your own warehouse, you'll want to be able flip the levers and configure it yourself. And the images will be yours, all yours.
  2. Complexity: "What's to manage?" you ask. "We're just spinning up a dozen or so dev and stage servers in an Amazon EC2 public cloud to get our feet wet. Plenty of companies do this without all the fuss of additional management tools." When your initial objectives are modest and you want your DevOps team to learn how things work first, cloud computing management looks like just a nice-to-have.
  3. Conquest: Speaking of DevOps, plenty of them want the challenge of understanding what's possible and how to build it. Or some of them say, "If I can't get approval, I'll just do it myself on the side," and they jump in. Because we are still in the early stages of cloud uptake, your engineers and managers are experimenting to discover just how far DIY and cobbling different tool sets together will take them, as opposed to researching what is available in the market. Who can blame them? The cloud presents itself as one of the most powerful APIs ever, and what self-respecting developer or IT department can resist that?
  4. Cost: Everybody's eye is on cost. "For about $200 a month, I can get five large, on-demand Linux instances for 100 hours, 50GB of storage and 10GB of data in and out. With pricing like that, we'll figure out the management piece ourselves. Most of what we'd buy in a cloud management product is stuff we can do ourselves anyway." It certainly looks like that at first glance, and if the budget is tight and your cloud-based DIY deployment is humming along, you probably can't make the case for spending money on management expertise that you're convinced you can provide on your own.
  5. Convenience: It's convenient and easy to think about DIY cloud management because Amazon, for instance, gives you so many options and tools to start with. If there were many long-standing public clouds from which to choose, the water would be muddier, you'd use a sharper pencil in the vetting process, and doing it yourself would seem like less of a slam dunk.

People stack these five Cs against off-the-shelf cloud management offerings all the time. Our industry spends a lot of time hearing these arguments, patiently nodding our heads and repeating the counter-arguments, based on our own experience with customers:

If you really need control, you'd be surprised by how much control and customization you can have with a cloud management product, even when you start with pre-configured images and templates.

Cloud computing management is complex, but cloud management products are designed to shield you from most of the complexity. Besides, the sooner you see how simple it is to automate the management of five or ten servers, the sooner you can get up to the 50 or 100 your business really needs.

There's plenty of technology to conquer in cloud computing management, but the industry has already conquered most of it, which is why off-the-shelf products are so comprehensive and accessible.

The cost of DIY is usually a lot more than your monthly fees. There's recruiting, training, non-recurring engineering expenses, headcount and the risk involved in building and maintaining your team. People don't always think that far down the road.

The flip-side of convenience is lock-in. It's hard to resist the ease of spinning up a cloud with just a browser and a credit card; however, is the product you conveniently start with today going to grow and scale gracefully with your business? Do you want all of your eggs in one basket? You've got to ask yourself, "Do I feel lucky?"

Let's examine the technology more closely.

Eyes Wide Open: Know What You're in For
What does cloud computing management mean? How many different layers are there to it, and when will you hit each one? Figure 2 depicts the layers of cloud computing management you'll traverse eventually, whether you DIY or license a product.

Basic cloud offerings from Amazon and open source tools can cover the entry-level echelon of service (bottom of Figure 2). A combination of DIY and open source is not especially dangerous at these levels:

  1. You provision basic or pre-configured images to meet your specifications and the needs of your users and customers, with memory, computing power, storage, OS and geographic proximity.
  2. Once you've installed applications in your cloud, you'll want to monitor them. Are they running properly? Have any of them gone down? Can you get alerts if something goes wrong?
  3. After you've tweaked your images, you'll want to clone them, say, for development/staging/production, for multiple developers or to meet increasing demand and traffic.
    Once you've gotten this far, you can also try products for cloud auto-scaling (originally invented by RightScale). With a few months and several servers under your belt, you arrive at the advanced echelon of service in need of yet another tool, because entry-level products don't cover these (middle of Figure 2):
  4. It's easy to take snapshots of images; sometimes too easy. You forget why you created them, what's inside them and whether anybody is still using them. You need configuration management to deal with image sprawl. Once you have that under control, you also need configuration management to create repeatable applications and services (for your own mini Platforms-as-a-Service).

With people all over the organization clamoring for cloud-based apps, you're ready for user management to set permissions and audit activity.

The cloud may be inexpensive, but it isn't free. When Finance asks you about cost allocation and ROI by department, project and region, you need tools that can break out your expenses and revenues from activity in the cloud.

The last three items are the state of the art in cloud computing management, and if you can get that far on DIY, you deserve a big raise. On the horizon is a final layer, still in its infancy (top of Figure 2).

Most of your cloud assets need to work together; e.g., start the database server first, then start up the application tier, then start the proxy, run some tests to make sure the whole app is working, and then turn on the website. Orchestration and workflow automation will soon allow you to code how your system should operate so that you don't need to intervene.

That's the long view of cloud computing management. Set your expectations accordingly.

Are You Close to the Tipping Point?
The market is torn at the moment. On the DIY side, cloud computing is not very old, and lots of organizations are scrambling to figure out what it can do for them and their business. There are plenty of eager, curious engineers ready to dive in, fire up a few servers, cut their teeth on cloud computing and a few open source tools, and do their company (and their résumé) some good.

On the off-the-shelf side, the market is filled with entrants and it's growing up fast. Cloud management products are feature-rich because those of us who focus on them have already hit most of the roadblocks.

For some companies, the choice of off-the-shelf is obvious. They've looked at Figure 2 and decided that they don't want to have to do it themselves. They tried rolling their own CRM until they saw how effortless Salesforce.com made it, and they remember tinkering for ages with their own Web servers until they realized Apache had nailed it. They've learned that lesson.

What's the tipping point for everybody else? When do features, capabilities and price point tip in favor of off-the-shelf products? From our experience, here are three telling metrics:

  1. Forty images - Once people try to keep track of more than 40 images, DIY cloud management tools start to get creaky. "I see a whole slew of servers, and some have really short names...I've forgotten what that one does..."
  2. Fifteen users - If they have 15 people operating in the cloud after only a couple of months, they're liable to be at 200 in a year. User management tools need to scale and work with all the other DIY tools in use.
  3. Five accounts - To sort out their billing and ROI headaches, they write discovery apps or go through the Amazon API trying to figure out which instances are in use and which images they come from so they can allocate costs correctly.

Alas, some organizations stick with DIY past that tipping point, to the point of no (easy) return. They've made it well into the advanced echelon, but only by cobbling together a patchwork of three to four disparate tool sets and growing their DevOps teams to 50 or more. Or, perhaps they decide to move all their DIY stuff to a different public cloud provider, or to their own private cloud. An off-the-shelf cloud computing management can still help them when they get to these points, but the effort will cost much more time and money than if they had started there in the first place.

Do All the Math, Not Just Some of It
Many customers come to us after they've outgrown their DIY efforts. Eventually, they discover that there are too many things to stitch together: configuration management, systems automation, monitoring, application automation, provisioning, user permissions, reporting and more.

Even if you're happy with the DIY cloud computing management you've put in place, are you really sure that it's worth the investment in time, money and manpower, compared to an off-the-shelf offering? Not only that, but are you sure you're far enough away from the tipping point that DIY will still look good a year from now?


Cloud Management - Obstacles Overcome in Off-the-Shelf Products

  • Removal of a single user's SSH key from all managed instances
  • Volume striping for better database performance
  • Image fingerprinting to identify pre-rolled images in private clouds accurately

More Stories By Darryl Eaton

Darryl Eaton is the director of product management at RightScale Inc. Not only does he ship products that are ahead of their time and blog about it, but he has also been known to play the Carillon, the largest and loudest musical instrument in the world. He wants you to make the right decision about DIY and off-the-shelf, so contact him at [email protected] and find out more about cloud management and the Five Cloud Commandments in RightScale’s library.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
"BSQUARE is in the business of selling software solutions for smart connected devices. It's obvious that IoT has moved from being a technology to being a fundamental part of business, and in the last 18 months people have said let's figure out how to do it and let's put some focus on it, " explained Dave Wagstaff, VP & Chief Architect, at BSQUARE Corporation, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com), moderated by Ashar Baig, Research Director, Cloud, at Gigaom Research, Nate Gordon, Director of T...
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...
SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, phone and digital TV services to consumers primarily in rural areas.

ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...

The BPM world is going through some evolution or changes where traditional business process management solutions really have nowhere to go in terms of development of the road map. In this demo at 15th Cloud Expo, Kyle Hansen, Director of Professional Services at AgilePoint, shows AgilePoint’s unique approach to dealing with this market circumstance by developing a rapid application composition or development framework.
The Internet of Things is not new. Historically, smart businesses have used its basic concept of leveraging data to drive better decision making and have capitalized on those insights to realize additional revenue opportunities. So, what has changed to make the Internet of Things one of the hottest topics in tech? In his session at @ThingsExpo, Chris Gray, Director, Embedded and Internet of Things, discussed the underlying factors that are driving the economics of intelligent systems. Discover how hardware commoditization, the ubiquitous nature of connectivity, and the emergence of Big Data a...
“In the past year we've seen a lot of stabilization of WebRTC. You can now use it in production with a far greater degree of certainty. A lot of the real developments in the past year have been in things like the data channel, which will enable a whole new type of application," explained Peter Dunkley, Technical Director at Acision, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nigeria has the largest economy in Africa, at more than US$500 billion, and ranks 23rd in the world. A recent re-evaluation of Nigeria's true economic size doubled the previous estimate, and brought it well ahead of South Africa, which is a member (unlike Nigeria) of the G20 club for political as well as economic reasons. Nigeria's economy can be said to be quite diverse from one point of view, but heavily dependent on oil and gas at the same time. Oil and natural gas account for about 15% of Nigera's overall economy, but traditionally represent more than 90% of the country's exports and as...
The Internet of Things is a misnomer. That implies that everything is on the Internet, and that simply should not be - especially for things that are blurring the line between medical devices that stimulate like a pacemaker and quantified self-sensors like a pedometer or pulse tracker. The mesh of things that we manage must be segmented into zones of trust for sensing data, transmitting data, receiving command and control administrative changes, and peer-to-peer mesh messaging. In his session at @ThingsExpo, Ryan Bagnulo, Solution Architect / Software Engineer at SOA Software, focused on desi...
"At our booth we are showing how to provide trust in the Internet of Things. Trust is where everything starts to become secure and trustworthy. Now with the scaling of the Internet of Things it becomes an interesting question – I've heard numbers from 200 billion devices next year up to a trillion in the next 10 to 15 years," explained Johannes Lintzen, Vice President of Sales at Utimaco, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
"For over 25 years we have been working with a lot of enterprise customers and we have seen how companies create applications. And now that we have moved to cloud computing, mobile, social and the Internet of Things, we see that the market needs a new way of creating applications," stated Jesse Shiah, CEO, President and Co-Founder of AgilePoint Inc., in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
Today’s enterprise is being driven by disruptive competitive and human capital requirements to provide enterprise application access through not only desktops, but also mobile devices. To retrofit existing programs across all these devices using traditional programming methods is very costly and time consuming – often prohibitively so. In his session at @ThingsExpo, Jesse Shiah, CEO, President, and Co-Founder of AgilePoint Inc., discussed how you can create applications that run on all mobile devices as well as laptops and desktops using a visual drag-and-drop application – and eForms-buildi...
We certainly live in interesting technological times. And no more interesting than the current competing IoT standards for connectivity. Various standards bodies, approaches, and ecosystems are vying for mindshare and positioning for a competitive edge. It is clear that when the dust settles, we will have new protocols, evolved protocols, that will change the way we interact with devices and infrastructure. We will also have evolved web protocols, like HTTP/2, that will be changing the very core of our infrastructures. At the same time, we have old approaches made new again like micro-services...
Code Halos - aka "digital fingerprints" - are the key organizing principle to understand a) how dumb things become smart and b) how to monetize this dynamic. In his session at @ThingsExpo, Robert Brown, AVP, Center for the Future of Work at Cognizant Technology Solutions, outlined research, analysis and recommendations from his recently published book on this phenomena on the way leading edge organizations like GE and Disney are unlocking the Internet of Things opportunity and what steps your organization should be taking to position itself for the next platform of digital competition.
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
As the Internet of Things unfolds, mobile and wearable devices are blurring the line between physical and digital, integrating ever more closely with our interests, our routines, our daily lives. Contextual computing and smart, sensor-equipped spaces bring the potential to walk through a world that recognizes us and responds accordingly. We become continuous transmitters and receivers of data. In his session at @ThingsExpo, Andrew Bolwell, Director of Innovation for HP's Printing and Personal Systems Group, discussed how key attributes of mobile technology – touch input, sensors, social, and ...