Click here to close now.


Microservices Expo Authors: Liz McMillan, Jason Bloomberg, Lori MacVittie, Elizabeth White, SmartBear Blog

Related Topics: Microservices Expo, Java IoT

Microservices Expo: Article

Crowdsourcing - A Best Practice or a Worst Practice?

How to know if crowdsourcing is for you

For the record, I am neither pro-crowdsourcing nor anti-crowdsourcing. For purposes of this article, I'm going to assume that we're talking about software development; however, crowdsourcing could be just as appropriate for situations as diverse as architecture, graphical design, PowerPoint slides or even developing a great pizza recipe.

The Ethics of Crowdsourcing
There has been some criticism of the crowdsourcing model on ethical grounds - for example, that it is unfair to the participants. For the most part, I'm going to stay out of that controversy. Though, if the rules are made clear and are posted up front, then each party can decide on their own whether to participate. To me, that makes it fair. If someone is cheated, that's a different story, and there are other (legal) remedies for that.

We've all engaged in things like job interviews or pitches to venture capital firms where we knew going into it that we would bear the burden of travel, preparation, etc., and that there were no guarantees whatsoever of results. I don't see how crowdsourcing is materially different. But the purpose of this article is to discuss crowdsourcing as a method of creating software - not to debate the ethical points.

Crowdsourcing: Deja Vu?
Crowdsourcing is in some ways like using a swear word. In some situations, it's appropriate and helpful, and in other situations, it's simply a terrible idea and entirely counter-productive. It's up to the business person to intelligently determine whether a particular issue is crowdsource-appropriate.

This is certainly not unique to crowdsourcing - the same can be said for outsourcing (or offshore outsourcing). In fact there are some similarities between the two, in that some years ago, offshore outsourcing grew in popularity - and briefly seemed to be promoted by many as a solution to every problem out there. I see the same thing happening with crowdsourcing today. I'm highly critical of organizations which promote crowdsourcing as a near-panacea. Apart from Duck Tape, nothing is a near-panacea.

With offshore outsourcing, there were some spectacular successes and even more spectacular disasters. If crowdsourcing truly takes off, we'll see the same pattern happen there too.  In time, rationality mostly prevailed and outsourcing/offshoring software development became "yet another tool" to be leveraged by the savvy business user.

Shortcomings of Crowdsourcing
Crowdsourcing has many of the same shortcomings as outsourcing, and a few more to boot. Some issues I see:

1) For the most part, the really talented architects and developers are busy with lucrative and demanding jobs. So if you think you're going to get someone to create something better than Hadoop without any compensation guarantees, think again. I'm not saying that there aren't some highly skilled people out there. But one needs to be realistic. You can't open something up to crowdsourcing and expect the world to come to you. On the other hand, your particular problem might be perfect for someone out there.

Although "contests", such as those promoted by organizations like TopCoder, can be useful for improving the quality of potential solutions, the quality of the end-solution is mostly determined by the quality of the talent pool and how well the contest is set up and managed. The fact that there is a contest doesn't change anything to any significant degree - it's still crowdsourcing.

And don't forget the overhead of managing and "judging" any such competition.  It's not as easy as determining the winner in a 100 metre dash.  As always, a badly defined and poorly run competition is almost certainly a failed one.

2) It's human nature that results and motivation go hand in hand.  In the working world, motivation takes many forms, and every person is different.  In general, people are motivated by compensation (monetary and non-monetary), respect and visibility from others, a sense of being part of a team, and the satisfaction of a job well done.

Because the nature of the relationship between a crowdsourcing participant and the business is so very ephemeral, the typical person simply doesn't have the underlying motivation.

3) The mathematics of complex systems are against outsourcing in general.  By this I mean that the effort to successfully manage such a project increases geometrically relative to the complexity of the problem trying to be solved.  Specifically, the effort to fully document the requirements of a project, to quality assure the results, to ensure that the proposed solution not only meets the requirements as set forth,  and provide a sound and extensible architectural base for the future increases MUCH faster than the actual effort required to solve the problem.

If you get involved in trying to manage multiple disconnected or semi-connected parties with varying competing or complementary solutions, that makes things even worse.

4) Who's going to maintain all that code?  Years ago, I had to write a utility that extracted data and metadata from a proprietary column-store database and stuff it into Oracle 10.  A year after I wrote the utility, I needed to go back and modify it.  It was a difficult task.  It's difficult to maintain your own code.  Maintaining someone else's code is far more difficult.  Although turnover is common in any organization, supportability of in-house developed code is significantly easier than externally acquired code.

5) Collaboration with internal team members is an important part of most development efforts, and this is something that is difficult with a crowdsourcing solution.

Advantages of Crowdsourcing
All that said, crowdsourcing has some real benefits.  True advantages.  For some projects (or project components), it's a great idea.

1) It can be highly cost effective when used properly (i.e. managed properly and employed for an appropriate class of problem).

2) The wide range of potential problem-solvers out there mean that some very interesting and innovative solutions can pop up.

3) External people aren't constrained by the mental "boxes" and preconceived notions that internal people create for themselves.

4) When used properly, the downside of crowdsourcing can be minimized.

Is Crowdsourcing Right for Your Project? How Do You Minimize Risk?
Anyhow, all this comes down to the old maxim: use the right tool for the job.  Some projects just aren't cut out for crowdsourcing, and some are.

That being said, you can't fix stupid, and some people insist on using a hammer where a screwdriver is the correct implement.  So if you cut off your finger using a bandsaw while trying to open up a can of tuna fish, don't go blaming the bandsaw.

If you are considering crowdsourcing, consider your answers to the following questions.  They may help you determine if your project is truly crowdsource-appropriate, and help you minimize the risk and/or impact of failure.

a) How much effort is required to appropriately define the problem as well as requirements for the solution...up front.

b) How much effort will it take to manage the crowdsourcing process?

c) How much calendar time will be expended if the crowdsourcing process fails to yield a useful solution?

d) What is the impact on the business if crowdsourcing doesn't work for this particular problem?

e) What is the impact on the business of using internal people instead of crowdsourcing? What is the "opportunity cost" of using internal people?

f) Can you define "pull the plug" points for a crowdsourcing project?  For example, how many parties express interest in working on your problem?

g) Can you establish some preliminary indicators that allow you to predict success likelihood for the final solution?

h) How do you decide between "pull the plug" vs. "go back and try to remedy the situation" if the leading indicators aren't looking good?

i) Can you define "success"? Do you have criteria in place for judging and testing solutions?

Crowdsourcing: Post Script
For an excellent followup on this article, please read "Maximizing CrowdSource Success" - perspectives from CrowdSourcing pioneer and TopCoder founder Jack Hughes.  Jack gives additional insight into many of the "must ask" questions to help readers determine whether crowdsourcing is appropriate for them - and, if so, how to maximize chance of success.

More Stories By Hollis Tibbetts

Hollis Tibbetts, or @SoftwareHollis as his 50,000+ followers know him on Twitter, is listed on various “top 100 expert lists” for a variety of topics – ranging from Cloud to Technology Marketing, Hollis is by day Evangelist & Software Technology Director at Dell Software. By night and weekends he is a commentator, speaker and all-round communicator about Software, Data and Cloud in their myriad aspects. You can also reach Hollis on LinkedIn – His latest online venture is OnlineBackupNews - a free reference site to help organizations protect their data, applications and systems from threats. Every year IT Downtime Costs $26.5 Billion In Lost Revenue. Even with such high costs, 56% of enterprises in North America and 30% in Europe don’t have a good disaster recovery plan. Online Backup News aims to make sure you all have the news and tips needed to keep your IT Costs down and your information safe by providing best practices, technology insights, strategies, real-world examples and various tips and techniques from a variety of industry experts.

Hollis is a regularly featured blogger at ebizQ, a venue focused on enterprise technologies, with over 100,000 subscribers. He is also an author on Social Media Today "The World's Best Thinkers on Social Media", and maintains a blog focused on protecting data: Online Backup News.
He tweets actively as @SoftwareHollis

Additional information is available at

All opinions expressed in the author's articles are his own personal opinions vs. those of his employer.

@MicroservicesExpo Stories
Apps and devices shouldn't stop working when there's limited or no network connectivity. Learn how to bring data stored in a cloud database to the edge of the network (and back again) whenever an Internet connection is available. In his session at 17th Cloud Expo, Bradley Holt, Developer Advocate at IBM Cloud Data Services, will demonstrate techniques for replicating cloud databases with devices in order to build offline-first mobile or Internet of Things (IoT) apps that can provide a better, ...
Culture is the most important ingredient of DevOps. The challenge for most organizations is defining and communicating a vision of beneficial DevOps culture for their organizations, and then facilitating the changes needed to achieve that. Often this comes down to an ability to provide true leadership. As a CIO, are your direct reports IT managers or are they IT leaders? The hard truth is that many IT managers have risen through the ranks based on their technical skills, not their leadership ab...
Despite all the talk about public cloud services and DevOps, you would think the move to cloud for enterprises is clear and simple. But in a survey of almost 1,600 IT decision makers across the USA and Europe, the state of the cloud in enterprise today is still fraught with considerable frustration. The business case for apps in the real world cloud is hybrid, bimodal, multi-platform, and difficult. Download this report commissioned by NTT Communications to see the insightful findings – registra...
Application availability is not just the measure of “being up”. Many apps can claim that status. Technically they are running and responding to requests, but at a rate which users would certainly interpret as being down. That’s because excessive load times can (and will be) interpreted as “not available.” That’s why it’s important to view ensuring application availability as requiring attention to all its composite parts: scalability, performance, and security.
Several years ago, I was a developer in a travel reservation aggregator. Our mission was to pull flight and hotel data from a bunch of cryptic reservation platforms, and provide it to other companies via an API library - for a fee. That was before companies like Expedia standardized such things. We started with simple methods like getFlightLeg() or addPassengerName(), each performing a small, well-understood function. But our customers wanted bigger, more encompassing services that would "do ...
“All our customers are looking at the cloud ecosystem as an important part of their overall product strategy. Some see it evolve as a multi-cloud / hybrid cloud strategy, while others are embracing all forms of cloud offerings like PaaS, IaaS and SaaS in their solutions,” noted Suhas Joshi, Vice President – Technology, at Harbinger Group, in this exclusive Q&A with Cloud Expo Conference Chair Roger Strukhoff.
The APN DevOps Competency highlights APN Partners who demonstrate deep capabilities delivering continuous integration, continuous delivery, and configuration management. They help customers transform their business to be more efficient and agile by leveraging the AWS platform and DevOps principles.
As we increasingly rely on technology to improve the quality and efficiency of our personal and professional lives, software has become the key business differentiator. Organizations must release software faster, as well as ensure the safety, security, and reliability of their applications. The option to make trade-offs between time and quality no longer exists—software teams must deliver quality and speed. To meet these expectations, businesses have shifted from more traditional approaches of d...
Ten years ago, there may have been only a single application that talked directly to the database and spit out HTML; customer service, sales - most of the organizations I work with have been moving toward a design philosophy more like unix, where each application consists of a series of small tools stitched together. In web example above, that likely means a login service combines with webpages that call other services - like enter and update record. That allows the customer service team to writ...
As the world moves towards more DevOps and microservices, application deployment to the cloud ought to become a lot simpler. The microservices architecture, which is the basis of many new age distributed systems such as OpenStack, NetFlix and so on, is at the heart of Cloud Foundry - a complete developer-oriented Platform as a Service (PaaS) that is IaaS agnostic and supports vCloud, OpenStack and AWS. In his session at 17th Cloud Expo, Raghavan "Rags" Srinivas, an Architect/Developer Evangeli...
All we need to do is have our teams self-organize, and behold! Emergent design and/or architecture springs up out of the nothingness! If only it were that easy, right? I follow in the footsteps of so many people who have long wondered at the meanings of such simple words, as though they were dogma from on high. Emerge? Self-organizing? Profound, to be sure. But what do we really make of this sentence?
In today's digital world, change is the one constant. Disruptive innovations like cloud, mobility, social media, and the Internet of Things have reshaped the market and set new standards in customer expectations. To remain competitive, businesses must tap the potential of emerging technologies and markets through the rapid release of new products and services. However, the rigid and siloed structures of traditional IT platforms and processes are slowing them down – resulting in lengthy delivery ...
Last month, my partners in crime – Carmen DeArdo from Nationwide, Lee Reid, my colleague from IBM and I wrote a 3-part series of blog posts on We titled our posts the Simple Math, Calculus and Art of DevOps. I would venture to say these are must-reads for any organization adopting DevOps. We examined all three ascpects – the Cultural, Automation and Process improvement side of DevOps. One of the key underlying themes of the three posts was the need for Cultural change – things like t...
There once was a time when testers operated on their own, in isolation. They’d huddle as a group around the harsh glow of dozens of CRT monitors, clicking through GUIs and recording results. Anxiously, they’d wait for the developers in the other room to fix the bugs they found, yet they’d frequently leave the office disappointed as issues were filed away as non-critical. These teams would rarely interact, save for those scarce moments when a coder would wander in needing to reproduce a particula...
In a report titled “Forecast Analysis: Enterprise Application Software, Worldwide, 2Q15 Update,” Gartner analysts highlighted the increasing trend of application modernization among enterprises. According to a recent survey, 45% of respondents stated that modernization of installed on-premises core enterprise applications is one of the top five priorities. Gartner also predicted that by 2020, 75% of
It is with great pleasure that I am able to announce that Jesse Proudman, Blue Box CTO, has been appointed to the position of IBM Distinguished Engineer. Jesse is the first employee at Blue Box to receive this honor, and I’m quite confident there will be more to follow given the amazing talent at Blue Box with whom I have had the pleasure to collaborate. I’d like to provide an overview of what it means to become an IBM Distinguished Engineer.
SYS-CON Events announced today that G2G3 will exhibit at SYS-CON's @DevOpsSummit Silicon Valley, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Based on a collective appreciation for user experience, design, and technology, G2G3 is uniquely qualified and motivated to redefine how organizations and people engage in an increasingly digital world.
The cloud has reached mainstream IT. Those 18.7 million data centers out there (server closets to corporate data centers to colocation deployments) are moving to the cloud. In his session at 17th Cloud Expo, Achim Weiss, CEO & co-founder of ProfitBricks, will share how two companies – one in the U.S. and one in Germany – are achieving their goals with cloud infrastructure. More than a case study, he will share the details of how they prioritized their cloud computing infrastructure deployments ...
If you are new to Python, you might be confused about the different versions that are available. Although Python 3 is the latest generation of the language, many programmers still use Python 2.7, the final update to Python 2, which was released in 2010. There is currently no clear-cut answer to the question of which version of Python you should use; the decision depends on what you want to achieve. While Python 3 is clearly the future of the language, some programmers choose to remain with Py...
Opinions on how best to package and deliver applications are legion and, like many other aspects of the software world, are subject to recurring trend cycles. On the server-side, the current favorite is container delivery: a “full stack” approach in which your application and everything it needs to run are specified in a container definition. That definition is then “compiled” down to a container image and deployed by retrieving the image and passing it to a container runtime to create a running...