Welcome!

Microservices Expo Authors: Pat Romanski, Karthick Viswanathan, Liz McMillan, Elizabeth White, Automic Blog

Related Topics: Microservices Expo, Containers Expo Blog, @CloudExpo

Microservices Expo: Blog Feed Post

Information Explosion – So What?

How are we still surviving the overload?

With the exponential growth of information, why are we not paralyzed yet?

IDC had predicted 40-60% Y-O-Y growth of information. This translates to a ten times growth in about 6 years time. Here are some interesting analogies on information explosion from Infographic: The Information Explosion, Forecasts, and the Cloud.

In spite of such growth we still continue to find that piece of information that we are looking for – we still have place to store data, picture, music, and video. We still reply to the important emails and catch up on Facebook. We are still able to afford to store the content within the budget and space constraint.

Three things have saved us! No, cloud storage in not one of them.

Reduction in the cost of storage

Storage cost has come down at about the same rate as the rate of increase of the data volume. One seems to have compensated the other. Matthew Komorowski has collected the cost data and has arrived at a formula for cost reduction – see the post A History of Storage Cost and see this post by Volkan Tunalı say that price become half every 14 months.

The reduction works out to about one order of magnitude every 4 years. Here is couple of other documents produced in 1992!

Improvement in search technology


Getting Information off the Internet is like taking drink from a fire hydrant … Mitch Kapor

Thanks primarily to Google, we are able to find our needle in the haystack or we are able to drink comfortably from the fire hydrant!

Today we can move away for from the data classification of structured vs. unstructured to:

  • Machine processable data: Such data is typically stored in a database or in a data warehouse. Computer can process, transform, analyze and aggregate such data. All transactional data fall in this category
  • Machine searchable data: Though machine can read this class of data and index it for people to search, computers cannot directly make sense of the data. Documents (word, excel, pdf etc) and web pages fall into this category. The semantic wave or the web 3.0 movement is aimed at making the searchable data into processable data.
  • Machine opaque data: Most of the audio, image and video data falls in this category. For many years now AI community has been working on trying to make opaque data into searchable data. OCR (optical character recognition) and Google similar images is a result of such effort. Opaque data can also be made partially searchable by adding meta-data to it.

Interestingly, the volume of searchable data is an order of magnitude more than processable data and the volume of opaque data is an order of magnitude more than searchable data. Gartner believes that over the next three years video will become a commonplace content type and interaction model for most users, and by 2013, more than 25 percent of the content that workers see in a day will be dominated by pictures, video or audio. (http://www.gartner.com/it/page.jsp?id=1454221)

Multi-factor knowledge acquisition

We have entered an era where we have adapted our ability to gather bits and pieces of information from multiple sources and build our understanding or find an accurate answer to a question we are asking. The beauty of this process is that the individual pieces of information may be partial and may even me erroneous. Our brain is able to resolve the contradiction and make sense and learn from this cacophony. This is the best way to combine what computers can do well (index and search) with what human can do well (make sense from multiple, incomplete and even contradictory set of information).

This process is like multifactor authentication where more than one piece of information about user identity is used to get more secure authentication.

Some of us follow this process with our own contributing to this cacophony. We go back and publish our understanding. We curate the content and create new content. As per Business Insider – Content Is No Longer King: Curation Is King.

Coming back to cloud storage and for it to remain viable the reduction in the cost of cloud storage needs to be visible.

That is the storage cost should be half every 14 months. However, such trend in not yet visible. I had done an analysis one year back – Cloud Economics – A Platform Comparison. You can use data from that post to do your own calculation.

Read the original blog entry...

More Stories By Udayan Banerjee

Udayan Banerjee is CTO at NIIT Technologies Ltd, an IT industry veteran with more than 30 years' experience. He blogs at http://setandbma.wordpress.com.
The blog focuses on emerging technologies like cloud computing, mobile computing, social media aka web 2.0 etc. It also contains stuff about agile methodology and trends in architecture. It is a world view seen through the lens of a software service provider based out of Bangalore and serving clients across the world. The focus is mostly on...

  • Keep the hype out and project a realistic picture
  • Uncover trends not very apparent
  • Draw conclusion from real life experience
  • Point out fallacy & discrepancy when I see them
  • Talk about trends which I find interesting
Google

@MicroservicesExpo Stories
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"At the keynote this morning we spoke about the value proposition of Nutanix, of having a DevOps culture and a mindset, and the business outcomes of achieving agility and scale, which everybody here is trying to accomplish," noted Mark Lavi, DevOps Solution Architect at Nutanix, in this SYS-CON.tv interview at @DevOpsSummit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
From manual human effort the world is slowly paving its way to a new space where most process are getting replaced with tools and systems to improve efficiency and bring down operational costs. Automation is the next big thing and low code platforms are fueling it in a significant way. The Automation era is here. We are in the fast pace of replacing manual human efforts with machines and processes. In the world of Information Technology too, we are linking disparate systems, softwares and tool...
"We focus on SAP workloads because they are among the most powerful but somewhat challenging workloads out there to take into public cloud," explained Swen Conrad, CEO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
"I think DevOps is now a rambunctious teenager – it’s starting to get a mind of its own, wanting to get its own things but it still needs some adult supervision," explained Thomas Hooker, VP of marketing at CollabNet, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Peak 10 is a hybrid infrastructure provider across the nation. We are in the thick of things when it comes to hybrid IT," explained , Chief Technology Officer at Peak 10, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"I will be talking about ChatOps and ChatOps as a way to solve some problems in the DevOps space," explained Himanshu Chhetri, CTO of Addteq, in this SYS-CON.tv interview at @DevOpsSummit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
With continuous delivery (CD) almost always in the spotlight, continuous integration (CI) is often left out in the cold. Indeed, it's been in use for so long and so widely, we often take the model for granted. So what is CI and how can you make the most of it? This blog is intended to answer those questions. Before we step into examining CI, we need to look back. Software developers often work in small teams and modularity, and need to integrate their changes with the rest of the project code b...
If you are thinking about moving applications off a mainframe and over to open systems and the cloud, consider these guidelines to prioritize what to move and what to eliminate. On the surface, mainframe architecture seems relatively simple: A centrally located computer processes data through an input/output subsystem and stores its computations in memory. At the other end of the mainframe are printers and terminals that communicate with the mainframe through protocols. For all of its apparen...
SYS-CON Events announced today that Calligo has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo is an innovative cloud service provider offering mid-sized companies the highest levels of data privacy. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalized support service from its globally located cloud platfor...
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
DevOps is good for organizations. According to the soon to be released State of DevOps Report high-performing IT organizations are 2X more likely to exceed profitability, market share, and productivity goals. But how do they do it? How do they use DevOps to drive value and differentiate their companies? We recently sat down with Nicole Forsgren, CEO and Chief Scientist at DORA (DevOps Research and Assessment) and lead investigator for the State of DevOps Report, to discuss the role of measure...
DevOps sees the coming together of practices, philosophies, and tools that allow you to create services and applications very quickly. This means that you can improve on your apps and evolve them at a much faster rate than those developers who are using traditional software development processes. We’ve talked about DevOps, in general, a great deal, but today, we’re going to dig a little deeper and take a look at Java DevOps specifically.
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
From personal care products to groceries and movies on demand, cloud-based subscriptions are fulfilling the needs of consumers across an array of market sectors. Nowhere is this shift to subscription services more evident than in the technology sector. By adopting an Everything-as-a-Service (XaaS) delivery model, companies are able to tailor their computing environments to shape the experiences they want for customers as well as their workforce.