Welcome!

Microservices Expo Authors: Liz McMillan, Pat Romanski, Gregor Petri, Elizabeth White, AppDynamics Blog

Related Topics: @BigDataExpo, Microservices Expo, Containers Expo Blog, @CloudExpo, Apache, SDN Journal

@BigDataExpo: Blog Feed Post

Big Data = Dropping the Big One?

Big Data is certainly both marvelous and terrifying

Editor’s note: This article was first published on Analyst One, a site focused on analysts and topics of interest to the analytical community.-bg

Eminent network scientist Laszlo Barabasi recently penned an op-ed calling on fellow scientists to spearhead the ethical use of big data. Comparing big data to the atom bomb, Barabasi persuasively argued that the technology and methodologies he and other social network theorists had created had far outstripped societal controls on its use.

Barabasi’s op-ed is part of a growing backlash against big data technologies and methodologies While Barabasi and historian of science George Dyson have the historical perspective, technical insight, and scientific stature to write insightfully about the problems of pervasive data collection and algorithms that structure human decisions, other criticisms have been less than edifying. Frustrated Harvard Business Review blogger Andrew McAfee recently called on pundits to “stop sounding ignorant about big data.” Big data, McAfee points out, is held to unrealistic standards and often the victim of strawmanning. Critics expect big data to eliminate uncertainty (spoiler: it doesn’t), falsely overestimate the power of qualitative thinking, make broad criticisms against quantification itself, and overestimate the willingness of big data advocates to automate important decisions. Listening to some critics talk, you’d think that Palantir or Recorded Future = Skynet.

While insightful in many aspects, Barabasi’s op-ed also fails to fully investigate the real implications of his Hadoop ~ ICBM analogy. Many scientists sought to influence the use of nuclear weapons, understandably believing themselves the most well-informed about the dangers they posed. However, even the most effective of their well-meaning efforts were superseded by Cold War politics. It is within the American political system — teetering between fear of terrorism, fear of big government, love of capitalism, and fear of capitalism — that big data’s societal impact will be decided. And if the rising tide of anti-science sentiment is any proof, politicians couldn’t care less about science or the men and women who practice it.  Scientists are no longer viewed as unimpeachable figures of authority — and to some extent it’s doubtful they ever really were in predictably populist America.

Second, if big data is a weapon of mass destruction, you aren’t going to see Hans Blix suddenly busting down the doors of startups for snap inspections of Apache software or NoSQL. The only thing inherently more “dual use” than offensive cyber tools are big data technologies and methodologies. They are quickly becoming an integral part of modern business, academic research, and intelligence practice. Barabasi and others are correct that in a world in which the individual is more vulnerable than ever to government and corporate usage of data science, we arguably should try to mitigate current and potential harm. The problem with analogizing data to nukes (besides the fact that Google never destroyed a Japanese city) is that the former are clumsy weapons of last resort that even bitter enemies had a stake in controlling and the latter are ubiquitous aspects of modern life.

While Barabasi and others may have pioneered the techniques industry and government demand, big data has long since ceased to be a purely academic endeavor. The men and women who use them mostly aren’t scientists. Big data is heavily driven by corporate and government needs. Even the most talented PhDs often leave the academy to pursue higher salaries and greater freedom in the corporate world. Perhaps the best big data analogy is not to the atomic science of Einstein or Oppenheimer, but to the mathematics of Newton, Leibnitz, and Fourier. Were they alive today, even these eminent scientists would be powerless to prevent their mathematics from being used for military operations research on how to kill more efficiently or from being inputted into faulty and investor-bankrupting financial models. A Taylor Series or a differential equation — once out in the wild — belongs to anyone with a pen, paper, and calculator. Likewise, with open-source tools like Python machine learning library scikit-learn, anyone with the requisite technical training can utilize some canonical data science techniques.

Big data is certainly both marvelous and terrifying. It offers the opportunity to make money, make new scientific discoveries, and enhance political endeavors from development to national security. It also puts the individual at the mercy of companies and governments. But at the end of the day it is “neither a atomic bomb nor a holy grail.” It should neither be held to unrealistic standards nor feared as a weapon of mass destruction. And everyone who cares about the ethics of data — from the scientist to the layperson — must understand that control over its use is a function of the messy and dysfunctional domestic political scene and the anarchic international system.

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder and partner at Cognitio Corp and publsher of CTOvision.com

@MicroservicesExpo Stories
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management solutions, helping companies worldwide activate their data to drive more value and business insight and to transform moder...
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. In his session at @DevOpsSummit at 18th Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, showed how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningful f...
About a year ago we tuned into “the need for speed” and how a concept like "serverless computing” was increasingly catering to this. We are now a year further and the term “serverless” is taking on unexpected proportions. With some even seeing it as the successor to cloud in general or at least as a successor to the clouds’ poorer cousin in terms of revenue, hype and adoption: PaaS. The question we need to ask is whether this constitutes an example of Hype Hopping: to effortlessly pivot to the ...
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
More and more companies are looking to microservices as an architectural pattern for breaking apart applications into more manageable pieces so that agile teams can deliver new features quicker and more effectively. What this pattern has done more than anything to date is spark organizational transformations, setting the foundation for future application development. In practice, however, there are a number of considerations to make that go beyond simply “build, ship, and run,” which changes ho...
SYS-CON Events announced today that Tintri Inc., a leading producer of VM-aware storage (VAS) for virtualization and cloud environments, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Tintri VM-aware storage is the simplest for virtualized applications and cloud. Organizations including GE, Toyota, United Healthcare, NASA and 6 of the Fortune 15 have said “No to LUNs.” With Tintri they mana...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Digitization is driving a fundamental change in society that is transforming the way businesses work with their customers, their supply chains and their people. Digital transformation leverages DevOps best practices, such as Agile Parallel Development, Continuous Delivery and Agile Operations to capitalize on opportunities and create competitive differentiation in the application economy. However, information security has been notably absent from the DevOps movement. Speed doesn’t have to negat...
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lo...
Your business relies on your applications and your employees to stay in business. Whether you develop apps or manage business critical apps that help fuel your business, what happens when users experience sluggish performance? You and all technical teams across the organization – application, network, operations, among others, as well as, those outside the organization, like ISPs and third-party providers – are called in to solve the problem.
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, will compare the Jevons Paradox to modern-day enterprise IT, e...
As applications are promoted from the development environment to the CI or the QA environment and then into the production environment, it is very common for the configuration settings to be changed as the code is promoted. For example, the settings for the database connection pools are typically lower in development environment than the QA/Load Testing environment. The primary reason for the existence of the configuration setting differences is to enhance application performance. However, occas...
SYS-CON Events announced today the Kubernetes and Google Container Engine Workshop, being held November 3, 2016, in conjunction with @DevOpsSummit at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA. This workshop led by Sebastian Scheele introduces participants to Kubernetes and Google Container Engine (GKE). Through a combination of instructor-led presentations, demonstrations, and hands-on labs, students learn the key concepts and practices for deploying and maintainin...
While DevOps promises a better and tighter integration among an organization’s development and operation teams and transforms an application life cycle into a continual deployment, Chef and Azure together provides a speedy, cost-effective and highly scalable vehicle for realizing the business values of this transformation. In his session at @DevOpsSummit at 19th Cloud Expo, Yung Chou, a Technology Evangelist at Microsoft, will present a unique opportunity to witness how Chef and Azure work tog...
When scaling agile / Scrum, we invariable run into the alignment vs autonomy problem. In short: you cannot have autonomous self directing teams if they have no clue in what direction they should go, or even shorter: Alignment breeds autonomy. But how do we create alignment? and what tools can we use to quickly evaluate if what we want to do is part of the mission or better left out? Niel Nickolaisen created the Purpose Alignment model and I use it with innovation labs in large enterprises to de...
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...