Welcome!

Microservices Expo Authors: Elizabeth White, Liz McMillan, Yeshim Deniz, Cynthia Dunlop, Carmen Gonzalez

Related Topics: @BigDataExpo, Microservices Expo, Containers Expo Blog, @CloudExpo, Apache, SDN Journal

@BigDataExpo: Blog Feed Post

Big Data = Dropping the Big One?

Big Data is certainly both marvelous and terrifying

Editor’s note: This article was first published on Analyst One, a site focused on analysts and topics of interest to the analytical community.-bg

Eminent network scientist Laszlo Barabasi recently penned an op-ed calling on fellow scientists to spearhead the ethical use of big data. Comparing big data to the atom bomb, Barabasi persuasively argued that the technology and methodologies he and other social network theorists had created had far outstripped societal controls on its use.

Barabasi’s op-ed is part of a growing backlash against big data technologies and methodologies While Barabasi and historian of science George Dyson have the historical perspective, technical insight, and scientific stature to write insightfully about the problems of pervasive data collection and algorithms that structure human decisions, other criticisms have been less than edifying. Frustrated Harvard Business Review blogger Andrew McAfee recently called on pundits to “stop sounding ignorant about big data.” Big data, McAfee points out, is held to unrealistic standards and often the victim of strawmanning. Critics expect big data to eliminate uncertainty (spoiler: it doesn’t), falsely overestimate the power of qualitative thinking, make broad criticisms against quantification itself, and overestimate the willingness of big data advocates to automate important decisions. Listening to some critics talk, you’d think that Palantir or Recorded Future = Skynet.

While insightful in many aspects, Barabasi’s op-ed also fails to fully investigate the real implications of his Hadoop ~ ICBM analogy. Many scientists sought to influence the use of nuclear weapons, understandably believing themselves the most well-informed about the dangers they posed. However, even the most effective of their well-meaning efforts were superseded by Cold War politics. It is within the American political system — teetering between fear of terrorism, fear of big government, love of capitalism, and fear of capitalism — that big data’s societal impact will be decided. And if the rising tide of anti-science sentiment is any proof, politicians couldn’t care less about science or the men and women who practice it.  Scientists are no longer viewed as unimpeachable figures of authority — and to some extent it’s doubtful they ever really were in predictably populist America.

Second, if big data is a weapon of mass destruction, you aren’t going to see Hans Blix suddenly busting down the doors of startups for snap inspections of Apache software or NoSQL. The only thing inherently more “dual use” than offensive cyber tools are big data technologies and methodologies. They are quickly becoming an integral part of modern business, academic research, and intelligence practice. Barabasi and others are correct that in a world in which the individual is more vulnerable than ever to government and corporate usage of data science, we arguably should try to mitigate current and potential harm. The problem with analogizing data to nukes (besides the fact that Google never destroyed a Japanese city) is that the former are clumsy weapons of last resort that even bitter enemies had a stake in controlling and the latter are ubiquitous aspects of modern life.

While Barabasi and others may have pioneered the techniques industry and government demand, big data has long since ceased to be a purely academic endeavor. The men and women who use them mostly aren’t scientists. Big data is heavily driven by corporate and government needs. Even the most talented PhDs often leave the academy to pursue higher salaries and greater freedom in the corporate world. Perhaps the best big data analogy is not to the atomic science of Einstein or Oppenheimer, but to the mathematics of Newton, Leibnitz, and Fourier. Were they alive today, even these eminent scientists would be powerless to prevent their mathematics from being used for military operations research on how to kill more efficiently or from being inputted into faulty and investor-bankrupting financial models. A Taylor Series or a differential equation — once out in the wild — belongs to anyone with a pen, paper, and calculator. Likewise, with open-source tools like Python machine learning library scikit-learn, anyone with the requisite technical training can utilize some canonical data science techniques.

Big data is certainly both marvelous and terrifying. It offers the opportunity to make money, make new scientific discoveries, and enhance political endeavors from development to national security. It also puts the individual at the mercy of companies and governments. But at the end of the day it is “neither a atomic bomb nor a holy grail.” It should neither be held to unrealistic standards nor feared as a weapon of mass destruction. And everyone who cares about the ethics of data — from the scientist to the layperson — must understand that control over its use is a function of the messy and dysfunctional domestic political scene and the anarchic international system.

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder and partner at Cognitio Corp and publsher of CTOvision.com

@MicroservicesExpo Stories
SoftLayer operates a global cloud infrastructure platform built for Internet scale. With a global footprint of data centers and network points of presence, SoftLayer provides infrastructure as a service to leading-edge customers ranging from Web startups to global enterprises. SoftLayer's modular architecture, full-featured API, and sophisticated automation provide unparalleled performance and control. Its flexible unified platform seamlessly spans physical and virtual devices linked via a world...
SYS-CON Events announced today that EastBanc Technologies will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. EastBanc Technologies has been working at the frontier of technology since 1999. Today, the firm provides full-lifecycle software development delivering flexible technology solutions that seamlessly integrate with existing systems – whether on premise or cloud. EastBanc Technologies partners with p...
SYS-CON Events announced today that Tintri Inc., a leading producer of VM-aware storage (VAS) for virtualization and cloud environments, will exhibit at the 18th International CloudExpo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
The initial debate is over: Any enterprise with a serious commitment to IT is migrating to the cloud. But things are not so simple. There is a complex mix of on-premises, colocated, and public-cloud deployments. In this power panel at 18th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists will look at the present state of cloud from the C-level view, and how great companies and rock star executives can use cloud computing to meet their most ambitious and disruptive business ...
Agile teams report the lowest rate of measuring non-functional requirements. What does this mean for the evolution of quality in this era of Continuous Everything? To explore how the rise of SDLC acceleration trends such as Agile, DevOps, and Continuous Delivery are impacting software quality, Parasoft conducted a survey about measuring and monitoring non-functional requirements (NFRs). Here's a glimpse at what we discovered and what it means for the evolution of quality in this era of Continuo...
SYS-CON Events announced today that Isomorphic Software will exhibit at SYS-CON's [email protected] at Cloud Expo New York, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Isomorphic Software provides the SmartClient HTML5/AJAX platform, the most advanced technology for building rich, high-productivity enterprise web applications for any device. SmartClient couples the industry’s broadest, deepest UI component set with a java server framework to deliver an end-...
SYS-CON Events announced today that AppNeta, the leader in performance insight for business-critical web applications, will exhibit and present at SYS-CON's @DevOpsSummit at Cloud Expo New York, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. AppNeta is the only application performance monitoring (APM) company to provide solutions for all applications – applications you develop internally, business-critical SaaS applications you use and the networks that deli...
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
You might already know them from theagileadmin.com, but let me introduce you to two of the leading minds in the Rugged DevOps movement: James Wickett and Ernest Mueller. Both James and Ernest are active leaders in the DevOps space, in addition to helping organize events such as DevOpsDays Austinand LASCON. Our conversation covered a lot of bases from the founding of Rugged DevOps to aligning organizational silos to lessons learned from W. Edwards Demings.
Application development and delivery methods have undergone radical changes in recent years to improve scalability and resiliency. Container images are the new build and deployment artifacts that are used to ship and run software. While startups have long been comfortable experimenting with and embracing new technologies, even large enterprises are now re-architecting their software systems so that they can benefit from container-enabled micro services architectures. With the launch of DC/OS, w...
In the rush to compete in the digital age, a successful digital transformation is essential, but many organizations are setting themselves up for failure. There’s a common misconception that the process is just about technology, but it’s not. It’s about your business. It shouldn’t be treated as an isolated IT project; it should be driven by business needs with the committed involvement of a range of stakeholders.
SYS-CON Events announced today that BMC Software has been named "Siver Sponsor" of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. BMC is a global leader in innovative software solutions that help businesses transform into digital enterprises for the ultimate competitive advantage. BMC Digital Enterprise Management is a set of innovative IT solutions designed to make digital business fast, seamless, and optimized from mainframe to mo...
Earlier this week, we hosted a Continuous Discussion (#c9d9) on Continuous Delivery (CD) automation and orchestration, featuring expert panelists Dondee Tan, Test Architect at Alaska Air, Taco Bakker, a LEAN Six Sigma black belt focusing on CD, and our own Sam Fell and Anders Wallgren. During this episode, we discussed the differences between CD automation and orchestration, their challenges with setting up CD pipelines and some of the common chokepoints, as well as some best practices and tips...
When I talk about driving innovation with self-organizing teams, I emphasize that such self-organization includes expecting the participants to organize their own teams, give themselves their own goals, and determine for themselves how to measure their success. In contrast, the definition of skunkworks points out that members of such teams are “usually specially selected.” Good thing he added the word usually – because specially selecting such teams throws a wrench in the entire works, limiting...
SYS-CON Events announced today TechTarget has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. TechTarget is the Web’s leading destination for serious technology buyers researching and making enterprise technology decisions. Its extensive global networ...
Automation is a critical component of DevOps and Continuous Delivery. This morning on #c9d9 we discussed CD Automation and how you can apply Automation to accelerate release cycles, improve quality, safety and governance? What is the difference between Automation and Orchestration? Where should you begin your journey to introduce both?
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...
As AT&Ts VP of Domain 2.0 architecture writes one aspect of their Domain 2.0 strategy is a goal to embrace a Microservices Application Architecture. One page 9 they describe how these envisage them fitting into the ECOMP architecture: "The initial steps of the recipes include a homing and placement task using constraints specified in the requests. ‘Homing and Placement' are micro-services involving orchestration, inventory, and controllers responsible for infrastructure, network, and applicati...
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...