Welcome!

Microservices Expo Authors: Pat Romanski, Liz McMillan, Mehdi Daoudi, Elizabeth White, Jason Bloomberg

Related Topics: @CloudExpo, Containers Expo Blog, Log Management

@CloudExpo: Article

Cloud Computing - Yahoo, HP & Intel Embark on Joint Cloud Research

Yahoo, HP and Intel are going to do cloud research together using a global, multi-data center, open source Cloud Computing Test

Yahoo, HP and Intel are going to do cloud research together using a global, multi-data center, open source Cloud Computing Test Bed bigger than anything put together for such a purpose before, they said.

The whole testbed could potentially scale to 24,000 cores, 18 terabytes of memory and 9 petabytes of disk, roughly 164 teraFLOPS of power, big enough, the threesome said, for Internet-scale tests, at least tests of short duration.

There will be six - God willing always-available - sites: one at each of the vendors and one each at the state-run Infocomm Development Authority of Singapore (IDA), the University of Illinois at Urbana-Champaign and Karlsruhe Institute of Technology (KIT) in Germany.

Each site - and, remember, academic researchers have lacked the hardware and software infrastructure to support Internet-scale systems software research before - is supposed to consist of 1,000-4,000 cores.

The hardware will be almost exclusively Intel-based HP widgetry, though what exactly they're not saying since one of the points of the exercise is to figure out works best.

HP of course has its new cloud-bound Xeon-based ProLiant BL2x220c G5, the first server blade to combine two independent servers in a single blade, and its StorageWorks 9100 Extreme Data Storage System (ExDS9100), a highly scalable storage system designed to simplify the management of multiple petabytes.

Intel, which already supports Tashi, the open source cluster management system for cloud computing, will test and perhaps concoct some mojo we haven't seen yet beyond its Data Center Management Interface (DCMI), Node Manager (NM) and virtualization stuff.

There's always the possibility that a radically new architecture could merge.

Yahoo's supercomputing cluster, dubbed M45 after one of the star clusters, has been up and working since November when Yahoo opened it up to research by Carnegie Mellon. It appears to be the model or proof point for the other five data centers, all which are supposed to be operational by the end of the year.

As Yahoo said in November, it planned to make M45 available to researchers from other universities for "open, collaborative research."

Yahoo's purpose with M45 was to advance Hadoop, the Apache Software Foundation's open source sub-project, an open source distributed file system and parallel execution environment that processes massive amounts of data.

Yahoo has been Hadoop's primary contributor and it's looking for other contributions so Hadoop will be integral to the HP-Intel-Yahoo R&D effort. Unless contributions are governed by an open source license the IP will belong to the developer.

The vendors, who are kicking in research talent themselves, said the breath of their research would be wider than, say, what IBM and Google are doing, which appears to be limited to the applications layer.

Besides hardware testing, Haddop will form the basis of the systems software research and the trio, particularly Intel, is interested in advancing the cause of parallel programming and software management.

Yahoo is also interested in advancing its Yahoo Research-developed Pig open source parallel programming language,

The trio wants to understand how systems software and hardware function in a cloud environment.

Obviously the results should turn up in applications software and services.

HP Labs says it will use the test bed for advanced research into intelligent infrastructure and dynamic cloud services, and stuff like massive storage and software deployment.

Under HP's concept of "Everything as a Service," devices and services are supposed to interact seamlessly through the cloud, and it figures businesses and individuals will use services that anticipate their needs based on location, preferences, calendar and communities.

According to Prith Banerjee, HP's senior vice-president of research and director of HP Labs, "To realize the full potential of cloud computing, the technology industry must think about the cloud as a platform for creating new services and experiences. This requires an entirely new approach to the way we design, deploy and manage cloud infrastructure and services."

In answer to a question on a conference call, Intel replied that the founders might be willing - pending discussions - to take in other partners.

They declined to talk about the size of the investment but it appears the National Science Foundation is picking up the chit for the University of Illinois.

Supposedly however the goal of the initiative is to "promote open collaboration among industry, academia and governments by removing the financial and logistical barriers to research in data-intensive, Internet-scale computing."

More Stories By Maureen O'Gara

Maureen O'Gara the most read technology reporter for the past 20 years, is the Cloud Computing and Virtualization News Desk editor of SYS-CON Media. She is the publisher of famous "Billygrams" and the editor-in-chief of "Client/Server News" for more than a decade. One of the most respected technology reporters in the business, Maureen can be reached by email at maureen(at)sys-con.com or paperboy(at)g2news.com, and by phone at 516 759-7025. Twitter: @MaureenOGara

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Microservices Articles
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lav...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
TCP (Transmission Control Protocol) is a common and reliable transmission protocol on the Internet. TCP was introduced in the 70s by Stanford University for US Defense to establish connectivity between distributed systems to maintain a backup of defense information. At the time, TCP was introduced to communicate amongst a selected set of devices for a smaller dataset over shorter distances. As the Internet evolved, however, the number of applications and users, and the types of data accessed and...
The Software Defined Data Center (SDDC), which enables organizations to seamlessly run in a hybrid cloud model (public + private cloud), is here to stay. IDC estimates that the software-defined networking market will be valued at $3.7 billion by 2016. Security is a key component and benefit of the SDDC, and offers an opportunity to build security 'from the ground up' and weave it into the environment from day one. In his session at 16th Cloud Expo, Reuven Harrison, CTO and Co-Founder of Tufin, ...
Many organizations are now looking to DevOps maturity models to gauge their DevOps adoption and compare their maturity to their peers. However, as enterprise organizations rush to adopt DevOps, moving past experimentation to embrace it at scale, they are in danger of falling into the trap that they have fallen into time and time again. Unfortunately, we've seen this movie before, and we know how it ends: badly.
Kin Lane recently wrote a couple of blogs about why copyrighting an API is not common. I couldn’t agree more that copyrighting APIs is uncommon. First of all, the API definition is just an interface (It is the implementation detail … Continue reading →
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbraith, a principal engineer at HPE, discussed how to build a fully functional Kubernetes cluster on a number of virtual machines or bare-metal hosts. Also included will be a brief demonstration of running a Galera MyS...
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Day 2 Keynote at 17th Cloud Expo, Sandy Ca...