Click here to close now.

Welcome!

Microservices Expo Authors: John Wetherill, Liz McMillan, Elizabeth White, Rex Morrow, Datical, Plutora Blog

Related Topics: CloudExpo® Blog

CloudExpo® Blog: Blog Feed Post

Cloud Storage for Your File Server

How much bandwidth you need to run file server with cloud storage?

As I was explaining Gladinet CloudAFS to a group of IT people as a file server with cloud storage attached, one question I got was:

How much bandwidth you need to run file server with cloud storage?

It is very obvious where the concern is coming from. If I had an on-premise file server, with cloud storage attached as tier 2 storage, will the slowness of the Internet (as compared to LAN) slow down the file server activities?

Rest assured, Gladinet CloudAFS takes care of this. Specifically, it takes care of two issues.

A. Slow Internet to the Cloud Storage – that is why it is tier2 storage.

B. There is a glitch, Internet is down.

The secret to both maintain local LAN performance and leverage Cloud Storage is to have a tier 1 storage locally on the hard drive (or NAS or SAN). When people read and write to the file server, it is writing to and reading from the tier 1 storage, thus gives you an impression of local storage because it is indeed local storage.

Gladinet CloudAFS will then schedule transfer to and from the tier 2 storage asynchronously.  You can schedule the transfer to be 12 hours later than the tier 1 storage hit so it can happen at night when nobody is working. If the Internet is down at a moment, the transfer will resume and retry later on.

Use Case 1 – File Server Backup

When your Internet is slow or the Internet is not quite reliable, you need to put cloud storage to a more secondary role, such as backup.

In this use case, you can configure Gladinet CloudAFS tier 1 storage to have the same size as tier2 cloud storage. This way, the read/write is always from local and you are guaranteed to have 100% up time and 100% local performance. The cloud storage is something nice to have just in case that your local server crash or out of capacity, you can get it back immediately by provision another CloudAFS server (5 minute install when OS is ready).

The benefit of this use case is that you can always quickly re-provision your file server in case something happens to it.

Use Case 2 – Cloud Storage Gateway

When your Internet is fast and the Internet is reliable, you can put cloud storage to a more prominent role, or on the other hand, put CloudAFS to a more secondary role such as a Cloud Storage Gateway with cache.

For example, if you file server is with Amazon EC2 and your storage is Amazon S3, you know you have fast and reliable connections in between, you have a fit for the Cloud Storage Gateway use case.

Another example, if you are hosting with Peer1 server and also using Peer1’s CloudOne storage, you have a fast and reliable connection in between too. This case fit you as well.

In this case, the local cache on the CloudAFS can be smaller than the actual cloud storage capacity.

When the cache is smaller, you will incur cache miss if a read to the CloudAFS server doesn’t already have the file locally. This case the file needs to be fetched from the cloud storage over the Internet. Since you have fast and reliable connection, this first time and one time cost is OK.

The benefit of this use case is that you can ship a lot of data over to cloud storage. For example, you can turn local backup software to cloud backup software by pointing the backup destination to CloudAFS.

CloudAFS Parameters

The two biggest parameter for CloudAFS are

A. How big the local cache is. The slower and less reliable the Internet is, the bigger the local cache you will need to assign.

B. Scheduled flush time. CloudAFS use a write-through cache with flush schedule. The slower the Internet you have, the more delay you will need to push the synchronization to night time.

At this time, CloudAFS supports a wide variety of cloud storage providers, including Amazon S3, Windows Azure, Rackspace Cloud Files, EMC Atmos, Mezeo, OpenStack, Nirvanix, AT&T Synaptic Storage, Peer 1 CloudOne and more.

Speed is an important factor when you apply cloud storage to file server. Visit other earlier posts about how to pick a cloud storage providers.

Read the original blog entry...

More Stories By Jerry Huang

Jerry Huang, an engineer and entrepreneur, founded Gladinet with his close friends and is pursuing interests in the cloud computing. He has published articles on the company blog as well as following up on the company twitter activities. He graduated from the University of Michigan in 1998 and has lived in West Palm Beach, Florida since.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@MicroservicesExpo Stories
In the first four parts of this series I presented an introduction to microservices along with a handful of emerging microservices patterns, and a discussion of some of the downsides and challenges to using microservices. The most recent installment of this series looked at ten ways that PaaS facilitates microservices development and adoption. In this post I’ll cover some words of wisdom, advice intended for individuals, teams, and organizations considering a move to microservices. I've gleaned...
T-Mobile has been transforming the wireless industry with its “Uncarrier” initiatives. Today as T-Mobile’s IT organization works to transform itself in a like manner, technical foundations built over the last couple of years are now key to their drive for more Agile delivery practices. In his session at DevOps Summit, Martin Krienke, Sr Development Manager at T-Mobile, will discuss where they started their Continuous Delivery journey, where they are today, and where they are going in an effort ...
SYS-CON Events announced today that EnterpriseDB (EDB), the leading worldwide provider of enterprise-class Postgres products and database compatibility solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. EDB is the largest provider of Postgres software and services that provides enterprise-class performance and scalability and the open source freedom to divert budget from more costly traditiona...
Do you think development teams really update those BMC Remedy tickets with all the changes contained in a release? They don't. Most of them just "check the box" and move on. They rose a Risk Level that won't raise questions from the Change Control managers and they work around the checks and balances. The alternative is to stop and wait for a department that still thinks releases are rare events. When a release happens every day there's just not enough time for people to attend CAB meeting...
I read an insightful article this morning from Bernard Golden on DZone discussing the DevOps conundrum facing many enterprises today – is it better to build your own DevOps tools or go commercial? For Golden, the question arose from his observations at a number of DevOps Days events he has attended, where typically the audience is composed of startup professionals: “I have to say, though, that a typical feature of most presentations is a recitation of the various open source products and compo...
There is no question that the cloud is where businesses want to host data. Until recently hypervisor virtualization was the most widely used method in cloud computing. Recently virtual containers have been gaining in popularity, and for good reason. In the debate between virtual machines and containers, the latter have been seen as the new kid on the block – and like other emerging technology have had some initial shortcomings. However, the container space has evolved drastically since coming on...
Containers Expo Blog covers the world of containers, as this lightweight alternative to virtual machines enables developers to work with identical dev environments and stacks. Containers Expo Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Bookmark Containers Expo Blog ▸ Here Follow new article posts on Twitter at @ContainersExpo
Node.js and io.js are increasingly being used to run JavaScript on the server side for many types of applications, such as websites, real-time messaging and controllers for small devices with limited resources. For DevOps it is crucial to monitor the whole application stack and Node.js is rapidly becoming an important part of the stack in many organizations. Sematext has historically had a strong support for monitoring big data applications such as Elastic (aka Elasticsearch), Cassandra, Solr, S...
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud en...
In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, president of Intellyx, panelists Roberto Medrano, Executive Vice President at Akana; Lori MacVittie, IoT_Microservices Power PanelEvangelist for F5 Networks; and Troy Topnik, ActiveState’s Technical Product Manager; will peel away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of ...
The 5th International DevOps Summit, co-located with 17th International Cloud Expo – being held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the...
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
This is the final installment of the six-part series Microservices and PaaS. It seems like forever since I attended Adrian Cockroft's meetup focusing on microservices. It's actually only been a couple of months, but much has happened since then: countless articles, meetups, and conference sessions focusing on microservices have been delivered, many meetings and design efforts at companies moving towards a microservices-based approach have been endured, and five installments of this blog series ...
SYS-CON Media named Andi Mann editor of DevOps Journal. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. DevOps Journal brings valuable information to DevOps professionals who are transforming the way enterprise IT is done. Andi Mann, Vice President, Strategic Solutions, at CA Technologies, is an accomplished digital business executive with extensive global expertise as a strategist, technologist, innovator, marketer, communicator, and thought lea...
Even though it’s now Microservices Journal, long-time fans of SOA World Magazine can take comfort in the fact that the URL – soa.sys-con.com – remains unchanged. And that’s no mistake, as microservices are really nothing more than a new and improved take on the Service-Oriented Architecture (SOA) best practices we struggled to hammer out over the last decade. Skeptics, however, might say that this change is nothing more than an exercise in buzzword-hopping. SOA is passé, and now that people are ...
Enterprises are fast realizing the importance of integrating SaaS/Cloud applications, API and on-premises data and processes, to unleash hidden value. This webinar explores how managers can use a Microservice-centric approach to aggressively tackle the unexpected new integration challenges posed by proliferation of cloud, mobile, social and big data projects. Industry analyst and SOA expert Jason Bloomberg will strip away the hype from microservices, and clearly identify their advantages and d...
It's 2:15pm on a Friday, and I'm sitting in the keynote hall at PyCon 2013 fidgeting through a succession of lightning talks that have very little relevance to my life. Topics like "Python code coverage techniques" (ho-hum) and "Controlling Christmas lights with Python” (yawn - I wonder if there's anything new on Hacker News)...when Solomon Hykes takes the stage, unveils Docker, and the world shifts. If you haven't seen it yet, you should watch the video of Solomon's Pycon The Future of Linux C...
You often hear the two titles of "DevOps" and "Immutable Infrastructure" used independently. In his session at DevOps Summit, John Willis, Technical Evangelist for Docker, will cover the union between the two topics and why this is important. He will cover an overview of Immutable Infrastructure then show how an Immutable Continuous Delivery pipeline can be applied as a best practice for "DevOps." He will end the session with some interesting case study examples.
Virtualization is everywhere. Enormous and highly profitable companies have been built on nothing but virtualization. And nowhere has virtualization made more of an impact than in Cloud Computing, the rampant and unprecedented adoption of which has been the direct result of the wide availability of virtualization software and techniques that enabled it. But does the cloud actually require virtualization?
An effective way of thinking in Big Data is composed of a methodical framework for dealing with the predicted shortage of 50-60% of the qualified Big Data resources in the U.S. This holistic model comprises the scientific and engineering steps that are involved in accelerating Big Data solutions: problem, diagnosis, facts, analysis, hypothesis, solution, prototype and implementation. In his session at Big Data Expo®, Tony Shan focused on the concept, importance, and considerations for each of t...