Click here to close now.




















Welcome!

Microservices Expo Authors: Pat Romanski, Elizabeth White, Liz McMillan, Jason Bloomberg, SmartBear Blog

Related Topics: @CloudExpo

@CloudExpo: Blog Feed Post

Cloud Storage for Your File Server

How much bandwidth you need to run file server with cloud storage?

As I was explaining Gladinet CloudAFS to a group of IT people as a file server with cloud storage attached, one question I got was:

How much bandwidth you need to run file server with cloud storage?

It is very obvious where the concern is coming from. If I had an on-premise file server, with cloud storage attached as tier 2 storage, will the slowness of the Internet (as compared to LAN) slow down the file server activities?

Rest assured, Gladinet CloudAFS takes care of this. Specifically, it takes care of two issues.

A. Slow Internet to the Cloud Storage – that is why it is tier2 storage.

B. There is a glitch, Internet is down.

The secret to both maintain local LAN performance and leverage Cloud Storage is to have a tier 1 storage locally on the hard drive (or NAS or SAN). When people read and write to the file server, it is writing to and reading from the tier 1 storage, thus gives you an impression of local storage because it is indeed local storage.

Gladinet CloudAFS will then schedule transfer to and from the tier 2 storage asynchronously.  You can schedule the transfer to be 12 hours later than the tier 1 storage hit so it can happen at night when nobody is working. If the Internet is down at a moment, the transfer will resume and retry later on.

Use Case 1 – File Server Backup

When your Internet is slow or the Internet is not quite reliable, you need to put cloud storage to a more secondary role, such as backup.

In this use case, you can configure Gladinet CloudAFS tier 1 storage to have the same size as tier2 cloud storage. This way, the read/write is always from local and you are guaranteed to have 100% up time and 100% local performance. The cloud storage is something nice to have just in case that your local server crash or out of capacity, you can get it back immediately by provision another CloudAFS server (5 minute install when OS is ready).

The benefit of this use case is that you can always quickly re-provision your file server in case something happens to it.

Use Case 2 – Cloud Storage Gateway

When your Internet is fast and the Internet is reliable, you can put cloud storage to a more prominent role, or on the other hand, put CloudAFS to a more secondary role such as a Cloud Storage Gateway with cache.

For example, if you file server is with Amazon EC2 and your storage is Amazon S3, you know you have fast and reliable connections in between, you have a fit for the Cloud Storage Gateway use case.

Another example, if you are hosting with Peer1 server and also using Peer1’s CloudOne storage, you have a fast and reliable connection in between too. This case fit you as well.

In this case, the local cache on the CloudAFS can be smaller than the actual cloud storage capacity.

When the cache is smaller, you will incur cache miss if a read to the CloudAFS server doesn’t already have the file locally. This case the file needs to be fetched from the cloud storage over the Internet. Since you have fast and reliable connection, this first time and one time cost is OK.

The benefit of this use case is that you can ship a lot of data over to cloud storage. For example, you can turn local backup software to cloud backup software by pointing the backup destination to CloudAFS.

CloudAFS Parameters

The two biggest parameter for CloudAFS are

A. How big the local cache is. The slower and less reliable the Internet is, the bigger the local cache you will need to assign.

B. Scheduled flush time. CloudAFS use a write-through cache with flush schedule. The slower the Internet you have, the more delay you will need to push the synchronization to night time.

At this time, CloudAFS supports a wide variety of cloud storage providers, including Amazon S3, Windows Azure, Rackspace Cloud Files, EMC Atmos, Mezeo, OpenStack, Nirvanix, AT&T Synaptic Storage, Peer 1 CloudOne and more.

Speed is an important factor when you apply cloud storage to file server. Visit other earlier posts about how to pick a cloud storage providers.

Read the original blog entry...

More Stories By Jerry Huang

Jerry Huang, an engineer and entrepreneur, founded Gladinet with his close friends and is pursuing interests in the cloud computing. He has published articles on the company blog as well as following up on the company twitter activities. He graduated from the University of Michigan in 1998 and has lived in West Palm Beach, Florida since.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@MicroservicesExpo Stories
Puppet Labs has announced the next major update to its flagship product: Puppet Enterprise 2015.2. This release includes new features providing DevOps teams with clarity, simplicity and additional management capabilities, including an all-new user interface, an interactive graph for visualizing infrastructure code, a new unified agent and broader infrastructure support.
SYS-CON Events announced today the Containers & Microservices Bootcamp, being held November 3-4, 2015, in conjunction with 17th Cloud Expo, @ThingsExpo, and @DevOpsSummit at the Santa Clara Convention Center in Santa Clara, CA. This is your chance to get started with the latest technology in the industry. Combined with real-world scenarios and use cases, the Containers and Microservices Bootcamp, led by Janakiram MSV, a Microsoft Regional Director, will include presentations as well as hands-on...
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies leverage disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 17th Cloud Expo, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advance...
It’s been proven time and time again that in tech, diversity drives greater innovation, better team productivity and greater profits and market share. So what can we do in our DevOps teams to embrace diversity and help transform the culture of development and operations into a true “DevOps” team? In her session at DevOps Summit, Stefana Muller, Director, Product Management – Continuous Delivery at CA Technologies, answered that question citing examples, showing how to create opportunities for ...
What does “big enough” mean? It’s sometimes useful to argue by reductio ad absurdum. Hello, world doesn’t need to be broken down into smaller services. At the other extreme, building a monolithic enterprise resource planning (ERP) system is just asking for trouble: it’s too big, and it needs to be decomposed.
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
Several years ago, I was a developer in a travel reservation aggregator. Our mission was to pull flight and hotel data from a bunch of cryptic reservation platforms, and provide it to other companies via an API library - for a fee. That was before companies like Expedia standardized such things. We started with simple methods like getFlightLeg() or addPassengerName(), each performing a small, well-understood function. But our customers wanted bigger, more encompassing services that would "do ...
Culture is the most important ingredient of DevOps. The challenge for most organizations is defining and communicating a vision of beneficial DevOps culture for their organizations, and then facilitating the changes needed to achieve that. Often this comes down to an ability to provide true leadership. As a CIO, are your direct reports IT managers or are they IT leaders? The hard truth is that many IT managers have risen through the ranks based on their technical skills, not their leadership ab...
SYS-CON Events announced today that DataClear Inc. will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. The DataClear ‘BlackBox’ is the only solution that moves your PC, browsing and data out of the United States and away from prying (and spying) eyes. Its solution automatically builds you a clean, on-demand, virus free, new virtual cloud based PC outside of the United States, and wipes it clean...
Whether you like it or not, DevOps is on track for a remarkable alliance with security. The SEC didn’t approve the merger. And your boss hasn’t heard anything about it. Yet, this unruly triumvirate will soon dominate and deliver DevSecOps faster, cheaper, better, and on an unprecedented scale. In his session at DevOps Summit, Frank Bunger, VP of Customer Success at ScriptRock, will discuss how this cathartic moment will propel the DevOps movement from such stuff as dreams are made on to a prac...
In his session at 17th Cloud Expo, Ernest Mueller, Product Manager at Idera, will explain the best practices and lessons learned for tracking and optimizing costs while delivering a cloud-hosted service. He will describe a DevOps approach where the applications and systems work together to track usage, model costs in a granular fashion, and make smart decisions at runtime to minimize costs. The trickier parts covered include triggering off the right metrics; balancing resilience and redundancy ...
Docker containerization is increasingly being used in production environments. How can these environments best be monitored? Monitoring Docker containers as if they are lightweight virtual machines (i.e., monitoring the host from within the container), with all the common metrics that can be captured from an operating system, is an insufficient approach. Docker containers can’t be treated as lightweight virtual machines; they must be treated as what they are: isolated processes running on hosts....
DevOps has traditionally played important roles in development and IT operations, but the practice is quickly becoming core to other business functions such as customer success, business intelligence, and marketing analytics. Modern marketers today are driven by data and rely on many different analytics tools. They need DevOps engineers in general and server log data specifically to do their jobs well. Here’s why: Server log files contain the only data that is completely full and accurate in th...
The pricing of tools or licenses for log aggregation can have a significant effect on organizational culture and the collaboration between Dev and Ops teams. Modern tools for log aggregation (of which Logentries is one example) can be hugely enabling for DevOps approaches to building and operating business-critical software systems. However, the pricing of an aggregated logging solution can affect the adoption of modern logging techniques, as well as organizational capabilities and cross-team ...
In today's digital world, change is the one constant. Disruptive innovations like cloud, mobility, social media, and the Internet of Things have reshaped the market and set new standards in customer expectations. To remain competitive, businesses must tap the potential of emerging technologies and markets through the rapid release of new products and services. However, the rigid and siloed structures of traditional IT platforms and processes are slowing them down – resulting in lengthy delivery ...
Early in my DevOps Journey, I was introduced to a book of great significance circulating within the Web Operations industry titled The Phoenix Project. (You can read our review of Gene’s book, if interested.) Written as a novel and loosely based on many of the same principles explored in The Goal, this book has been read and referenced by many who have adopted DevOps into their continuous improvement and software delivery processes around the world. As I began planning my travel schedule last...
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, discussed why containers should be paired with new architectural practices such as microservices rathe...
Any Ops team trying to support a company in today’s cloud-connected world knows that a new way of thinking is required – one just as dramatic than the shift from Ops to DevOps. The diversity of modern operations requires teams to focus their impact on breadth vs. depth. In his session at DevOps Summit, Adam Serediuk, Director of Operations at xMatters, Inc., will discuss the strategic requirements of evolving from Ops to DevOps, and why modern Operations has begun leveraging the “NoOps” approa...
The Microservices architectural pattern promises increased DevOps agility and can help enable continuous delivery of software. This session is for developers who are transforming existing applications to cloud-native applications, or creating new microservices style applications. In his session at DevOps Summit, Jim Bugwadia, CEO of Nirmata, will introduce best practices, patterns, challenges, and solutions for the development and operations of microservices style applications. He will discuss ...
Before becoming a developer, I was in the high school band. I played several brass instruments - including French horn and cornet - as well as keyboards in the jazz stage band. A musician and a nerd, what can I say? I even dabbled in writing music for the band. Okay, mostly I wrote arrangements of pop music, so the band could keep the crowd entertained during Friday night football games. What struck me then was that, to write parts for all the instruments - brass, woodwind, percussion, even k...