Welcome!

Microservices Expo Authors: Pat Romanski, Liz McMillan, Elizabeth White, Kevin Benedict, Carmen Gonzalez

Related Topics: Machine Learning , Java IoT, Industrial IoT, Microservices Expo, PowerBuilder, Containers Expo Blog

Machine Learning : Blog Post

Where Is My Disk Space?

An HTML 5 File System Visualizer

by Nick Mueller, Zetta.net

Hello new users! The file system visualizer can be found at wheresmydiskspace.com - continue reading to learn more about the development of the tool and the visualization options.

Before buying more storage space it's a good idea to make sure your existing space isn't filled with redundant or old data - or hundreds of downloaded cat videos.

Disk capacity is increasing and while prices continue to drop, those savings are offset by demands for new capacity to store more and larger files. Not only does this mean more primary disk space, but 2x that amount for backups.

Zetta co-founder Lou Montulli may have the answer to this problem. Recently Lou combined his experience with browsers and storage in creating an open-source tool - a File System Visualizer (www.wheresmydiskspace.com) - for analyzing storage usage.

Lou was a founding engineer at Netscape in 1994 when he helped create the first commercial web browser Netscape Navigator. Over the years he's been responsible for the development of many browser related innovations, and co-founded Zetta.net in 2008 - where he continues to serve as VP of Engineering and Chief Scientist.

"The tool was conceived as a method for visualizing multiple aspects of any large file set: an existing file system, a backup or an archive," he says. "This can be a great tool to use if you find yourself running low on disk space and need to find files to delete to free up space."

The tool makes heavy use of the Data-Driven Documents JavaScript library together with jQuery, Dojo, PrettyPhoto, JavaScript and Scalable Vector Graphics. Sponsored by Zetta, all the source code for the File System Visualizer is available and a BSD license allows anyone to use it, commercially and non-commercially, free of charge.

"Part of the challenge and opportunity of this tool was writing it in JavaScript and using HTML as the user interface," Lou says. "I was part of the team who wrote the very first web browsers, so I was personally motivated to design a tool that takes advantage of some of the great new technologies coming out of HTML5, Mozilla.org and the broader web community."

Getting Started
The File System Visualizer is free to use and doesn't require installing any software. You just need a web browser that supports SVG and has a fast JavaScript engine. Go to www.wheresmydiskspace.com. The home page has a few video demonstration of the product that you can view before running the software.

Or you can:

  1. Click the link at the top of the page to take you directly to the visualizer.
  2. There you have three options: you can look at some sample data sets, use a Java applet to collect the data from your local machine and create a manifest file detailing what is in the file system, or you can load a manifest file created in a previous scan.
  3. If you choose to do a new scan, and there are a large number of folders, the software will prompt you to save the manifest to your disk rather than keeping it in the browser.

Test Setup
We recently had the File System Visualizer tested on a Windows 7 desktop with a third generation Intel Core i7 processor and 16 GB RAM. The scan took approximately 5 minutes. When completed, a message came up that there were 52,993 folders.

The software can analyze a local disk, or an administrator can run it remotely on any mountable drive. At this point it runs on Windows (32-bit and 64-bit) and OSX.

Visualizing Your Data
After running the scan, the software then presents seven different views of the data. The views are illustrated at the top of the page and you can click on any of the images to access that view of the data.

Summary Page - This showed that the test computer had 353.1 GB of data in 52,993 folders containing 364,931 items, with an average file size of 967.7 KB.

Visual Tree - This gives a hierarchical tree visualization of the data. On the left is a pull-down box where you can select to view the data by size, by type or by date. There is also a slider where you can select the tree display depth from one to seven levels.

Screenshot of the Tree View

Viewing by size shows a hierarchical view of the file system and the amount of data in each folder with up to seven levels of depth. To look at just the contents of a single folder, rather than the entire file system at once, just click on the dot next to that folder.

Viewing by type at the first level divided the data into known types and uncategorized. Going to the second depth level divided the uncategorized by their file extension and the categorized into groups such as disk images, games, database, software development, fonts, plugins, office types, settings, executables, media, backup and system. For most of those categories, going to the next level would give the file extensions, but some categories (media, office types and encodings) would further subdivide before getting to their final level.

Viewing by date, the first level divides the data into "1 year and older" and "within 1 year" and shows the GB of data in each category. Taking it to the second level splits the "within 1 year" branch into five levels and the "1 year and older" into each of the years for which you have data. There is no third level available.

Hierarchical List - This view presents the data in list rather than tree format. To get to deeper levels, click the + sign next to any of the categories. In addition to the file names, there are columns for Size in Directory, Total Size and % with children. When you click on the headers for the columns, up and down arrows appear, making it look like the data is sortable by those columns, but it isn't.

Flattened List - This is a sortable, non-hierarchical list of the folders. When viewing by Size, in addition to File Name, there are seven other sortable columns of data in each folder, including Size and Number of Items. The Type and Date views are similarly sortable. In none of these views can you look at a subtree, only at the entire file system. To view a subtree, go to one of the other views and narrow it down to the subtree and view type you want, and then click on the Flattened List visualization.

Your hard drive in "sun burst" view.

Sunburst - A type of pie chart, with rings showing each of the levels of depth. The chart can display each slice as an even size, or can adjust the sizes by the file count or amount of data in the slice. Clicking on any of the slices will move that folder or data point into the center circle, with the rings showing the subfolders or subcategories of that particular subdirectory.

Tree Map - A box type view of the data. As with the Sunburst, the boxes can be sized equally, or sized by data size or number of files. Clicking on any of the boxes will show the details within that subdirectory or data type.

Bubble Chart - This gives two layout options for showing the data: Bubble Chart or Circle Pack. The Bubble Chart shows bubbles for all the items in that category sized by the amount of data in that folder or file type. The Circle Pack presents a hierarchical view of the bubbles. In either view, clicking on a bubble or circle will give the bubbles showing the subcategories of that item.

Conclusion
The File System Visualizer is a quick and easy way to gain understanding of what's on your file system. It's intuitive to use and within minutes, you can start locating what is taking up disk space. Then you can delete or archive anything that is no longer needed, or establish policies to prevent wasted space. Then, if additional storage space is still needed, you can give management a clear visual presentation of how storage is being used in your environment. You can start visualizing your hard drive right now.

Nick is Zetta's Corporate Reporter, and has been writing and telling stories about technology with blogs, social media, and content marketing since the days when the BBS reigned.

More Stories By Derek Kol

Derek Kol is a technology specialist focused on SMB and enterprise IT innovations.

@MicroservicesExpo Stories
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the work...
2016 has been an amazing year for Docker and the container industry. We had 3 major releases of Docker engine this year , and tremendous increase in usage. The community has been following along and contributing amazing Docker resources to help you learn and get hands-on experience. Here’s some of the top read and viewed content for the year. Of course releases are always really popular, particularly when they fit requests we had from the community.
In his General Session at DevOps Summit, Asaf Yigal, Co-Founder & VP of Product at Logz.io, will explore the value of Kibana 4 for log analysis and will give a real live, hands-on tutorial on how to set up Kibana 4 and get the most out of Apache log files. He will examine three use cases: IT operations, business intelligence, and security and compliance. This is a hands-on session that will require participants to bring their own laptops, and we will provide the rest.
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
"We're bringing out a new application monitoring system to the DevOps space. It manages large enterprise applications that are distributed throughout a node in many enterprises and we manage them as one collective," explained Kevin Barnes, President of eCube Systems, in this SYS-CON.tv interview at DevOps at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
The evolution of JavaScript and HTML 5 to support a genuine component based framework (Web Components) with the necessary tools to deliver something close to a native experience including genuine realtime networking (UDP using WebRTC). HTML5 is evolving to offer built in templating support, the ability to watch objects (which will speed up Angular) and Web Components (which offer Angular Directives). The native level support will offer a massive performance boost to frameworks having to fake all...
In their general session at 16th Cloud Expo, Michael Piccininni, Global Account Manager - Cloud SP at EMC Corporation, and Mike Dietze, Regional Director at Windstream Hosted Solutions, reviewed next generation cloud services, including the Windstream-EMC Tier Storage solutions, and discussed how to increase efficiencies, improve service delivery and enhance corporate cloud solution development. Michael Piccininni is Global Account Manager – Cloud SP at EMC Corporation. He has been engaged in t...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
As the race for the presidency heats up, IT leaders would do well to recall the famous catchphrase from Bill Clinton’s successful 1992 campaign against George H. W. Bush: “It’s the economy, stupid.” That catchphrase is important, because IT economics are important. Especially when it comes to cloud. Application performance management (APM) for the cloud may turn out to be as much about those economics as it is about customer experience.
When you focus on a journey from up-close, you look at your own technical and cultural history and how you changed it for the benefit of the customer. This was our starting point: too many integration issues, 13 SWP days and very long cycles. It was evident that in this fast-paced industry we could no longer afford this reality. We needed something that would take us beyond reducing the development lifecycles, CI and Agile methodologies. We made a fundamental difference, even changed our culture...
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
@DevOpsSummit taking place June 6-8, 2017 at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @DevOpsSummit at Cloud Expo New York Call for Papers is now open.
SYS-CON Events announced today that Dataloop.IO, an innovator in cloud IT-monitoring whose products help organizations save time and money, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Dataloop.IO is an emerging software company on the cutting edge of major IT-infrastructure trends including cloud computing and microservices. The company, founded in the UK but now based in San Fran...
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and E...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
The unique combination of Amazon Web Services and Cloud Raxak, a Gartner Cool Vendor in IT Automation, provides a seamless and cost-effective way of securely moving on-premise IT workloads to Amazon Web Services. Any enterprise can now leverage the cloud, manage risk, and maintain continuous security compliance. Forrester's analysis shows that enterprises need automated security to lower security risk and decrease IT operational costs. Through the seamless integration into Amazon Web Services, ...
Software development is a moving target. You have to keep your eye on trends in the tech space that haven’t even happened yet just to stay current. Consider what’s happened with augmented reality (AR) in this year alone. If you said you were working on an AR app in 2015, you might have gotten a lot of blank stares or jokes about Google Glass. Then Pokémon GO happened. Like AR, the trends listed below have been building steam for some time, but they’ll be taking off in surprising new directions b...
Updating DevOps to the latest production data slows down your development cycle. Probably it is due to slow, inefficient conventional storage and associated copy data management practices. In his session at @DevOpsSummit at 20th Cloud Expo, Dhiraj Sehgal, in Product and Solution at Tintri, will talk about DevOps and cloud-focused storage to update hundreds of child VMs (different flavors) with updates from a master VM in minutes, saving hours or even days in each development cycle. He will also...
The word—or rather, the contraction—is simple: DevOps. But DevOps, which refers to the increased communication and collaboration between development and IT operations, is an ever-changing, sometimes complicated term. While “dev” and “ops” used to be siloed into separate philosophies, practices, tools, and work flows, they’re merging into one. The result? A more efficient, reliable process and product.