Managing Unstructured Data With VSI - Angel Events

Managing Unstructured Data with VSIVan Symons

Managing Unstructured Data with Visual Storage Intelligence There are various “types” of data that enterprises utilize today in order to manage theirbusiness effectively. Typically, data is divided into 2 categories:-Structured DataUnstructured DataStructured data is used to describe data when multipe records are stored in a single file.Databases are an example of this type of data. Structured data is sometimes referred to asblock storage because of the way the data is accessed; record or block level data access isneeded.Unstructured data is file data. There are no records ineach file - rather the file itself is a self contained entity.Examples of this type of data include documents,spreadsheets, video and audio files. Because each file isits own self contained entity, managing millions andmillions of these small and large files effectively is amuch different process than managing stuctured data –both from a storage usage and performanceperspective.Unstructured Data GrowthThe reason this distinction is important is:-Unstructured Data is growing exponentiallyStorage arrays are optimized for one or the other – not bothManagement tools / processes are different for eachStorage arrays are typically optimized for one of these types of data structures but not both.For example, many arrays are optimized to block storage access and have a “NAS” (networkattached storage) appliance that can be added specifically to support file (or unstrucutureddata) access. These arrays are optimized for managing structured data, but accommodatesmall amounts of unstrucutured data.Not only are storage arrays optimized to support one or the other of these data types – so tooare management tools. Most clients have some combination of both of these data types andno way to uniformly analyze, report on and manage their storage information across theenterprise. Visual Storage Intelligence (VSI) does exactly this. VSI collects data from all types ofstorage arrays and tracks usage patterns over time - enabling rapid decision making based onreal data.The Challenge:1

To collect data across all storage environments and organize it in a manner which allows clientsto:-See trends over timePredict when additional capacity or performance is neededTrack usage patterns at the array and business unit levelSee the macro trends as well as the detailed File analyticsThe chart below is from one of the VSI “enterprise view” reports across all storage deviceswithin an enterprise. Notice this report shows storage usage by “Data Type”.Storage by Data TypeThis “Storage by Data Type” report reflects how much storage within the enterprise isstructured vs unstructured. Unstructured is broken down even further to enable businessdecisions based on the type of data and how it is growing. This example is typical of manyenvironments and most of our clients were unaware of just how much space is being consumedby storing unstructured data.NAS storage is the term used for managing unstructured data – and this type of storage isdivided into 2 types:2

NAS – Files storageNAS – BackupThis is specifically done within VSI because the types of devices and the decisions for backupdevices are different than that of a general storage array. All storage on EMC Data Domain forexample is classified NAS.The “Storage by Data Type” chart shows:-Block storage1.1 PB (1.2 PB Allocated)NAS Storage (Files) 3.1 PB (3.5 PB Allocated)NAS Backup164 TB (807 TB Allocated)(25% of Usable Storage)(68% of Usable Storage)(2% of Usable)VSI not only gives you an overall picture of your environment, but breaks it down automaticallyby type of data stored (block or file) while tracking usage over time. The reason this breakdownis important is because typical Block devices are configured for high performance with featureslike Flash drives or all flash arrays to maximize performance - while NAS storage is typicallyconfigured for high capacity with performance secondary.VSI has an interactive interface. The previous chart shows allocated and used space, but each ofthese portions can be deselected based on your need. The “Usable and Free Space by DataType” chart on the next page reflects the usable storage for each data type. When used spaceis removed (deselected) you see the amount of freespace available for new data to be added toeach type of storage.3

Usable & Free space by Data TypeFrom this single chart you can see NAS storage has about 700 TB of free space while the blockstorage environment has about 400 TB.With the VSI trend charts like the one below, you can drill into the trends by tier of storage foreach of these environments.NAS Storage Trends4

This NAS Storage Trends chart shows:Usable CapacityUsedFreeBeginning2.5 PB1.9 PB687TBCurrent3.1 PB2.3 PB716TBGrowth600 TB400 TB29TB%Growth24%21%4%This chart is critical because many NAS arrays have features like compression and datadeduplication built in to minimize the amount of space taken up by duplicate data. This meansthe arrays “effective capacity” will vary over time based on the space reduction ratios. Ifvariances occur in these ratios, the amount of “effective free space” can be significantlyreduced without changing the amout of storage used on the array.Below is an example of an array where the “effective capacity” (purple) changed over time toreduce overall free space while the used remained fairly constant.Sample Data Domain Storage TrendsThe change in data reduction is what causes the reduced overall freespace. VSI automaticallytracks and projects this usage over time for all storage devices. This is particularly important forNAS devices because of the built in data reduction technologies.5

The final piece of data analysis at the enterprise level gives visibility into the type of storageavailable for each data type. The “Free Space by Storage Tier” chart below shows theUnstructured Data (NAS storage) has a variety of technologies available based on specific userrequirements.Free Space by Storage Tier (NAS Storage)Tier 1 (SSD/Flash)Tier 2 (Blue – SAS10K/15K)Tier 3 (Red – SATA)MultiTier (SSD SATA/SAS)very little avalable – and this is as expected193TB (this tier has shown expansion over time)170 TB (high density)346 TBNow I not only see what is available, but I also see the types of storage available. In many casesthe issue is not if storage is available, but is if “the right storage” is available. Notice VSI givesyou instant visibility with this information across your enterprise.6

Array Storage AvailabilityThe same detailed analysis and set of reports are available at the storage array level. Once youselect the type of storage you need there are a variety of ways to analyze the data interactivelyand select the proper location. The chart below shows free space by individual storage array.Free Space by ArrayData for each array is kept at the Array, Controller, Pool, Volume and Business Unit levels. Thisallows you to view, track and analyze the storage at whatever level necessary.7

This device level chart below shows the pattern of growth in terms of performance and capacityover time. It also forecasts the next 6 months growth so that the appropriate decisions andplans can be made without a lot of complex data calculations, taking the guess-work out of yourstorage management.Device Trends (Capacity & Performance)Share File AnalysisThe final phase of analysis for unstructured data is usage patterns. You need to know how oftenindividual files are being accessed so you can make decisions including what should be movedoff premise, to tape, to the cloud, or not saved at all.File Share Analysis by Data Type and Last Access Date8

The initial screen for “File Share Analysis by Data Type” on the previous page summarizes thedata in tabular format showing the following:Overall Space:Total Files:Files Not Access in 3 yearsØ 3yr Old File Space1.6 PB27 Million24 Million1.4 PBFile Space by Date of Last AccessThe report above shows the same data in chart format. It says 64% of the 1.6 PB share is neverbeing accessed. This screen allowed the client to migrate this data off of spinning disk to archivestorage which reduced their IT spend by over 200,000 in this particular area. The client’scomment was:“Without this information we would never have made this decision and moving this low accessdata allowed us to avoid additional on premise storage purchases – thanks to VSI we avoided allof that and even better we were able to make the decisions in a matter of days.”9

Forecasting & Business Unit AnalysisOne benefit of having all storage devices reported on in a single tool is the ability to:-Do Business Unit Analysis across all storageBuild forecasts that integrate and leverage all storageBusiness Unit AnalysisVSI captures and stores all of your physical host mappings, virtual host mappings andvolume/file share mappings. This data is categorized using customer defined conventions whichthen allows an executive to view which applications and business areas are using what and howmuch storage.If this is only done with Block or File (unstructured data) independently it does not provide acomplete picture and may result in less than optimal decisions.Usage by Business ApplicationArray 1Array 2Array 3#4Array 5Array 6Array 7Array 8We see in the “Usage by Business Application” chart above an example of storage for differentareas of the business as defined by the client. BC (Business Continuity ) takes up a significantamount of space. Once this chart was drawn it was realized by the customer they had duplicateefforts underway by different areas of the business. They were able to consolidate thoseactivites which resulted in:-Reducing the overall space used in BC by over 2/3Improve device freespaceEliminating potential storage purchases on both Block and BAS storage arrays10

The same Business Unit growth patterns are tracked over time so you can be sure tounderstand and meet the unique needs of each business area in a timely and cost effectivemanner.Forecasting by Data TypeThe forecasting models are automatically generated each week with trends being calculatedautomatically.The forecasting screen allows clients to allocate the amount of free space they desire on top ofthe space used to get an accurate indication of their storage requirements. By tracking actualconsumption the forecasting chart gives a much more accurate picture of storage demand. VSIalso supports the ability to forecast at the business unit and application level.11