Storage and Stuff

Photo: http://funthingstodowhileyourewaiting.com/wp-content/uploads/2013/12/overpacked-car.jpg

Try and remember back to the day when you could fit all of your worldly goods into your car and you moved into your first apartment or non-parentally owned dwelling.  It was good to have all of that space for yourself and you could never imagine having enough stuff to fill it up.

As you progressed through life you started accumulating stuff.  And more stuff.  Pretty soon that dwelling was bursting at the seams with your stuff.  So rather than get rid of your stuff, you got a bigger place to store your stuff.

Then you had life events (marriage, children, pets, hobbies) that forced you to get more stuff, which filled up your dwelling even faster… so once again, rather than get rid of stuff, you get a bigger place to store your stuff.

But sometimes, you can’t afford a bigger place to store your stuff, so rather than get rid of stuff, you opt for the self-storage rental space and load a whole bunch of stuff in your car and drive it over to the offsite storage location knowing that if ever there was a remote chance you may need to access your stuff, while time consuming, you could still get at it.

In many ways, this little story parallels my life experience with institutional data.  In the beginning, most of what we needed to store and backup were on floppy disks (yes, I’m THAT old).

Then network storage became available, and we soon filled that up with our data, documents and other files.  The challenge then became managing that storage.  It was expensive so we quickly learned to archive old data and files to storage tapes, but doing so ensured the files weren’t readily available.

Then the price of storage dropped like a stone.  Suddenly we could afford drive arrays that would mirror our data over multiple disks to ensure continuity and integrity, and make retrieval lighting fast.  We could buy appliances that indexed our data and files so we could find things. 

Even better, the price of offsite, hosted storage became a viable alternative for the SME (Small to Medium Enterprise).  Climate controlled data centers manned by an army of highly qualified and certified network engineers (or so the brochures say) who would provide secure access to your institutional data, 24 hours a day, 99.999% of the time.

It was the best of both worlds.  I could keep all my institutional data “stuff”, and potentially reduce my OpEx line in my storage budget.  

It made sense to move my data over there… or maybe not… perhaps I should reconsider.

On the ‘plus’ side:

• I have a scalable computing and storage solution that can grow (and shrink) depending on business needs.
• My staffing requirement for high end support techs is lessened.
• My power costs and space requirements go down.
• I can store lots of stuff cheaply, and be confident that it is backed up.

On the ‘negative’ side:

• Turnaround times for service response can increase. You are now one of many customers in the queue.
• Internet connectivity can impede performance.  You are not likely going to have 1GB connectivity to the offsite storage.  Query responses will take longer. If you are a user of big data, be sure to factor your network traffic capacity into the equation. 
• If your Internet connection goes down, you do not have access to your data.
• If you try to save money by leveraging a single site hosted storage solution, your enterprise will be offline should the host site go down.  This has been known to happen with even the large players and is never a good thing.
• I can store lots of stuff cheaply.

You will notice that being able to store a lot of stuff cheaply is on both sides of the equation.  Just like your household stuff, it is not always a good thing to hang onto things.  You do need to purge occasionally.

In the enterprise world, keeping data for a long time can actually be a serious problem.  Should your organization ever find itself in litigation and the lawyers start their eDiscovery process, you can be assured of a significant legal bill if they get to sort through years of email, documents, and data.  If you do not have an electronic document retention policy (which includes emails) your proclivity to store lots of stuff can come back to bite you… but that’s a topic for another post.

Personally, I have found a hybrid solution works best for my institution.  We have struck a balance between the storage that support quick access to data, while leveraging the benefits of offsite storage solutions.

Now… if only I could bring some order to my basement, or else consider moving again.