Our goal is to provide everyone with the storage space that they need while also managing a scarce resource. Consumers have seen the prices of mass storage devices drop precipitously over the past few years and, while the prices of server storage have also fallen, the drop in price has not been as pronounced. Furthermore, server storage is significantly different in many ways, as are the systems that are used to provide you with access to this storage.
Initial Space Allocation
Our initial space allocation on the SAS e-mail servers is 100MB. This may seem small when you compare it to free hosted services like Google that offer in excess of 7TB, but this limited allocation serves multiple purposes. Since there are a number of different e-mail systems at Rutgers as well as a number of free e-mail servers like Google, many of our users will receive an account on our systems that they will never use. In the best case, those users will have their mail forwarded to another account and not store any mail on our servers; however, in many cases users simply ignore those accounts. The account sits idle and collects e-mail such as departmental bulletins and spam. Since no one is using the account, the mail simply builds up until the quota is reached. By providing a small quota, we minimize the amount of "junk" mail that we store for no reason.
In cases where the mail account is used, providing a large amount of space to all users imperils the reliability of the mail system. We currently have over 1000 e-mail account on our mail server. If every user were granted 1GB of default quota, any malicious user could send repeated large e-mail messages to multiple users on our system and quickly fill all available space allocated for e-mail. This would bring the entire e-mail system down. By limiting user quotas we are able to minimize the risk of this happening by decreasing the overall available free space that can be exploited by anyone with an e-mail account and a list of addresses.
These small initial allocations do not preclude the allocation of additional space when needed. There are users on our systems with excess of 7GB of mail and, as they use more space, we will allocate them more quota when they need it. In some cases, we will charge a nominal fee for large amounts of drive space to offset our costs but this only happens in rare cases.
File Storage
Our initial space allocation on the SAS Novell servers is also 100MB in each user's home directory. For the vast majority of users, this is sufficient but we will increase this allocation as needed upon request. Since the home directory is primarily used for personal files that are not shared with other users, most additional space requests are for the Workgroup drive.
There is no default allocation on the Workgroup drive but departments can request up to 5GB of space on this drive without incurring any additional fees. Requests for space in excess of 5GB are charged at $15/gig/year. This charge is assessed for two reasons. First and foremost, it helps us offset the cost of the space; and second, it encourages users to use space efficiently. For example, we routinely find users that attempt to backup their entire local hard drive to the server simply because they don't realize that perhaps only their My Documents directory is worth copying to the server. By monitoring individual space allocation and charging for large amounts of space, we're often able to help users store their data more efficiently while simultaneously decreasing costs.
Why are costs for the storage so high when I can buy a 1TB hard drive for $100?
With consumer storage prices going down at an ever increasing rate, people are increasingly hard pressed to understand why we are so "stingy" with our storage.
The fact is that the drives you can buy in the store bear little in common with those used for our servers. You can walk into a store and get a 1TB hard drive for $79 while the same amount of space on our servers costs about $7000.
The drives you purchase on the consumer markets are inexpensive SATA drives coupled with relatively slow technology like USB 2.0 to transfer data. They also tend to be single units that have absolutely no redundancy and rely on cheap components like a $5 power supply commonly referred to as a "wall wart".
We purchase storage arrays -- systems that contain 15-20-30 disk drives in a chassis that has completely redundant power supplies, redundant paths for data access and even redundant hard drives that immediately take over should an individual drive in the chassis fail. They are also configured to stripe data across the drives so that the loss of one drive doesn't result in the loss of data.
The array chassis is then connected to a SAN fabric that transfers data between the storage and the connected systems over fiber optic cabling that runs at between 2gbps and 8gbps as opposed to the 480mbps of those USB drives.
That entire infrastructure is then covered with a 24x7x365 warranty with four hour response time so, even though the storage system is entirely redundant (which prevents the users from even knowing about the failure of a component), we have the replacement for the bad part in our hands in four hours or less and can install it with no disruption in service.
Also consider that we use separate storage arrays with similar redundancy and connectivity to backup all that data on a nightly basis and we keep those backups for 60 days. That infrastructure is then duplicated one more time on our offsite backup server which also keeps 60 days of backups to guard against a catastrophe that destroys our entire data center.
This means that for every gigabyte you store on the main server, two more gigabytes are used on our backup and offsite backup servers respectively along with 60 days of old versions for all files that have changed.
As you can see, the storage we are using goes well beyond what you can purchase in the store and this results in a higher cost. There are also other critical components that are part of this infrastructure that haven't been mentioned. They include servers that host the primary and backup storage as well as the network and physical infrastructure that make sure that it is secure and in appropriate environmental conditions. Finally, there is the staff time that is involved in implementing, supporting, and maintaining this infrastructure.
In short
We control the allocation of space closely but we're not opposed to giving out larger chunks of space when needed. All you need to do is let us know how much you'll need, how long you think you'll need it for and what the proposed use is, and we'll be happy to give you what you need so long as we have it available. (We ask for the proposed use because we can provide different types of space depending on use.) In cases where large amounts of space are needed for extended periods of time we do charge a nominal fee to help offset the cost.
We're careful not to hand out large amounts of space on a whim because it seems that the amount people store seems to be driven, in part, by the amount of free space they have immediately available. The policy we've developed helps us manage space usage so those that truly need the additional space can have it.
