Data Storage Problems

This week the New York Times published a long article about the problem of data storage, and I would like to summarize some of their findings. The article is available here in Saturday’s technology section.

The article is an attack at what the author sees as wasteful use of resources in data storage centres. There are now hundreds of thousands of these huge centres spread throughout the world, and the problem is they use an incredible amount of electricity. The servers have to be kept cool and they have to have spare capacity so that we can download whatever we want whenever we want.

Inside a US data centre

Inside a US data centre

Worldwide these centres use about 30 billion watts of electricity, and that is about 30 nuclear power plants worth of power. A single data center uses about the same amount as a small town, and the main criticism is the nature of the usage.

In the US 2% of all electricity used goes to these data centers, but the vast majority of this resource is wasted. Typically many servers are left to run 24 a day but never or rarely used (more than half in this study), and the average machine in operation uses less than 10% of its capacity. Servers are left running obsolete programs or in ‘comatose’ because nobody wants to risk a mistake and turn them off.

All of this means that any data center might use 30 times as much electricity as is needed to carry out the functions it performs.

All of these centres also have to have a back up in case of power failure, and so are surrounded by diesel generators and stacks of batteries, and many have been found in breach of environmental regulations and fined. The article gives details but the companies are names that we all know and use.

If you read the more than 300 comments however you will discover that a lot of people do not agree with the findings as reported. Many technicians argue that the companies cited are investing huge amounts of money into making the storage of data more efficient, and are constructing wind farms and using solar power in an attempt to cut costs and emissions. The article has its agenda and exploits it fully, but the problem is real.

I personally believe that we are witnessing the results of a digital culture change. We no longer have to store data on our machines, we can store it in some mythical cloud out there in the cyber-universe. This makes us think that it somehow exists without the need for a hard drive, but this is not true. As a result we keep things that we do not need. I have 500 e mails in my inbox, with attachments, photos that I will never again look at and other useless things, and they are all in storage somewhere.

Technology advances, storage gets cheaper and uses less space, but the amount of data created is growing at an incredible rate. My question is, can we do anything about it? Are we not the ones who should take some responsibility and think about the consequences of our actions. We think about not using paper to print emails but we don’t think about not sending them!

7 thoughts on “Data Storage Problems

    • there won’t be a drop off. More and more data is produced all of the time, not doubling but multiplying hundredfold and advances in storage capacity are not growing at such a pace. can you imagine how much is involved in downloading a 3D film?

      • When you put it that way I agree it is only going to get worse. Interestingly, I have been researching cheap hosting recently and noticed that a lot of the big players are putting themselves forward as green companies. I was a little perplexed as to why they would be doing this however after reading your article it is obvious that they are expecting all of this to become public knowledge pretty soon.

  1. 2% of the total US consumption seems a huge amount I believe but “the vast majority of this resource is wasted” doesn’t seems to be logical if we experience the explosion of networking node over the globe.
    Imagine if we try to shut down these data centers & if some of them were used in a CDN (Content Delivery network), we’ll then be waiting for our turn to access the content. It is the technology that accelerates our experience of accessing the booming internet and if this cost saves the time (which is equivalent to “money”), it’s not at all a problem to let them work 24X7.
    Isn’t it?

    • They have to work constantly and have over capacity, but some people argue that they are rune inefficiently.There have been experiments in running centres at 90% capacity that have gone without any problems.

  2. The statistics are alarming especially where the writer writes that the combined electricity consumption of the data centres is equivalent to 30 nuclear power plants worth of power! The excess electricity used in running the data centres could be mobilized and utilized in supplying electricity to towns in countries which face constant electricity shortages or sometimes have access to no electricity at all.
    Funny how
    USB thumb drive like this one will hold 100 times what a data center would have 20 years ago.

Leave a Reply

Your email address will not be published. Required fields are marked *