Despite moving on from the University, Charles has come through with a follow-up article with further details of the UNSW data centre. Thanks Charles!

Charles writes:

Last blog we covered “Switching”, The new UNSW R1 data centre is designed as a modern 2nd generation High Density data centre. This article covers the components that make up a High Density data centre, and its capabilities within UNSW.

2.) HIGH DENSITY (power in kW as a measure):
The data centre is designed for High Density – between 7.5-30kw per rack. The first stage is 7.5kW average per rack, with an overall capability of 1500kW from the substation in Stage One West Hall. The R1 data centre is designed to grow to 3000mW with the addition of a second power feed of 1500kW from the existing campus substation.

The data centre contains two APC “PODs” and 10 free-standing racks per hall. Each POD is a self-contained cooling and racking system(of 30 racks each),  with connections back to the electrical supply and chillers outside.

With higher density rack servers, (say 6x6U quad CPU 12-core servers)a rack can be handling over 10kW of load however, with blade chassis (with 4xchassis x 8 blade quad-cpu 12-core takes us to 16-20 kW in a rack. High Performance Computing (HPC) servers for faculty research  go up to 30kW per rack.

Storage density has also increased with a terabyte in a rack, weighing around 1 tonne bring not only power considerations but also weight issues. Another reason for not having false floors in the data centre – holding all that weight is better spread directly to a ground floor slab such as in UNSW’s new R1 data centre.

With the increasing density within the data centre, (numbers of servers/rack and power requirements), we need to manage the cooling, electrical supply, POD configuration as well as the server, storage and switching configuration. Rather than increase gradually across the entire data centre, the aim is to target highest density per rack from Day One, and then to spread the numbers of racks.  To get rack density higher (and therefore power/cooling load)we need to alter the configuration of the In-Row Coolers, and/or we can later expand into the East hall and add another 70 racks.

The common theme around the modern high density data centre is that everything is connected and interdependent. Its not just a matter of square metres – the “Kennards Storage” approach. We take into account the server configuration, power usage, cooling configuration, switching and cabling density before making an integrated decision. This central theme will continue throughout the blog, as is part of our UNSW Data Centre Strategy, and incorporated into our new Data centre designs.

Next blog, we will look at “Cooling” and how the APC In-Row cooling units give us the capability to cool up to 30kW in a rack, and why our design is based upon cooling the rack and not the room. We say “cool the refrigerator not the kitchen”.

Until next time,

Charles

Charles Nolan
Ex-IT Consultant for UNSW