Supercomputers have many applications, including modelling simulations, performing complex scientific calculations, and rendering high definition 3D graphics.

iVEC operates multiple supercomputers that use complementary technologies. These include traditional CPUs and GPU accelerators.

The supercomputing hardware currently managed by iVEC includes the systems located within the Pawsey Centre (Magnus and Galaxy) and the Epic and Fornax supercomputers.

MAGNUS

Magnus, from the Latin ‘Great’, is a latest-generation Cray XC30 system that will be used for supercomputing projects across the entire range of scientific fields serviced by iVEC. Magnus currently comprises two compute cabinets, each holding 52 compute blades, with four nodes per blade and each node supporting two 8-core Intel Sandy Bridge Xeon E5-2670 processors running at 2.6 GHz, for a total of 3328 cores delivering around 69 TFLOPS. The compute nodes communicate amongst themselves over Cray’s high-speed, low-latency Aries interconnect. Storage consists of two petabytes of scratch file system space, provided by Cray’s Sonexion Lustre appliances, and is connected via 56 Gbit/s FDR Infiniband. Magnus will be expanded in late 2014 with additional cabinets and processors to deliver final performance in excess of one PFLOPS.

Vital statistics
Manufacturer: Cray Inc. (USA)
Model: XC30 Series Supercomputer
Compute Processors: Intel Xeon E5-2670 “Sandy Bridge”
Computing Power: 69 teraFLOPS, from 208 dual-socket compute nodes
Memory: 13 terabytes (64 gigabytes of DDR3-1866 per compute node, at 1,600 MHz)
Interconnect: Cray Aries interconnect, at 72 gigabits/sec per node
Network Topology: Cray Dragonfly
Weight: 1.7 tonnes/cabinet
Power consumption: circa. 50 kWatts/cabinet
Local storage: Sonnexion 1600 Data Storage System, 2 petabytes

Interesting facts

•The XC30 is the first Cray series to feature Intel Xeon processors.
•Magnus Phase 1 is equivalent to 36,315 Cray -2 systems.
•Internode communications are more than 9,300-times faster than the NBN.
•Magnus (Phase 1) is 70-times more powerful than iVEC’s first teraFLOPS-capable supercomputer, named Cognac, which began production in 2005.
•Each cabinet of Magnus (Phase 1) has 3 chassis. Each chassis contains 16 blades. Each blade has 4 nodes. Each node has 2 sockets and each socket has an 8-core Sandy Bridge processor, giving a total core count of 3,328.

GALAXY

Galaxy is a Cray XC30 system that will be used to support the efforts of the Square Kilometre Array precursor projects currently underway in the north-west of Western Australia. It is a Real-Time Computer, allowing direct processing of data delivered to the Pawsey Centre from the Murchison Radio Astronomy Observatory. Galaxy consists of three cabinets which contain 118 compute blades, each of which has four nodes. Each node supports two 10-core Intel Ivy Bridge Xeon E5-2960 processors operating at 3.00 GHz, for a total of 9440 cores delivering around 200 TFLOPS of compute power. Communication between the nodes occurs via Cray’s high-speed, low-latency Aries interconnect. Galaxy has its own one petabyte Lustre scratch file system, provided by Cray’s Sonexion appliance storage system and connected via FDR Infiniband.

Vital statistics
Manufacturer: Cray Inc. (USA)
Model: XC30 Series Supercomputer
Compute Processors: Intel Xeon E5-2690v2 “Ivy Bridge”
Computing Power: 200 TeraFLOPS, from 472 dual-socket compute nodes
Memory: 29.5 Terabytes (64 Gigabytes of DDR3-1866 per compute node, at 1,866 MHz)
Interconnect: Cray Aries interconnect, at 72 Gigabits/sec per node
Network Topology: Cray Dragonfly
Weight: 1.7 tonnes/cabinet
Power consumption: Circa. 50 kWatts/cabinet
Local storage: Sonnexion 1600 Data Storage System, 2 Petabytes

Epic

Epic is a HP commodity linux cluster housed in a Performance Optimised Datacentre (POD). It runs CentOS Linux. It comprises 25 HP BladeSystem c7000 Enclosures each of which contains 16 double-dense HP ProLiant BL2x220c G6 blades. Each of the compute node (blade) contains two hex-core Intel Xeon X5660 processors operating at 2.8 GHz with 24 GB of RAM. In total, there are 800 compute nodes providing 9600 cores and 18 TB of RAM to iVEC researchers. Each node contains a quad data rate (QDR) Infiniband card.

Fornax

Fornax is Latin for “furnace” and is the name of a southern hemisphere constellation identified by Nicolas Louis de Lacaille in 1756. This name is representative of the supercomputer’?s capability for dealing with data-intensive problems, which is expected to be of particular use to the radio astronomy computing community.

The SGI supercomputer comprises 96 nodes, each containing two 6-core Intel Xeon X5650 CPUs, an NVIDIA Tesla C2075 GPU and 72 GB RAM, resulting in a system containing 1152 cores and 96 GPUs.

The system also has a 500 TB global filesystem, in addition to each node having 7 TB of local disk space. One distinguishing feature of the system, allowing it to tackle data-intensive problems, is the presence of two InfiniBand networks. The first allows each node to access the global filesystem, while the second allows access to neighbouring nodes. This dual-rail system also allows separation of MPI traffic from storage traffic.