Meet Wiener, the University of Queensland supercomputer that Dell kept quiet
- 02 October, 2019 00:01
In 2017, the University of Queensland (UQ) department of molecular biology, neuroscience and translational research embarked on a journey to build Wiener, the Dell Technologies high-performance computer (HPC).
With the capacity to process up to 80 terabytes of data per day, Wiener – named after mathematician Norbert Wiener – was kept under wraps until this week when it was unveiled at Dell Technologies Forum in Sydney.
The HPC is a GPU-accelerated supercomputer that was built by Dell for the university’s Research Computing Centre (RCC).
The delivery took two and a half weeks and another two were then required to turn it on back in November 2017.
It was initially built on Dell EMC PowerEdge R740 servers with the visualisation nodes featuring two PowerEdge R740, two Intel Xeon Gold 6132 (28 cores per node), 1 TB DDR4 RAM, three NVIDIA Tesla V100 Accelerator units, 1.6 TB Dell EMC NVMe flash storage, 100 Gbps Mellanox EDR InfiniBand HCA’s and Dell EMC ProSupport Plus: (7x24) Technical Support & Assistance.
The compute and analysis nodes had 17 PowerEdge R740, two Intel Xeon Gold 6132 (28 cores per node), 384 GB DDR4 RAM, two NVIDIA Tesla V100 Accelerator unit, 1.6 TB Dell EMC NVMe flash storage, 100 Gbps Mellanox EDR InfiniBand HCA’s and Dell EMC support.
At the time, NVIDIA was yet to be shipped for commercial sales, hence the university was unable to make a broad public announcement about the supercomputer.
The Queensland Brain Institute (QBI), the university’s neuroscience research institute, is using the Wiener system to model the behaviour of ultrasound using an analysis technique called Finite Element Method (FEM). The modelling calculates what happens to each element of the brain when an ultrasound is passed through the skull.
It is hoped that ultrasound can be used to temporarily allow direct delivery of therapeutic drugs to the brain, something not currently possible due to the presence of a blood-brain barrier and activate cells that can digest the plaques that are a hallmark of Alzheimer’s disease.
UQ is not on the human trial stage but it it is working towards it. For now though, the results will be tested on sheep , an animal with similar skull properties to humans, according to UQ.
The demand for Wiener's capabilities has expanded its use to climate modelling, psychological testing and learning, and disease identification.
"We underestimated how much people cared [about Wiener]," said Jake Carroll, CTO Research Computing Centre at the University of Queensland.
“It’s become a whole ecosystem. Wiener has become a plethora of massive machine learning and deep learning capabilities in the organisation. It’s the focal point of AI computing infrastructure at The University of Queensland.”
Elesewhere, the University of Queensland’s School of Information Technology and Electrical Engineering departments are using Wiener to develop new digital pathology techniques for faster blood samples results, while another machine learning algorithm will be able to diagnose the presence of skin cancer from histology slides with the accuracy of a trained pathologist.
The Wiener supercomputer also supports the university’s lattice light sheet microscope (LLSM), using an image restoration technique called deconvolution to provide clear, real-time 4D biology imaging.
To cater for these, the system went through an expansion in January 2019 using PowerEdge C4140, doubling the initial nodes and tripling the compute performance. Called phase two, this includes 15 PowerEdge C4140 which features two Intel Xeon Gold 6132 (28 cores per node), 384 GB DDR4 RAM, four NVIDIA SXM2 Tesla 32 GB V100 Accelerator units, 1.6 TB Dell EMC NVMe flash storage, 100 Gbps Mellanox EDR InfiniBand HCA’s and Dell assistance.
The upgrade resulted in 11 per cent faster training for machine learning and allowing for lower latency internodal communication for distributed deep learning (DL) workloads.
“We are seeing exponential growth in both velocity and volume of data being generated by scientific instruments around the world,” said Andrew Underwood, field chief technology officer, HPC and Artificial Intelligence at Dell Technologies APJ.
“This is driving Dell Technologies and our customers to collaborate on approaches that leverage parallel processing power from our Dell EMC PowerEdge platforms, in order to unlock the confluence of simulation, analytics and machine learning.”
Back in 2017, when Wiener was built, it was the first supercomputer outside the US specifically designed for extremely high-resolution and fast microscopy image processing.
At the time, QBI and School of Mathematics and Physics researcher Professor Geoff Goodhill said Wiener would dramatically increase the speed at which we can derive new insights from experimental data, and will also test new hypotheses about nervous system function.
In July, Australian National University (ANU) announced its own "fastest supercomputer" in Australia was about to get 10-times more powerful, thanks to a multi-million dollar upgrade being delivered by Fujitsu Australia. The new machine, dubbed 'Gadi', will replace the current model named 'Raijin', also build by Fujitsu back in 2012.