Meta is building the 'world's fastest AI SUPERCOMPUTER'

Meta is building the ‘world’s fastest AI supercomputer’ to help Facebook in its transformation into a ‘metaverse’, it has announced today. 

The new supercomputer, called the AI Research SuperCluster (RSC), will be the fastest supercomputer in the world when it’s fully built by mid 2022, according to the tech giant. 

RSC will pave the way toward building technology for ‘the next major computing platform’ – the metaverse.

Facebook (the company, not the product) renamed itself Meta in October, as part of its long-term project to turn its social media platform into a metaverse. 

In the future, the social media platform will be accessible within the metaverse using virtual reality (VR) and augmented reality (AR) headsets and smart glasses.   

According to Meta, RSC will help its researchers build better AI models that can work across hundreds of different languages, develop new AR tools and seamlessly analyse text, images and video simultaneously. 

This could enable new AI systems that can power real-time voice translations to large groups of people in the metaverse, each speaking a different language, so they can seamlessly collaborate on a research project or play an AR game together.  

In a blog published today, Kevin Lee, Meta’s director of product management, and Shubho Sengupta, a software engineer for AI Research, said the new supercomputer is already partly operational but will be fully built by the middle of this year. 

Meta declined to say where exactly the supercomputer is based, although it’s worth noting the company’s headquarters are in Menlo Park, California.

With RSC, an AI model with tens of billions of parameters can finish training in three weeks – a third of the time it currently takes with Meta’s infrastructure. 

‘Developing the next generation of advanced AI will require powerful new computers capable of quintillions of operations per second,’ Meta explained.

‘Today, Meta is announcing that we’ve designed and built the AI Research SuperCluster (RSC) – which we believe is among the fastest AI supercomputers running today and will be the fastest AI supercomputer in the world when it’s fully built out in mid-2022.

‘We wanted this infrastructure to be able to train models with more than a trillion parameters on data sets as large as an exabyte – which, to provide a sense of scale, is the equivalent of 36,000 years of high-quality video.’

Meta researchers have already started using RSC to train large models in natural language processing (NLP) and computer vision for research, they say. 

NLP involves building machines that understand and respond to text or voice data, and respond with text or speech of their own – just like humans do. 

RSC is packed with equipment manufactured by US technology company Nvidia, including high-end graphics processing units (GPUs) – electronic circuits that break complex problems into thousands or millions of separate tasks and work them out at once.

AI supercomputers are built by combining multiple graphics processing units (GPUs) into compute nodes, which are then connected by a powerful network to allow fast communication between those GPUs. 

‘RSC today comprises a total of 760 Nvidia DGX A100 systems as its compute nodes, for a total of 6,080 GPUs – with each A100 GPU being more powerful than the V100 used in our previous system,’ Sengupta and Lee say. 

‘Through 2022, we’ll work to increase the number of GPUs from 6,080 to 16,000, which will increase AI training performance by more than 2.5 times.’ 

Because of the coronavirus pandemic, RSC began as a completely remote project that the team took from a simple shared document to a functioning ‘cluster’ – a set of computers that work together – in about a year and a half.  

‘Covid-19 and industry-wide wafer supply constraints also brought supply chain issues that made it difficult to get everything from chips to components like optics and GPUs, and even construction materials – all of which had to be transported in accordance with new safety protocols,’ they say. 

‘To build this cluster efficiently, we had to design it from scratch, creating many entirely new Meta-specific conventions and rethinking previous ones along the way.’ 

Meta says it has committed to long-term investment in AI since 2013, when it created the Facebook AI Research lab.

But it’s taken the focus a whole step further since Mark Zuckerberg’s new goal of making the metaverse a reality. 

The term ‘metaverse’, coined in the 1992 dystopian novel ‘Snow Crash’, is used to describe immersive, shared spaces accessed across different platforms where the physical and digital converge. 

Zuckerberg, who co-founded Facebook in his dormitory room at Harvard University in 2004, has described the metaverse as an ’embodied internet’. 

Within years, Facebook users will be able to use the platform not on their phone or computer, but by donning a VR headset. 

Rather than swiping the screen of a device, they could potentially meet up with a Facebook friend in a virtual shared space – like an ultra-realistic simulation of another planet or an idyllic garden – and vocally chat to each other’s avatars.  

‘It’ll be characterised by social presence, the feeling that you’re right there with another person, no matter where in the world you happen to be,’ the firm says.

‘The metaverse is still a ways off, but parts of it are already here and even more are on the horizon.’