Today, the British semiconductor and software design company Arm is holding the Arm Everywhere China event in Shanghai to unveil its latest contributions to the AI computing landscape.
The company’s aim is to provide a full ecosystem for AI-native computing, including the new AI-native compute platform CSS for Mobile 2, as well as the Arm Mali G2-Ultra NX GPU – a new chip built around neural computing, which promises improved graphics performance at a much lower cost in terms of heat and power consumption. Arm claims its on-device AI will boost graphical performance via a method it positions as more open than Nvidia’s divisive DLSS 5 technology, and closer to the developers’ original vision.
The firm would not confirm which devices will be using the Mali G2-Ultra NX GPU, but the earlier Mali G1-Ultra chip is found in devices from Xiaomi, Vivo, and OnePlus.
Deyan Lazarov is in Arm’s GPU product team as part of the Edge AI business unit, which deals with consumer devices like phones, tablets, laptops, and consoles. He says Arm has been working on the Mali G2-Ultra NX for the past two and a half years, and the thinking behind the chip is to allow mobile developers to take advantage of compute-intensive technology like ray tracing.
“There was actually growing adoption of ray tracing in mobile titles,” he says. “The technology has been a bit slow to get going, but now we’re seeing more and more titles actually interested in enabling ray tracing.”
As GamesIndustry.biz reported earlier this year, the new GPU allows mobile developers to take advantage of Unreal Engine technologies such as MegaLights, and Arm has worked with Sumo Digital to create the game Neural Dawn, which showcases the power of the Mali G2-Ultra NX.
Lazarov says the chip is aimed to support titles that are “mobile first, but then they want to look great: they want to look like desktop titles.”
“Then we also have other titles which are just cross-platform by design, so you can play them on pretty much any gaming device that you have. And then those types of titles naturally want to provide a unified experience to their players, so that when you’re gaming on your console at home, and then you jump on a train to London or whatever, you don’t get a significant drop in the experience.”
A few of the games that will be taking advantage of the power of the new chip include Arena Breakout Infinite from Tencent, Where Winds Meet from NetEase, and Infinity Nikki from Infold Games.
Too hot to handle
“The thing that’s different about mobile phones is that they have different thermals and different power budgets from a console,” says Lazarov. “They cannot just dissipate heat as much as a console can, as much as a PC can, and they are operating off of a battery all the time.”
Arm’s solution to this involves integrating neural processing with the more traditional rendering pipe. “It can reduce the GPU workload and it can also reduce the power that the GPU consumes, which then also has knock-on effects on how efficient and capable your device is at the system level.”
“So what that basically means is that you get this new efficiency, which helps remove some of the limitations that developers face. And it also helps them not have to worry about making trade-offs as much as they would normally do.”
To encourage developers to embrace the possibilities of AI-native graphics, Arm is adopting an open software approach, as well as providing a suite of tools. “We’ve been working really hard with the ecosystem over the past two years or so,” says Lazarov. “The goal that we had was, let’s find a way to use AI which is really developer friendly and allows developers to actually have access to these really big efficiency gains – they can then take that extra budget and then reinvest it in visual fidelity if they want to, performance if they want to, battery life if they want to, or a combination of those three things.”
The key technologies include Neural Super Sampling (NSS), Neural Super Sampling and Denoising (NSSD), and Neural Frame Rate Upscaling (NFRU). Arm’s technology is essentially able to upscale images and cast additional rays, as well as generate further frames using a form of AI reconstruction.
Lazarov gives an example of two frames. “In the first frame, what’s happening is that the GPU is actually rendering just a quarter of the image, and then the remaining three quarters are being reconstructed on our neuro accelerators thanks to NSS,” he says. “Then in the second frame, there’s actually zero work happening in terms of traditional rendering on the GPU. All the pixels are coming thanks to the neuro accelerators, so everything is AI reconstructed.”
Not like DLSS 5
Nvidia has also been working on AI-enabled technology with DLSS 5, which is able to add extra detail to images – but the technology has been met with controversy owing to the way it can potentially deviate from the developers’ original intentions. Lazarov is at pains to point out that Arm’s approach is nothing like Nvidia’s.
“Our networks are not generative, so they’re not from the same nature as the latest DLSS model,” he says. “Our networks are convolution type networks, which means that they don’t imagine anything, they don’t create anything, they just kind of know what things need to look like.
Our networks have been trained on real content, as well as on datasets that are open and public. All of our networks, including resolution networks and frame rate generation networks, are accessible to the public on GitHub and Hugging Face.
This accessibility means that developers can take our networks and customize them to fit their specific needs. For example, if you have a unique art style in your game, you can capture datasets from your game and retrain our network to enhance the visuals according to your style. This way, the network can upscale resolution and insert frames in a way that aligns perfectly with your game’s aesthetics. By retraining the network yourself, you retain control over your intellectual property.
“DLSS is not open. It’s not like our network. It’s proprietary, it’s closed, it’s a black box, you don’t know what’s happening”
This stands in stark contrast to DLSS 5, which is closed and proprietary, offering limited visibility into its operations. With DLSS, users have no insight into the training data or the ability to retrain the network. It’s a take it or leave it approach, unlike our network, which empowers users to tailor it to their preferences.
Furthermore, our technology is mobile-centric, designed to function efficiently within mobile constraints such as power and bandwidth limitations.
For instance, our Mali G2-Ultra NX chip focuses on reducing power consumption significantly. By minimizing external memory access, we achieve not only impressive performance and efficiency gains but also a substantial reduction in power usage.
An illustration of this progress is seen in the enhanced performance of our new GPU compared to its predecessor. Even under maximum load, last year’s GPU could only run Neural Dawn at 40 fps, while the new GPU effortlessly achieves 60 fps.
“In the future we’re going to be where the developers are, we’re going to make it as easy as we can for them”
Regarding development tools, we offer Unreal plug-ins currently, with plans to expand to other game engines like Unity. Our goal is to simplify integration for developers, ensuring a seamless process for implementing our technology into their games.
With our SDK, even developers using custom game engines can benefit from our AI-native graphics solutions. Integrating our technology into existing titles can be accomplished in as little as three months, enabling developers to enhance their games quickly and efficiently.
For example, Neural Dawn, developed by a small team in less than 18 months, showcases the potential of our technology to elevate existing games and accelerate the creation of innovative titles.
Please rewrite the following sentence so it is clearer:
“I have a lot of things to do today and I don’t know how I’m going to get them all done.”