MaterLab

Artificial intelligence (AI) is becoming increasingly ubiquitous in daily life, but can it also form part of the solution to manage its own high energy use? Jonathan Spencer Jones finds out.

The fast-growing use of AI, and generative AI in particular, is making headlines almost daily, both for the new applications that are emerging and for their high energy consumption.

While AI’s use in enterprise analytics applications is steadily increasing, its emergence in the consumer domain, for example in the latest generation of mobile phones, is set to skyrocket – with an impact that is yet to become clear.

In a 2023 review of the available literature on ‘Intelligent Computing: The Latest Advances, Challenges and Future’, researchers found that the computational power needed for AI’s rise was doubling roughly every 100 days, and could increase by more than a million times over the next five years.

Generative AI may add to this need. Its increased complexity in relation to the standard predictive AI means it is significantly more computationally intensive and requires at least ten times, and possibly up to 30 times the energy.

For example, a traditional Google query is calculated to consume 0.3Wh, whereas a ChatGPT query is estimated at 2.9Wh – and more complex generative AI content creation tasks would use even more.

Data center challenges

The data centers which underpin the use of AI currently consume a relatively small amount of electricity, with the IEA estimates (excluding cryptomining) standing at 1-1.3% of the global electricity demand in 2022 and projected to increase to 1.5-3% by 2026.

For context, EVs are projected to reach around 2% by 2026, while aluminum production consumes around 4% of the global demand.

Schneider Electric’s white paper on challenges for data center design in an environment of AI disruption paints a similar picture, projecting the total data center power consumption, i.e. demand, to grow at an annual rate of around 10%, from 57GW in 2023 to 93GW in 2028.

But notably, over the same period, AI – representing 4.5GW of power consumption in 2023 – is projected to grow from 25%, to 33% or 14-18.7GW by 2028, taking it from an 8% share to up to 20%.

This growth, which comes with a good deal of uncertainty given the uncertain future uptake of generative AI, is requiring increasingly frequent demand projections from utilities – and the impact is set to be more significant in some countries than in others.

For example, EPRI has estimated in its Powering Intelligence: Analyzing Artificial Intelligence and Data Center Energy Consumption report that data center energy consumption in the US could grow from around 4% to up to 9% of electricity generation annually by 2030.

A further impact comes from the geographic concentration of the industry within certain countries and the local challenges this growth can create. Again in the US, EPRI points to 15 states which accounted for 80% of the national data center load in 2023, while in one state, Virginia, data centers comprised as much as a quarter of the electric load.

Data centers are also increasing in size, as EPRI highlights. New centers are commonly built with capacities from 100MW to 1000MW, roughly equivalent to the load from 80,000 to 800,000 homes.

“Data centers have become a drain on natural resources, not only for energy but also for the water for cooling, and we need to find a way to make them more sustainable,” says Ellie Morris, Engagement Manager of the Energy Transition and Environment team at London-based Faculty AI.

“Then there is the accountability side, and reliably tracking the emissions from a specific model run or usage of a computer or the cloud is a grey area. And any reporting can often obscure the reality of what’s going on if there is any kind of carbon offsetting in a different location to which the energy is being used.”

This opaqueness and inability to reliably track data through the supply chain make it very difficult to set any sort of target, she points out.

AI energy demand

A whole range of solutions is being implemented to manage data center energy demand, and at the same time to decarbonize what has been described as one of the fastest-growing industries worldwide.

The most widely implemented approach for clean energy in the sector has been power purchase agreements (PPAs), with nuclear and geothermal energies now emerging as increasingly popular options alongside the traditional renewables, solar and wind.

Some also look far ahead, such as Microsoft’s 50MW PPA with Helion Energy, which has marked a first for fusion, with its plant expected to come online in 2028.

Then there is onsite renewable generation and storage, and solutions such as demand response and flexibility provision.

But as important as these strategies are for sustainability, they are insufficient and must be accompanied by efficiency improvements, including the use of AI itself.

The two key areas are data management and data manipulation.

DNA storage

One of the main challenges for data centres is storing and archiving the data to be indexed and searchable. The more efficiently this is done, the less the compute time and the lower the corresponding energy consumption for a given use case.

Enter DNA storage as the anticipated next-generation storage medium. DNA storage mimics how the body stores genetic information, offering the prospect of storing almost unlimited quantities of data at a million+ times higher density than an SSD, HDD or tape, in a reliable and durable form. For scale, DNA storage is like being able to fit the data from 150 smartphones – almost 10,000GB – on the head of a pin.

The first foundational specifications for DNA storage – for storing basic vendor and CODEC information – have been released by the DNA Data Storage Alliance, which is backed by companies including Microsoft and Western Digital.

French startup Biomemory, whose goal is to leverage molecular engineering “to build cost-effective and energy-efficient exabyte rackable appliances for data centers”, has reportedly already shipped a DNA storage device, albeit with only 1kB of capacity.

But the company is targeting a 100PB card – in more familiar terms, 100,000TB – by 2026 and a 1,000PB or 1EB card by 2030, and has in its sights a target cost of $1/TB compared to the ten-year cost of $17/TB for magnetic tapes.

Erfane Arwani, CEO of Biomemory, has projected that in ten years, the addressable market for DNA data storage could be around $10 billion.

Powerful and efficient

The second key area for improvement is data center processing. AI accelerators such as graphics processing units (GPUs) are becoming increasingly energy efficient even as they grow more powerful.

US-based NVIDIA, which currently accounts for around 95% of the global market share for AI accelerators, has reported that its new-generation chips are 25 times more energy efficient than the previous generation.

However, NVIDIA’s market dominance is open to challenge as startups emerge, such as Silicon Valley-based Groq.

Groq’s ‘AI inference engine’ is designed to overcome what have become the two main bottlenecks: compute density and memory bandwidth. It aims to speed up AI applications and, in particular, large language models such as ChatGPT.

In April 2024, Groq partnered with data center solution provider Earth Wind & Power to develop a European generative AI ‘compute center’ in Norway, committing to deploy and operate 21,600 of its language processing units (LPUs) in 2024 with the option to increase the number to 129,600 LPUs in 2025.

With the LPUs estimated to use as little as one-tenth the power of traditional GPUs, Groq CEO and founder Jonathan Ross has said that besides the size, what makes this deployment particularly exciting is that it will require a fraction of the power to run compared to a GPU cluster.

Groq is targeting the deployment of more than 1.5 million LPUs worldwide by the end of 2025.

The AI factor

AI is clearly here to stay, and while it comes with the stated energy challenges, it is not necessarily negative, says Guilherme Castro, Senior Manager of Faculty AI’s Energy Transition and Environment team.

“In a country like Iceland, where for example Google can reach 99.9% of renewable energy generation supply in its data centers, there is a clean energy source that is not competing with housing or other electricity-consuming sources.”

So there may be tradeoffs, but ultimately it is about efficiencies, he continues.

“We are working with a company that is utilizing software that, as part of the simulations, requires some to be done manually, and sometimes the analyst will need to run ten or 15 simulations to achieve the result. If we can use AI to do just one simulation to achieve the same outcome, that is a real gain in efficiency, with the AI reducing the energy consumption of that software.

“We work on this daily, and see the impact of AI in not only improving the repetitive tasks of the analyst but achieving the same outcome in a shorter time.”

Looking for efficiencies

Morris prefaces recommendations for AI by stating that there are multiple ways of structuring solutions to address a particular problem.

“We always look for efficiencies, but in one case it may be cost efficiencies and in another carbon efficiencies in the way we set up the algorithms, whereas an alternate company may do it in a different way,” she comments.

But perhaps most important is to evaluate the use case to assess whether AI is the best solution.

“With every new technology there is experimentation, and although AI isn’t new, it’s in the hype phase and everyone is experimenting to test and understand where it can take them,” says Castro.

“But at the end of the day, we need to use AI for the use cases with a clear value return, whether it’s a cost or carbon benefit.

“In the context of the energy transition, if one can implement AI to enable renewable energy to deliver flexibility to overcome a bottleneck, as humans cannot compute all the required amount of information to optimise the grid, it’s a perfect use case.”

Author: Jonathan Spencer Jones
10/10/2024

News