ChatGPT may not be as power-hungry as once assumed

ChatGPT, OpenAI’s chatbot platform, may not be as power-hungry as once assumed. But its appetite largely depends on how ChatGPT is being used, and the AI models that are answering the queries, according to a new study.

A recent analysis by Epoch AI, a nonprofit AI research institute, attempted to calculate how much energy a typical ChatGPT query consumes. A commonly-cited stat is that ChatGPT requires around 3 watt-hours of power to answer a single question, or 10 times as much as a Google search.

Epoch believes that’s an overestimate.

Using OpenAI’s latest default model for ChatGPT, GPT-4o, as a reference, Epoch found the average ChatGPT query consumes around 0.3 watt-hours — less than many household appliances.

“The energy use is really not a big deal compared to using normal appliances or heating or cooling your home, or driving a car,” Joshua You, the data analyst at Epoch who conducted the analysis, told TechCrunch.

AI’s energy usage — and its environmental impact, broadly speaking — is the subject of contentious debate as AI companies look to rapidly expand their infrastructure footprints. Just last week, a group of over 100 organizations published an open letter calling on the AI industry and regulators to ensure that new AI data centers don’t deplete natural resources and force utilities to rely on non-renewable sources of energy.

You told TechCrunch his analysis was spurred by what he characterized as outdated previous research. You pointed out, for example, that the author of the report that arrived at the 3-watt-hours estimate assumed OpenAI used older, less efficient chips to run its models.

“I’ve seen a lot of public discourse that correctly recognized that AI was going to consume a lot of energy in the coming years, but didn’t really accurately describe the energy that was going to AI today,” You said. “Also, some of my colleagues noticed that the most widely-reported estimate of 3 watt-hours per query was based on fairly old research, and based on some napkin math seemed to be too high.”

Granted, Epoch’s 0.3 watt-hours figure is an approximation, as well; OpenAI hasn’t published the details needed to make a precise calculation.

The analysis also doesn’t consider the additional energy costs incurred by ChatGPT features like image generation, or input processing. You acknowledged that “long input” ChatGPT queries — queries with long files attached, for instance — likely consume more electricity upfront than a typical question.

You said he does expect baseline ChatGPT power consumption to rise, however.

“[The] AI will get more advanced, training this AI will probably require much more energy, and this future AI may be used much more intensely — handling much more tasks, and more complex tasks, than how people use ChatGPT today,” You said.

While there have been remarkable breakthroughs in AI efficiency in recent months, the scale at which AI is being deployed is expected to drive enormous, power-hungry infrastructure expansion. In the next two years, AI data centers may need close to all of California’s 2022 power capacity (68 GW), according to a Rand report. By 2030, training a frontier model could demand power output equivalent to that of eight nuclear reactors (8 GW), the report predicted.

ChatGPT alone reaches an enormous — and expanding — number of people, making its server demands similarly massive. OpenAI, along with several investment partners, plans to spend billions of dollars on new AI data center projects over the next few years.

OpenAI’s attention — along with the rest of the AI industry’s — is also shifting to so-called reasoning models, which are generally more capable in terms of the tasks they can accomplish, but require more computing to run. As opposed to models like GPT-4o, which respond to queries nearly instantaneously, reasoning models “think” for seconds to minutes before answering, a process that sucks up more computing — and thus power.

“Reasoning models will increasingly take on tasks that older models can’t, and generate more [data] to do so, and both require more data centers,” You said.

OpenAI has begun to release more power-efficient reasoning models like o3-mini. But it seems unlikely, at least at this juncture, the efficiency gains will offset the increased power demands from reasoning models’ “thinking” process and growing AI usage around the world.

You suggested that people worried about their AI energy footprint use apps such as ChatGPT infrequently, or select models that minimize the computing necessary — to the extent that’s realistic.

“You could try using smaller AI models like [OpenAI’s] GPT-4o-mini,” You said, “and sparingly use them in a way that requires processing or generating a ton of data.”