Artificial Intelligence and the Carbon Cost

The AI Revolution

There's no doubt that the explosive growth of large language models has had a positive impact in the business world. From transcription to number crunching, from the most annoying chatbots right through to nightmare IVR's that trap customers in a never ending maze of stupidity.

GEN invested in AI a good few years ago and we're now running a GPU chassis with 32 A100's based on 76 cores over 4 CPU's, and this is pretty powerful for running internal workloads, handling API requests and data processing and analysis for our clients, but like everything in tech, its not enough.

Our next platform will probably be based on Nvidia H100 processors, and we suspect will be distributed across a series of chassis. But there's a problem...

The Problem

GEN has for the last decade or so been focused on our carbon footprint, providing our clients with the assurance of green powered systems for their virtualisation and service requirements, and this is not something that we're looking to change. However, our current AI engine consumes more power than the two data-centres combined, and that's on a quiet day.

This outrageous power demand is not something we can fix, since our combined solar is only sufficient to power the HPE clusters, and so for AI we're taking power from the Grid. This *IS* certified green power from the grid, provided by a company who themselves invest heavily in green power generation, but, our current AI platform takes 14Kw when fully utilised, and that's a problem.

The new H100 processors, whilst faster are not more energy efficient, because it seems energy efficiency is a secondary objective in today's race for power.

The Bigger Problem

The bigger problem here is that we are small in comparison to some companies, and as we see AI hardware becoming more affordable, companies are going to start deploying their own GPU clusters to handle their internal needs, and the power cost for that is what?

So let's assume, for the sake of argument that in the coming years 10% of UK businesses will have some sort of AI hardware, then that's about 560,000 companies. If they each had a fairly small 4 core setup, then that's

78,4Mw of Power or 550Mwh per year

This staggering amount of power will of course be on the rise in the coming decade, and I'm not sure our expansion of solar and wind will out-pace it. GEN's own planned expansion will consume almost 100Mwh in a year and the power alone will cost us £30k, which cannot be sustainable, yet, we need to stay current and compete in the AI market. We could of course just offload all our AI to a third party, like OpenAI or Anthropic, but then its just someone else's problem, and there are massive privacy concerns that make it pretty much unfeasible. UK Companies are coming to us for workloads because they want privacy and isolation, and a guarantee that nothing that's sent is stored or logged.

The Future

We need companies like Nvidia to be thinking about power consumption as a primary factor in new processing hardware. Companies should be looking at the most power efficient platforms for their investment, and indeed in server's we're seeing this happen, but not for AI and GPU. Unfortunately, only the UK currently has ridiculous energy prices, due to government stupidity that's keeping the cost of energy pegged to the price of GAS, but even when this changes, and it will, we, as a country should be pushing everything into sustainability, through energy efficient buildings, hardware, generation, and everything in between.

Blog Index   PermaLink   Comment           24 Votes