Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Content provided by Asim Hussain and Green Software Foundation. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Asim Hussain and Green Software Foundation or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Player FM - Podcast App
Go offline with the Player FM app!
Go offline with the Player FM app!
The Week in Green Software: AI Energy Scores & Leaderboards
MP3•Episode home
Manage episode 501503742 series 3336430
Content provided by Asim Hussain and Green Software Foundation. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Asim Hussain and Green Software Foundation or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Host Chris Adams is joined by Asim Hussain to explore the latest news from The Week in Green Software. They look at Hugging Face’s AI energy tools, Mistral’s lifecycle analysis, and the push for better data disclosure in the pursuit for AI sustainability. They discuss how prompt design, context windows, and model choice impact emissions, as well as the role of emerging standards like the Software Carbon Intensity for AI, and new research on website energy use.
Learn more about our people:
Find out more about the GSF:
News:
- A Gift from Hugging Face on Earth Day: ChatUI-Energy Lets You See Your AI Chat’s Energy Impact Live [04:02]
- Our contribution to a global environmental standard for AI | Mistral AI [19:47]
- AI Energy Score Leaderboard - a Hugging Face Space by AIEnergyScore [30:42]
- Challenges Related to Approximating the Energy Consumption of a Website | IEEE [55:14]
- National Drought Group meets to address “nationally significant” water shortfall - GOV.UK
Resources:
- GitHub - huggingface/chat-ui: Open source codebase powering the HuggingChat app [07:47]
- General policy framework for the ecodesign of digital services version 2024 [29:37]
- Software Carbon Intensity (SCI) Specification Project | GSF [37:35]
- Neural scaling law - Wikipedia [45:26]
- Software Carbon Intensity for Artificial Intelligence | GSF [52:25]
Announcement:
- Green Software Movement | GSF [01:01:45]
If you enjoyed this episode then please either:
- Follow, rate, and review on Apple Podcasts
- Follow and rate on Spotify
- Watch our videos on The Green Software Foundation YouTube Channel!
- Connect with us on Twitter, Github and LinkedIn!
TRANSCRIPT BELOW:
Asim Hussain: ChatGPT, they're all like working towards a space of how do we build a tool where people can literally pour junk into it, and it will figure something out.
Whereas what we should be doing, is how do you use that context window very carefully. And it is like programming.
Chris Adams: Hello, and welcome to Environment Variables, brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding green software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software.
I'm your host, Chris Adams.
Hello and welcome to this week in Green Software where we look at the latest news in sustainable software development. I am joined once again by my friend and partner in crime or occasionally crimes, Asim Hussain, of the Green Software Foundation. My name is Chris Adams. I am the Director of Policy and Technology at the Green Web Foundation, no longer the executive director there,
and, as we've moved to a co-leadership model. And, Asim, really lovely to see you again, and I believe this is the first time we've been on a video podcast together, right?
Asim Hussain: Yeah. I have to put clothes on now, so, so that's,
Chris Adams: That raises all kinds of questions to how intimate our podcast discussions were before. Maybe they had a different meaning to you than they did to me, actually.
Asim Hussain: Maybe you didn't know I was naked, but anyway.
Chris Adams: No, and that makes it fine. That's what, that's what matters. I also have to say, this is the first time we get to, I like the kind of rocking the Galactus style headphones that you've got on here.
Asim Hussain: These are my, yeah, no, these are old ones that I posted recently. I actually repaired them. I got my soldering iron and I repaired the jack at the end there. So, I'm very proud of myself for having repaired. I had the right to repair. Chris. I had the right to repair it.
Chris Adams: Yeah. This is why policy matters.
Asim Hussain: I also have the capability.
Chris Adams: Good. So you can get, so, good on you for saving a bunch of embodied carbon and, how that's calculated is something we might touch on. So, yes. So if you are new to this podcast, my friends, we're just gonna be reviewing some of the news and stories that are kinda showed up on our respective radars as we work in our kind of corresponding roles in both the Green Software Foundation and the Green Web Foundation.
And hopefully this will be somewhat interesting or at least diverting to people as they wash their dishes whilst listening to us. So that's the plan. Asim, should I give you a chance to just briefly introduce what you do at the Green Software Foundation before I go into this?
'Cause I realized, I've just assumed that everyone knows who you are. And I know who you are, but maybe there's people who are listening for the first time, for example.
Asim Hussain: Oh yeah. So, yeah. So my name's Asim Hussain. I am a technologist by trade. I've been building software for several decades now. I formed the green software, yeah, Green Software Foundation, you know, four years ago. And, now I'm the executive director and I'm basically in charge of, yeah, just running the foundation and making sure we deliver against our vision of a future where software has zero harmful environmental impacts.
Chris Adams: That's a noble goal to be working for. And Asim, I wanted to check. How long is it now? Is it three years or four years? 'Cause we've been doing this a while.
Asim Hussain: We, yeah. So we just fin, well, four years was May, so yeah, four years. So next birthday's the fifth birthday.
Chris Adams: Wow. Time flies when
the world is burning, I suppose.
Alright, so anyway, as per usual, what we'll do, we share all the show notes and any links that we discuss or projects we discuss, we'll do our damnedest to make sure that they're available for anyone who wants to continue their quest and learning more about sustainability in the field of software.
And I suppose, Asim, it looks like you're sitting comfortably now. Should we start looking at some of the news stories?
Asim Hussain: Let's go for it.
Chris Adams: Alright. Okay. The first one we have, is a story from Hugging Face. This is actually a few months back, but it's one to be aware of if it missed you the first time. So, Hugging Face released a new tool called Chat UI Energy that essentially lets you see, the energy impact live from using a kind of chat session,
a bit like ChatGPT or something like that. Asim, I think we both had a chance to play around with this, and we'll share a link to the actual story around this as well as the actual repo that's online. What do you think of this? what's your immediate take when you see this and have a little poke around with this?
Asim Hussain: Well, it's good. I wanna make sure. It's a really nice addition to a chat interface. So just so the audience who's not seeing it, every time you do a prompt, it tells you the energy in, well, in watt hours, what I'm seeing right now. But then also, you know, some other stats as well.
And then also kind of how much of a phone charge it is. And that's probably the most surprising one. I just did a prompt, which was 5.7% of a phone charge, which was, that's pretty significant. Actually, I dunno, is that significant? So, one of the things is, I'm trying to, what I'm trying to find out from it though is how does that calculation, 'cause that's my world, it's like, how does, what do you really mean by a calculation?
Is it cumulative? Is it session based? Is it just, you know, how, what have you calculated in terms of the energy emissions? The little info on the side is just the energy of the GPU during inference. So it's not the energy of kind of anything else in the entire user journey of me using a UI to ask a prompt.
But we also know that's probably the most significant. And I'm kind of quite interested in figuring out, as I'm prompting it, I'm one, I'm, one of the things I'm seeing is that every single prompt is actually, the emissions are bigger than the previous prompt. Oh no, it's not actually, that's not true.
Yeah, it is.
Chris Adams: Ah, this is the thing you've been mentioning about cumulative,
Asim Hussain: Cumulative. Yeah. Which is a confusing one. 'Cause I've had a lot of people who are really very good AI engineers go, "Asim, no, that's not true." And other people going, "yeah, it kind of is true." But they've just optimized it to the point where the point at which you get hit with that is at a much larger number.
But the idea is that there's, there, it used to be an n squared issue for your prompt and your prompt session history. So every time you put a new prompt in all of your past session history was sent with your next prompt. And if you are actually building, like a your own chat system, if you are actually building like your own chat solution for your company or wherever, that is typically how you would implement it as a very toy solution to begin with is just, you know, take all the texts that was previous and the new text and send it, in the next session.
But I think what, they were explaining to me, which was actually in the more advanced solutions, you know, the ones from Claude or ChatGPT, there's a lot of optimization that happens behind the scenes. So it doesn't really, it doesn't really happen that way, but I was trying to figure out whether it happens with this interface and I haven't quite figured it out yet.
Chris Adams: Oh, okay. So I think what you might be referring to is the fact that when you have like a GPU card or something like that, there's like new tokens and kind of cashed tokens, which are priced somewhat differently now. And this is one of the things that we've seen.
'Cause it's using maybe a slightly different kind of memory, which might be slightly faster or is slightly kind of is slightly lower cost to service in that sense. Yeah. Okay. So this is one thing that we don't see. What I, the good news is we can share a link to this, for anyone listening, this source code is all on GitHub, so we can have a look at some of this.
And one of the key things you'll see actually is, well this is sending a message. When you see the actual numbers update, the, it's not actually, what it's actually doing is it's calculating all this stuff client site based on how big each model is likely to be. 'Cause when you look at this, you can A,
Asim Hussain: It's a model.
Chris Adams: You can work out the, I mean, so when people talk about should I be using the word please or thank you, and am I making the things worse by treating this like a human or should I just be prompting the machine like a machine, is there a carbon footprint to that? This will display some numbers that you can see there, but this has all been calculated inside your browser rather than actually on the server.
So like you said, Asim, there is a bit of a model that's taking place here, but as a kind of way to like mess around and kind of have a way into this. This is quite interesting and even now it's kind of telling that there are so few providers that make any of this available, right now. We're still struggling even in like the third quarter of 2025,
to have a commercial service that will expose these numbers to you in a way that you can actually meaningfully change the environmental footprint of through either your prompting behavior or well maybe model choice. But that's one of the key things that I see. I can't think, I can't think of any large commercial service that's doing this.
The only one is possibly GreenPT,
which is basically put a front end on Scaleway's, inference service and I'm not sure how much is being exposed there for them to make some assumptions as well.
Asim Hussain: Do you know how bad, do you know how,
I feel very uncomfortable with the idea of a future where a whole bunch of people are not saying please or thank you, and the reason for it is they're proudly saying, "well, I care about, I care about sustainability, so I'm not gonna say please or thank you anymore 'cause it's costing too many, too much carbon." I find that very uncomfortable. I personally, I don't wanna, we could, choose not to say please or thank you in all of our communications because it causes, emissions no matter what you do. I don't know.
Chris Adams: I'm glad you weren't there, Asim. 'Cause I was thinking about that too. There's a carbon cost to breathing out and if, you, I guess maybe that's 'cause we're both English and it's kinda hardwired into us. It's like the same way that, you know, if you were to step on my toe, I would apologize to you stepping on my toe because I'm just English and I, and it's a muscle memory, kind of like impulsing.
Okay.
Asim Hussain: Yeah.
Chris Adams: That's, what we found. We will share some couple, a couple of links to both the news article, the project on Hugging Face, and I believe it's also on GitHub, so we can like, check this out and possibly make a PR to account for the different kinds of caching that we just discussed to see if that does actually make a meaningful difference on this.
For other people who are just looking, curious about this, this is one of the tools which also allows you to look at a, basically not only through weird etiquette, how etiquette can of impact the carbon footprint of using a tool, but also your choice of model. So some models might be, say 10 times the size of something, but if they're 10, if they're not 10 times as good, then there's an open question about whether it's really worth using them, for example.
And I guess that might be a nice segue to the next story that we touch on. But Asim, I'll let you, you gotta say something. I
Asim Hussain: No, I was gonna say, because I, this is, 'cause I've been diving into this like a lot recently, which is, you know, how do you efficiently use AI? Because I think a lot of the, a lot of the content that's out there about, you know, oh, AI's emissions and what to do to reduce AI's emissions, there are all the choices that as a consumer of AI, you have absolutely no ability to affect. I mean, unless you are somebody who's quite comfortable, you know, taking an open source model and rolling out your own infrastructure or this or that or the other. If you're just like an everyday, not even an everyday person, but just somebody who works in a company who's, you know, the company bought Claude, you know, you're using Claude,
end of story, what are you, like, what do you do? And I think that's really, it is a really interesting area. I might just derail our whole conversation to talk about this, but I think it's a really interesting area because, what it's really boiling down to is your use of the context window.
And so you have a certain number of tokens in a chat before that chat implodes, and you can't use that chat anymore. And historically, those number of tokens were quite low. Relative to, because of all the caching stuff hadn't been invented yet and this and that and the other. So the tokens were quite low.
What, didn't mean they didn't mean they were, the prompts were cheaper before. I think they were still causing a lot of emissions. But because they've improved the efficiency and rather than just said, I've improved the efficiency, leave it at that, I've improved the efficiency, Jevons paradox, I've improved the efficiency,
let's just give people more tokens to play around with before we lock them out.
So the game that we're always playing is how to actually efficiently use that context. And the please or thank you question is actually, see this is, I don't think it's that good one. 'Cause it's two tokens in a context window of a million now, is what's coming down the pipeline.
The whole game. And I think this is where we're coming from as you know, if you wanna be in the green software space and actually have something positive to say about how to actually have a relationship with AI, it's all about managing that context. 'Cause the way context works is you're just trying to, it's like you've got this intern and if you flash a document at this intern, you can't then say, "oh, ignore that.
Forget it I didn't mean to show you that." It's too late. They've got it and it's in their memory and you can't get rid of it. the only solution is to literally execute that intern and bury their body and get a new intern and then make sure they see the information in the order and only the information they need to see so that when you finally ask 'em that question, they give you the right answer. And so what a lot of people do is they just, because there's a very limited understanding of how to play, how to understand, how to play with this context space, what people end up doing is they're just going, "listen, here's my entire fricking document. It's actually 50,000 words long. You've got it, and now I'm gonna ask you, you know, what did I do last Thursday?"
So it's, and all of that context is wasted. And I think that's, and it's also like a very simplistic way of using an AI, which is why like a lot of companies are, kind of moving towards that space because they know that it means their end user doesn't have to be very well versed in the use of the tool in order to get benefit out of it.
So that's why ChatGPT, they're all like working towards a space of how do we build a tool where people can literally pour junk into it, and It will figure something out.
Whereas what we should be doing and what I'm like, and I think it's not only what we should be doing, it's, what the people who are like really looking at how to actually get real benefit from AI,
is how do you use that context window very carefully. And it is like programming. It is really like program. That's what, that's my experience with it so far. It's like, I want this, I need to feed this AI information. It's gonna get fed in an order that matters. It's gonna get fed in a format that matters.
I need to make sure that the context I'm giving it is exactly right and minimal. Minimal for the question that I wanna answer, get it answered at the end of it. So we're kind of in this like space of abundance where, because every AI provider's like, "well do what you want. Here's a million tokens.
Do what you want, do what you want."
And they're all, we're all just chucking money. These we're just chucking all our context tokens at it. They're burning money on the other side because they're not about making a profit at the moment. They're just about becoming the winner. So they don't really care about kind of profitability to that level.
So what us It's all about, I'm just getting back to it again. I think, we need to eventually be telling that story of like, how do you actually use the context window very carefully? And again, it's annoyed me that the conversation has landed at please and thank you. 'Cause the actual conversation should be, you know, turning that Excel file into a CSV because it knows how to parse a CSV and it uses fewer tokens to parse a CSV than an Excel file. Don't dump the whole Excel file, export the sheet that you need in order for it to, answer that question. If you f up, don't just kill the session and start a new session.
This is, there's this advice that we need to be giving that I don't even know yet.
Chris Adams: MVP. Minimal viable prompt.
Asim Hussain: Minimal viable prompt! Yeah. What is the minimal viable prompt and the, what's frustrating me is that like one of the things that we use Claude and I use Claude a lot, and Claude's got a very limited context window and I love that.
It was like Twitter when you had to, remember Twitter when you had to like have 160 characters?
It was beautiful.
Chris Adams: to 280, and then you're prepared to be on that website, you can be as, you can monologue as much as you want
Asim Hussain: Yeah. You can now monologue, but it was beautiful having to express an idea in this short, like short, I love that whole, how do I express this complex thing in a tweet? And so with the short context windows, were kind of forced to do that, and now I'm really scared because now everybody, Claude literally two days ago has now gone, right, you've got a million context window, and I'm like, oh, damn it.
Now I don't even, now I don't have personally
Chris Adams: That's a million token context window when you say that. Right. So that's enough for a small book basically. I can dump entire book into it, then ask questions about it. Okay. Well, I guess it depends on the size of your book really, but yeah, so that's, what you're referring to when you talk about a million context window there.
Asim Hussain: Yeah, yeah. And it's kind of an energy question, but the energy doesn't really, kind of, knowing how much, like I've just looked at chat UI window and I've checked a couple of prompts and it's told me the energy, and it's kinda that same world.
It's just it's just there to make me feel guilty, whereas the actual advice you should be getting is well, actually no, I, what do I do? How am I supposed to prompt this thing to actually make it consume less energy? And that's the,
Chris Adams: Oh, I see. So this is basically, so this is, you're showing me the thing and now you're making me feel bad. And this may be why various providers have hosted chat tools who want people to use them more, don't automatically ship the features that make people feel bad without giving 'em a thing they can actually do to improve that experience.
And it may be that it's harder to share some of the guidance like you've just shared about making minimum viable prompt or kind of clear prompt. I mean, to be honest, in defence of Anthropic, they do actually have some pretty good guidance now, but I'm not aware of any of it that actually talks about in terms of here's how to do it for the lowest amount of potential tokens, for example.
Asim Hussain: No, I don't see them. I don't see them. I mean, they, yeah, they do have like stuff, which is how to optimize your context window, but at the same time, they're living in this world where everybody's now
working to a bigger, that's what they have to do.
And I don't know, it's kinda like, where do we, because we, 'cause the AI advice we would typically have given in the past, or we would typically give is listen, just run your AI in a cleaner region. And you are like, well, I can't bloody do that with Anthropic, can I? It's just, it's whatever it is, it's, you know.
Chris Adams: That's a soluble problem though. Like,
Asim Hussain: Like what I'm just saying or,
Chris Adams: Yeah. You know, but like the idea they're saying, "Hey, I want to use the service. And I want to have some control over where this is actually served from."
That is a thing that you can plausibly do. And that's maybe a thing that's not exposed by end users, but that is something that is doable.
And, I mean, we can touch on, we actually did speak about, we've got Mistral's LCA reporting as one of the things, where they do offer some kind of control, not directly, but basically by saying, "well, because we run our stuff in France, we're already using a low carbon grid."
So it's almost like by default you're choosing this rather than you explicitly opting in to have like the kind of greener one by, the greener one through an active choice,
I suppose.
Asim Hussain: They're building some data centers over there as well, aren't they? So it's a big, it's a big advantage for Mistral to be in France, to be honest with you. It's yeah, they're in
Chris Adams: this definitely does help, there's, I mean, okay. Well, we had this on our list, actually, so maybe this is something we can talk about for our next story, because another one on our list since we last spoke was actually a blog post from Mistral.ai talking about, they refer to, in a rather grandiose terms, our contribution to a global environmental standard for AI.
And this is them sharing for the first time something like a lifecycle analysis data about using their models. And, it's actually one that has, it's not just them who've been sharing this. They actually did work with a number of organizations, both France's agency, ADM. They were following a methodology specifically set out by AFNOR, which is a little bit like one of the French kind of, environmental agency, the frugal AI methodology.
And they've also, they were working with I think, two organizations. I think it's Sopra Steria, and I forget the name of the other one who was mentioned here, but it's not just like a kind of throwaway quote from say Sam Altman. It's actually, yeah, here we are is working with Hubblo, which is a nonprofit consultancy based in Paris and Resilio who are a Swiss organization, who are actually also quite, who are quite very well respected and peer reviewed inside this.
So you had something, some things to share about this one as well. 'Cause I, this felt like it was a real step forward from commercial operators, but still falling somewhat short of where we kind of need to be. So, Asim, what, when you read this, what were the first things that occurred to you, I suppose, were there any real takeaways for you?
Asim Hussain: Well, I'd heard about this, on the grapevine, last year because I think, one of the researchers from Resilio was at greenIO, yeah, in Singapore. And I was there and he gave a little a sneak. They didn't say who it was gonna be, they didn't say it was Mistral, but they said, we are working on one.
And he had like enough to tease some of the aspects of it. I suspect once it's got released, some of the actual detail work has not, that's what I'm, I think I'm, unless I, unless there's a paper I'm missing. But yeah, there is kind of more work I think here that didn't end up to actually get released once it's, once it got announced, but there was, it was a large piece of work.
It's good. It's the first AI company in the world of this, you know, size that has done any work in this space and released it. Other than like a flippant comment from Sam Altman, "I heard some people seem to care about the emission, energy consumption of AI." So, so that's good. And I think we're gonna use this, it's gonna be used in as a, as I'd say, a proxy or an analog for kind of many other, situations.
I think it's, it is lacking a little bit in the detail. But that's okay. I think we, every single company that starts, we should celebrate every organization that leads forward with some of this stuff. it's always very, when you're inside these organizations, It's always a very hard headwind to push against.
'Cause there's a lot of negative reasons to release stuff like this, especially when you're in a very competitive space like AI. So they took the lead, we just celebrate that. I think we're going to, there's some data here that we can use as models for other, as, you know, when we now want to look at what are the emissions of Anthropic or OpenAI or Gemini or something like that,
there's some more, you know, analogs that we can use. But also not a huge amount of surprise, I'd say, it's kind of a training and inference,
Chris Adams: Yep.
That turns be where the environmental footprint is.
Asim Hussain: Yeah. Training and inference, which is kind of, which is good. I mean, I think obviously hardware and embodied impacts is, they kind of separate kind of the two together.
I suspect, the data center construction is probably gonna be, I don't know
that is quite low. Yeah, yeah,
Chris Adams: I looked at this, I mean this is, it's been very difficult to actually find any kind of meaningful numbers to see what share this might actually make. 'Cause as the energy gets cleaner, it's likely that this will be a larger share of emissions. But one thing that was surprising here was like, this is, you know, France, which is a relatively cr clean grid, like maybe between 40 and say 60 grams of CO2 per kilowatt hour, which is, that's 10 times better than the global average, right?
Or maybe 9, between 8 and 10 times cleaner than the global average. And even then it's, so with the industry being that clean, you would expect the embodied emissions from like data centers and stuff to represent a larger one. But the kind of high level, kind of pretty looking graphic that we see here shows that in, it's less than 2% across all these different kind of impact criteria like carbon emissions or water consumption or materials, for example.
This is one thing that, I was expecting it to be to be larger, to be honest. The other thing that I noticed when I looked at this is that, dude, there's no energy numbers.
Asim Hussain: Oh, yeah.
Chris Adams: Yeah. And this is the thing that it feels like a, this is the thing that everyone's continually asking for.
Asim Hussain: It's an LCA. So they use the LCAs specification, so
Chris Adams: That's, a very good point. You're right. that is, that's a valid response, I suppose. 'Cause energy by itself doesn't have a, doesn't have a carbon footprint, but the results of generating that energy does, electricity does have that impact. So yeah.
Okay. Maybe that's
For
Asim Hussain: the audience, they use like a well known, well respected, standardized way of reporting the lifecycle emissions using the LCA lifecycle analysis methodology, which is like an ISO certified standard of doing it. So they adhere to a standard.
Chris Adams: So this actually made me realize, if this is basically here and you are a customer of a AI provider, 'cause we were looking at this ourselves trying to figure out, okay, well what people speak to us about a AI policies? And we realized well, we should probably, you know, what would you want to have inside one?
The fact that you have a provider here who's actually done this work, does suggest that for that it's possible to actually request this information if you're a customer under NDAs. In the same way that with, if you're speaking to Amazon or probably any of the large providers, if you're spending enough money with them, you can have information that is disclosed to you directly under NDA.
So it may not be great for the world to see, but if you are an organization and you are using, say, Mistral, for example, or Mistral services, this would make me think that they're probably more able to provide much more detailed information so that you can at least make some informed decisions in a way that you might not be able to get from some of the other competing providers.
So maybe that's one thing that we actually do see that is a kind of. Not really a published benefit in this sense, but it's something that you're able to do if you are in a decision making position yourself and you're looking to choose a particular provider, for example.
Asim Hussain: I mean, you should always be picking the providers who've actually got some, you know,
Chris Adams: optimize for disclosure,
Asim Hussain: optimize for disclosure. Yeah. Always be picking the providers if you optimize for disclosure. I mean, if we, the people listening to this, that is the thing that you can do. And Mistral, They're also, they have some arguments in here as well, which is kind of, they did kind of also surface that it is like a pretty linear relationship between your emissions and the size of the model, which is a very useful piece of information for us to know, as a consumer.
Because then we can go, well actually I've heard all these stories about use Smaller models use smaller models and now you actually have some data behind it, which is supporting the fact that, yeah, using a smaller model isn't, it's not got some weird non-linearity to it, so a half size model is only like 10% less, emissions.
A half size model is half the emissions. So that's pretty, that's a pretty good thing to know. Helps Mistral, the fact that they have a lot of small models that you can pick and choose, is not, so a lot of this stuff really benefits Mistral. They are the kind of the kind of organization which has a product offering which is benefited, which does benefit a sustainability community.
So they have like small models you can use. I think, I wonder actually, Chris, 'cause they do say that they're building their own data center in France, but they've never said where there exists, where they until now, where they've been running their AI. So that might be the reason for, they might have been running it in East Coast US or something
like
Chris Adams: I think that would be quite unlike, wouldn't be very likely, given that most of their provider, most of their customers are based in probably Western Europe still. Right. There is very much a kinda like Gaelic kind of flavor to the tooling. And I've, I mean actually Mistral, or Mistral's tools are ones which I've been using myself personally over the last, like few months, for example.
And it's also worth bearing in mind that they, took on a significant amount of investment from Microsoft a few years back and I would be very surprised if they weren't, or if they weren't using a French data center serving French providers. 'Cause if you were to choose between two countries, okay, if, France or like France actually has, and since 2021, I believe, has had actually a law specifically about measuring the environmental footprint of digital services.
So they've got things that they, I think it's called, I'm going to, I'm just gonna share a link to that, to the name of the law because I'm gonna butcher the French pronunciation, but it basically, it translates to Reduce the Environmental Footprint of Digital Services Law.
That's pretty much it. And that's where, as a follow on from that, that's what, that's what the RGESN, the kind of general guidance that it shares across kind of government websites in general for France. They've already got a bunch of this stuff out there for like how to do greener IT. I suspect that France is probably gonna be one of, well, probably the premier country, if you'd run, be running a startup to see something like this happening much more so than, well probably the US right now, especially given the current kind of push with its current kind of federal approach, which is basically calling into doubt climate change in the wider sense basically.
We were talking about disclosure, right? And we said an optimization for disclosure. And that's probably a nice segue to talk about, another link we had here, which was the energy score leaderboard. Because this is one thing that we frequently point to. And this is one thing that we've suggested in my line of work, that if you are looking to find some particular models, one of the places to look would be the AI Energy Score Leaderboard, which is actually maintained by Hugging Face.
And, I share this 'cause it's one of the few places where you can say, I'm looking for a model to help me maybe do something like image generation or captioning text or generating text or doing various things like this. And you can get an idea of how much power these use on a standardized setup.
Plus, how satisfied, you know, what the kind of satisfaction score might be, based on these tools and based on a kind of standardized set of like tests, I suppose. The thing is though, this looks like it hasn't been updated since February. So for a while I was thinking, oh, Jesus, does this mean we actually need to, do we have to be careful about who we, how we recommend this?
But it turns out that there's a new release that will be coming out in September. It's updated every six months. And, now that I do have to know about AI, this is one thing that I'm looking forward to
seeing some of the releases on because if you look at the leaderboard for various slices, you'll see things like Microsoft Phi 1 or Google Gemma 2 or something like that.
Asim Hussain: That quite old?
Chris Adams: yeah, these are old now, it's six months in generative AI land is quite a long time. There's Phi 4 now, for example, and there's a bunch of these out there. So I do hope that we'll see this actually. And if you feel the same way, then
yeah, go on.
Asim Hussain: Is it, 'cause, is I always assume this was like a, live leaderboard. So as soon as a model, I suppose once a model, like the emissions of a model are linked to the model and the version of it. So once you've computed that and put on the leaderboard, it's not gonna change. So then it's just the case of as new models come out, you just measure and it just sees how it goes on the leaderboard.
Because I'm seeing something here. I'm, I thought open, I'm seeing OpenAI, GPT. Isn't that the one they just released?
Chris Adams: No, you're thinking GPT-OSS, perhaps
Asim Hussain: Oh.
Chris Adams: One thing they had from a while ago. So that one, for example, came out less than two weeks ago, I believe. That isn't showing up here.
Asim Hussain: That isn't showing up
Chris Adams: The, I'm, I was actually looking at this thinking, oh, hang on, it's six months, something being updated, six months,
that's, it'd be nice if there was a way, a faster way to expedite kind of getting things disclosed to this. For example, let's say I'm working in a company and I've, someone's written in a policy that says only choose models that disclose in the public somewhere. This is one of the logical places where you might be looking for this stuff right now, for example, and there's a six month lag, and I can totally see a bunch of people saying, no, I don't wanna do that.
But right now there's a six month kind of update process for this.
Asim Hussain: In the AI realm is an eternity. Yeah.
Chris Adams: Yeah. But at the same time, this is, it feels like a thing that this is a thing that should be funded, right? I mean, it's, it feels :I wish there was a mechanism by which organizations that do want to list the things, how to make them to kind of pay for something like that so they can actually get this updated so that you've actually got some kind of meaningful, centralized way to see this.
Because whether we like it or not, people are basically rolling this stuff out, whether we like it or not, and I feel In the absence of any kind of meaningful information or very patchy disclosure, you do need something. And like this is one of the best resources I've seen so far, but it would be nice to have it updated.
So this is why I'm looking forward to seeing what happens in September. And if you think, if you too realize that like models and timely access to information models might be useful, it's worth getting in touch with these folks here because, I asked 'em about this when I was trying to see when they were, what the update cycle was.
And basically the thing they said was like, yeah, we're, really open to people speaking to us to figure out a way to actually create a faster funded mechanism for actually getting things listed so that you can have this stuff visible. Because as I'm aware, as I understand it, this is a labor of love by various people, you know, between their day jobs, basically.
So it's not like they've got two or three FTE all day long working on this, but it's something that is used by hundreds of people. It's the same kind of open source problem that we see again and again. But this is like one of the pivotal data sources that you could probably cite in the public domain right now.
So this is something that would be really nice to actually have resolved.
Asim Hussain: Because there is actually, 'cause the way Hugging Face works is, they have a lab and they have their own infrastructure. Is that how it works? Yeah. So that's
Chris Adams: this would, that was be, that was either, that was physically theirs, or it was just some space.
Asim Hussain: Spin up. But yeah. But yeah, but they have to effectively like to get the score here. It's not self certified, I presume, but there's a, you know, each of these things has got to get run against the benchmark. So there's basically, if I remember, there was a way of like self certifying.
There was literally a way for
Chris Adams: You could upload your stuff.
Asim Hussain: Yeah. OpenAI could disclose to the Hugging Face to the, what the emissions of, you know, what the energy of it was. But most of it is, there's actually, you gotta run against the H100 and there's a benchmark
Chris Adams: Yep, exactly. So there's a bit of manual. There's a bit of manual steps to do that, and this is precisely the thing that you'd expect that really, it's not like an insoluble problem to have some way to actually expedite this so that people across the industry have some mechanism to do this. 'cause right now it's really hard to make informed decisions about either model choice or anything like that.
Even if you were to architect a more responsibly designed system, particularly in terms of environmental impact here.
Asim Hussain: Because if you were to release a new model and you wanted it listed in the leaderboard, you would have to run every other model against. Why would you need to do that? You need to
Chris Adams: You wouldn't need to do that. You just need to, you, because you don't have control over when it's released, you have to wait six months until the people who are working in that get round to doing that.
Asim Hussain: Just the time. It's just a time. Yeah. Someone's
Chris Adams: If you're gonna spend like a millions of dollars on something like this, it feels like this is not, even if you were to drop say, if, even if it was to cost, maybe say a figure in the low thousands to do something like this, just to get that listed and get that visible, that would be worth it.
So that you've actually got like a functioning way for people to actually disclose this information, to inform decisions. 'Cause right now there's, nothing that's easy to find. This is probably the easiest option I've seen so far and we've only just seen like the AI code of practice that's actually kind of been kind of pub that came into effect in August in Europe for example.
But even then, you still don't really have that much in the way of like public ways to filter or look for something based on the particular task you're trying to achieve.
I wanted to ask you actually, Asim, so I think, I can't remember last time if I was speaking to you, if this came up, I know that in your, with your GSF hat on, there's been some work to create a software carbon intensity for AI spec, right. Now, I know that there's a thing where like court cases, you don't wanna kind of prejudice the discussions too much by having things internally.
Although you're probably not, there isn't like AI court, you can be in contempt of, but I mean, yeah, not yet, but, who knows? Give it another six months. Is there anything that, is there anything, any, juicy gossip or anything you can share that people have been learning? 'cause like you folks have been diving into this with a bunch of domain experts so far, and this isn't my, like, while I do some of this, I'm not involved in those discussions.
So I mean, and I'm aware that there has been a bunch of work trying to figure out, okay, how do you standardize around this? What do you measure? You know, do you count tokens? Do you count like a prompt? What's the thing? Is there anything that you can share that you're allowed to talk about before it goes?
Asim Hussain: Yeah. I think, we, I think that what we've landed on is that as long as I'm not discussing stuff which is in, you know, active discussion and it's kind of made its way into the spec and there's been, you know, broad consensus over, I think it's pretty safe to talk about it.
If there's something that's kind of, and what we do, we do everything in GitHub. So if there's something which is like, I won't, I won't discuss anything which has only been discussed in like an issue or a discussion or comment thread or something. If it's actually made its way into the actual spare, that's pretty safe.
So yeah, the way it's really landed is that there's, there was a lot of conversations at the start. There was a lot of conversations and I was very confused. I didn't really know where things were gonna end up with. But you know, at the start there was a lot of conversations around well, how do we deal with training?
How do we deal with training? There's this thing called inference. And it's interesting 'cause when we look at a lot of other specs that have been created, even the way the Mistral LCA was done, so they, they gave a per inference, or per request. I've forgotten what they did. It, they didn't do per token.
So per
Chris Adams: they do per chat session or per task, right. I think it's something along those lines. Yeah.
Asim Hussain: Something along that, it wasn't a per token thing. But even then they, they added the training cost to it. And like those, some of the questions we were adding, can you add, is there a way of adding like the training? The training happened like ages ago. Can you somehow, is there a function that you can use to amortize that training to like future inference runs?
And we explored like lots of conversations. There's like a decay function. So if you were the first person to use a new model, the emissions per token would be higher because you are amortizing more of the training cost and the older models, the, so you explored like a decay function, we explored, yeah.
There's lots of ideas.
Chris Adams: Similar to the embodied usage, essentially like what we have with embodied versus, embodied carbon versus like use time carbon. You're essentially doing the same thing for training, being like the embodied bit and inference being the usage. And if you had training and you had three inferences, each of those inferences is massive.
Like in terms of the car embodied carbon, if there's like a billion, it's gonna much lower per, for each one.
Asim Hussain: But then you get into really weird problems because I mean it, we do that with the embodied carbon hardware, but we do that by saying, do you know what? The lifespans gone be four years and that's it. And we're just gonna pretend it's an equal waiting every single day for four years.
Chris Adams: Not with the GHG protocol. You can't do it with the GHG protocol. You can't amortize it out like that. You can, you have to do it the same year, so it, your emissions look awful one year
Asim Hussain: Ah, the year that you bought it from.
Chris Adams: So this is actually one of the reasons, but yeah, this is actually one of the problems with the kind of default way of measuring embodied carbon versus other things inside this is, it's not, like Facebook for example, they've proposed another way of measuring it, which does that, this kind of amortization approach, which is quite a bit closer to how you might do, I guess, like typical amortization of capital, capital
Asim Hussain: Cap, yeah.
Chris Adams: So that's the, that's the difference in the models. And this is, these are some of the kind of honestly sometimes tedious details that actually have quite a significant impact. Because if you did have to, that's gonna have totally different incentive incentives. If you, especially at the beginning of something, if you said, well, if you pay the full cost, then you are incentivized not to use this shiny new model.
'Cause it makes you look awful compared to you using an existing one for example.
Asim Hussain: And that's one of the other questions like, is like, how do you, I mean, a lot of these questions were coming up like what do you... A we never, we didn't pick that solution. and we also didn't pick the solution of we had the, we actually had the conversation of you amortize it over a year, and then there's a cliff.
And then that was like, we're gonna incentivize people to use older models with this idea that older models were the thing.
There were questions that pop up all the time. Like, what do you do when you have an open source model? If you were to, if I was to fine tune an open source model and then make a service based off of that, is the emissions of the model the open source model that I got Llama whatever it was, am I responsible for that?
Or is the,
and there was like, if you were to say, if you were to say no, then you're incentivizing people to just like open source their models and go, "meh well the emissions are free now 'cause I'm using an open source model." So there's lots of these, it's very nuanced. Kind of the, a lot of the conversations we have in the standards space, is like a small decision can actually have a cascading series of unintended consequences.
So the thing that we really like sat down was like, what, well, what actually, what do you want to incentivize? Let's just start there. What do we want to incentivize? Okay, we've listed those things we wanna incentivize. Right. Now, let's design a metric, which through no accident incentivizes those things. And where they ended up was basically two,
there's gonna be two measures. So we didn't, we didn't solve the training one because there isn't a solution to it. It's a different audience cares about the training emissions than that doesn't, consumers, it's not important to you because it doesn't really matter. It doesn't change how you behave with a model.
It doesn't change how you prompt a model just because it had some training emissions in the past. What matters to you most is your direct emissions from your actions you're performing at that given moment in time. So it's likely gonna be like two SCI scores for AI, a consumer and a provider. So the consumer is like inference plus everything else.
and also what is the functional unit? There's a lot of conversations here as well, and that's likely to land that now very basically the same as how you sell an AI model. So if you are an LLM, you're typically selling by token. And so why for us to pick something which isn't token in a world where everybody else is thinking token, token, token, token, it would be a very strange choice and it would make the decision really hard for people when they're evaluating certain models. They'd be like, oh, it's this many dollars per token for this one and this many dollars per token for that one. But it's a carbon per growth. And it's a carbon per growth,
I can't rationalize that. Where, if it's well look, that's $2 per token, but one gram per token of emissions and that's $4 per token, but half a gram per token for emissions. I can evaluate the kind of cost, carbon trade off, like a lot easier. The cognitive load is a lot easier.
Chris Adams: So you're normalizing on the same units, essentially, right?
Asim Hussain: Yeah. As how, however it's sold, however, it's, 'cause that's sort of, it's a fast, AI is also a very fast moving space and we dunno where it's gonna land in six months, but we are pretty sure that people are gonna figure out how to sell it, in a way that makes sense. So lining up the carbon emissions to how it's sold.
And the provider one is going to be, that's gonna include like the training emissions, but also like data and everything else. And that's gonna be probably per version of an AI. And that will, so you can imagine like OpenAI, like ChatGPT would have a consumer score of carbon per token and also a provider score of ChatGPT 5 has, and it's gonna be probably like per flop or something,
so per flop of generating ChatGPT 5, it was this many, this much carbon. And that's really like how it's gonna,
it's also not gonna be total totals are like, forget about totals. Totals are pointless when it comes to, to change the behavior.
You really want to have a, there's this thing called neural scaling laws.
The paper.
Chris Adams: Is that the one that you double the size of the model when it's supposed to double the performance? Is that the thing?
Asim Hussain: It's not double, but yeah, got relationship. Yeah. So there's this logarithmic, perfectly logarithmic relationship between model accuracy and model size, model accuracy, and the data, the number of training you put into it, and model size and the amount of compute you put into, it's all logarithmic.
So it's often used as the reason, the rationale for like why we need to, yeah, larger models is because we can prove it. So, but that basically comes down to like really then, you know, like if like I care more about, but for instance, I don't particularly, it doesn't matter to me how much, it's not that important to know the total training emissions of ChatGPT 5 versus ChatGPT 4.
What's far more useful, is to know, well, what was the carbon per flop of training for 4 versus the carbon per flop of training for 5? 'Cause then that gives you more interesting information. Have you, did you,
Chris Adams: What does that allow?
Asim Hussain: Bother to do anything? Huh?
Chris Adams: Yeah. What does that allow me to do? If I know if 5 is 10 times worse per flop than 4,
what that incentivize me to do differently? 'Cause I think I might need a bit of hand help here making this call here.
Asim Hussain: Because I think, 'cause it, what, let's say ChatGPT 6 is going to come along. The one thing we know absolutely sure is it's just gonna be in terms of total bigger than ChatGPT 5. So as like a metric, it's not, if you are an engineer, if you are somebody trying to make decisions regarding what do I do to actually train this model with causing less emissions, it doesn't really help me because it's just, a number that goes higher and higher.
Chris Adams: Oh, it's a bit like carbon intensity of a firm versus, absolute emissions. Is that the much, the argument you're using? So it doesn't matter that Amazon's emissions have increased by 20%, the argument is well, at least if they've got more efficient per dollar of revenue, then that's still improvement.
That's the line of reasoning that's using, right?
Asim Hussain: Yeah. So it's,
because of the way the SCI is, it's not if you want to do a total, there are LCAs, like the thing that Mistral did, there's existing standards that are very well used. They're very well respected. There's a lot of, there's a lot of information about how to do them.
You can just use those mechanisms to calculate a total. What the SCI is all about is what is a,
KPI that a team can use and they can optimize against, so over time, the product gets more and more efficient?
Obviously, you should also be calculating your totals and be making a decision based upon both.
But just having a total is, I've gotta be honest with you, it's just, I don't see totals having, in terms of changing behavior, I don't think it changes any behavior. Full stop.
Chris Adams: Okay. I wanna put aside the whole, we live in a physical world with physical limits and everything like that, but I think the argument you're making is essentially that, because the, you need something to at least allow you to course correct on the way to reducing emissions in absolute terms, for example. And your argument you're making is if you at least have an efficiency figure, that's something you can kind of calibrate and change over time in a way that you can't with absolute figures, which might be like having a, you know, a budget between now and 2030, for example.
That's the thinking behind it, right?
Asim Hussain: Yeah. I mean, if you, I've actually got an example here from 'cause we, so we don't have actual compute. They, no, no one's ever disclosed like the actual compute that they used per model. But they have, or they used to disclose the number of parameters per model. And we know that there's a relationship.
So there's a really interesting, so for 2, 3 and 4, we have some idea regarding the training emissions and the parameters, not from a disclosure, from like research as well, so between, but when you compute the emissions per billion parameters of the model, so per billion parameters of the model, GPT two was 33.3 tons of carbon per billion parameters of the model.
Chris Adams: Okay.
Asim Hussain: GPT-3 went down to 6.86 tons of carbon per billion parameters. So it went down from 33 to 6. So that was a good thing. It feels like a good thing, but we know the total emissions of 3 was higher. Interestingly, GPT-4 went up to 20 tons of carbon per billion parameters. So that's like an interesting thing to know.
It's like you did something efficient between two and three. You did something good. Whatever it was, we don't know what it was, we did something good actually the carbon emissions per parameter reduced. Then you did something. Maybe it was bad. Maybe I, some, maybe it was necessary. Maybe it was architectural. But for some reason your emissions,
Chris Adams: You became massively less efficient in the set, in that
next
Asim Hussain: In terms of carbon. In terms of carbon, you became a lot less efficient in GPT-4. We have no information about GPT 5. I hope it's less than 20 metric tons per billion parameters.
Chris Adams: I think I'm starting to wanna step, follow your argument and I'm not, I'm not gonna say I agree with it or not, but I, the, I think the argument you're making is essentially by switching from, you know, that that in itself is a useful signal that you can then do something with. there was maybe like a regression or a bug that happened in that one that you can say, well, what change that I need to do so I can actually start working my way towards, I don't know, us careering less forcefully towards oblivion, for example, or something like that.
Right.
Asim Hussain: Yeah.
Chris Adams: Okay. That makes, I think I understand that now. And, let's, and I suppose the question I should ask from following on from that is that this is, some of this is, we're talking about, we got into this, 'cause we were talking about the SCI for AI, this kind of standard or presumably an ISO standard that we published.
Is there a kind of rough like roadmap for when this is gonna be in the public domain, for example, or people might be requesting this in commercial agreements or something like that?
Asim Hussain: I mean, I can tell you what my hope is. So I think, I mean, cause everything is based upon consensus and if anybody objects then everything or all the plans basically, you know, put on the back burner. But everything's looking very positive. I'm very hopeful that by the end of Q3, so the end of September, we will have gone into draft and then, there hasn't been a full agreement yet as to what we'll actually publish for that. But I'm hoping we'll be able to actually publish the whole specification, because what we wanna start doing is get, I mean this maybe if anybody's interested, we wanna start running case studies because right now it's like the outline of what we want the calculation to be is being agreed on.
But we need a lot of use cases of very different types of products that have computed using it. Not just, you know, I'm a major player and I've got a gazillion servers and we also want, need people, there's lots of organizations we're talking to or listen, we've just, we are, AI is not our central business, but we've built like AI solutions internally and we want to be able to measure that.
Or even smaller organizations or people who are not even training in AI, but just consuming APIs then build like an AI solution on top of that. So there's like a whole range of things that we wanna measure and we want to publish, go into draft in September, and then work on a number of case studies. Hopefully, dream,
my dream, and I, no one holds me to this is by kind of Q1, Q2 next year where we're out and we start the ISO process then, but when we come out, we want to come out with here's a specification. It'll come out with a training course that you can take to learn how to compute the specification. It will come out with a tooling.
So you can just plug in values and then you'll be able to get your numbers and also come out with a number of case studies from organizations who, this is how exactly we calculated it, and maybe you can learn from, how we did it. So that's our goal.
Chris Adams: Okay, well that, so we're looking at basically, okay, first half of 2026, so there's still time to be involved and there's, and presumably later on in Q3, Q4, some of this will be going out in public for people to kind of respond to or have this some, something like the consultation there.
Asim Hussain: Yeah, It'd be a public consultation coming up soon.
Chris Adams: This is useful to know because this takes it to our last story we were looking at, which is actually also talking about the challenges related to the working on the environmental footprint of other things, particularly websites.
This is our final link of the podcast, which is a link to, the IEEE, where there's a post by, I believe it's Janne Kalliola. And, oh dear. I'm not gonna pronounce the other person's name very well. Juho Vepsäläinen. Oh dear. I'm so sorry for mispronouncing your names. I'm drawing attention to this 'cause this is the first time In a while I've seen a peer reviewed article in the IEEE specifically, which I think is the.
It's the Instutute of Electrical and Electronics Engineers. I forget what it stands for. Yes, thank you. They looked at both, Firefox Profiler and Website Carbon. They basically started looking at the environmental footprint, what kind of, what does using these website calculators actually tell you and what can you use?
And they had some recommendations about, okay, we've tried using these tools, what can we learn from that? And the thing that was actually particularly interesting was that they were using Firefox's Firefox profiler specifically to look at the footprint of, they're basically saying that there's two, three insights that have probably come away from this, which I thought was interesting.
One of them, it's really hard to get meaningful numbers around data transfer, which I think is actually something that we've shared and we've covered in a lot of detail and I'm finding very helpful for future discussions around creating something like a software, carbon intensity for Web for this.
But the other thing they did was they spoke about the use of, like tools out there, like profilers, which do provide this direct measurement that does give you some meaningful numbers. But when you look at the charts, the differences aren't that high. For example, they were showing comparisons with things like website carbon, which shows massively different, massively different kind of readings for the carbon footprint of one site versus another.
And then when they used other tools like say Firefox Profiler, the differences were somewhat more modest between these two things. So this kind of gives the impression that tool, some of the tools that use like the soft, the sustainable web design model may, they may be overestimating the effectiveness of changes you might be making as an engineer versus what gets measured directly.
Now, there's obviously a elephant in the room and that this isn't measuring what's happening server side, but this is the first time I've seen a really, kind of a real deep dive by, some people who are actually looking into this to come up with some things you can, you can test, I suppose, or you can kind of, you can like, reproduce to see if they get, you're getting the same numbers from these people here.
And, this is actually quite a useful, it's, I found it quite noteworthy and really nice to see and I would've found out about it because, Janne actually shared it inside the Climateaction.tech Slack.
Asim Hussain: So it was a paper inside IEEE or, an article inside that
Chris Adams: It's, a paper. So it's a peer reviewed paper in volume 13 of IEEE and they basically, they talk about the current state of the art, how people currently try to measure energy consumption on the Web. Then they talk about some of the tools you can use for the end user devices. Talk about some of the issues related to trying to go on just data transfer alone and why that isn't necessarily the best thing to be using, but, what kind of statements you could plausibly make.
But as someone who ends up, you know, we, the organization I work for, we implemented the sustainable web design model for this. Having something like this is so, so useful because we can now cite other peer reviewed work that's in the public domain that we can say, hey, we need to update this, based on this, or possibly do some, or an idea, which I believe that Professor Daniel Sheen shared with me.
He said, well, if we know, if we've got figures for the top million websites, the top thousand websites, maybe you could actually just experimentally validate those versus what you have in the, in a model already. So you can get better numbers for this. There's a bunch of steps. Yeah, exactly. If you were to measure the top thousand ones compared to the model figures, then that will give you an idea of the gap between the model figure and the ground truth, so you can end up with a slightly better, a better figure.
There's a bunch of things that you could do out there, which would, might make it easier to make these, this tooling much, much easier to use and much more likely to give people the signals they are craving to kind of build websites in a more kind of climate compatible fashion, for example.
Asim Hussain: And I think it's important because I think people like when you use a, when you use a tool and it gives you a, it gives you a value, it's incentivizing a behavior. And it might be incentivizing the wrong behavior. And it's, and I think that's one of the things I find that when people get excited about a measurement, I don't, because I'm, I need to know the details behind it.
'Cause I know that if you're a little bit wrong, you're incentivizing the wrong thing. And you shouldn't just, you shouldn't just take it face value. But it's really hard. I also, in the sense it's really bloody hard even for the tool makers to even figure out what to do here.
So this isn't really a, you know, but it's not really criticism of anybody. But, yeah, it's just really hard to figure this stuff out. But the Firefox stuff is using yours isn't, it's using CO2.js, isn't it?
Chris Adams: I'm not sure if this actually uses the carbon figures we use 'cause we're just, we basically package up the numbers from Ember, which is a non-profit think tank who already published stuff. I can't remember if this one is actually use using the energy or the carbon figures basically.
But we update the carbon figures, every month anyway. So it may, it might be our, I'll need to kind of check if they measure in terms, if they, I think they report this in energy, not carbon actually. It's what they used inside this.
Actually, I'll need to reread and we're coming up to time actually.
Asim Hussain: Here we come time, so this, but also I think maybe just call out a little bit. So we are gonna be running the, and you are leading it, the SCI for Web assembly shortly in the foundation. And I think this is, this can be a very, this looks, my brief scan of it, like a very important pre-read, I presume for a lot of the people who are gonna be attending that assembly.
Chris Adams: Yeah, I'm actually really pleased this came out. That was initially what I saw, oh great, this is a really nice, concise piece that covers this. This was another piece from Daniel Sheen talking about, okay, well how do you measure network figures, for example? 'cause he's put some really, good interesting stuff inside that we don't have enough time to talk about, but it's a really, but we'll share links to that inside that because yes, this is something that we'll be doing and I'm looking forward to doing it.
And oh, I've just realized we've gone way over.
Asim Hussain: We're well over. You've gotta go, on. Let's just, let's wrap
Chris Adams: Dude, really lovely catching up with you again. Oh, the final thing I need to give is this, just quickly talking about this GSM, the Green Software Movement thing that you were talking about here. Maybe I can just give you space to do that before we cl before we wrap up.
'Cause I know this is the project you're working on at the moment.
Asim Hussain: Yeah. So the movement is a platform that we've created, so it's movement.greensoftware.foundation. So this is where we, will be putting a lot more of our tension moving forward in terms of engaging with the broader community. It's also where all of our training is going to be.
So our current training is moving over there, and we just now have a, now that we've got like a real platform to publish training to. We're gonna get training for all of our products and services, so for SCI, Impact Framework, SOFT, RTC. We're gonna do training for all of them and have them available on the platform.
And you'll be able to go in, you'll be able to learn about the products that we've created, learn about the foundation, get certified for your training. But also it's a platform where you can connect with other people as well. So you can meet people, have chats, have conversations, connect with people who are local to you.
We've had over 130,000 people take our previous training, which unfortunately is on a previous, another platform. So we're gonna be trying to move everybody over. So hopefully our goal is ultimately for this to be the platform where you go, at least from terms of the Green Software Foundation to learn about our products, our standards get involved would be, our champions programs moving over there as well.
And we're just kind of like having, this will be where we put a lot of our effort moving forward, and I recommend people go to it, join, sign up, take the training, and connect with others.
Chris Adams: Alright. Okay. Well, Asim, lovely catching up with you. And I hope you have a lovely rest of the week. And I guess I'll see you in the Slacks or the Zulips or whichever online tools we use to across paths.
Asim Hussain: Zulips. I don't know what that is. Yeah. Sounds good. right, mate.
Chris Adams: our open source chat tool inside the Green Web Foundation. It runs on Django and it's wonderful.
Yeah, it's really good. I cannot recommend it enough. If you are using Slack and you are sick of using Slack, then use Zulips. Zulips is wonderful. Yeah. It's really, good.
Asim Hussain: I can check it out. Yeah. All right.
Chris Adams: Take man. See you Bye.
Asim Hussain: Bye.
Chris Adams: Hey everyone, thanks for listening. Just a reminder to follow Environment Variables on Apple Podcasts, Spotify, or wherever you get your podcasts. And please do leave a rating and review if you like what we're doing. It helps other people discover the show, and of course, we'd love to have more listeners.
To find out more about the Green Software Foundation, please visit greensoftware.foundation. That's greensoftware.foundation in any browser. Thanks again, and see you in the next episode.
118 episodes
MP3•Episode home
Manage episode 501503742 series 3336430
Content provided by Asim Hussain and Green Software Foundation. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Asim Hussain and Green Software Foundation or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Host Chris Adams is joined by Asim Hussain to explore the latest news from The Week in Green Software. They look at Hugging Face’s AI energy tools, Mistral’s lifecycle analysis, and the push for better data disclosure in the pursuit for AI sustainability. They discuss how prompt design, context windows, and model choice impact emissions, as well as the role of emerging standards like the Software Carbon Intensity for AI, and new research on website energy use.
Learn more about our people:
Find out more about the GSF:
News:
- A Gift from Hugging Face on Earth Day: ChatUI-Energy Lets You See Your AI Chat’s Energy Impact Live [04:02]
- Our contribution to a global environmental standard for AI | Mistral AI [19:47]
- AI Energy Score Leaderboard - a Hugging Face Space by AIEnergyScore [30:42]
- Challenges Related to Approximating the Energy Consumption of a Website | IEEE [55:14]
- National Drought Group meets to address “nationally significant” water shortfall - GOV.UK
Resources:
- GitHub - huggingface/chat-ui: Open source codebase powering the HuggingChat app [07:47]
- General policy framework for the ecodesign of digital services version 2024 [29:37]
- Software Carbon Intensity (SCI) Specification Project | GSF [37:35]
- Neural scaling law - Wikipedia [45:26]
- Software Carbon Intensity for Artificial Intelligence | GSF [52:25]
Announcement:
- Green Software Movement | GSF [01:01:45]
If you enjoyed this episode then please either:
- Follow, rate, and review on Apple Podcasts
- Follow and rate on Spotify
- Watch our videos on The Green Software Foundation YouTube Channel!
- Connect with us on Twitter, Github and LinkedIn!
TRANSCRIPT BELOW:
Asim Hussain: ChatGPT, they're all like working towards a space of how do we build a tool where people can literally pour junk into it, and it will figure something out.
Whereas what we should be doing, is how do you use that context window very carefully. And it is like programming.
Chris Adams: Hello, and welcome to Environment Variables, brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding green software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software.
I'm your host, Chris Adams.
Hello and welcome to this week in Green Software where we look at the latest news in sustainable software development. I am joined once again by my friend and partner in crime or occasionally crimes, Asim Hussain, of the Green Software Foundation. My name is Chris Adams. I am the Director of Policy and Technology at the Green Web Foundation, no longer the executive director there,
and, as we've moved to a co-leadership model. And, Asim, really lovely to see you again, and I believe this is the first time we've been on a video podcast together, right?
Asim Hussain: Yeah. I have to put clothes on now, so, so that's,
Chris Adams: That raises all kinds of questions to how intimate our podcast discussions were before. Maybe they had a different meaning to you than they did to me, actually.
Asim Hussain: Maybe you didn't know I was naked, but anyway.
Chris Adams: No, and that makes it fine. That's what, that's what matters. I also have to say, this is the first time we get to, I like the kind of rocking the Galactus style headphones that you've got on here.
Asim Hussain: These are my, yeah, no, these are old ones that I posted recently. I actually repaired them. I got my soldering iron and I repaired the jack at the end there. So, I'm very proud of myself for having repaired. I had the right to repair. Chris. I had the right to repair it.
Chris Adams: Yeah. This is why policy matters.
Asim Hussain: I also have the capability.
Chris Adams: Good. So you can get, so, good on you for saving a bunch of embodied carbon and, how that's calculated is something we might touch on. So, yes. So if you are new to this podcast, my friends, we're just gonna be reviewing some of the news and stories that are kinda showed up on our respective radars as we work in our kind of corresponding roles in both the Green Software Foundation and the Green Web Foundation.
And hopefully this will be somewhat interesting or at least diverting to people as they wash their dishes whilst listening to us. So that's the plan. Asim, should I give you a chance to just briefly introduce what you do at the Green Software Foundation before I go into this?
'Cause I realized, I've just assumed that everyone knows who you are. And I know who you are, but maybe there's people who are listening for the first time, for example.
Asim Hussain: Oh yeah. So, yeah. So my name's Asim Hussain. I am a technologist by trade. I've been building software for several decades now. I formed the green software, yeah, Green Software Foundation, you know, four years ago. And, now I'm the executive director and I'm basically in charge of, yeah, just running the foundation and making sure we deliver against our vision of a future where software has zero harmful environmental impacts.
Chris Adams: That's a noble goal to be working for. And Asim, I wanted to check. How long is it now? Is it three years or four years? 'Cause we've been doing this a while.
Asim Hussain: We, yeah. So we just fin, well, four years was May, so yeah, four years. So next birthday's the fifth birthday.
Chris Adams: Wow. Time flies when
the world is burning, I suppose.
Alright, so anyway, as per usual, what we'll do, we share all the show notes and any links that we discuss or projects we discuss, we'll do our damnedest to make sure that they're available for anyone who wants to continue their quest and learning more about sustainability in the field of software.
And I suppose, Asim, it looks like you're sitting comfortably now. Should we start looking at some of the news stories?
Asim Hussain: Let's go for it.
Chris Adams: Alright. Okay. The first one we have, is a story from Hugging Face. This is actually a few months back, but it's one to be aware of if it missed you the first time. So, Hugging Face released a new tool called Chat UI Energy that essentially lets you see, the energy impact live from using a kind of chat session,
a bit like ChatGPT or something like that. Asim, I think we both had a chance to play around with this, and we'll share a link to the actual story around this as well as the actual repo that's online. What do you think of this? what's your immediate take when you see this and have a little poke around with this?
Asim Hussain: Well, it's good. I wanna make sure. It's a really nice addition to a chat interface. So just so the audience who's not seeing it, every time you do a prompt, it tells you the energy in, well, in watt hours, what I'm seeing right now. But then also, you know, some other stats as well.
And then also kind of how much of a phone charge it is. And that's probably the most surprising one. I just did a prompt, which was 5.7% of a phone charge, which was, that's pretty significant. Actually, I dunno, is that significant? So, one of the things is, I'm trying to, what I'm trying to find out from it though is how does that calculation, 'cause that's my world, it's like, how does, what do you really mean by a calculation?
Is it cumulative? Is it session based? Is it just, you know, how, what have you calculated in terms of the energy emissions? The little info on the side is just the energy of the GPU during inference. So it's not the energy of kind of anything else in the entire user journey of me using a UI to ask a prompt.
But we also know that's probably the most significant. And I'm kind of quite interested in figuring out, as I'm prompting it, I'm one, I'm, one of the things I'm seeing is that every single prompt is actually, the emissions are bigger than the previous prompt. Oh no, it's not actually, that's not true.
Yeah, it is.
Chris Adams: Ah, this is the thing you've been mentioning about cumulative,
Asim Hussain: Cumulative. Yeah. Which is a confusing one. 'Cause I've had a lot of people who are really very good AI engineers go, "Asim, no, that's not true." And other people going, "yeah, it kind of is true." But they've just optimized it to the point where the point at which you get hit with that is at a much larger number.
But the idea is that there's, there, it used to be an n squared issue for your prompt and your prompt session history. So every time you put a new prompt in all of your past session history was sent with your next prompt. And if you are actually building, like a your own chat system, if you are actually building like your own chat solution for your company or wherever, that is typically how you would implement it as a very toy solution to begin with is just, you know, take all the texts that was previous and the new text and send it, in the next session.
But I think what, they were explaining to me, which was actually in the more advanced solutions, you know, the ones from Claude or ChatGPT, there's a lot of optimization that happens behind the scenes. So it doesn't really, it doesn't really happen that way, but I was trying to figure out whether it happens with this interface and I haven't quite figured it out yet.
Chris Adams: Oh, okay. So I think what you might be referring to is the fact that when you have like a GPU card or something like that, there's like new tokens and kind of cashed tokens, which are priced somewhat differently now. And this is one of the things that we've seen.
'Cause it's using maybe a slightly different kind of memory, which might be slightly faster or is slightly kind of is slightly lower cost to service in that sense. Yeah. Okay. So this is one thing that we don't see. What I, the good news is we can share a link to this, for anyone listening, this source code is all on GitHub, so we can have a look at some of this.
And one of the key things you'll see actually is, well this is sending a message. When you see the actual numbers update, the, it's not actually, what it's actually doing is it's calculating all this stuff client site based on how big each model is likely to be. 'Cause when you look at this, you can A,
Asim Hussain: It's a model.
Chris Adams: You can work out the, I mean, so when people talk about should I be using the word please or thank you, and am I making the things worse by treating this like a human or should I just be prompting the machine like a machine, is there a carbon footprint to that? This will display some numbers that you can see there, but this has all been calculated inside your browser rather than actually on the server.
So like you said, Asim, there is a bit of a model that's taking place here, but as a kind of way to like mess around and kind of have a way into this. This is quite interesting and even now it's kind of telling that there are so few providers that make any of this available, right now. We're still struggling even in like the third quarter of 2025,
to have a commercial service that will expose these numbers to you in a way that you can actually meaningfully change the environmental footprint of through either your prompting behavior or well maybe model choice. But that's one of the key things that I see. I can't think, I can't think of any large commercial service that's doing this.
The only one is possibly GreenPT,
which is basically put a front end on Scaleway's, inference service and I'm not sure how much is being exposed there for them to make some assumptions as well.
Asim Hussain: Do you know how bad, do you know how,
I feel very uncomfortable with the idea of a future where a whole bunch of people are not saying please or thank you, and the reason for it is they're proudly saying, "well, I care about, I care about sustainability, so I'm not gonna say please or thank you anymore 'cause it's costing too many, too much carbon." I find that very uncomfortable. I personally, I don't wanna, we could, choose not to say please or thank you in all of our communications because it causes, emissions no matter what you do. I don't know.
Chris Adams: I'm glad you weren't there, Asim. 'Cause I was thinking about that too. There's a carbon cost to breathing out and if, you, I guess maybe that's 'cause we're both English and it's kinda hardwired into us. It's like the same way that, you know, if you were to step on my toe, I would apologize to you stepping on my toe because I'm just English and I, and it's a muscle memory, kind of like impulsing.
Okay.
Asim Hussain: Yeah.
Chris Adams: That's, what we found. We will share some couple, a couple of links to both the news article, the project on Hugging Face, and I believe it's also on GitHub, so we can like, check this out and possibly make a PR to account for the different kinds of caching that we just discussed to see if that does actually make a meaningful difference on this.
For other people who are just looking, curious about this, this is one of the tools which also allows you to look at a, basically not only through weird etiquette, how etiquette can of impact the carbon footprint of using a tool, but also your choice of model. So some models might be, say 10 times the size of something, but if they're 10, if they're not 10 times as good, then there's an open question about whether it's really worth using them, for example.
And I guess that might be a nice segue to the next story that we touch on. But Asim, I'll let you, you gotta say something. I
Asim Hussain: No, I was gonna say, because I, this is, 'cause I've been diving into this like a lot recently, which is, you know, how do you efficiently use AI? Because I think a lot of the, a lot of the content that's out there about, you know, oh, AI's emissions and what to do to reduce AI's emissions, there are all the choices that as a consumer of AI, you have absolutely no ability to affect. I mean, unless you are somebody who's quite comfortable, you know, taking an open source model and rolling out your own infrastructure or this or that or the other. If you're just like an everyday, not even an everyday person, but just somebody who works in a company who's, you know, the company bought Claude, you know, you're using Claude,
end of story, what are you, like, what do you do? And I think that's really, it is a really interesting area. I might just derail our whole conversation to talk about this, but I think it's a really interesting area because, what it's really boiling down to is your use of the context window.
And so you have a certain number of tokens in a chat before that chat implodes, and you can't use that chat anymore. And historically, those number of tokens were quite low. Relative to, because of all the caching stuff hadn't been invented yet and this and that and the other. So the tokens were quite low.
What, didn't mean they didn't mean they were, the prompts were cheaper before. I think they were still causing a lot of emissions. But because they've improved the efficiency and rather than just said, I've improved the efficiency, leave it at that, I've improved the efficiency, Jevons paradox, I've improved the efficiency,
let's just give people more tokens to play around with before we lock them out.
So the game that we're always playing is how to actually efficiently use that context. And the please or thank you question is actually, see this is, I don't think it's that good one. 'Cause it's two tokens in a context window of a million now, is what's coming down the pipeline.
The whole game. And I think this is where we're coming from as you know, if you wanna be in the green software space and actually have something positive to say about how to actually have a relationship with AI, it's all about managing that context. 'Cause the way context works is you're just trying to, it's like you've got this intern and if you flash a document at this intern, you can't then say, "oh, ignore that.
Forget it I didn't mean to show you that." It's too late. They've got it and it's in their memory and you can't get rid of it. the only solution is to literally execute that intern and bury their body and get a new intern and then make sure they see the information in the order and only the information they need to see so that when you finally ask 'em that question, they give you the right answer. And so what a lot of people do is they just, because there's a very limited understanding of how to play, how to understand, how to play with this context space, what people end up doing is they're just going, "listen, here's my entire fricking document. It's actually 50,000 words long. You've got it, and now I'm gonna ask you, you know, what did I do last Thursday?"
So it's, and all of that context is wasted. And I think that's, and it's also like a very simplistic way of using an AI, which is why like a lot of companies are, kind of moving towards that space because they know that it means their end user doesn't have to be very well versed in the use of the tool in order to get benefit out of it.
So that's why ChatGPT, they're all like working towards a space of how do we build a tool where people can literally pour junk into it, and It will figure something out.
Whereas what we should be doing and what I'm like, and I think it's not only what we should be doing, it's, what the people who are like really looking at how to actually get real benefit from AI,
is how do you use that context window very carefully. And it is like programming. It is really like program. That's what, that's my experience with it so far. It's like, I want this, I need to feed this AI information. It's gonna get fed in an order that matters. It's gonna get fed in a format that matters.
I need to make sure that the context I'm giving it is exactly right and minimal. Minimal for the question that I wanna answer, get it answered at the end of it. So we're kind of in this like space of abundance where, because every AI provider's like, "well do what you want. Here's a million tokens.
Do what you want, do what you want."
And they're all, we're all just chucking money. These we're just chucking all our context tokens at it. They're burning money on the other side because they're not about making a profit at the moment. They're just about becoming the winner. So they don't really care about kind of profitability to that level.
So what us It's all about, I'm just getting back to it again. I think, we need to eventually be telling that story of like, how do you actually use the context window very carefully? And again, it's annoyed me that the conversation has landed at please and thank you. 'Cause the actual conversation should be, you know, turning that Excel file into a CSV because it knows how to parse a CSV and it uses fewer tokens to parse a CSV than an Excel file. Don't dump the whole Excel file, export the sheet that you need in order for it to, answer that question. If you f up, don't just kill the session and start a new session.
This is, there's this advice that we need to be giving that I don't even know yet.
Chris Adams: MVP. Minimal viable prompt.
Asim Hussain: Minimal viable prompt! Yeah. What is the minimal viable prompt and the, what's frustrating me is that like one of the things that we use Claude and I use Claude a lot, and Claude's got a very limited context window and I love that.
It was like Twitter when you had to, remember Twitter when you had to like have 160 characters?
It was beautiful.
Chris Adams: to 280, and then you're prepared to be on that website, you can be as, you can monologue as much as you want
Asim Hussain: Yeah. You can now monologue, but it was beautiful having to express an idea in this short, like short, I love that whole, how do I express this complex thing in a tweet? And so with the short context windows, were kind of forced to do that, and now I'm really scared because now everybody, Claude literally two days ago has now gone, right, you've got a million context window, and I'm like, oh, damn it.
Now I don't even, now I don't have personally
Chris Adams: That's a million token context window when you say that. Right. So that's enough for a small book basically. I can dump entire book into it, then ask questions about it. Okay. Well, I guess it depends on the size of your book really, but yeah, so that's, what you're referring to when you talk about a million context window there.
Asim Hussain: Yeah, yeah. And it's kind of an energy question, but the energy doesn't really, kind of, knowing how much, like I've just looked at chat UI window and I've checked a couple of prompts and it's told me the energy, and it's kinda that same world.
It's just it's just there to make me feel guilty, whereas the actual advice you should be getting is well, actually no, I, what do I do? How am I supposed to prompt this thing to actually make it consume less energy? And that's the,
Chris Adams: Oh, I see. So this is basically, so this is, you're showing me the thing and now you're making me feel bad. And this may be why various providers have hosted chat tools who want people to use them more, don't automatically ship the features that make people feel bad without giving 'em a thing they can actually do to improve that experience.
And it may be that it's harder to share some of the guidance like you've just shared about making minimum viable prompt or kind of clear prompt. I mean, to be honest, in defence of Anthropic, they do actually have some pretty good guidance now, but I'm not aware of any of it that actually talks about in terms of here's how to do it for the lowest amount of potential tokens, for example.
Asim Hussain: No, I don't see them. I don't see them. I mean, they, yeah, they do have like stuff, which is how to optimize your context window, but at the same time, they're living in this world where everybody's now
working to a bigger, that's what they have to do.
And I don't know, it's kinda like, where do we, because we, 'cause the AI advice we would typically have given in the past, or we would typically give is listen, just run your AI in a cleaner region. And you are like, well, I can't bloody do that with Anthropic, can I? It's just, it's whatever it is, it's, you know.
Chris Adams: That's a soluble problem though. Like,
Asim Hussain: Like what I'm just saying or,
Chris Adams: Yeah. You know, but like the idea they're saying, "Hey, I want to use the service. And I want to have some control over where this is actually served from."
That is a thing that you can plausibly do. And that's maybe a thing that's not exposed by end users, but that is something that is doable.
And, I mean, we can touch on, we actually did speak about, we've got Mistral's LCA reporting as one of the things, where they do offer some kind of control, not directly, but basically by saying, "well, because we run our stuff in France, we're already using a low carbon grid."
So it's almost like by default you're choosing this rather than you explicitly opting in to have like the kind of greener one by, the greener one through an active choice,
I suppose.
Asim Hussain: They're building some data centers over there as well, aren't they? So it's a big, it's a big advantage for Mistral to be in France, to be honest with you. It's yeah, they're in
Chris Adams: this definitely does help, there's, I mean, okay. Well, we had this on our list, actually, so maybe this is something we can talk about for our next story, because another one on our list since we last spoke was actually a blog post from Mistral.ai talking about, they refer to, in a rather grandiose terms, our contribution to a global environmental standard for AI.
And this is them sharing for the first time something like a lifecycle analysis data about using their models. And, it's actually one that has, it's not just them who've been sharing this. They actually did work with a number of organizations, both France's agency, ADM. They were following a methodology specifically set out by AFNOR, which is a little bit like one of the French kind of, environmental agency, the frugal AI methodology.
And they've also, they were working with I think, two organizations. I think it's Sopra Steria, and I forget the name of the other one who was mentioned here, but it's not just like a kind of throwaway quote from say Sam Altman. It's actually, yeah, here we are is working with Hubblo, which is a nonprofit consultancy based in Paris and Resilio who are a Swiss organization, who are actually also quite, who are quite very well respected and peer reviewed inside this.
So you had something, some things to share about this one as well. 'Cause I, this felt like it was a real step forward from commercial operators, but still falling somewhat short of where we kind of need to be. So, Asim, what, when you read this, what were the first things that occurred to you, I suppose, were there any real takeaways for you?
Asim Hussain: Well, I'd heard about this, on the grapevine, last year because I think, one of the researchers from Resilio was at greenIO, yeah, in Singapore. And I was there and he gave a little a sneak. They didn't say who it was gonna be, they didn't say it was Mistral, but they said, we are working on one.
And he had like enough to tease some of the aspects of it. I suspect once it's got released, some of the actual detail work has not, that's what I'm, I think I'm, unless I, unless there's a paper I'm missing. But yeah, there is kind of more work I think here that didn't end up to actually get released once it's, once it got announced, but there was, it was a large piece of work.
It's good. It's the first AI company in the world of this, you know, size that has done any work in this space and released it. Other than like a flippant comment from Sam Altman, "I heard some people seem to care about the emission, energy consumption of AI." So, so that's good. And I think we're gonna use this, it's gonna be used in as a, as I'd say, a proxy or an analog for kind of many other, situations.
I think it's, it is lacking a little bit in the detail. But that's okay. I think we, every single company that starts, we should celebrate every organization that leads forward with some of this stuff. it's always very, when you're inside these organizations, It's always a very hard headwind to push against.
'Cause there's a lot of negative reasons to release stuff like this, especially when you're in a very competitive space like AI. So they took the lead, we just celebrate that. I think we're going to, there's some data here that we can use as models for other, as, you know, when we now want to look at what are the emissions of Anthropic or OpenAI or Gemini or something like that,
there's some more, you know, analogs that we can use. But also not a huge amount of surprise, I'd say, it's kind of a training and inference,
Chris Adams: Yep.
That turns be where the environmental footprint is.
Asim Hussain: Yeah. Training and inference, which is kind of, which is good. I mean, I think obviously hardware and embodied impacts is, they kind of separate kind of the two together.
I suspect, the data center construction is probably gonna be, I don't know
that is quite low. Yeah, yeah,
Chris Adams: I looked at this, I mean this is, it's been very difficult to actually find any kind of meaningful numbers to see what share this might actually make. 'Cause as the energy gets cleaner, it's likely that this will be a larger share of emissions. But one thing that was surprising here was like, this is, you know, France, which is a relatively cr clean grid, like maybe between 40 and say 60 grams of CO2 per kilowatt hour, which is, that's 10 times better than the global average, right?
Or maybe 9, between 8 and 10 times cleaner than the global average. And even then it's, so with the industry being that clean, you would expect the embodied emissions from like data centers and stuff to represent a larger one. But the kind of high level, kind of pretty looking graphic that we see here shows that in, it's less than 2% across all these different kind of impact criteria like carbon emissions or water consumption or materials, for example.
This is one thing that, I was expecting it to be to be larger, to be honest. The other thing that I noticed when I looked at this is that, dude, there's no energy numbers.
Asim Hussain: Oh, yeah.
Chris Adams: Yeah. And this is the thing that it feels like a, this is the thing that everyone's continually asking for.
Asim Hussain: It's an LCA. So they use the LCAs specification, so
Chris Adams: That's, a very good point. You're right. that is, that's a valid response, I suppose. 'Cause energy by itself doesn't have a, doesn't have a carbon footprint, but the results of generating that energy does, electricity does have that impact. So yeah.
Okay. Maybe that's
For
Asim Hussain: the audience, they use like a well known, well respected, standardized way of reporting the lifecycle emissions using the LCA lifecycle analysis methodology, which is like an ISO certified standard of doing it. So they adhere to a standard.
Chris Adams: So this actually made me realize, if this is basically here and you are a customer of a AI provider, 'cause we were looking at this ourselves trying to figure out, okay, well what people speak to us about a AI policies? And we realized well, we should probably, you know, what would you want to have inside one?
The fact that you have a provider here who's actually done this work, does suggest that for that it's possible to actually request this information if you're a customer under NDAs. In the same way that with, if you're speaking to Amazon or probably any of the large providers, if you're spending enough money with them, you can have information that is disclosed to you directly under NDA.
So it may not be great for the world to see, but if you are an organization and you are using, say, Mistral, for example, or Mistral services, this would make me think that they're probably more able to provide much more detailed information so that you can at least make some informed decisions in a way that you might not be able to get from some of the other competing providers.
So maybe that's one thing that we actually do see that is a kind of. Not really a published benefit in this sense, but it's something that you're able to do if you are in a decision making position yourself and you're looking to choose a particular provider, for example.
Asim Hussain: I mean, you should always be picking the providers who've actually got some, you know,
Chris Adams: optimize for disclosure,
Asim Hussain: optimize for disclosure. Yeah. Always be picking the providers if you optimize for disclosure. I mean, if we, the people listening to this, that is the thing that you can do. And Mistral, They're also, they have some arguments in here as well, which is kind of, they did kind of also surface that it is like a pretty linear relationship between your emissions and the size of the model, which is a very useful piece of information for us to know, as a consumer.
Because then we can go, well actually I've heard all these stories about use Smaller models use smaller models and now you actually have some data behind it, which is supporting the fact that, yeah, using a smaller model isn't, it's not got some weird non-linearity to it, so a half size model is only like 10% less, emissions.
A half size model is half the emissions. So that's pretty, that's a pretty good thing to know. Helps Mistral, the fact that they have a lot of small models that you can pick and choose, is not, so a lot of this stuff really benefits Mistral. They are the kind of the kind of organization which has a product offering which is benefited, which does benefit a sustainability community.
So they have like small models you can use. I think, I wonder actually, Chris, 'cause they do say that they're building their own data center in France, but they've never said where there exists, where they until now, where they've been running their AI. So that might be the reason for, they might have been running it in East Coast US or something
like
Chris Adams: I think that would be quite unlike, wouldn't be very likely, given that most of their provider, most of their customers are based in probably Western Europe still. Right. There is very much a kinda like Gaelic kind of flavor to the tooling. And I've, I mean actually Mistral, or Mistral's tools are ones which I've been using myself personally over the last, like few months, for example.
And it's also worth bearing in mind that they, took on a significant amount of investment from Microsoft a few years back and I would be very surprised if they weren't, or if they weren't using a French data center serving French providers. 'Cause if you were to choose between two countries, okay, if, France or like France actually has, and since 2021, I believe, has had actually a law specifically about measuring the environmental footprint of digital services.
So they've got things that they, I think it's called, I'm going to, I'm just gonna share a link to that, to the name of the law because I'm gonna butcher the French pronunciation, but it basically, it translates to Reduce the Environmental Footprint of Digital Services Law.
That's pretty much it. And that's where, as a follow on from that, that's what, that's what the RGESN, the kind of general guidance that it shares across kind of government websites in general for France. They've already got a bunch of this stuff out there for like how to do greener IT. I suspect that France is probably gonna be one of, well, probably the premier country, if you'd run, be running a startup to see something like this happening much more so than, well probably the US right now, especially given the current kind of push with its current kind of federal approach, which is basically calling into doubt climate change in the wider sense basically.
We were talking about disclosure, right? And we said an optimization for disclosure. And that's probably a nice segue to talk about, another link we had here, which was the energy score leaderboard. Because this is one thing that we frequently point to. And this is one thing that we've suggested in my line of work, that if you are looking to find some particular models, one of the places to look would be the AI Energy Score Leaderboard, which is actually maintained by Hugging Face.
And, I share this 'cause it's one of the few places where you can say, I'm looking for a model to help me maybe do something like image generation or captioning text or generating text or doing various things like this. And you can get an idea of how much power these use on a standardized setup.
Plus, how satisfied, you know, what the kind of satisfaction score might be, based on these tools and based on a kind of standardized set of like tests, I suppose. The thing is though, this looks like it hasn't been updated since February. So for a while I was thinking, oh, Jesus, does this mean we actually need to, do we have to be careful about who we, how we recommend this?
But it turns out that there's a new release that will be coming out in September. It's updated every six months. And, now that I do have to know about AI, this is one thing that I'm looking forward to
seeing some of the releases on because if you look at the leaderboard for various slices, you'll see things like Microsoft Phi 1 or Google Gemma 2 or something like that.
Asim Hussain: That quite old?
Chris Adams: yeah, these are old now, it's six months in generative AI land is quite a long time. There's Phi 4 now, for example, and there's a bunch of these out there. So I do hope that we'll see this actually. And if you feel the same way, then
yeah, go on.
Asim Hussain: Is it, 'cause, is I always assume this was like a, live leaderboard. So as soon as a model, I suppose once a model, like the emissions of a model are linked to the model and the version of it. So once you've computed that and put on the leaderboard, it's not gonna change. So then it's just the case of as new models come out, you just measure and it just sees how it goes on the leaderboard.
Because I'm seeing something here. I'm, I thought open, I'm seeing OpenAI, GPT. Isn't that the one they just released?
Chris Adams: No, you're thinking GPT-OSS, perhaps
Asim Hussain: Oh.
Chris Adams: One thing they had from a while ago. So that one, for example, came out less than two weeks ago, I believe. That isn't showing up here.
Asim Hussain: That isn't showing up
Chris Adams: The, I'm, I was actually looking at this thinking, oh, hang on, it's six months, something being updated, six months,
that's, it'd be nice if there was a way, a faster way to expedite kind of getting things disclosed to this. For example, let's say I'm working in a company and I've, someone's written in a policy that says only choose models that disclose in the public somewhere. This is one of the logical places where you might be looking for this stuff right now, for example, and there's a six month lag, and I can totally see a bunch of people saying, no, I don't wanna do that.
But right now there's a six month kind of update process for this.
Asim Hussain: In the AI realm is an eternity. Yeah.
Chris Adams: Yeah. But at the same time, this is, it feels like a thing that this is a thing that should be funded, right? I mean, it's, it feels :I wish there was a mechanism by which organizations that do want to list the things, how to make them to kind of pay for something like that so they can actually get this updated so that you've actually got some kind of meaningful, centralized way to see this.
Because whether we like it or not, people are basically rolling this stuff out, whether we like it or not, and I feel In the absence of any kind of meaningful information or very patchy disclosure, you do need something. And like this is one of the best resources I've seen so far, but it would be nice to have it updated.
So this is why I'm looking forward to seeing what happens in September. And if you think, if you too realize that like models and timely access to information models might be useful, it's worth getting in touch with these folks here because, I asked 'em about this when I was trying to see when they were, what the update cycle was.
And basically the thing they said was like, yeah, we're, really open to people speaking to us to figure out a way to actually create a faster funded mechanism for actually getting things listed so that you can have this stuff visible. Because as I'm aware, as I understand it, this is a labor of love by various people, you know, between their day jobs, basically.
So it's not like they've got two or three FTE all day long working on this, but it's something that is used by hundreds of people. It's the same kind of open source problem that we see again and again. But this is like one of the pivotal data sources that you could probably cite in the public domain right now.
So this is something that would be really nice to actually have resolved.
Asim Hussain: Because there is actually, 'cause the way Hugging Face works is, they have a lab and they have their own infrastructure. Is that how it works? Yeah. So that's
Chris Adams: this would, that was be, that was either, that was physically theirs, or it was just some space.
Asim Hussain: Spin up. But yeah. But yeah, but they have to effectively like to get the score here. It's not self certified, I presume, but there's a, you know, each of these things has got to get run against the benchmark. So there's basically, if I remember, there was a way of like self certifying.
There was literally a way for
Chris Adams: You could upload your stuff.
Asim Hussain: Yeah. OpenAI could disclose to the Hugging Face to the, what the emissions of, you know, what the energy of it was. But most of it is, there's actually, you gotta run against the H100 and there's a benchmark
Chris Adams: Yep, exactly. So there's a bit of manual. There's a bit of manual steps to do that, and this is precisely the thing that you'd expect that really, it's not like an insoluble problem to have some way to actually expedite this so that people across the industry have some mechanism to do this. 'cause right now it's really hard to make informed decisions about either model choice or anything like that.
Even if you were to architect a more responsibly designed system, particularly in terms of environmental impact here.
Asim Hussain: Because if you were to release a new model and you wanted it listed in the leaderboard, you would have to run every other model against. Why would you need to do that? You need to
Chris Adams: You wouldn't need to do that. You just need to, you, because you don't have control over when it's released, you have to wait six months until the people who are working in that get round to doing that.
Asim Hussain: Just the time. It's just a time. Yeah. Someone's
Chris Adams: If you're gonna spend like a millions of dollars on something like this, it feels like this is not, even if you were to drop say, if, even if it was to cost, maybe say a figure in the low thousands to do something like this, just to get that listed and get that visible, that would be worth it.
So that you've actually got like a functioning way for people to actually disclose this information, to inform decisions. 'Cause right now there's, nothing that's easy to find. This is probably the easiest option I've seen so far and we've only just seen like the AI code of practice that's actually kind of been kind of pub that came into effect in August in Europe for example.
But even then, you still don't really have that much in the way of like public ways to filter or look for something based on the particular task you're trying to achieve.
I wanted to ask you actually, Asim, so I think, I can't remember last time if I was speaking to you, if this came up, I know that in your, with your GSF hat on, there's been some work to create a software carbon intensity for AI spec, right. Now, I know that there's a thing where like court cases, you don't wanna kind of prejudice the discussions too much by having things internally.
Although you're probably not, there isn't like AI court, you can be in contempt of, but I mean, yeah, not yet, but, who knows? Give it another six months. Is there anything that, is there anything, any, juicy gossip or anything you can share that people have been learning? 'cause like you folks have been diving into this with a bunch of domain experts so far, and this isn't my, like, while I do some of this, I'm not involved in those discussions.
So I mean, and I'm aware that there has been a bunch of work trying to figure out, okay, how do you standardize around this? What do you measure? You know, do you count tokens? Do you count like a prompt? What's the thing? Is there anything that you can share that you're allowed to talk about before it goes?
Asim Hussain: Yeah. I think, we, I think that what we've landed on is that as long as I'm not discussing stuff which is in, you know, active discussion and it's kind of made its way into the spec and there's been, you know, broad consensus over, I think it's pretty safe to talk about it.
If there's something that's kind of, and what we do, we do everything in GitHub. So if there's something which is like, I won't, I won't discuss anything which has only been discussed in like an issue or a discussion or comment thread or something. If it's actually made its way into the actual spare, that's pretty safe.
So yeah, the way it's really landed is that there's, there was a lot of conversations at the start. There was a lot of conversations and I was very confused. I didn't really know where things were gonna end up with. But you know, at the start there was a lot of conversations around well, how do we deal with training?
How do we deal with training? There's this thing called inference. And it's interesting 'cause when we look at a lot of other specs that have been created, even the way the Mistral LCA was done, so they, they gave a per inference, or per request. I've forgotten what they did. It, they didn't do per token.
So per
Chris Adams: they do per chat session or per task, right. I think it's something along those lines. Yeah.
Asim Hussain: Something along that, it wasn't a per token thing. But even then they, they added the training cost to it. And like those, some of the questions we were adding, can you add, is there a way of adding like the training? The training happened like ages ago. Can you somehow, is there a function that you can use to amortize that training to like future inference runs?
And we explored like lots of conversations. There's like a decay function. So if you were the first person to use a new model, the emissions per token would be higher because you are amortizing more of the training cost and the older models, the, so you explored like a decay function, we explored, yeah.
There's lots of ideas.
Chris Adams: Similar to the embodied usage, essentially like what we have with embodied versus, embodied carbon versus like use time carbon. You're essentially doing the same thing for training, being like the embodied bit and inference being the usage. And if you had training and you had three inferences, each of those inferences is massive.
Like in terms of the car embodied carbon, if there's like a billion, it's gonna much lower per, for each one.
Asim Hussain: But then you get into really weird problems because I mean it, we do that with the embodied carbon hardware, but we do that by saying, do you know what? The lifespans gone be four years and that's it. And we're just gonna pretend it's an equal waiting every single day for four years.
Chris Adams: Not with the GHG protocol. You can't do it with the GHG protocol. You can't amortize it out like that. You can, you have to do it the same year, so it, your emissions look awful one year
Asim Hussain: Ah, the year that you bought it from.
Chris Adams: So this is actually one of the reasons, but yeah, this is actually one of the problems with the kind of default way of measuring embodied carbon versus other things inside this is, it's not, like Facebook for example, they've proposed another way of measuring it, which does that, this kind of amortization approach, which is quite a bit closer to how you might do, I guess, like typical amortization of capital, capital
Asim Hussain: Cap, yeah.
Chris Adams: So that's the, that's the difference in the models. And this is, these are some of the kind of honestly sometimes tedious details that actually have quite a significant impact. Because if you did have to, that's gonna have totally different incentive incentives. If you, especially at the beginning of something, if you said, well, if you pay the full cost, then you are incentivized not to use this shiny new model.
'Cause it makes you look awful compared to you using an existing one for example.
Asim Hussain: And that's one of the other questions like, is like, how do you, I mean, a lot of these questions were coming up like what do you... A we never, we didn't pick that solution. and we also didn't pick the solution of we had the, we actually had the conversation of you amortize it over a year, and then there's a cliff.
And then that was like, we're gonna incentivize people to use older models with this idea that older models were the thing.
There were questions that pop up all the time. Like, what do you do when you have an open source model? If you were to, if I was to fine tune an open source model and then make a service based off of that, is the emissions of the model the open source model that I got Llama whatever it was, am I responsible for that?
Or is the,
and there was like, if you were to say, if you were to say no, then you're incentivizing people to just like open source their models and go, "meh well the emissions are free now 'cause I'm using an open source model." So there's lots of these, it's very nuanced. Kind of the, a lot of the conversations we have in the standards space, is like a small decision can actually have a cascading series of unintended consequences.
So the thing that we really like sat down was like, what, well, what actually, what do you want to incentivize? Let's just start there. What do we want to incentivize? Okay, we've listed those things we wanna incentivize. Right. Now, let's design a metric, which through no accident incentivizes those things. And where they ended up was basically two,
there's gonna be two measures. So we didn't, we didn't solve the training one because there isn't a solution to it. It's a different audience cares about the training emissions than that doesn't, consumers, it's not important to you because it doesn't really matter. It doesn't change how you behave with a model.
It doesn't change how you prompt a model just because it had some training emissions in the past. What matters to you most is your direct emissions from your actions you're performing at that given moment in time. So it's likely gonna be like two SCI scores for AI, a consumer and a provider. So the consumer is like inference plus everything else.
and also what is the functional unit? There's a lot of conversations here as well, and that's likely to land that now very basically the same as how you sell an AI model. So if you are an LLM, you're typically selling by token. And so why for us to pick something which isn't token in a world where everybody else is thinking token, token, token, token, it would be a very strange choice and it would make the decision really hard for people when they're evaluating certain models. They'd be like, oh, it's this many dollars per token for this one and this many dollars per token for that one. But it's a carbon per growth. And it's a carbon per growth,
I can't rationalize that. Where, if it's well look, that's $2 per token, but one gram per token of emissions and that's $4 per token, but half a gram per token for emissions. I can evaluate the kind of cost, carbon trade off, like a lot easier. The cognitive load is a lot easier.
Chris Adams: So you're normalizing on the same units, essentially, right?
Asim Hussain: Yeah. As how, however it's sold, however, it's, 'cause that's sort of, it's a fast, AI is also a very fast moving space and we dunno where it's gonna land in six months, but we are pretty sure that people are gonna figure out how to sell it, in a way that makes sense. So lining up the carbon emissions to how it's sold.
And the provider one is going to be, that's gonna include like the training emissions, but also like data and everything else. And that's gonna be probably per version of an AI. And that will, so you can imagine like OpenAI, like ChatGPT would have a consumer score of carbon per token and also a provider score of ChatGPT 5 has, and it's gonna be probably like per flop or something,
so per flop of generating ChatGPT 5, it was this many, this much carbon. And that's really like how it's gonna,
it's also not gonna be total totals are like, forget about totals. Totals are pointless when it comes to, to change the behavior.
You really want to have a, there's this thing called neural scaling laws.
The paper.
Chris Adams: Is that the one that you double the size of the model when it's supposed to double the performance? Is that the thing?
Asim Hussain: It's not double, but yeah, got relationship. Yeah. So there's this logarithmic, perfectly logarithmic relationship between model accuracy and model size, model accuracy, and the data, the number of training you put into it, and model size and the amount of compute you put into, it's all logarithmic.
So it's often used as the reason, the rationale for like why we need to, yeah, larger models is because we can prove it. So, but that basically comes down to like really then, you know, like if like I care more about, but for instance, I don't particularly, it doesn't matter to me how much, it's not that important to know the total training emissions of ChatGPT 5 versus ChatGPT 4.
What's far more useful, is to know, well, what was the carbon per flop of training for 4 versus the carbon per flop of training for 5? 'Cause then that gives you more interesting information. Have you, did you,
Chris Adams: What does that allow?
Asim Hussain: Bother to do anything? Huh?
Chris Adams: Yeah. What does that allow me to do? If I know if 5 is 10 times worse per flop than 4,
what that incentivize me to do differently? 'Cause I think I might need a bit of hand help here making this call here.
Asim Hussain: Because I think, 'cause it, what, let's say ChatGPT 6 is going to come along. The one thing we know absolutely sure is it's just gonna be in terms of total bigger than ChatGPT 5. So as like a metric, it's not, if you are an engineer, if you are somebody trying to make decisions regarding what do I do to actually train this model with causing less emissions, it doesn't really help me because it's just, a number that goes higher and higher.
Chris Adams: Oh, it's a bit like carbon intensity of a firm versus, absolute emissions. Is that the much, the argument you're using? So it doesn't matter that Amazon's emissions have increased by 20%, the argument is well, at least if they've got more efficient per dollar of revenue, then that's still improvement.
That's the line of reasoning that's using, right?
Asim Hussain: Yeah. So it's,
because of the way the SCI is, it's not if you want to do a total, there are LCAs, like the thing that Mistral did, there's existing standards that are very well used. They're very well respected. There's a lot of, there's a lot of information about how to do them.
You can just use those mechanisms to calculate a total. What the SCI is all about is what is a,
KPI that a team can use and they can optimize against, so over time, the product gets more and more efficient?
Obviously, you should also be calculating your totals and be making a decision based upon both.
But just having a total is, I've gotta be honest with you, it's just, I don't see totals having, in terms of changing behavior, I don't think it changes any behavior. Full stop.
Chris Adams: Okay. I wanna put aside the whole, we live in a physical world with physical limits and everything like that, but I think the argument you're making is essentially that, because the, you need something to at least allow you to course correct on the way to reducing emissions in absolute terms, for example. And your argument you're making is if you at least have an efficiency figure, that's something you can kind of calibrate and change over time in a way that you can't with absolute figures, which might be like having a, you know, a budget between now and 2030, for example.
That's the thinking behind it, right?
Asim Hussain: Yeah. I mean, if you, I've actually got an example here from 'cause we, so we don't have actual compute. They, no, no one's ever disclosed like the actual compute that they used per model. But they have, or they used to disclose the number of parameters per model. And we know that there's a relationship.
So there's a really interesting, so for 2, 3 and 4, we have some idea regarding the training emissions and the parameters, not from a disclosure, from like research as well, so between, but when you compute the emissions per billion parameters of the model, so per billion parameters of the model, GPT two was 33.3 tons of carbon per billion parameters of the model.
Chris Adams: Okay.
Asim Hussain: GPT-3 went down to 6.86 tons of carbon per billion parameters. So it went down from 33 to 6. So that was a good thing. It feels like a good thing, but we know the total emissions of 3 was higher. Interestingly, GPT-4 went up to 20 tons of carbon per billion parameters. So that's like an interesting thing to know.
It's like you did something efficient between two and three. You did something good. Whatever it was, we don't know what it was, we did something good actually the carbon emissions per parameter reduced. Then you did something. Maybe it was bad. Maybe I, some, maybe it was necessary. Maybe it was architectural. But for some reason your emissions,
Chris Adams: You became massively less efficient in the set, in that
next
Asim Hussain: In terms of carbon. In terms of carbon, you became a lot less efficient in GPT-4. We have no information about GPT 5. I hope it's less than 20 metric tons per billion parameters.
Chris Adams: I think I'm starting to wanna step, follow your argument and I'm not, I'm not gonna say I agree with it or not, but I, the, I think the argument you're making is essentially by switching from, you know, that that in itself is a useful signal that you can then do something with. there was maybe like a regression or a bug that happened in that one that you can say, well, what change that I need to do so I can actually start working my way towards, I don't know, us careering less forcefully towards oblivion, for example, or something like that.
Right.
Asim Hussain: Yeah.
Chris Adams: Okay. That makes, I think I understand that now. And, let's, and I suppose the question I should ask from following on from that is that this is, some of this is, we're talking about, we got into this, 'cause we were talking about the SCI for AI, this kind of standard or presumably an ISO standard that we published.
Is there a kind of rough like roadmap for when this is gonna be in the public domain, for example, or people might be requesting this in commercial agreements or something like that?
Asim Hussain: I mean, I can tell you what my hope is. So I think, I mean, cause everything is based upon consensus and if anybody objects then everything or all the plans basically, you know, put on the back burner. But everything's looking very positive. I'm very hopeful that by the end of Q3, so the end of September, we will have gone into draft and then, there hasn't been a full agreement yet as to what we'll actually publish for that. But I'm hoping we'll be able to actually publish the whole specification, because what we wanna start doing is get, I mean this maybe if anybody's interested, we wanna start running case studies because right now it's like the outline of what we want the calculation to be is being agreed on.
But we need a lot of use cases of very different types of products that have computed using it. Not just, you know, I'm a major player and I've got a gazillion servers and we also want, need people, there's lots of organizations we're talking to or listen, we've just, we are, AI is not our central business, but we've built like AI solutions internally and we want to be able to measure that.
Or even smaller organizations or people who are not even training in AI, but just consuming APIs then build like an AI solution on top of that. So there's like a whole range of things that we wanna measure and we want to publish, go into draft in September, and then work on a number of case studies. Hopefully, dream,
my dream, and I, no one holds me to this is by kind of Q1, Q2 next year where we're out and we start the ISO process then, but when we come out, we want to come out with here's a specification. It'll come out with a training course that you can take to learn how to compute the specification. It will come out with a tooling.
So you can just plug in values and then you'll be able to get your numbers and also come out with a number of case studies from organizations who, this is how exactly we calculated it, and maybe you can learn from, how we did it. So that's our goal.
Chris Adams: Okay, well that, so we're looking at basically, okay, first half of 2026, so there's still time to be involved and there's, and presumably later on in Q3, Q4, some of this will be going out in public for people to kind of respond to or have this some, something like the consultation there.
Asim Hussain: Yeah, It'd be a public consultation coming up soon.
Chris Adams: This is useful to know because this takes it to our last story we were looking at, which is actually also talking about the challenges related to the working on the environmental footprint of other things, particularly websites.
This is our final link of the podcast, which is a link to, the IEEE, where there's a post by, I believe it's Janne Kalliola. And, oh dear. I'm not gonna pronounce the other person's name very well. Juho Vepsäläinen. Oh dear. I'm so sorry for mispronouncing your names. I'm drawing attention to this 'cause this is the first time In a while I've seen a peer reviewed article in the IEEE specifically, which I think is the.
It's the Instutute of Electrical and Electronics Engineers. I forget what it stands for. Yes, thank you. They looked at both, Firefox Profiler and Website Carbon. They basically started looking at the environmental footprint, what kind of, what does using these website calculators actually tell you and what can you use?
And they had some recommendations about, okay, we've tried using these tools, what can we learn from that? And the thing that was actually particularly interesting was that they were using Firefox's Firefox profiler specifically to look at the footprint of, they're basically saying that there's two, three insights that have probably come away from this, which I thought was interesting.
One of them, it's really hard to get meaningful numbers around data transfer, which I think is actually something that we've shared and we've covered in a lot of detail and I'm finding very helpful for future discussions around creating something like a software, carbon intensity for Web for this.
But the other thing they did was they spoke about the use of, like tools out there, like profilers, which do provide this direct measurement that does give you some meaningful numbers. But when you look at the charts, the differences aren't that high. For example, they were showing comparisons with things like website carbon, which shows massively different, massively different kind of readings for the carbon footprint of one site versus another.
And then when they used other tools like say Firefox Profiler, the differences were somewhat more modest between these two things. So this kind of gives the impression that tool, some of the tools that use like the soft, the sustainable web design model may, they may be overestimating the effectiveness of changes you might be making as an engineer versus what gets measured directly.
Now, there's obviously a elephant in the room and that this isn't measuring what's happening server side, but this is the first time I've seen a really, kind of a real deep dive by, some people who are actually looking into this to come up with some things you can, you can test, I suppose, or you can kind of, you can like, reproduce to see if they get, you're getting the same numbers from these people here.
And, this is actually quite a useful, it's, I found it quite noteworthy and really nice to see and I would've found out about it because, Janne actually shared it inside the Climateaction.tech Slack.
Asim Hussain: So it was a paper inside IEEE or, an article inside that
Chris Adams: It's, a paper. So it's a peer reviewed paper in volume 13 of IEEE and they basically, they talk about the current state of the art, how people currently try to measure energy consumption on the Web. Then they talk about some of the tools you can use for the end user devices. Talk about some of the issues related to trying to go on just data transfer alone and why that isn't necessarily the best thing to be using, but, what kind of statements you could plausibly make.
But as someone who ends up, you know, we, the organization I work for, we implemented the sustainable web design model for this. Having something like this is so, so useful because we can now cite other peer reviewed work that's in the public domain that we can say, hey, we need to update this, based on this, or possibly do some, or an idea, which I believe that Professor Daniel Sheen shared with me.
He said, well, if we know, if we've got figures for the top million websites, the top thousand websites, maybe you could actually just experimentally validate those versus what you have in the, in a model already. So you can get better numbers for this. There's a bunch of steps. Yeah, exactly. If you were to measure the top thousand ones compared to the model figures, then that will give you an idea of the gap between the model figure and the ground truth, so you can end up with a slightly better, a better figure.
There's a bunch of things that you could do out there, which would, might make it easier to make these, this tooling much, much easier to use and much more likely to give people the signals they are craving to kind of build websites in a more kind of climate compatible fashion, for example.
Asim Hussain: And I think it's important because I think people like when you use a, when you use a tool and it gives you a, it gives you a value, it's incentivizing a behavior. And it might be incentivizing the wrong behavior. And it's, and I think that's one of the things I find that when people get excited about a measurement, I don't, because I'm, I need to know the details behind it.
'Cause I know that if you're a little bit wrong, you're incentivizing the wrong thing. And you shouldn't just, you shouldn't just take it face value. But it's really hard. I also, in the sense it's really bloody hard even for the tool makers to even figure out what to do here.
So this isn't really a, you know, but it's not really criticism of anybody. But, yeah, it's just really hard to figure this stuff out. But the Firefox stuff is using yours isn't, it's using CO2.js, isn't it?
Chris Adams: I'm not sure if this actually uses the carbon figures we use 'cause we're just, we basically package up the numbers from Ember, which is a non-profit think tank who already published stuff. I can't remember if this one is actually use using the energy or the carbon figures basically.
But we update the carbon figures, every month anyway. So it may, it might be our, I'll need to kind of check if they measure in terms, if they, I think they report this in energy, not carbon actually. It's what they used inside this.
Actually, I'll need to reread and we're coming up to time actually.
Asim Hussain: Here we come time, so this, but also I think maybe just call out a little bit. So we are gonna be running the, and you are leading it, the SCI for Web assembly shortly in the foundation. And I think this is, this can be a very, this looks, my brief scan of it, like a very important pre-read, I presume for a lot of the people who are gonna be attending that assembly.
Chris Adams: Yeah, I'm actually really pleased this came out. That was initially what I saw, oh great, this is a really nice, concise piece that covers this. This was another piece from Daniel Sheen talking about, okay, well how do you measure network figures, for example? 'cause he's put some really, good interesting stuff inside that we don't have enough time to talk about, but it's a really, but we'll share links to that inside that because yes, this is something that we'll be doing and I'm looking forward to doing it.
And oh, I've just realized we've gone way over.
Asim Hussain: We're well over. You've gotta go, on. Let's just, let's wrap
Chris Adams: Dude, really lovely catching up with you again. Oh, the final thing I need to give is this, just quickly talking about this GSM, the Green Software Movement thing that you were talking about here. Maybe I can just give you space to do that before we cl before we wrap up.
'Cause I know this is the project you're working on at the moment.
Asim Hussain: Yeah. So the movement is a platform that we've created, so it's movement.greensoftware.foundation. So this is where we, will be putting a lot more of our tension moving forward in terms of engaging with the broader community. It's also where all of our training is going to be.
So our current training is moving over there, and we just now have a, now that we've got like a real platform to publish training to. We're gonna get training for all of our products and services, so for SCI, Impact Framework, SOFT, RTC. We're gonna do training for all of them and have them available on the platform.
And you'll be able to go in, you'll be able to learn about the products that we've created, learn about the foundation, get certified for your training. But also it's a platform where you can connect with other people as well. So you can meet people, have chats, have conversations, connect with people who are local to you.
We've had over 130,000 people take our previous training, which unfortunately is on a previous, another platform. So we're gonna be trying to move everybody over. So hopefully our goal is ultimately for this to be the platform where you go, at least from terms of the Green Software Foundation to learn about our products, our standards get involved would be, our champions programs moving over there as well.
And we're just kind of like having, this will be where we put a lot of our effort moving forward, and I recommend people go to it, join, sign up, take the training, and connect with others.
Chris Adams: Alright. Okay. Well, Asim, lovely catching up with you. And I hope you have a lovely rest of the week. And I guess I'll see you in the Slacks or the Zulips or whichever online tools we use to across paths.
Asim Hussain: Zulips. I don't know what that is. Yeah. Sounds good. right, mate.
Chris Adams: our open source chat tool inside the Green Web Foundation. It runs on Django and it's wonderful.
Yeah, it's really good. I cannot recommend it enough. If you are using Slack and you are sick of using Slack, then use Zulips. Zulips is wonderful. Yeah. It's really, good.
Asim Hussain: I can check it out. Yeah. All right.
Chris Adams: Take man. See you Bye.
Asim Hussain: Bye.
Chris Adams: Hey everyone, thanks for listening. Just a reminder to follow Environment Variables on Apple Podcasts, Spotify, or wherever you get your podcasts. And please do leave a rating and review if you like what we're doing. It helps other people discover the show, and of course, we'd love to have more listeners.
To find out more about the Green Software Foundation, please visit greensoftware.foundation. That's greensoftware.foundation in any browser. Thanks again, and see you in the next episode.
118 episodes
All episodes
×Welcome to Player FM!
Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.