Hi @rootfs, I've been working with @ross7 on the grid intensity go SDK thing. I've tried to provide some more background to the answers

[swaradgat19]

    Hi @rootfs, I've been working with @ross7 on the grid intensity go SDK thing. I've tried to provide some more background to the answers

How does the carbon API help? If all the deployments are in the same data center/same geo, does carbon intensity change the anything in compute? It looks to me the best use case is the multi-cluster, multi data center case (see some prior study here )

The above example works by moving workloads geographically (as in, it moves them through space).

You can also move workloads temporally (as in move them through time).

The carbon intensity changes based on the time of day, so the same workload run at different times will have different emissions figures.

The issue referred to one paper titled Let's Wait Awhile: How Temporal Workload Shifting Can Reduce Carbon Emissions in the Cloud, and it's a fun read, going into this in more detail.

Last month at the recent ACM SIGEnergy workshop, there was a talk from some folks at VMware sharing some new findings, called Breaking the Barriers of Stranded Energy through Multi-cloud and Federated Data Centers. It's really worth a watch but this quote from the abstract gives an idea of why the time element is worth being able to act upon:

many computation workloads (such as some learning or big data) can be flexible in time (scheduled for delayed execution) and space (transferred across any geographical distance with limited cost). This opens the possibility of shifting workloads in time and space to take advantage in real time of any amount of excess renewable energy, which otherwise would be curtailed and wasted. Initial results show that a single datacenter that time shifts load can reduce its emissions by 19% or more annually

There's also some work by Facebook/Meta, where they have shared some results from using this same carbon aware workload scheduling as part of their sustainabilty strategy - see their recent carbon explorer repo. I think they might use their own scheduler, rather than Kubernetes, but the principle is the same - move work through space to make the most of cheaper green energy for your compute.

Does it make sense to schedule every deployment or only selectively pick those that have a high energy consumption, thus high carbon impact ones? That brings the question of how to measure the energy consumption of deployments.

For the suitability question, that's down to the person running the cluster, and the job. Some jobs are better fits for moving through time (low latency, pause-able jobs), and some jobs better for moving through space (ones that don't have to be run within a specific jurisdiction). These are somewhat independent of the energy consumption. If you're curious about the the energy consumption part, I think Scaphandre provides some numbers you can use and labelling of jobs for k8s, and this piece here from the BBC gives an example of it in use.

Hope that helps!

Originally posted by @mrchrisadams in #3467 (comment)

[swaradgat19]

Hey. Can you provide a link to the paper titled "Breaking the Barriers of Stranded Energy through Multi-cloud and Federated Data Centers". I 'm unable to find it anywhere on the internet. It would be really useful if you do!

[mrchrisadams]

hey folks, I've asked to see if it's published anywhere - it might be going through peer review, or be scheduled to be published in the near future.

[mrchrisadams]

OK, I heard back - here's the closet thing:

Hi Chris, the “Breaking the Barriers of Stranded Energy through Multi-cloud and Federated Data Center” talk we gave at ACM e-Energy was an invited keynote, so there’s no paper specifically tied to it. We are working on something that we hope to submit to a conference in early 2023!

The only peer-reviewed work we have at the moment is the short HotCarbon paper.

There's a bunch here too:
https://podcast.greensoftware.foundation/e/xnvm3598-from-carbon-aware-to-carbon-intelligent

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.