How could I reflect 'commodity' set in the 'emission_factor' parameter? (for specific model)

[MESSAGEix-SK-Woo]

Hi, I'm currently building an Industry sector model in korea using MESSAGEix.

It's more specific model than global model, as a result, many of the technologies have multi inputs.

ex) kiln tec has various input such as "raw_material, coalB, biomass, waste, elec, .... ". but output is only 'clinker'

but some sets('var_cost' or 'emission_factor', ....) don't have 'commodity' set in their code.

I would like to reflect a 'commodity' set in these parameters.

and, To revise GAMS code is not to difficult. 'cuz I know how use it.

Although revising the GAMS code is not difficult for me, as I know how to use it, I found that even after adding a 'commodity' in the 'emission_factor' parameter, the python code did not reflect this change.

Here's what I did:

1. I changed the GAMS code (emission_factor, added commodity).
2. However, it did not reflect in the Python code. When I checked using "scenario.par('emission_factor')" in Python, I noticed that there wasn't a 'commodity' set in the 'emission_factor' parameter.

I suspect that this may be a Python issue.
Can you please help me to reflect the 'commodity' in the 'emission_factor' parameter?
I would really appreciate your help.

Thank you for your time, and I look forward to hearing back from you soon.

[glatterf42]

Hi @MESSAGEix-SK-Woo, thanks for your question! I think there is a fundamental misunderstanding here: message_ix uses ixmp in the background, with which you can connect to a database (via the ixmp.Platform()). This database contains the data for different scenarios, accessible via message_ix.Scenario(). You are probably working on one of them in your code!
Scenarios are used to handle different kind of model data, among them parameters. Following the baseline westeros tutorial, if you set up your scenario as

import pandas as pd
import ixmp
import message_ix

mp = ixmp.Platform()
scenario = message_ix.Scenario(
    mp, model="Westeros Electrified", scenario="baseline", version="new"
)

you can add parameters to the scenario like this:

# Define the time frame of the model
history = [690]
model_horizon = [700, 710, 720]
scenario.add_horizon(year=history + model_horizon, firstmodelyear=model_horizon[0])

# Define the locations of the model
country = "Westeros"
scenario.add_spatial_sets({"country": country}) 

# Define possible values of some categories used to define parameters
scenario.add_set("commodity", ["electricity", "light"])
scenario.add_set("level", ["secondary", "final", "useful"])

# Assume a GDP profile
gdp_profile = pd.Series([1.0, 1.5, 1.9], index=pd.Index(model_horizon, name="Time"))

# Estimate baseline demand
demand_per_year = 40 * 12 * 1000 / 8760
light_demand = pd.DataFrame(
    {
        "node": country,
        "commodity": "light",
        "level": "useful",
        "year": model_horizon,
        "time": "year",
        "value": (demand_per_year * gdp_profile).round(),
        "unit": "GWa",
    }
)

# Use `add_par` for adding data to a MESSAGEix parameter
scenario.add_par("demand", light_demand)

The interesting part is that you can use scenario.add_par() to change existing parameter data as well. So if you were to define a new commodity for the light_demand and want to overwrite the existing light_demand, you would also define a new pd.DataFrame (e.g. called new_light_demand and store that in the scenario using scenario.add_par("demand", new_light_demand). Afterwards, scenario.par("demand") should return the updated parameter data.

[MESSAGEix-SK-Woo]

Thanks for your answer @glatterf42 !

I think my descrption is not enough.

I'll ask in the examples.
I want to change

light_demand = pd.DataFrame(
    {  "node": country,
        "commodity": "light",
        "level": "useful",
        "year": model_horizon,
        "time": "year",
        "value": (demand_per_year * gdp_profile).round(),
        "unit": "GWa",
    }
)

to below one.

light_demand = pd.DataFrame(
    {
        "node": country,
        "commodity": "light",
        "level": "useful",
        "technology" : '~~", 
        "year": model_horizon,
        "time": "year",
        "value": (demand_per_year * gdp_profile).round(),
        "unit": "GWa",
    }
)

I add the 'technology' set in the demand.

but it doens't apply. because gams code is like this. (there isn't 'technology' in the DEMAND)

Variables
* auxiliary variables for demand, prices, costs and GDP (for reporting when MESSAGE is run with MACRO)
    DEMAND(node,commodity,level,year_all,time) demand

so I change gams code like this. and also change 'demand' Related Code

Variables
* auxiliary variables for demand, prices, costs and GDP (for reporting when MESSAGE is run with MACRO)
    DEMAND(node,commodity,level,technology,year_all,time) demand

but it doesn't work. because 'demand' in python is not changed.

I would really appreciate your help!!

Thanks :)

[khaeru]

Hi @MESSAGEix-SK-Woo —

First, I've converted the "issue" you opened to a "discussion" in the "MESSAGEix modeling" section. This is the appropriate place for Q&A related to your individual modeling efforts, especially since there is no particular bug in the code per se.

Second, directly to your question, there is indeed a place in the Python code where the dimensionality of each parameter is recorded, particularly here:

message_ix/message_ix/models.py

Line 176 in 248bcbd

"emission_factor": item("par", "nl t yv ya m e"),

—you would need to install the code from source and manually adjust this line to add the extra dimension, corresponding to whatever changes you have made in the GAMS code. (Since you say you are familiar with GAMS, I assume you also know that you need to modify all the places in the GAMS code where emission_factor is used; not merely its definition.) You haven't shared your modified GAMS code, so I can't provide more specific guidance than that.

Third, however, I question whether you have understood the existing features of the model. Maybe if you use these fully, it is not necessary at all to expand the dimensionality of emission_factor.

In particular, emissions are computed as:
$ACT \times \mathrm{ef}$

With the full dimensions of this parameter and variable:
$ACT_{n^L,t,y^V,y^A,m,h} \times \mathrm{ef}_{n^L,t,y^V,y^A,m,e}$

For simplicity, let's consider only one technology (your $t = \text{'kiln'}$), in one node ($n^L$), vintage ($y^V$), and period of activity ($y^A$), and assume you are using a model without time slices, so ($h = \text{'year'}$):
$ACT_{m} \times \mathrm{ef}_{m,e}$

This means for every 1 unit of $ACT$ for $t = \text{'kiln'}$, a different amount of emissions can be specified for multiple emissions species ($e$). For any one choice of $(t, n^L, y^V, y^A, m)$, this value is fixed.

Now this technology would also have a corresponding input efficiency:
$input_{n^L,t,y^V,y^A,m,n^O,c,l,h,h^O}$
again simplified to:
$input_{m,c,l}$

What this means is that, for every 1 unit of $ACT$, a different amount of commodity input can be specified for multiple commodities ($c$).

As you say, it's possible for it to have multiple inputs by setting distinct values for various $c \in \{\text{'raw material'}, \text{'coalB'}, \text{'biomass'}, \text{'waste'}, \ldots\}$.

For an example, let's suppose that in a particular mode ($m$) of operation, the kiln uses 0.2 units of coalB and 0.3 units of biomass per unit of ACT. You will have data on the emissions intensity (call this $ei_e$) of these commodities (say, for the species CO₂) individually. Then it is simple to compute that the emission_factor for one unit of ACT is $0.2 * ei_{\text{coalB}} + 0.3 * ei_{\text{biomass}}$.

This value will always be valid given the corresponding input data, which cannot change within the optimization. If you wanted to represent a different situation where the technology used, for instance, 0.4 units of coalB and 0.1 units of biomass per unit of ACT, you would need to enter these values for a different technology $t$ or mode $m$, and then you should calculate the corresponding emission_factor. The expression in the previous paragraph would not be valid for these different input efficiencies.

Please let me know if this explanation is unclear. It doesn't appear so from your original question, but please be more specific if there is something else in particular you are trying to achieve, and you think that the features as described are not sufficient to model it.

[MESSAGEix-SK-Woo]

First, thanks for helping me improve my github literacy,
For the second answer, thank you for clarifying my questions,
For the third answer, I really appreciate your kindness and professionalism.

Here are my answers.

First, as you mentioned, I solved the problem by reflecting commodity in 'emission' in emission_factor.

• emission_factor(location,tec,vintage,year,mode,emission)
• emission: below it.

emissions_set = [
#CO2 by commodity     
   'CO2_limestone', 
   'CO2_clinker', 
   'CO2_slag', 
   'CO2_crushed_limestone', 
   'CO2_clay', 
   'CO2_iron_material',
   'CO2_combinded_material',
   'CO2_coalB',
   'CO2_fueloil', 
   'CO2_biomass', 
   'CO2_elec', 
    .....

So I tried to build the model without modifying GAMS.

However, the problem is with var_cost. My model applied fuel cost to var_cost.
If it is an integrated model, the fuel cost(=var cost) will be endogenously determined and reflected in the model at the primary level, but the current model is not an integrated model.
Therefore, we need to reflect var_cost exogenously.

So, I checked var_cost.

The var_cost is var_cost(node,tec,vintage,year,mode,time), and there is no set to reflect 'commodity' like 'emission' in the emission_factor parameter.
I think I need to change the Python code and GAMS code for this, so I asked.
(Since I'm interested in the messageix model, I thought it would be good to know in order to better utilize it).

Secondly, I would like to respond to your comment that even if it is a MULTI-INPT, you can calculate and reflect the emission_factor by weighted average.
The multi-input contains fuel(elec, coal, ...) and raw_material(limestone, ..) together. The units are also different: tco2/toe, tco2/ton.

So I thought it was necessary to split them out, if only to distinguish between raw materials and fuels.
(Of course, if the input consists only of fuel, I think it would be sufficient to have just one emission_factor, calculated as a weighted average).

I think it would be okay not to modify the python code for the sake of simplicity of the model.

I think this is something I need to think about a bit more, but I'd like to practice modifying the python code and GAMS code as an exercise to get better at using MESSAGEix.

I'm not sure if I've answered your question correctly, so please correct me if I'm wrong.

I know you must be very busy, but again, thank you for your kind and detailed response.

[khaeru]

Here's another example:

• A technology $t = \text{'baker'}$ produces $c = \text{'apple pie'}$.
• The activity ($\text{ACT}$) of this technology is measured in units of [hour].
• The output efficiency for $c = \text{'apple pie'}$ is measured in [pie / hour]; so $\text{ACT} \times \text{output}$ gives [hour] × [pie / hour] = [pie] (in the current period $y$).
• For instance, suppose the output efficiency is 2.0 [pie / hour]. If the baker works 3.5 hours ($\text{ACT}$), the total output (note: the total, not the efficiency) is 3.5 [hour] × 2.0 [pie / hour] = 7.0 [pie].
• The baker has (for simplicity) two inputs:
  • $c = \text{'apple'}$; the input efficiency for this commodity is measured in [apple / hour], say 4.0 [apple / hour].
  • $c = \text{'flour'}$; the input efficiency for this commodity is measured in [gram / hour], say 800 [gram / hour].
  • So if ACT is 3.5 [hour], then the total inputs are 3.5 × 4.0 = 14.0 [apple] and 3.5 × 800 = 2800 [gram] of flour.
  • (I leave it as an exercise, but if you knew that there were 2.0 [apple / pie], then you could also derive the input efficiency in [apple / hour] and find the same total input. We often see values like this 2.0 that are neither the input nor output efficiency.)
• Now suppose "emissions" are food waste: the apple peels and spilled flour. We're interested in the total mass of food waste across different input with different units. Let's say:
  • The peel of 1 apple weighs 20 g.
  • The baker spills 2% of the flour they use.
• Then for 1.0 hour of baking:
  • 4.0 [apple / hour] × 20 [g / apple] = 80 [g / hour] in "emissions" from apple inputs.
  • 800 [g / hour] × 0.02 = 16 [g / hour] in "emissions" from flour inputs.
• Thus, we can enter a MESSAGEix emission_factor of 80 + 16 = 96 [g / hour] for $e = \text{'food waste'}$.
• A baker who bakes for 3.5 hours will generate 3.5 [hour] × 96 [g / hour] = 336 [g] of food waste.

As you can see, having input and output efficiencies in different physical units does not prevent computing a total emission factor, so long as the emissions (here food waste) are measured in the same units. (I would not call this a “weighted average”, since we don't divide by a total count as we would when computing an average.)

Keeping the units/dimensions and checking the dimensions of these computed quantities is a good habit to ensure you've done the calculations correctly.

[MESSAGEix-SK-Woo]

The model I was using was a GAMS model with the following equations.

ACT * (Input / Output) * Emission_Factor 
ACT * (Input / Output) * Var_cost
...

Since the emission factor and var_cost are reflected for each commodity by tec, I didn't need to calculate the value by tec separately.

However, since the MESSAGEix model is a global model, I think it is configured without commodity in consideration of model running time, etc.

Using your method, I will try to utilize both emission factor and var_cost(and other parameters) as the parameter set defined in MESSAGEix.

Thanks for your kindness as always @khaeru