diff --git a/_images/rmcat.png b/_images/rmcat.png index 86c2bf6..9b6a9ae 100644 Binary files a/_images/rmcat.png and b/_images/rmcat.png differ diff --git a/_sources/environments/congestion_control.md.txt b/_sources/environments/congestion_control.md.txt index 481b6ba..62f9f78 100644 --- a/_sources/environments/congestion_control.md.txt +++ b/_sources/environments/congestion_control.md.txt @@ -16,7 +16,7 @@ congestion_control/rmcat :link: cards-rmcat :link-type: ref -In this environment, the agent performs periodic actions to control the sending rate of a real-time traffic RMCAT (NADA) flow. +In this environment, the agent performs periodic actions to adjust the sending rate of a real-time traffic flow using RTP Media Congestion Avoidance Techniques (RMCAT). ::: :::: diff --git a/_sources/environments/congestion_control/rmcat.md.txt b/_sources/environments/congestion_control/rmcat.md.txt index f9a0d31..5650d82 100644 --- a/_sources/environments/congestion_control/rmcat.md.txt +++ b/_sources/environments/congestion_control/rmcat.md.txt @@ -5,14 +5,15 @@ title: RMCAT # RMCAT ```{figure} rmcat.png --- -width: 50% +width: 100% --- ``` [config.json]: https://github.com/IntelLabs/networkgym/blob/main/network_gym_client/envs/rmcat/config.json The RMCAT environment is part of the congestion control environments and provides general information about the environment. -In this environment, the agent performs periodic actions to control the sending rate of a real-time traffic RMCAT (NADA) flow. +In this environment, the agent performs periodic actions to adjust the sending rate of a real-time traffic flow using RTP Media Congestion Avoidance Techniques (RMCAT). +This environment is based on the GitHub Repo: [cisco/ns3-rmcat](https://github.com/cisco/ns3-rmcat). | | | | ----- | ---- | @@ -23,7 +24,7 @@ In this environment, the agent performs periodic actions to control the sending ## Description -The RMCAT environment represents a congestion control problem for real-time traffic. The goal of congestion control is to strategically controls sending rate, aiming to achieve high throughput, low latency and low loss. +The RMCAT environment represents a congestion control problem for real-time traffic. The goal of congestion control is to strategically control the sending rate, aiming to achieve high throughput, low latency, and low loss. If no action is given to the environment, RMCAT uses [NADA](https://datatracker.ietf.org/doc/html/draft-ietf-rmcat-nada-05) as its default algorithm. ## Prerequisite @@ -31,27 +32,9 @@ Ensure that you have access to the NetworkGym Server on [vLab](https://registrat ## Observation Space -The observation is an `ndarray` with shape `(3,)` representing three features. The first feature is `rtt`, representing the round trip time. The second feature is `xcurr`, represeting the congestion level. The third feature is the received throughput `rrate`. +The observation is an `ndarray` with shape `(3,)` representing three features selected from the available measurements for this environment. -| Feature | Observation | Min | Max | -| ----- | ---- | ----- | ---- | -| 0 | rtt (ms) | 0.0 | Inf | -| 1 | xcurr | 0.0 | 500.0 | -| 2 | rrate (bps) | 0.0 | Inf| - -## Action Space -The action space is a `ndarray` with shape `(1,)` representing the sending rate of rmcat flow. The min value is 150000 and max value is 1500000. - - -## Reward - -TODO - -## Custom Observation Space and Reward -1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. -To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `rmcat::rtt` to the list would enable receiving measurements with the name `rtt` from the source `rmcat`. - -:::{dropdown} Click to view a demo of all measurements for this environment. +:::{dropdown} Click to view available measurements for this environment. Refer to [Working with Measurements](../../tutorials/working_with_measurements.md) for further elaboration of the following measurements. @@ -72,6 +55,25 @@ Refer to [Working with Measurements](../../tutorials/working_with_measurements.m ::: +The first feature is `rtt`, representing the round trip time. The second feature is `xcurr`, represeting the congestion level. The third feature is the received throughput `rrate`. + +| Feature | Observation | Min | Max | +| ----- | ---- | ----- | ---- | +| 0 | rtt (ms) | 0.0 | Inf | +| 1 | xcurr | 0.0 | 500.0 | +| 2 | rrate (bps) | 0.0 | Inf| + +## Action Space +The action space is a `ndarray` with shape `(1,)` representing the sending rate of rmcat flow. The min value is 150000 and max value is 1500000. + + +## Reward + +Not implemented yet. + +## Customize Observation Space and Reward + +1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `rmcat::rtt` to the list would enable receiving measurements with the name `rtt` from the source `rmcat`. ```{eval-rst} 2. Customize Observation Space in the :meth:`network_gym_client.envs.rmcat.Adapter.get_observation` function. 3. Customize Reward in the :meth:`network_gym_client.envs.rmcat.Adapter.get_reward` function. diff --git a/_sources/environments/mx_network_slicing/cellular_network_slicing.md.txt b/_sources/environments/mx_network_slicing/cellular_network_slicing.md.txt index 732c103..9bbaef3 100644 --- a/_sources/environments/mx_network_slicing/cellular_network_slicing.md.txt +++ b/_sources/environments/mx_network_slicing/cellular_network_slicing.md.txt @@ -33,53 +33,9 @@ Ensure that you have access to the NetworkGym Server on [vLab](https://registrat ## Observation Space -The observation space is represented by a multidimensional array (`ndarray`) with a shape of `(5, N)`, where each of the `N` slices contains five distinct features: -1. **Total Throughput**: This measures the total data rate for each slice in Mbps, ranging from `0` to `Inf`. -2. **Resource Block Usage Rate**: It represents the fraction of resource blocks utilized by each slice, expressed as a percentage in the range of `[0, 1]`, with the sum across all slices equal to 1. -3. **Average Delay Violation Rate**: Initially measured user-wise and then averaged for each slice, this rate indicates the average proportion of delay violations, expressed as a percentage in the range of `[0, 1]`. -4. **Maximum Delay**: This records the maximum delay experienced by a slice at a specific moment, expressed in milliseconds and ranging from `0` to `Inf`. -5. **Average Delay**: It computes the average delay experienced by a slice, expressed in milliseconds and ranging from `0` to `Inf`. - -| Feature | Observation | Min | Max | -| --- | --- | --- | --- | -| 0 | Total Throughput per slice (mbps) | 0.0 | Inf | -| 1 | Resource Block Usage Rate per slice | 0.0 | 1.0 | -| 2 | Average Delay Violation Rate per slice | 0.0 | 1.0 | -| 3 | Maximum Delay per slice (ms) | 0.0 | Inf | -| 4 | Average Delay per slice (ms) | 0.0 | Inf | - - -## Action Space - -The action space is an `ndarray` with a shape of `(N,)`, indicating the allocation of resource blocks to `N` slices. It is important to note that the sum of the actions in this allocation should not exceed one. In case the sum surpasses one, the actions will be appropriately scaled down using the softmax function to ensure adherence to the constraint. This can be expressed mathematically as follows: -```{eval-rst} -if :math:`\sum_{i=1}^{N}a_i > 1, \quad a_i = \frac{e^{a_i}}{\sum_{j}e^{a_j}}.` -``` - -| Num | Action | Min | Max | -| --- | --- | --- | --- | -| 0 | Prioritized Rb allocated to slice 0 | 0.0 | 1.0 | -| 1 | Prioritized Rb allocated to slice 1 | 0.0 | 1.0 | -| ... | | | | -| N | Prioritized Rb allocated to slice N | 0.0 | 1.0 | - -## Reward - -The reward function in our system takes into account three factors: throughput, delay violation rate, and resource cost. It is formulated as follows: - -```{eval-rst} -:math:`R(t) = \sum_{i = 1}^N \left( \frac{{\text{throughput}_i}}{{\text{load}_i}} - \lambda \cdot \text{delay_violation_rate}_i - \gamma \cdot \text{rb_usage_rate}_i \right)` -``` - -```{eval-rst} -In this formulation, we can adjust the emphasis given to the delay violation rate and the resource block usage rate using the parameters (:math:`\lambda`) and (:math:`\gamma`) respectively. By default, (:math:`\lambda=1.5`) and (:math:`\gamma = 0.5`), which prioritize delay considerations. -``` - -## Custom Observation Space and Reward -1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. -To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::wifi::dl::traffic_ratio` to the list would enable receiving measurements with the name `wifi::dl::traffic_ratio` from the source `gma`. +The observation space is represented by a multidimensional array (`ndarray`) with a shape of `(5, N)`, where each of the `N` slices contains five distinct features selected from the available measurements for this environment: -:::{dropdown} Click to view a demo of all measurements for this environment. +:::{dropdown} Click to view available measurements for this environment. Refer to [Working with Measurements](../../tutorials/working_with_measurements.md) for further elaboration of the following measurements. @@ -129,6 +85,50 @@ Refer to [Working with Measurements](../../tutorials/working_with_measurements.m ::: +1. **Total Throughput**: This measures the total data rate for each slice in Mbps, ranging from `0` to `Inf`. +2. **Resource Block Usage Rate**: It represents the fraction of resource blocks utilized by each slice, expressed as a percentage in the range of `[0, 1]`, with the sum across all slices equal to 1. +3. **Average Delay Violation Rate**: Initially measured user-wise and then averaged for each slice, this rate indicates the average proportion of delay violations, expressed as a percentage in the range of `[0, 1]`. +4. **Maximum Delay**: This records the maximum delay experienced by a slice at a specific moment, expressed in milliseconds and ranging from `0` to `Inf`. +5. **Average Delay**: It computes the average delay experienced by a slice, expressed in milliseconds and ranging from `0` to `Inf`. + +| Feature | Observation | Min | Max | +| --- | --- | --- | --- | +| 0 | Total Throughput per slice (mbps) | 0.0 | Inf | +| 1 | Resource Block Usage Rate per slice | 0.0 | 1.0 | +| 2 | Average Delay Violation Rate per slice | 0.0 | 1.0 | +| 3 | Maximum Delay per slice (ms) | 0.0 | Inf | +| 4 | Average Delay per slice (ms) | 0.0 | Inf | + + +## Action Space + +The action space is an `ndarray` with a shape of `(N,)`, indicating the allocation of resource blocks to `N` slices. It is important to note that the sum of the actions in this allocation should not exceed one. In case the sum surpasses one, the actions will be appropriately scaled down using the softmax function to ensure adherence to the constraint. This can be expressed mathematically as follows: +```{eval-rst} +if :math:`\sum_{i=1}^{N}a_i > 1, \quad a_i = \frac{e^{a_i}}{\sum_{j}e^{a_j}}.` +``` + +| Num | Action | Min | Max | +| --- | --- | --- | --- | +| 0 | Prioritized Rb allocated to slice 0 | 0.0 | 1.0 | +| 1 | Prioritized Rb allocated to slice 1 | 0.0 | 1.0 | +| ... | | | | +| N | Prioritized Rb allocated to slice N | 0.0 | 1.0 | + +## Reward + +The reward function in our system takes into account three factors: throughput, delay violation rate, and resource cost. It is formulated as follows: + +```{eval-rst} +:math:`R(t) = \sum_{i = 1}^N \left( \frac{{\text{throughput}_i}}{{\text{load}_i}} - \lambda \cdot \text{delay_violation_rate}_i - \gamma \cdot \text{rb_usage_rate}_i \right)` +``` + +```{eval-rst} +In this formulation, we can adjust the emphasis given to the delay violation rate and the resource block usage rate using the parameters (:math:`\lambda`) and (:math:`\gamma`) respectively. By default, (:math:`\lambda=1.5`) and (:math:`\gamma = 0.5`), which prioritize delay considerations. +``` + +## Custom Observation Space and Reward +1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. +To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::lte::dl::rate` to the list would enable receiving measurements with the name `lte::dl::rate` from the source `gma`. ```{eval-rst} 2. Customize Observation Space in the :meth:`network_gym_client.envs.network_slicing.Adapter.get_observation` function. 3. Customize Reward in the :meth:`network_gym_client.envs.network_slicing.Adapter.get_reward` function. diff --git a/_sources/environments/mx_traffic_management/mx_qos_traffic_steering.md.txt b/_sources/environments/mx_traffic_management/mx_qos_traffic_steering.md.txt index 2690c41..4e5c8cf 100644 --- a/_sources/environments/mx_traffic_management/mx_qos_traffic_steering.md.txt +++ b/_sources/environments/mx_traffic_management/mx_qos_traffic_steering.md.txt @@ -45,33 +45,9 @@ Ensure that you have access to the NetworkGym Server on [vLab](https://registrat ## Observation Space -The observation is an `ndarray` with shape `(3,N,)` representing three features for N users. The first feature is MAX LTE rate, representing the user's estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links. +The observation is an `ndarray` with shape `(3,N,)` representing three features for N users selected from the available measurement for this environment. -| Feature | Observation | Min | Max | -| ----- | ---- | ----- | ---- | -| 0 | MAX LTE rate (mbps) | 0.0 | Inf | -| 1 | MAX Wi-Fi rate (mbps) | 0.0 | Inf | -| 2 | throughput (mbps) | 0.0 | Inf| - -## Action Space - -The action is a `ndarray` with shape `(N,)` which can take values {0, 1} indicating the link to transmit data. N is the number of users. -- 0: Steer traffic over Cellular -- 1: Steer traffic over Wi-Fi - -## Reward - -```{eval-rst} - -We computes the number of Quality of Service (QoS) users over Wi-Fi as reward function :meth:`network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num`. Given that Cellular already implements an admission control mechanism to support the maximum number of QoS users, maximizing the QoS users over Wi-Fi also leads to the maximization of the total number of users in the network that meet the QoS requirements. In essence, optimizing the QoS users over Wi-Fi indirectly ensures the best possible overall network performance with the desired QoS levels for all users. - -``` - -## Custom Observation Space and Reward -1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. -To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::wifi::dl::traffic_ratio` to the list would enable receiving measurements with the name `wifi::dl::traffic_ratio` from the source `gma`. - -:::{dropdown} Click to view a demo of all measurements for this environment. +:::{dropdown} Click to view available measurements for this environment. Refer to [Working with Measurements](../../tutorials/working_with_measurements.md) for further elaboration of the following measurements. @@ -138,6 +114,31 @@ Refer to [Working with Measurements](../../tutorials/working_with_measurements.m ::: +The first feature is MAX LTE rate, representing the user's estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links. + +| Feature | Observation | Min | Max | +| ----- | ---- | ----- | ---- | +| 0 | MAX LTE rate (mbps) | 0.0 | Inf | +| 1 | MAX Wi-Fi rate (mbps) | 0.0 | Inf | +| 2 | throughput (mbps) | 0.0 | Inf| + +## Action Space + +The action is a `ndarray` with shape `(N,)` which can take values {0, 1} indicating the link to transmit data. N is the number of users. +- 0: Steer traffic over Cellular +- 1: Steer traffic over Wi-Fi + +## Reward + +```{eval-rst} + +We computes the number of Quality of Service (QoS) users over Wi-Fi as reward function :meth:`network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num`. Given that Cellular already implements an admission control mechanism to support the maximum number of QoS users, maximizing the QoS users over Wi-Fi also leads to the maximization of the total number of users in the network that meet the QoS requirements. In essence, optimizing the QoS users over Wi-Fi indirectly ensures the best possible overall network performance with the desired QoS levels for all users. + +``` + +## Custom Observation Space and Reward +1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. +To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::wifi::dl::traffic_ratio` to the list would enable receiving measurements with the name `wifi::dl::traffic_ratio` from the source `gma`. ```{eval-rst} 2. Customize Observation Space in the :meth:`network_gym_client.envs.qos_steer.Adapter.get_observation` function. 3. Customize Reward in the :meth:`network_gym_client.envs.qos_steer.Adapter.get_reward` function. diff --git a/_sources/environments/mx_traffic_management/mx_traffic_splitting.md.txt b/_sources/environments/mx_traffic_management/mx_traffic_splitting.md.txt index 64c0728..2593268 100644 --- a/_sources/environments/mx_traffic_management/mx_traffic_splitting.md.txt +++ b/_sources/environments/mx_traffic_management/mx_traffic_splitting.md.txt @@ -32,41 +32,9 @@ Ensure that you have access to the NetworkGym Server on [vLab](https://registrat ## Observation Space -The observation is an `ndarray` with shape `(3,N,)` representing three features for N users. The first feature is MAX LTE rate, representing the user's estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links. +The observation is an `ndarray` with shape `(3,N,)` representing three features for N users selected from the available measurements for this environment. -| Feature | Observation | Min | Max | -| ----- | ---- | ----- | ---- | -| 0 | MAX LTE rate (mbps) | 0.0 | Inf | -| 1 | MAX Wi-Fi rate (mbps) | 0.0 | Inf | -| 2 | throughput (mbps) | 0.0 | Inf| - -## Action Space -The action space is a `ndarray` with shape `(N,)` representing the traffic ratio over Wi-Fi for N users. The traffic ratio over Cellular equals (1.0 - action). - -| Num | Action | Min | Max | -| ----- | ---- | ----- | ---- | -| 0 | Wi-Fi traffic ratio for user 0 | 0.0 | 1.0 | -| 1 | Wi-Fi traffic ratio for user 1| 0.0 | 1.0 | -| ... | | | | -| N-1 | Wi-Fi traffic ratio for user N-1| 0.0 | 1.0 | - -## Transition Dynamics -Given an action (Wi-Fi split ratio for user i: R[i]), transmitter i follows the following transition dynamics: -- transmit R[i] traffic over Wi-Fi link. -- transmit (1-R[i]) traffic over LTE link. - -## Reward - -```{eval-rst} -We compute a utility function :math:`f = 0.5 \times log(throughput)-0.5 \times log(OWD)` using the reward function :meth:`network_gym_client.envs.nqos_split.Adapter.netowrk_util`, where OWD is the one-way delay. -The goal of the utility function is to maximize the throughput and minimizing delay. -``` - -## Custom Observation Space and Reward -1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. -To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::wifi::dl::traffic_ratio` to the list would enable receiving measurements with the name `wifi::dl::traffic_ratio` from the source `gma`. - -:::{dropdown} Click to view a demo of all measurements for this environment. +:::{dropdown} Click to view available measurements for this environment. Refer to [Working with Measurements](../../tutorials/working_with_measurements.md) for further elaboration of the following measurements. @@ -133,6 +101,39 @@ Refer to [Working with Measurements](../../tutorials/working_with_measurements.m ::: +The first feature is MAX LTE rate, representing the user's estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links. + +| Feature | Observation | Min | Max | +| ----- | ---- | ----- | ---- | +| 0 | MAX LTE rate (mbps) | 0.0 | Inf | +| 1 | MAX Wi-Fi rate (mbps) | 0.0 | Inf | +| 2 | throughput (mbps) | 0.0 | Inf| + +## Action Space +The action space is a `ndarray` with shape `(N,)` representing the traffic ratio over Wi-Fi for N users. The traffic ratio over Cellular equals (1.0 - action). + +| Num | Action | Min | Max | +| ----- | ---- | ----- | ---- | +| 0 | Wi-Fi traffic ratio for user 0 | 0.0 | 1.0 | +| 1 | Wi-Fi traffic ratio for user 1| 0.0 | 1.0 | +| ... | | | | +| N-1 | Wi-Fi traffic ratio for user N-1| 0.0 | 1.0 | + +## Transition Dynamics +Given an action (Wi-Fi split ratio for user i: R[i]), transmitter i follows the following transition dynamics: +- transmit R[i] traffic over Wi-Fi link. +- transmit (1-R[i]) traffic over LTE link. + +## Reward + +```{eval-rst} +We compute a utility function :math:`f = 0.5 \times log(throughput)-0.5 \times log(OWD)` using the reward function :meth:`network_gym_client.envs.nqos_split.Adapter.netowrk_util`, where OWD is the one-way delay. +The goal of the utility function is to maximize the throughput and minimizing delay. +``` + +## Custom Observation Space and Reward +1. Edit the "subscribed_network_stats" in the [config.json] to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of 'unused' data and reduces overhead. +To subscribe to a specific measurement, add its `source`::`name` combination to the "subscribed_network_stats" list. For example, appending `gma::wifi::dl::traffic_ratio` to the list would enable receiving measurements with the name `wifi::dl::traffic_ratio` from the source `gma`. ```{eval-rst} 2. Customize Observation Space in the :meth:`network_gym_client.envs.nqos_split.Adapter.get_observation` function. 3. Customize Reward in the :meth:`network_gym_client.envs.nqos_split.Adapter.get_reward` function. diff --git a/environments/congestion_control.html b/environments/congestion_control.html index 6362d3d..df6cd61 100644 --- a/environments/congestion_control.html +++ b/environments/congestion_control.html @@ -264,7 +264,7 @@

Congestion Control
RMCAT
-

In this environment, the agent performs periodic actions to control the sending rate of a real-time traffic RMCAT (NADA) flow.

+

In this environment, the agent performs periodic actions to adjust the sending rate of a real-time traffic flow using RTP Media Congestion Avoidance Techniques (RMCAT).

 diff --git a/environments/congestion_control/rmcat.html b/environments/congestion_control/rmcat.html index 1304488..394ccb6 100644 --- a/environments/congestion_control/rmcat.html +++ b/environments/congestion_control/rmcat.html @@ -255,10 +255,11 @@

RMCAT#

-../../_images/rmcat.png +../../_images/rmcat.png

The RMCAT environment is part of the congestion control environments and provides general information about the environment. -In this environment, the agent performs periodic actions to control the sending rate of a real-time traffic RMCAT (NADA) flow.

+In this environment, the agent performs periodic actions to adjust the sending rate of a real-time traffic flow using RTP Media Congestion Avoidance Techniques (RMCAT). +This environment is based on the GitHub Repo: cisco/ns3-rmcat.

@@ -284,7 +285,7 @@

Description#

-

The RMCAT environment represents a congestion control problem for real-time traffic. The goal of congestion control is to strategically controls sending rate, aiming to achieve high throughput, low latency and low loss.

+

The RMCAT environment represents a congestion control problem for real-time traffic. The goal of congestion control is to strategically control the sending rate, aiming to achieve high throughput, low latency, and low loss. If no action is given to the environment, RMCAT uses NADA as its default algorithm.

Prerequisite#

@@ -292,7 +293,31 @@

Prerequisite

Observation Space#

-

The observation is an ndarray with shape (3,) representing three features. The first feature is rtt, representing the round trip time. The second feature is xcurr, represeting the congestion level. The third feature is the received throughput rrate.

+

The observation is an ndarray with shape (3,) representing three features selected from the available measurements for this environment.

+
+ +Click to view available measurements for this environment.
+
+
+
+
+

Refer to Working with Measurements for further elaboration of the following measurements.

+
     id        name source      ts                value
+0   [0]      loglen  rmcat  100710               [58.0]
+1   [0]        qdel  rmcat  100710               [30.0]
+2   [0]         rtt  rmcat  100710               [55.0]
+3   [0]       ploss  rmcat  100710                [0.0]
+4   [0]         plr  rmcat  100710                [0.0]
+5   [0]       xcurr  rmcat  100710  [30.17099952697754]
+6   [0]       rrate  rmcat  100710         [774531.375]
+7   [0]       srate  rmcat  100710           [585416.0]
+8   [0]  nada_srate  rmcat  100710        [557536.1875]
+9   [0]      avgint  rmcat  100710                [0.0]
+10  [0]      curint  rmcat  100710                [0.0]
+
+
+
+

The first feature is rtt, representing the round trip time. The second feature is xcurr, represeting the congestion level. The third feature is the received throughput rrate.

@@ -328,38 +353,14 @@

Action Space

Reward#

-

TODO

+

Not implemented yet.

-
-

Custom Observation Space and Reward#

+
+

Customize Observation Space and Reward#

    -

  1. Edit the “subscribed_network_stats” in the config.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. -To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending rmcat::rtt to the list would enable receiving measurements with the name rtt from the source rmcat.

    2. +

  2. Edit the “subscribed_network_stats” in the config.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending rmcat::rtt to the list would enable receiving measurements with the name rtt from the source rmcat.

-
- -Click to view a demo of all measurements for this environment.
-
-
-
-
-

Refer to Working with Measurements for further elaboration of the following measurements.

-
     id        name source      ts                value
-0   [0]      loglen  rmcat  100710               [58.0]
-1   [0]        qdel  rmcat  100710               [30.0]
-2   [0]         rtt  rmcat  100710               [55.0]
-3   [0]       ploss  rmcat  100710                [0.0]
-4   [0]         plr  rmcat  100710                [0.0]
-5   [0]       xcurr  rmcat  100710  [30.17099952697754]
-6   [0]       rrate  rmcat  100710         [774531.375]
-7   [0]       srate  rmcat  100710           [585416.0]
-8   [0]  nada_srate  rmcat  100710        [557536.1875]
-9   [0]      avgint  rmcat  100710                [0.0]
-10  [0]      curint  rmcat  100710                [0.0]
-
-
-
-
    +

    1. Customize Observation Space in the network_gym_client.envs.rmcat.Adapter.get_observation() function.

    2. Customize Reward in the network_gym_client.envs.rmcat.Adapter.get_reward() function.

    @@ -448,7 +449,7 @@

    Episode EndObservation Space
  1. Action Space
  2. Reward
    3. -
  3. Custom Observation Space and Reward
    4. +
  4. Customize Observation Space and Reward
  5. Arguments
  6. Episode End
    7. diff --git a/environments/mx_network_slicing/cellular_network_slicing.html b/environments/mx_network_slicing/cellular_network_slicing.html index 76dae9b..836e329 100644 --- a/environments/mx_network_slicing/cellular_network_slicing.html +++ b/environments/mx_network_slicing/cellular_network_slicing.html @@ -292,8 +292,60 @@

    Prerequisite

    Observation Space#

    -

    The observation space is represented by a multidimensional array (ndarray) with a shape of (5, N), where each of the N slices contains five distinct features:

    -
      +

      The observation space is represented by a multidimensional array (ndarray) with a shape of (5, N), where each of the N slices contains five distinct features selected from the available measurements for this environment:

      +
      + +Click to view available measurements for this environment.
      +
      +
      +
      +
      +

      Refer to Working with Measurements for further elaboration of the following measurements.

      +
                       id                        name source    ts                                              value
+0   [1, 2, 3, 4, 5]                dl::max_rate    lte  3000                     [75.0, 55.0, 55.0, 75.0, 75.0]
+1   [1, 2, 3, 4, 5]                     cell_id    lte  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
+2   [1, 2, 3, 4, 5]                    slice_id    lte  3000                          [0.0, 0.0, 1.0, 1.0, 1.0]
+3   [1, 2, 3, 4, 5]                dl::rb_usage    lte  3000                     [2.88, 4.08, 4.32, 2.88, 2.88]
+4               [1]          dl::cell::max_rate    lte  3000  [{'slice': [0, 1], 'value': [65.0, 68.33333333...
+5               [1]          dl::cell::rb_usage    lte  3000        [{'slice': [0, 1], 'value': [6.96, 10.08]}]
+6   [1, 2, 3, 4, 5]           ul::missed_action    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+7   [1, 2, 3, 4, 5]          ul::measurement_ok    gma  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
+8   [1, 2, 3, 4, 5]                    ul::rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+9   [1, 2, 3, 4, 5]                ul::qos_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+10  [1, 2, 3, 4, 5]         ul::delay_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+11  [1, 2, 3, 4, 5]  ul::delay_test_1_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+12  [1, 2, 3, 4, 5]  ul::delay_test_2_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+13  [1, 2, 3, 4, 5]                     ul::owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
+14  [1, 2, 3, 4, 5]                 ul::max_owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
+15  [1, 2, 3, 4, 5]                 dl::tx_rate    gma  3000                [2.056, 2.056, 1.942, 2.056, 2.056]
+16  [1, 2, 3, 4, 5]           lte::dl::priority    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+17  [1, 2, 3, 4, 5]               lte::ul::rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+18  [1, 2, 3, 4, 5]           lte::ul::qos_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+19  [1, 2, 3, 4, 5]      lte::ul::traffic_ratio    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+20  [1, 2, 3, 4, 5]                lte::ul::owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
+21  [1, 2, 3, 4, 5]            lte::ul::max_owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
+22  [1, 2, 3, 4, 5]           dl::missed_action    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+23  [1, 2, 3, 4, 5]          dl::measurement_ok    gma  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
+24  [1, 2, 3, 4, 5]                    dl::rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
+25  [1, 2, 3, 4, 5]                dl::qos_rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
+26  [1, 2, 3, 4, 5]         dl::delay_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+27  [1, 2, 3, 4, 5]  dl::delay_test_1_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+28  [1, 2, 3, 4, 5]  dl::delay_test_2_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+29  [1, 2, 3, 4, 5]                     dl::owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
+30  [1, 2, 3, 4, 5]                 dl::max_owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
+31  [1, 2, 3, 4, 5]                 ul::tx_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+32  [1, 2, 3, 4, 5]                       x_loc    gma  3000              [19.83, 2.836, 78.31, 46.913, 27.607]
+33  [1, 2, 3, 4, 5]                       y_loc    gma  3000                [8.645, 0.388, 7.493, 4.089, 2.242]
+34  [1, 2, 3, 4, 5]           lte::ul::priority    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
+35  [1, 2, 3, 4, 5]               lte::dl::rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
+36  [1, 2, 3, 4, 5]           lte::dl::qos_rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
+37  [1, 2, 3, 4, 5]      lte::dl::traffic_ratio    gma  3000                [100.0, 100.0, 100.0, 100.0, 100.0]
+38  [1, 2, 3, 4, 5]                lte::dl::owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
+39  [1, 2, 3, 4, 5]            lte::dl::max_owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
+
      +
      +
      +

      1. Total Throughput: This measures the total data rate for each slice in Mbps, ranging from 0 to Inf.

      2. Resource Block Usage Rate: It represents the fraction of resource blocks utilized by each slice, expressed as a percentage in the range of [0, 1], with the sum across all slices equal to 1.

      3. Average Delay Violation Rate: Initially measured user-wise and then averaged for each slice, this rate indicates the average proportion of delay violations, expressed as a percentage in the range of [0, 1].

        4. @@ -387,61 +439,9 @@

        RewardCustom Observation Space and Reward#

        1. Edit the “subscribed_network_stats” in the config.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. -To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending gma::wifi::dl::traffic_ratio to the list would enable receiving measurements with the name wifi::dl::traffic_ratio from the source gma.

          2. +To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending gma::lte::dl::rate to the list would enable receiving measurements with the name lte::dl::rate from the source gma.

        -
        - -Click to view a demo of all measurements for this environment.
        -
        -
        -
        -
        -

        Refer to Working with Measurements for further elaboration of the following measurements.

        -
                         id                        name source    ts                                              value
-0   [1, 2, 3, 4, 5]                dl::max_rate    lte  3000                     [75.0, 55.0, 55.0, 75.0, 75.0]
-1   [1, 2, 3, 4, 5]                     cell_id    lte  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
-2   [1, 2, 3, 4, 5]                    slice_id    lte  3000                          [0.0, 0.0, 1.0, 1.0, 1.0]
-3   [1, 2, 3, 4, 5]                dl::rb_usage    lte  3000                     [2.88, 4.08, 4.32, 2.88, 2.88]
-4               [1]          dl::cell::max_rate    lte  3000  [{'slice': [0, 1], 'value': [65.0, 68.33333333...
-5               [1]          dl::cell::rb_usage    lte  3000        [{'slice': [0, 1], 'value': [6.96, 10.08]}]
-6   [1, 2, 3, 4, 5]           ul::missed_action    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-7   [1, 2, 3, 4, 5]          ul::measurement_ok    gma  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
-8   [1, 2, 3, 4, 5]                    ul::rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-9   [1, 2, 3, 4, 5]                ul::qos_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-10  [1, 2, 3, 4, 5]         ul::delay_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-11  [1, 2, 3, 4, 5]  ul::delay_test_1_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-12  [1, 2, 3, 4, 5]  ul::delay_test_2_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-13  [1, 2, 3, 4, 5]                     ul::owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
-14  [1, 2, 3, 4, 5]                 ul::max_owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
-15  [1, 2, 3, 4, 5]                 dl::tx_rate    gma  3000                [2.056, 2.056, 1.942, 2.056, 2.056]
-16  [1, 2, 3, 4, 5]           lte::dl::priority    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-17  [1, 2, 3, 4, 5]               lte::ul::rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-18  [1, 2, 3, 4, 5]           lte::ul::qos_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-19  [1, 2, 3, 4, 5]      lte::ul::traffic_ratio    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-20  [1, 2, 3, 4, 5]                lte::ul::owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
-21  [1, 2, 3, 4, 5]            lte::ul::max_owd    gma  3000                     [-1.0, -1.0, -1.0, -1.0, -1.0]
-22  [1, 2, 3, 4, 5]           dl::missed_action    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-23  [1, 2, 3, 4, 5]          dl::measurement_ok    gma  3000                          [1.0, 1.0, 1.0, 1.0, 1.0]
-24  [1, 2, 3, 4, 5]                    dl::rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
-25  [1, 2, 3, 4, 5]                dl::qos_rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
-26  [1, 2, 3, 4, 5]         dl::delay_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-27  [1, 2, 3, 4, 5]  dl::delay_test_1_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-28  [1, 2, 3, 4, 5]  dl::delay_test_2_violation    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-29  [1, 2, 3, 4, 5]                     dl::owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
-30  [1, 2, 3, 4, 5]                 dl::max_owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
-31  [1, 2, 3, 4, 5]                 ul::tx_rate    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-32  [1, 2, 3, 4, 5]                       x_loc    gma  3000              [19.83, 2.836, 78.31, 46.913, 27.607]
-33  [1, 2, 3, 4, 5]                       y_loc    gma  3000                [8.645, 0.388, 7.493, 4.089, 2.242]
-34  [1, 2, 3, 4, 5]           lte::ul::priority    gma  3000                          [0.0, 0.0, 0.0, 0.0, 0.0]
-35  [1, 2, 3, 4, 5]               lte::dl::rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
-36  [1, 2, 3, 4, 5]           lte::dl::qos_rate    gma  3000                [2.056, 2.056, 2.056, 2.056, 2.056]
-37  [1, 2, 3, 4, 5]      lte::dl::traffic_ratio    gma  3000                [100.0, 100.0, 100.0, 100.0, 100.0]
-38  [1, 2, 3, 4, 5]                lte::dl::owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
-39  [1, 2, 3, 4, 5]            lte::dl::max_owd    gma  3000                          [3.0, 3.0, 3.0, 3.0, 3.0]
-
        -
        -
        -
          +

          1. Customize Observation Space in the network_gym_client.envs.network_slicing.Adapter.get_observation() function.

          2. Customize Reward in the network_gym_client.envs.network_slicing.Adapter.get_reward() function.

          diff --git a/environments/mx_traffic_management/mx_qos_traffic_steering.html b/environments/mx_traffic_management/mx_qos_traffic_steering.html index e0bab18..85975f0 100644 --- a/environments/mx_traffic_management/mx_qos_traffic_steering.html +++ b/environments/mx_traffic_management/mx_qos_traffic_steering.html @@ -303,57 +303,10 @@

          Prerequisite

          Observation Space#

          -

          The observation is an ndarray with shape (3,N,) representing three features for N users. The first feature is MAX LTE rate, representing the user’s estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links.

          -
          -

- - - - - - - - - - - - - - - - - - - - - - - - -

Feature

Observation

Min

Max

0

MAX LTE rate (mbps)

0.0

Inf

1

MAX Wi-Fi rate (mbps)

0.0

Inf

2

throughput (mbps)

0.0

Inf

-
-
-
-

Action Space#

-

The action is a ndarray with shape (N,) which can take values {0, 1} indicating the link to transmit data. N is the number of users.

- -
-
-

Reward#

-

We computes the number of Quality of Service (QoS) users over Wi-Fi as reward function network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num(). Given that Cellular already implements an admission control mechanism to support the maximum number of QoS users, maximizing the QoS users over Wi-Fi also leads to the maximization of the total number of users in the network that meet the QoS requirements. In essence, optimizing the QoS users over Wi-Fi indirectly ensures the best possible overall network performance with the desired QoS levels for all users.

-
-
-

Custom Observation Space and Reward#

-
    -

  1. Edit the “subscribed_network_stats” in the config.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. -To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending gma::wifi::dl::traffic_ratio to the list would enable receiving measurements with the name wifi::dl::traffic_ratio from the source gma.

    2. -
+

The observation is an ndarray with shape (3,N,) representing three features for N users selected from the available measurement for this environment.

-Click to view a demo of all measurements for this environment.
+Click to view available measurements for this environment.
@@ -420,7 +373,55 @@

Custom Observation Space and Reward +

The first feature is MAX LTE rate, representing the user’s estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links.

+
+ + + + + + + + + + + + + + + + + + + + + + + + + +

Feature

Observation

Min

Max

0

MAX LTE rate (mbps)

0.0

Inf

1

MAX Wi-Fi rate (mbps)

0.0

Inf

2

throughput (mbps)

0.0

Inf

+
+
+
+

Action Space#

+

The action is a ndarray with shape (N,) which can take values {0, 1} indicating the link to transmit data. N is the number of users.

+ +
+
+

Reward#

+

We computes the number of Quality of Service (QoS) users over Wi-Fi as reward function network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num(). Given that Cellular already implements an admission control mechanism to support the maximum number of QoS users, maximizing the QoS users over Wi-Fi also leads to the maximization of the total number of users in the network that meet the QoS requirements. In essence, optimizing the QoS users over Wi-Fi indirectly ensures the best possible overall network performance with the desired QoS levels for all users.

+
+
+

Custom Observation Space and Reward#

+
    +

  1. Edit the “subscribed_network_stats” in the config.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. +To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending gma::wifi::dl::traffic_ratio to the list would enable receiving measurements with the name wifi::dl::traffic_ratio from the source gma.

    2. +
+

  1. Customize Observation Space in the network_gym_client.envs.qos_steer.Adapter.get_observation() function.

  2. Customize Reward in the network_gym_client.envs.qos_steer.Adapter.get_reward() function.

diff --git a/environments/mx_traffic_management/mx_traffic_splitting.html b/environments/mx_traffic_management/mx_traffic_splitting.html index 575e842..dd8ae7f 100644 --- a/environments/mx_traffic_management/mx_traffic_splitting.html +++ b/environments/mx_traffic_management/mx_traffic_splitting.html @@ -293,7 +293,77 @@

Prerequisite

Observation Space#

-

The observation is an ndarray with shape (3,N,) representing three features for N users. The first feature is MAX LTE rate, representing the user’s estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links.

+

The observation is an ndarray with shape (3,N,) representing three features for N users selected from the available measurements for this environment.

+
+ +Click to view available measurements for this environment.
+
+
+
+
+

Refer to Working with Measurements for further elaboration of the following measurements.

+
              id                        name source    ts                               value
+0   [1, 2, 3, 4]                dl::max_rate    lte  6000            [37.0, 37.0, 23.0, 37.0]
+1   [1, 2, 3, 4]                     cell_id    lte  6000                [1.0, 1.0, 1.0, 1.0]
+2   [1, 2, 3, 4]                    slice_id    lte  6000                [0.0, 0.0, 0.0, 0.0]
+3   [1, 2, 3, 4]                dl::rb_usage    lte  6000       [25.5, 26.625, 25.625, 22.25]
+4            [1]          dl::cell::max_rate    lte  6000   [{'slice': [0], 'value': [33.5]}]
+5            [1]          dl::cell::rb_usage    lte  6000  [{'slice': [0], 'value': [100.0]}]
+6   [1, 2, 3, 4]           ul::missed_action    gma  6000                [0.0, 0.0, 0.0, 0.0]
+7   [1, 2, 3, 4]          ul::measurement_ok    gma  6000                [1.0, 1.0, 1.0, 1.0]
+8   [1, 2, 3, 4]                    ul::rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
+9   [1, 2, 3, 4]                ul::qos_rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
+10  [1, 2, 3, 4]         ul::delay_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+11  [1, 2, 3, 4]  ul::delay_test_1_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+12  [1, 2, 3, 4]  ul::delay_test_2_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+13  [1, 2, 3, 4]                     ul::owd    gma  6000                [1.0, 1.0, 2.0, 2.0]
+14  [1, 2, 3, 4]                 ul::max_owd    gma  6000                [1.0, 1.0, 8.0, 6.0]
+15  [1, 2, 3, 4]                 dl::tx_rate    gma  6000    [12.429, 11.964, 21.721, 49.716]
+16  [1, 2, 3, 4]          wifi::dl::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
+17  [1, 2, 3, 4]              wifi::ul::rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
+18  [1, 2, 3, 4]          wifi::ul::qos_rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
+19  [1, 2, 3, 4]     wifi::ul::traffic_ratio    gma  6000        [100.0, 100.0, 100.0, 100.0]
+20  [1, 2, 3, 4]               wifi::ul::owd    gma  6000                [1.0, 1.0, 2.0, 2.0]
+21  [1, 2, 3, 4]           wifi::ul::max_owd    gma  6000                [1.0, 1.0, 8.0, 6.0]
+22  [1, 2, 3, 4]           lte::dl::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
+23  [1, 2, 3, 4]               lte::ul::rate    gma  6000                [0.0, 0.0, 0.0, 0.0]
+24  [1, 2, 3, 4]           lte::ul::qos_rate    gma  6000                [0.0, 0.0, 0.0, 0.0]
+25  [1, 2, 3, 4]      lte::ul::traffic_ratio    gma  6000                [0.0, 0.0, 0.0, 0.0]
+26  [1, 2, 3, 4]                lte::ul::owd    gma  6000            [-1.0, -1.0, -1.0, -1.0]
+27  [1, 2, 3, 4]            lte::ul::max_owd    gma  6000            [-1.0, -1.0, -1.0, -1.0]
+28  [1, 2, 3, 4]       wifi::dl::split_ratio    gma  6000             [21.0, 2.0, 28.0, 30.0]
+29  [1, 2, 3, 4]        lte::dl::split_ratio    gma  6000              [11.0, 30.0, 4.0, 2.0]
+30  [1, 2, 3, 4]           dl::missed_action    gma  6000                [0.0, 0.0, 0.0, 0.0]
+31  [1, 2, 3, 4]          dl::measurement_ok    gma  6000                [1.0, 1.0, 1.0, 1.0]
+32  [1, 2, 3, 4]                    dl::rate    gma  6000    [11.616, 11.616, 16.024, 57.144]
+33  [1, 2, 3, 4]                dl::qos_rate    gma  6000    [11.616, 11.616, 16.024, 57.144]
+34  [1, 2, 3, 4]         dl::delay_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+35  [1, 2, 3, 4]  dl::delay_test_1_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+36  [1, 2, 3, 4]  dl::delay_test_2_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
+37  [1, 2, 3, 4]                     dl::owd    gma  6000         [202.0, 118.0, 312.0, 40.0]
+38  [1, 2, 3, 4]                 dl::max_owd    gma  6000        [217.0, 161.0, 359.0, 119.0]
+39  [1, 2, 3, 4]                 ul::tx_rate    gma  6000        [0.208, 0.208, 0.287, 1.023]
+40  [1, 2, 3, 4]                       x_loc    gma  6000     [13.871, 6.703, 52.198, 33.715]
+41  [1, 2, 3, 4]                       y_loc    gma  6000        [7.964, 1.676, 9.021, 0.988]
+42  [1, 2, 3, 4]          wifi::ul::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
+43  [1, 2, 3, 4]              wifi::dl::rate    gma  6000       [8.24, 0.696, 15.216, 35.312]
+44  [1, 2, 3, 4]          wifi::dl::qos_rate    gma  6000       [8.24, 0.696, 15.216, 35.312]
+45  [1, 2, 3, 4]     wifi::dl::traffic_ratio    gma  6000             [48.0, 7.0, 73.0, 83.0]
+46  [1, 2, 3, 4]               wifi::dl::owd    gma  6000                [0.0, 0.0, 3.0, 5.0]
+47  [1, 2, 3, 4]           wifi::dl::max_owd    gma  6000               [1.0, 0.0, 8.0, 17.0]
+48  [1, 2, 3, 4]           lte::ul::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
+49  [1, 2, 3, 4]               lte::dl::rate    gma  6000        [9.056, 8.712, 5.568, 7.432]
+50  [1, 2, 3, 4]           lte::dl::qos_rate    gma  6000        [9.056, 8.712, 5.568, 7.432]
+51  [1, 2, 3, 4]      lte::dl::traffic_ratio    gma  6000            [52.0, 93.0, 27.0, 17.0]
+52  [1, 2, 3, 4]                lte::dl::owd    gma  6000         [202.0, 117.0, 310.0, 61.0]
+53  [1, 2, 3, 4]            lte::dl::max_owd    gma  6000        [217.0, 161.0, 359.0, 119.0]
+54  [1, 2, 3, 4]                     cell_id   wifi  6000                [0.0, 0.0, 1.0, 1.0]
+55  [1, 2, 3, 4]                dl::max_rate   wifi  6000            [78.0, 78.0, 65.0, 78.0]
+56  [1, 2, 3, 4]                ul::max_rate   wifi  6000            [78.0, 78.0, 78.0, 78.0]
+
+
+
+

The first feature is MAX LTE rate, representing the user’s estimation of channel capacity (e.g., the max rate if the user utilizes all resources). The second feature is the Max Wi-Fi rate, and the third feature is the received throughput, combining both links.

@@ -379,76 +449,7 @@

Custom Observation Space and Rewardconfig.json to include only the necessary measurements required for computing observations and rewards. This minimizes the transmission of ‘unused’ data and reduces overhead. To subscribe to a specific measurement, add its source::name combination to the “subscribed_network_stats” list. For example, appending gma::wifi::dl::traffic_ratio to the list would enable receiving measurements with the name wifi::dl::traffic_ratio from the source gma.

-
- -Click to view a demo of all measurements for this environment.
-
-
-
-
-

Refer to Working with Measurements for further elaboration of the following measurements.

-
              id                        name source    ts                               value
-0   [1, 2, 3, 4]                dl::max_rate    lte  6000            [37.0, 37.0, 23.0, 37.0]
-1   [1, 2, 3, 4]                     cell_id    lte  6000                [1.0, 1.0, 1.0, 1.0]
-2   [1, 2, 3, 4]                    slice_id    lte  6000                [0.0, 0.0, 0.0, 0.0]
-3   [1, 2, 3, 4]                dl::rb_usage    lte  6000       [25.5, 26.625, 25.625, 22.25]
-4            [1]          dl::cell::max_rate    lte  6000   [{'slice': [0], 'value': [33.5]}]
-5            [1]          dl::cell::rb_usage    lte  6000  [{'slice': [0], 'value': [100.0]}]
-6   [1, 2, 3, 4]           ul::missed_action    gma  6000                [0.0, 0.0, 0.0, 0.0]
-7   [1, 2, 3, 4]          ul::measurement_ok    gma  6000                [1.0, 1.0, 1.0, 1.0]
-8   [1, 2, 3, 4]                    ul::rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
-9   [1, 2, 3, 4]                ul::qos_rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
-10  [1, 2, 3, 4]         ul::delay_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-11  [1, 2, 3, 4]  ul::delay_test_1_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-12  [1, 2, 3, 4]  ul::delay_test_2_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-13  [1, 2, 3, 4]                     ul::owd    gma  6000                [1.0, 1.0, 2.0, 2.0]
-14  [1, 2, 3, 4]                 ul::max_owd    gma  6000                [1.0, 1.0, 8.0, 6.0]
-15  [1, 2, 3, 4]                 dl::tx_rate    gma  6000    [12.429, 11.964, 21.721, 49.716]
-16  [1, 2, 3, 4]          wifi::dl::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
-17  [1, 2, 3, 4]              wifi::ul::rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
-18  [1, 2, 3, 4]          wifi::ul::qos_rate    gma  6000        [0.192, 0.208, 0.272, 0.792]
-19  [1, 2, 3, 4]     wifi::ul::traffic_ratio    gma  6000        [100.0, 100.0, 100.0, 100.0]
-20  [1, 2, 3, 4]               wifi::ul::owd    gma  6000                [1.0, 1.0, 2.0, 2.0]
-21  [1, 2, 3, 4]           wifi::ul::max_owd    gma  6000                [1.0, 1.0, 8.0, 6.0]
-22  [1, 2, 3, 4]           lte::dl::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
-23  [1, 2, 3, 4]               lte::ul::rate    gma  6000                [0.0, 0.0, 0.0, 0.0]
-24  [1, 2, 3, 4]           lte::ul::qos_rate    gma  6000                [0.0, 0.0, 0.0, 0.0]
-25  [1, 2, 3, 4]      lte::ul::traffic_ratio    gma  6000                [0.0, 0.0, 0.0, 0.0]
-26  [1, 2, 3, 4]                lte::ul::owd    gma  6000            [-1.0, -1.0, -1.0, -1.0]
-27  [1, 2, 3, 4]            lte::ul::max_owd    gma  6000            [-1.0, -1.0, -1.0, -1.0]
-28  [1, 2, 3, 4]       wifi::dl::split_ratio    gma  6000             [21.0, 2.0, 28.0, 30.0]
-29  [1, 2, 3, 4]        lte::dl::split_ratio    gma  6000              [11.0, 30.0, 4.0, 2.0]
-30  [1, 2, 3, 4]           dl::missed_action    gma  6000                [0.0, 0.0, 0.0, 0.0]
-31  [1, 2, 3, 4]          dl::measurement_ok    gma  6000                [1.0, 1.0, 1.0, 1.0]
-32  [1, 2, 3, 4]                    dl::rate    gma  6000    [11.616, 11.616, 16.024, 57.144]
-33  [1, 2, 3, 4]                dl::qos_rate    gma  6000    [11.616, 11.616, 16.024, 57.144]
-34  [1, 2, 3, 4]         dl::delay_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-35  [1, 2, 3, 4]  dl::delay_test_1_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-36  [1, 2, 3, 4]  dl::delay_test_2_violation    gma  6000                [0.0, 0.0, 0.0, 0.0]
-37  [1, 2, 3, 4]                     dl::owd    gma  6000         [202.0, 118.0, 312.0, 40.0]
-38  [1, 2, 3, 4]                 dl::max_owd    gma  6000        [217.0, 161.0, 359.0, 119.0]
-39  [1, 2, 3, 4]                 ul::tx_rate    gma  6000        [0.208, 0.208, 0.287, 1.023]
-40  [1, 2, 3, 4]                       x_loc    gma  6000     [13.871, 6.703, 52.198, 33.715]
-41  [1, 2, 3, 4]                       y_loc    gma  6000        [7.964, 1.676, 9.021, 0.988]
-42  [1, 2, 3, 4]          wifi::ul::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
-43  [1, 2, 3, 4]              wifi::dl::rate    gma  6000       [8.24, 0.696, 15.216, 35.312]
-44  [1, 2, 3, 4]          wifi::dl::qos_rate    gma  6000       [8.24, 0.696, 15.216, 35.312]
-45  [1, 2, 3, 4]     wifi::dl::traffic_ratio    gma  6000             [48.0, 7.0, 73.0, 83.0]
-46  [1, 2, 3, 4]               wifi::dl::owd    gma  6000                [0.0, 0.0, 3.0, 5.0]
-47  [1, 2, 3, 4]           wifi::dl::max_owd    gma  6000               [1.0, 0.0, 8.0, 17.0]
-48  [1, 2, 3, 4]           lte::ul::priority    gma  6000                [0.0, 0.0, 0.0, 0.0]
-49  [1, 2, 3, 4]               lte::dl::rate    gma  6000        [9.056, 8.712, 5.568, 7.432]
-50  [1, 2, 3, 4]           lte::dl::qos_rate    gma  6000        [9.056, 8.712, 5.568, 7.432]
-51  [1, 2, 3, 4]      lte::dl::traffic_ratio    gma  6000            [52.0, 93.0, 27.0, 17.0]
-52  [1, 2, 3, 4]                lte::dl::owd    gma  6000         [202.0, 117.0, 310.0, 61.0]
-53  [1, 2, 3, 4]            lte::dl::max_owd    gma  6000        [217.0, 161.0, 359.0, 119.0]
-54  [1, 2, 3, 4]                     cell_id   wifi  6000                [0.0, 0.0, 1.0, 1.0]
-55  [1, 2, 3, 4]                dl::max_rate   wifi  6000            [78.0, 78.0, 65.0, 78.0]
-56  [1, 2, 3, 4]                ul::max_rate   wifi  6000            [78.0, 78.0, 78.0, 78.0]
-
-
-
-
    +

    1. Customize Observation Space in the network_gym_client.envs.nqos_split.Adapter.get_observation() function.

    2. Customize Reward in the network_gym_client.envs.nqos_split.Adapter.get_reward() function.

    diff --git a/searchindex.js b/searchindex.js index 88faf78..88ac9db 100644 --- a/searchindex.js +++ b/searchindex.js @@ -1 +1 @@ -Search.setIndex({"docnames": ["README", "api/network_gym_client", "api/network_gym_client/adapter", "api/network_gym_client/adapter/network_slicing", "api/network_gym_client/adapter/nqos_split", "api/network_gym_client/adapter/qos_steer", "api/network_gym_client/adapter/rmcat", "api/network_gym_client/env", "api/network_gym_client/northbound_interface", "api/network_gym_env", "api/network_gym_env/configure", "api/network_gym_env/simulator", "api/network_gym_env/southbound_interface", "api/network_gym_server", "contact", "content/motivation", "content/overview", "content/quickstart", "environments/congestion_control", "environments/congestion_control/rmcat", "environments/mx_network_slicing", "environments/mx_network_slicing/cellular_network_slicing", "environments/mx_traffic_management", "environments/mx_traffic_management/mx_qos_traffic_steering", "environments/mx_traffic_management/mx_traffic_splitting", "index", "tutorials/handling_time_limits", "tutorials/implementing_custom_environment", "tutorials/training_agents", "tutorials/working_with_measurements"], "filenames": ["README.md", "api/network_gym_client.md", "api/network_gym_client/adapter.md", "api/network_gym_client/adapter/network_slicing.md", "api/network_gym_client/adapter/nqos_split.md", "api/network_gym_client/adapter/qos_steer.md", "api/network_gym_client/adapter/rmcat.md", "api/network_gym_client/env.md", "api/network_gym_client/northbound_interface.md", "api/network_gym_env.md", "api/network_gym_env/configure.md", "api/network_gym_env/simulator.md", "api/network_gym_env/southbound_interface.md", "api/network_gym_server.md", "contact.md", "content/motivation.md", "content/overview.md", "content/quickstart.md", "environments/congestion_control.md", "environments/congestion_control/rmcat.md", "environments/mx_network_slicing.md", "environments/mx_network_slicing/cellular_network_slicing.md", "environments/mx_traffic_management.md", "environments/mx_traffic_management/mx_qos_traffic_steering.md", "environments/mx_traffic_management/mx_traffic_splitting.md", "index.md", "tutorials/handling_time_limits.md", "tutorials/implementing_custom_environment.md", "tutorials/training_agents.md", "tutorials/working_with_measurements.md"], "titles": ["Build Documentation Website", "network_gym_client", "Adapter", "network_slicing Adapter", "nqos_split Adapter", "qos_steer Adapter", "rmcat Adapter", "Env", "NorthBound", "network_gym_env", "Configure", "Dummy Simulator", "SouthBound", "network_gym_server", "Contact", "Motivation", "Overview", "Quickstart", "Congestion Control", "RMCAT", "Network Slicing", "Cellular Network Slicing", "Traffic Management", "Multi-Access QoS Traffic Steering", "Multi-Access Traffic Splitting", "NetworkGym: Democratizing Network AI via Sim-aaS.", "Handling Time Limits", "Implementing Custom Environment", "Training Agents", "Working with Measurements"], "terms": {"option": [0, 1, 7, 17, 19, 21, 23, 24, 25, 26], "creat": [0, 11, 17, 25, 27, 28], "new": [0, 11, 17, 29], "virtual": 0, "python": [0, 17, 25], "environ": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29], "python3": [0, 27], "m": [0, 19, 21, 27, 29], "venv": 0, "sphinx_venv": 0, "sourc": [0, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 19, 21, 23, 24, 25, 28], "bin": 0, "activ": [0, 9, 16, 17, 28], "requir": [0, 15, 19, 21, 23, 24, 29], "librari": 0, "pip": 0, "r": [0, 21, 24], "txt": 0, "In": [0, 9, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "doc": 0, "folder": 0, "run": [0, 7, 10, 11, 17, 19, 21, 23, 24, 26], "follow": [0, 1, 7, 10, 19, 21, 23, 24, 25, 27, 29], "command": [0, 27], "export": 0, "static": 0, "_build": 0, "html": 0, "make": [0, 17], "clean": 0, "network_gym_cli": [0, 2, 3, 4, 5, 6, 17, 19, 21, 23, 24, 26], "The": [0, 1, 7, 9, 10, 11, 13, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "contain": [0, 7, 17, 21, 29], "modul": [0, 9], "rm": 0, "sphinx": 0, "apidoc": 0, "o": [0, 17, 25], "d": 0, "2": [0, 7, 19, 21, 23, 24, 25, 27, 29], "markdown": 0, "e": [0, 1, 7, 10, 11, 15, 19, 21, 23, 24, 25, 26, 29], "g": [0, 1, 7, 10, 11, 15, 23, 24, 25, 29], "eval": 0, "autoclass": 0, "env": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 17, 19, 21, 23, 24, 26, 27, 28], "networkgym": [1, 7, 8, 13, 19, 21, 23, 24, 26, 27, 28], "client": [1, 7, 8, 9, 10, 11, 13, 19, 21, 23, 24, 25, 27], "includ": [1, 13, 17, 19, 21, 23, 24, 25, 28, 29], "three": [1, 16, 19, 21, 23, 24, 25, 26], "compon": [1, 9, 10, 13, 25], "custom": [1, 7, 10, 16, 17, 25, 26], "gymnasium": [1, 2, 7, 15, 16, 17, 28], "adapt": [1, 7, 16, 19, 21, 23, 24, 27, 29], "northbound": [1, 7, 13, 16, 17, 25], "interfac": [1, 7, 8, 9, 12, 13, 17, 25, 27], "inheret": 1, "class": [1, 3, 4, 5, 6, 7, 8, 10, 11, 17, 27, 29], "commun": [1, 7, 9, 16, 17], "agent": [1, 3, 4, 5, 6, 7, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29], "us": [1, 7, 10, 11, 15, 17, 20, 21, 24, 25, 26, 27, 28, 29], "standard": [1, 15, 25, 27], "exchang": [1, 9, 10, 17], "ob": [1, 2, 7, 17, 26], "reward": [1, 2, 7, 17, 25, 26, 29], "action": [1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 18, 20, 22, 25, 26, 27, 28, 29], "reset": [1, 7, 17, 26], "step": [1, 7, 19, 21, 23, 24, 26, 27], "function": [1, 2, 3, 4, 5, 6, 17, 19, 21, 23, 24, 25, 29], "import": [1, 17, 19, 21, 23, 24, 26], "gym": [1, 7, 8, 25], "def": 1, "__init__": 1, "self": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11], "arg1": 1, "arg2": 1, "super": 1, "return": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 17], "observ": [1, 3, 4, 5, 6, 7, 25, 28, 29], "termin": [1, 7, 9, 10, 11, 17, 19, 21, 23, 24, 27], "truncat": [1, 7, 17, 19, 21, 23, 24], "info": [1, 2, 7, 17, 26, 27, 28], "seed": [1, 7], "none": [1, 7], "transform": [1, 2, 16], "data": [1, 2, 5, 9, 16, 17, 19, 21, 23, 24, 25, 27, 28, 29], "format": [1, 2, 16, 17, 25, 27], "from": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 17, 19, 21, 23, 24, 25, 26, 28, 29], "network_gym": [1, 2, 7], "other": [1, 10, 26], "wai": [1, 7, 24, 29], "around": [1, 25], "network": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 17, 19, 23, 24, 27, 28, 29], "stat": [1, 2, 3, 4, 5, 6, 8, 11, 25], "chang": [1, 26], "polici": [1, 2, 3, 4, 5, 6, 8, 9, 17, 28, 29], "connect": [1, 7, 8, 9, 10, 11, 12, 13, 16, 17, 22, 23, 24, 25], "server": [1, 7, 8, 9, 10, 11, 12, 13, 17, 19, 21, 23, 24, 25, 27], "configur": [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 17, 19, 21, 23, 24, 25, 26, 29], "paramet": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 19, 21, 23, 24, 26], "between": [1, 2, 7, 9, 10, 16, 24, 25, 27], "envrion": [1, 13], "config_json": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26], "thi": [2, 3, 4, 5, 6, 7, 9, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 29], "i": [2, 3, 4, 7, 8, 10, 11, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "an": [2, 7, 9, 10, 13, 15, 16, 17, 19, 21, 23, 24, 25, 26, 28, 29], "It": [2, 7, 9, 16, 17, 19, 21, 23, 24, 25, 26], "measur": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 17, 19, 21, 23, 24, 25, 27], "json": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26, 27, 29], "space": [2, 3, 4, 5, 6, 7], "also": [2, 7, 15, 17, 21, 23, 24, 25, 26, 27], "can": [2, 7, 9, 10, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "appli": [2, 29], "initi": [2, 3, 4, 5, 6, 7, 8, 9, 10, 21], "file": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26, 27, 29], "wandb_log_buffer_append": 2, "add": [2, 13, 17, 19, 21, 23, 24, 29], "wandb": [2, 17], "log": [2, 7, 24], "buffer": 2, "send": [2, 7, 8, 9, 11, 17, 18, 19, 25, 27, 28, 29], "later": 2, "wandb_log": 2, "dict": [2, 7, 11, 17], "inform": [2, 7, 17, 19, 23, 24, 25], "append": [2, 19, 21, 23, 24, 29], "df_to_dict": 2, "df": [2, 3, 4, 5, 6, 29], "id_nam": 2, "id": [2, 7, 8, 10, 19, 21, 23, 24, 29], "datatyp": 2, "panda": [2, 5], "datafram": [2, 3, 4, 5, 6, 8], "dictionari": [2, 7, 17], "object": [2, 7, 21, 29], "descript": [2, 29], "string": 2, "descritpt": 2, "convert": 2, "fill_empty_featur": 2, "featur": [2, 3, 4, 5, 6, 19, 21, 23, 24, 25], "valu": [2, 4, 7, 17, 19, 21, 23, 24, 27, 29], "fill": 2, "miss": [2, 29], "input": [2, 17, 29], "pd": [2, 3, 4, 5, 6, 8], "int": [2, 3, 7, 8, 10, 11], "list": [2, 7, 10, 11, 17, 19, 21, 23, 24, 29], "result": [2, 7, 11, 17, 19, 21, 23, 24, 26], "after": [2, 7, 17, 19, 21, 23, 24, 26, 29], "replac": 2, "base": [3, 4, 5, 6, 9, 16, 17, 24, 25, 29], "get_observ": [3, 4, 5, 6, 19, 21, 23, 24, 29], "prepar": [3, 4, 5, 6], "should": [3, 4, 5, 6, 7, 17, 21, 27, 29], "same": [3, 4, 5, 6, 10, 21, 27], "number": [3, 4, 5, 6, 7, 11, 17, 19, 21, 23, 24, 26, 27], "defin": [3, 4, 5, 6, 7, 17, 19, 21, 23, 24, 25, 28], "get_observation_spac": [3, 4, 5, 6], "get_reward": [3, 4, 5, 6, 19, 21, 23, 24, 29], "get_polici": [3, 4, 5, 6], "rl": [3, 4, 5, 6, 7, 17, 25, 28, 29], "No": [3, 17, 25, 28], "yet": 3, "get_action_spac": [3, 4, 5, 6], "get": [3, 4, 5, 6, 7], "get_rbg_siz": 3, "bandwidth": [3, 23, 29], "comput": [3, 17, 19, 21, 23, 24, 25, 29], "resourc": [3, 20, 21, 23, 24, 29], "block": [3, 21, 29], "group": [3, 21, 27], "size": [3, 29], "bandwith": 3, "rb": [3, 21], "code": [3, 10, 28, 29], "coppi": 3, "ns3": 3, "pf": 3, "type": [3, 7, 17, 25, 27], "0": [3, 4, 7, 17, 19, 21, 23, 24, 26, 27, 29], "alloc": [3, 20, 21], "rbg": 3, "resouc": 3, "netowrk_util": [4, 24], "throughput": [4, 19, 21, 23, 24, 29], "delai": [4, 7, 21, 23, 24, 28, 29], "alpha": 4, "5": [4, 19, 21, 23, 24, 27, 29], "calcul": [4, 5, 23, 29], "util": [4, 10, 16, 21, 23, 24, 25, 27, 29], "specifi": [4, 7, 17, 19, 21, 23, 24, 26], "balanc": 4, "default": [4, 5, 7, 17, 21, 25, 29], "float": 4, "repres": [4, 17, 19, 21, 23, 24, 29], "bit": 4, "per": [4, 5, 19, 21, 23, 24, 26], "second": [4, 17, 19, 23, 24, 26], "metric": [4, 17], "calculate_wifi_qos_user_num": [5, 23], "qos_rat": [5, 21, 23, 24, 27, 29], "qo": [5, 15, 17, 22, 29], "user": [5, 7, 11, 17, 21, 22, 23, 24, 25, 27, 28, 29], "over": [5, 22, 23, 24, 29], "wi": [5, 22, 23, 24, 27], "fi": [5, 22, 23, 24, 27], "rate": [5, 18, 19, 21, 23, 24, 27, 29], "doubl": 5, "initil": [7, 11], "intern": [7, 17], "state": [7, 17, 25, 28], "": [7, 9, 10, 17, 19, 21, 23, 24, 26, 27, 28, 29], "prng": 7, "np_random": 7, "If": [7, 17, 23, 26, 29], "doe": [7, 25], "alreadi": [7, 23], "have": [7, 17, 19, 21, 23, 24, 25, 26, 28], "pass": 7, "chosen": 7, "some": [7, 17, 29], "entropi": 7, "timestamp": [7, 11, 29], "dev": 7, "urandom": 7, "howev": [7, 23, 29], "ha": [7, 17, 19, 21, 23, 24, 26], "you": [7, 16, 17, 19, 21, 23, 24, 26, 27, 28, 29], "integ": 7, "even": 7, "exist": [7, 29], "usual": 7, "want": 7, "right": [7, 21], "been": [7, 17, 28], "never": 7, "again": 7, "pleas": [7, 25, 27], "refer": [7, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "minim": [7, 19, 21, 23, 24], "exampl": [7, 17, 19, 21, 23, 24, 29], "abov": [7, 25], "see": [7, 19, 21, 23, 24], "paradigm": 7, "addit": [7, 17, 25], "how": [7, 16, 17, 25, 26, 27], "depend": [7, 27], "specif": [7, 15, 16, 19, 21, 23, 24, 25, 26, 28, 29], "obstyp": 7, "auxiliari": 7, "complement": 7, "analog": 7, "one": [7, 9, 11, 17, 21, 24, 26, 29], "timestep": [7, 17], "dynam": [7, 23, 29], "gamsim": 7, "check": 7, "last": [7, 26, 29], "episod": [7, 17, 26, 27], "done": [7, 17], "acttyp": 7, "provid": [7, 10, 16, 17, 19, 23, 24, 25, 26, 27, 28, 29], "updat": [7, 17, 22, 24], "element": [7, 16, 17, 29], "observation_spac": [7, 17], "next": [7, 17], "due": 7, "supportsfloat": 7, "take": [7, 17, 21, 23, 29], "bool": 7, "whether": 7, "reach": [7, 17], "under": 7, "mdp": 7, "task": [7, 27], "which": [7, 16, 17, 21, 23], "posit": [7, 21, 23, 24], "neg": 7, "goal": [7, 19, 23, 24, 29], "move": 7, "lava": 7, "sutton": 7, "barton": 7, "gridworld": 7, "true": [7, 17, 23], "need": [7, 15, 17, 21], "call": [7, 17], "condit": 7, "outsid": 7, "scope": 7, "satisfi": 7, "typic": 7, "timelimit": [7, 17], "could": [7, 17, 29], "indic": [7, 21, 23, 29], "physic": [7, 25], "go": 7, "out": [7, 17, 25, 29], "bound": [7, 17, 29], "end": [7, 9, 17, 25, 26], "prematur": 7, "befor": 7, "diagnost": 7, "help": [7, 17], "debug": 7, "learn": [7, 15, 25, 27, 28], "raw": 7, "flag": 7, "deprec": 7, "A": [7, 11, 17, 19, 21, 23, 24, 25, 26, 28, 29], "boolean": 7, "case": [7, 15, 17, 21, 25, 29], "further": [7, 17, 19, 21, 23, 24], "undefin": 7, "wa": [7, 29], "remov": 7, "openai": 7, "v26": 7, "favor": 7, "signal": [7, 9, 10, 17, 24, 29], "mai": [7, 17, 26], "emit": 7, "differ": [7, 17, 25, 26], "reason": 7, "mayb": 7, "underli": [7, 10], "solv": 7, "successfulli": [7, 27], "certain": [7, 19, 21, 23, 24, 26], "exceed": [7, 17], "simul": [7, 9, 10, 15, 16, 25, 29], "enter": 7, "invalid": 7, "action_spac": [7, 17, 26, 28], "correspond": [7, 16, 29], "valid": [7, 17, 29], "all": [7, 10, 17, 19, 20, 21, 23, 24, 27, 29], "For": [7, 17, 19, 21, 23, 24, 25, 27, 29], "discret": [7, 17], "give": 7, "mean": 7, "ar": [7, 9, 10, 17, 19, 21, 23, 24, 27], "two": [7, 9, 13, 16, 25, 26], "1": [7, 17, 19, 21, 23, 24, 26, 27, 29], "box": [7, 16, 17, 19, 21, 23, 24], "3": [7, 9, 10, 15, 16, 17, 19, 21, 23, 24, 25, 26, 27, 29], "4028234663852886e": 7, "38": [7, 21, 23, 24], "4": [7, 19, 21, 23, 24, 27, 29], "float32": [7, 19, 21, 23, 24], "shape": [7, 17, 19, 21, 23, 24], "denot": [7, 19, 21, 23, 24, 26, 29], "arrai": [7, 17, 21, 28], "we": [7, 13, 17, 21, 23, 24, 25, 26], "well": 7, "high": [7, 19, 23, 24, 25, 29], "8000002e": 7, "00": 7, "4028235e": 7, "1887903e": 7, "01": 7, "dtype": 7, "low": [7, 19, 24, 29], "northbound_interface_cli": 7, "traslat": 7, "dataformat": 7, "northboundcli": 8, "sever": 8, "repli": 8, "start": [8, 9, 10, 11, 19, 26, 27], "request": [8, 9], "where": [8, 17, 21, 24, 29], "load": [8, 17, 19, 21, 23, 24, 29], "recv": 8, "receiv": [8, 9, 11, 17, 19, 21, 23, 24, 27, 29], "messag": [8, 9, 10], "process_measur": 8, "reply_json": 8, "process": [8, 9, 26, 27], "compris": [9, 16], "core": 9, "These": [9, 28], "establish": [9, 10, 16, 17, 27], "via": [9, 12, 27], "design": [9, 21, 25, 28], "southbound": [9, 11, 13, 16, 17, 25, 27], "serv": [9, 21, 25], "bridg": 9, "facilit": [9, 10, 16, 17, 25], "statist": [9, 29], "keep": [9, 16], "aliv": 9, "period": [9, 10, 18, 19, 20, 22, 23, 24], "dispatch": [9, 10], "hello": [9, 10, 27], "upon": [9, 10], "disengag": 9, "n": [9, 10, 15, 16, 17, 21, 23, 24, 25], "smulat": 9, "instig": 9, "plan": [9, 13, 17], "incorpor": 9, "emul": [9, 16], "testb": [9, 16, 27, 29], "altern": 9, "earmark": 9, "futur": 9, "releas": [9, 13], "depict": [9, 16], "flow": [9, 18, 19, 23, 29], "complet": [9, 27], "cycl": 9, "within": [9, 16, 17, 21, 27, 29], "session": [9, 16, 17, 19, 21, 23, 24, 26], "launch": [9, 10, 17, 27], "env_config": [9, 10, 11, 26, 27], "trigger": [9, 28], "gener": [9, 11, 19, 21, 23, 24, 26, 27, 29], "incom": 9, "onc": [9, 17, 19, 21, 23, 24, 26, 27], "conclud": 9, "map": [9, 13, 16, 17], "onli": [9, 10, 19, 21, 23, 24, 27, 29], "env_sim": [9, 10, 11, 27], "ani": [9, 17, 23, 25], "given": [9, 21, 23, 24], "moment": [9, 21], "cannot": [9, 19, 21, 23, 24, 26], "occur": 9, "simultan": 9, "standpoint": 9, "treat": 9, "both": [9, 17, 19, 21, 22, 23, 24, 26, 29], "equival": 9, "identifi": [9, 29], "them": [9, 17, 21, 25], "entiti": [9, 10, 25, 27, 29], "networkgymsim": 10, "env_list": [10, 27], "recept": 10, "its": [10, 16, 19, 21, 23, 24, 29], "socket": [10, 11, 12, 27], "ident": [10, 11, 12], "identif": 10, "subsequ": [10, 19, 21, 23, 24, 26], "commenc": 10, "conclus": 10, "close": 10, "reconnect": 10, "point": [10, 19, 21, 23, 24, 26, 29], "perceiv": 10, "divis": 10, "advantag": [10, 28], "straightforward": [10, 17, 27], "expans": 10, "test": [10, 15, 23, 27, 29], "while": [10, 20, 21, 26], "str": [10, 11, 12], "support": [10, 16, 17, 23, 27], "non": [10, 17], "offic": 10, "account": [10, 21, 29], "name": [10, 11, 17, 19, 21, 23, 24, 27, 29], "env_ident": 11, "client_ident": 11, "msg_json": 11, "random": [11, 17, 21, 23, 24], "sampl": [11, 17, 26, 28], "when": [11, 13, 19, 21, 23, 24, 25, 27, 28, 29], "resum": 11, "env_idennt": 11, "env_port": 11, "first": [11, 17, 19, 23, 24, 26], "who": 11, "msg": [11, 27], "cliet": 11, "generate_dummy_measur": 11, "tag": [11, 27, 29], "num_us": 11, "ad": 11, "start_simul": 11, "api": [11, 15, 28], "report": 11, "run_one_interv": 11, "interv": [11, 17, 29], "southbound_interfac": 12, "southbound_connect": 12, "indent": 12, "zmq": 12, "manag": [13, 17, 23, 24, 25], "select": [13, 16, 17, 19, 21, 22, 23, 24, 27], "idl": 13, "instanc": [13, 17, 25, 27, 29], "rout": [13, 16], "tabl": 13, "At": [13, 16, 19, 21, 23, 24, 26], "present": [13, 16, 17, 23, 27], "do": 13, "softwar": [13, 25], "email": 14, "github": [14, 19, 21, 23, 24, 27], "slack": 14, "real": [15, 18, 19], "world": [15, 25], "dataset": 15, "control": [15, 19, 23, 25], "oper": [15, 19, 21, 23, 24, 25, 26], "difficult": 15, "acquir": 15, "align": [15, 27], "usag": [15, 21, 29], "itself": 15, "enough": 15, "train": [15, 17, 19, 21, 23, 24, 25], "reinforc": [15, 25, 28], "etc": [15, 25], "current": [15, 29], "enabl": [15, 16, 17, 19, 21, 23, 24, 25, 28, 29], "multi": [15, 29], "access": [15, 19, 21, 25, 27, 29], "traffic": [15, 18, 19, 29], "split": [15, 22, 29], "awar": 15, "steer": [15, 22], "cellular": [15, 20, 23, 24], "ran": [15, 17, 25], "slice": [15, 23, 24, 29], "tool": [15, 25], "often": 15, "veri": 15, "complex": [15, 29], "especi": 15, "research": [15, 16, 17, 25], "without": [15, 23, 25], "expertis": [15, 25], "lack": 15, "common": 15, "simpl": [15, 17], "evalu": [15, 23, 27], "benchmark": 15, "adher": [15, 21], "addition": [15, 16, 17, 27], "offer": [15, 16, 25, 28], "framework": [16, 25], "consist": [16, 17, 25], "essenti": [16, 17, 25, 27], "each": [16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29], "plai": [16, 25], "crucial": [16, 25], "role": [16, 25, 28], "system": [16, 17, 21, 25, 29], "encompass": 16, "seamless": 16, "interact": [16, 25, 26, 27, 28], "displai": 16, "below": [16, 17, 26, 28, 29], "graphic": 16, "architectur": [16, 25], "visual": [16, 17], "aid": 16, "divers": [16, 27], "collabor": [16, 17], "harmoni": 16, "form": 16, "streamlin": [16, 25, 27], "effect": [16, 17, 25], "servic": [16, 20, 21, 23, 25], "tailor": [16, 21, 25], "advanc": 16, "ai": [16, 17], "develop": [16, 17, 25], "By": [16, 21, 25, 29], "click": [16, 19, 21, 23, 24, 29], "blue": 16, "navig": 16, "directli": 16, "page": [16, 25], "respect": [16, 17, 21, 25], "main": 16, "respons": 16, "allow": [16, 17, 25, 28], "compat": 16, "like": [16, 21, 25], "stabl": [16, 25], "baselines3": [16, 25], "cleanrl": [16, 25], "desir": [16, 23], "central": 16, "maintain": [16, 25], "track": 16, "assign": [16, 21, 23, 24, 27], "dure": [16, 23], "either": [16, 17, 19, 21, 23, 24], "through": [16, 17, 19, 21, 23, 24, 25, 26, 27], "distinct": [16, 21], "machin": [17, 19, 21, 23, 24, 25], "requst": 17, "mlwin": 17, "ring": 17, "univers": 17, "public": 17, "internet": 17, "intel": 17, "issu": [17, 29], "feel": 17, "free": 17, "u": [17, 27], "netaigym": [17, 27], "gmail": [17, 27], "com": [17, 27], "sim": 17, "aa": 17, "devcloud": 17, "avail": [17, 23, 25, 28, 29], "meanwhil": [17, 21], "enhanc": 17, "capabl": 17, "5g": 17, "distribut": [17, 21, 23, 24], "energi": 17, "save": 17, "predict": 17, "more": [17, 19, 21, 23, 24, 25, 29], "gain": 17, "begin": 17, "download": [17, 19, 21, 23, 24], "your": [17, 25, 29], "prompt": 17, "collect": [17, 25, 28], "forward": [17, 27], "norhtbound": 17, "nb": 17, "algorithm": [17, 21, 25, 29], "store": [17, 29], "relev": 17, "transmit": [17, 19, 21, 23, 24, 27], "sb": [17, 27], "continu": [17, 19, 21, 23, 24, 26], "conveni": 17, "websit": 17, "demonstr": [17, 26, 27], "packag": 17, "networkgymenv": [17, 19, 21, 23, 24, 26], "client_id": [17, 19, 21, 23, 24, 26], "argument": 17, "classic": 17, "loop": [17, 25], "implement": [17, 21, 23, 25], "load_config_fil": [17, 19, 21, 23, 24, 26], "env_nam": [17, 26, 27], "nqos_split": [17, 24, 26], "num_step": [17, 26], "1000": [17, 26], "rang": [17, 21, 26, 28], "exit": [17, 19, 21, 23, 24, 26], "break": [17, 26], "epsiod": [17, 26], "up": [17, 26], "still": [17, 26], "anoth": [17, 26], "keyword": 17, "associ": 17, "dedic": [17, 20, 21], "worker": 17, "until": 17, "parallel": 17, "obtain": 17, "perform": [17, 18, 19, 20, 22, 23, 24, 25, 26], "As": 17, "along": 17, "One": 17, "multipl": [17, 21, 23, 24, 29], "fix": 17, "restart": 17, "scenario": [17, 23], "sequenti": 17, "success": 17, "insight": 17, "time": [17, 18, 19, 21, 23, 24, 29], "constraint": [17, 21], "found": 17, "handl": [17, 19, 21, 23, 24], "limit": [17, 19, 21, 23, 24, 27], "illustr": [17, 26, 29], "everi": [17, 21, 23, 24], "attribut": 17, "understand": 17, "expect": [17, 27], "output": [17, 27, 29], "instead": 17, "prefer": 17, "empti": [17, 25, 28], "shown": [17, 27, 28], "np": [17, 28], "inherit": 17, "major": 17, "possibl": [17, 23], "might": [17, 29], "describ": 17, "dimension": 17, "upper": 17, "lower": 17, "our": [17, 21, 27], "shift": 17, "tupl": 17, "multibinari": 17, "binari": 17, "multidiscret": [17, 23], "wrapper": 17, "maximum": [17, 21, 23, 29], "clipact": 17, "clip": 17, "li": 17, "rescaleact": 17, "rescal": 17, "lie": 17, "timeawareobserv": 17, "about": [17, 19, 21, 23, 24, 25], "index": [17, 29], "ensur": [17, 19, 21, 23, 24], "transit": 17, "markov": 17, "flattenobserv": 17, "flatten": 17, "normalizeobserv": 17, "normal": 17, "t": [17, 19, 21, 23, 24, 27, 29], "coordin": [17, 29], "center": [17, 25], "unit": [17, 27], "varianc": 17, "wrap": 17, "arg": 17, "normal_obs_env": 17, "rmcat": [18, 29], "nada": [18, 19, 29], "part": [19, 23, 24], "congest": [19, 29], "inf": [19, 21, 23, 24], "150000": 19, "1500000": 19, "config": [19, 21, 23, 24], "problem": [19, 24], "strateg": [19, 21, 24], "aim": [19, 23, 24], "achiev": [19, 23, 24, 25, 29], "latenc": [19, 24], "loss": [19, 23, 29], "vlab": [19, 21, 23, 24, 27], "ndarrai": [19, 21, 23, 24], "rtt": [19, 29], "round": [19, 29], "trip": [19, 29], "xcurr": [19, 29], "represet": 19, "level": [19, 20, 21, 23], "third": [19, 23, 24], "rrate": [19, 29], "min": [19, 21, 23, 24], "max": [19, 21, 23, 24], "500": [19, 23, 29], "bp": [19, 29], "todo": 19, "edit": [19, 21, 23, 24], "subscribed_network_stat": [19, 21, 23, 24, 29], "necessari": [19, 21, 23, 24], "transmiss": [19, 21, 23, 24], "unus": [19, 21, 23, 24], "reduc": [19, 21, 23, 24], "overhead": [19, 21, 23, 24], "To": [19, 21, 23, 24, 28, 29], "subscrib": [19, 21, 23, 24, 29], "combin": [19, 21, 23, 24, 29], "would": [19, 21, 23, 24, 27, 29], "view": [19, 21, 23, 24, 29], "demo": [19, 21, 23, 24, 27, 28], "work": [19, 21, 23, 24], "elabor": [19, 21, 23, 24], "loglen": [19, 29], "100710": 19, "58": 19, "qdel": [19, 29], "30": [19, 21, 23, 24], "55": [19, 21, 23, 24], "ploss": [19, 29], "plr": [19, 29], "17099952697754": 19, "6": [19, 21, 23, 24, 27], "774531": 19, "375": 19, "7": [19, 21, 23, 24, 27], "srate": [19, 29], "585416": 19, "8": [19, 21, 23, 24, 27], "nada_sr": [19, 29], "557536": 19, "1875": 19, "9": [19, 21, 23, 24, 27], "avgint": [19, 29], "10": [19, 21, 23, 24, 27], "curint": [19, 29], "repo": [19, 21, 23, 24, 27], "detail": [19, 21, 23, 24, 25, 28], "episodes_per_sess": [19, 21, 23, 24, 26], "steps_per_episod": [19, 21, 23, 24, 26], "l": [19, 21, 23, 24, 26], "happen": [19, 21, 23, 24], "length": [19, 21, 23, 24, 26], "note": [19, 21, 23, 24, 26, 29], "reconfigur": [19, 21, 23, 24, 26], "program": [19, 21, 23, 24, 25, 26], "entir": [19, 21, 23, 24, 26], "among": [20, 21], "priorit": [20, 21, 23, 29], "share": [20, 21, 27], "strive": [20, 21], "meet": [20, 21, 23, 29], "agreement": [20, 21], "partit": 21, "percept": 21, "possess": 21, "abil": 21, "great": 21, "effici": [21, 23], "belong": 21, "mac": 21, "schedul": 21, "proport": 21, "fair": 21, "guarante": 21, "equit": 21, "imparti": 21, "network_sl": 21, "pose": 21, "challeng": [21, 23], "compet": 21, "finit": 21, "primari": [21, 23], "multidimension": 21, "five": 21, "total": [21, 23], "mbp": [21, 23, 24, 27], "fraction": 21, "express": 21, "percentag": [21, 29], "sum": [21, 29], "across": [21, 23], "equal": [21, 24, 29], "averag": [21, 29], "violat": [21, 23, 29], "wise": 21, "record": [21, 29], "experienc": 21, "millisecond": 21, "exce": 21, "surpass": 21, "appropri": 21, "scale": 21, "down": 21, "softmax": 21, "mathemat": 21, "sum_": 21, "a_i": 21, "quad": 21, "frac": 21, "j": 21, "a_j": 21, "num": [21, 24], "factor": 21, "cost": 21, "formul": 21, "left": 21, "text": 21, "_i": 21, "lambda": 21, "cdot": 21, "delay_violation_r": 21, "gamma": 21, "rb_usage_r": 21, "adjust": [21, 23, 27], "emphasi": 21, "consider": 21, "gma": [21, 23, 24, 27, 29], "wifi": [21, 23, 24, 29], "dl": [21, 23, 24, 27, 29], "traffic_ratio": [21, 23, 24, 29], "max_rat": [21, 23, 24, 27, 29], "lte": [21, 22, 23, 24, 27, 29], "3000": 21, "75": 21, "cell_id": [21, 23, 24, 29], "slice_id": [21, 23, 24, 29], "rb_usag": [21, 23, 24, 29], "88": 21, "08": 21, "32": [21, 23, 24], "cell": [21, 23, 24, 29], "65": [21, 23, 24], "68": 21, "33333333": 21, "96": 21, "ul": [21, 23, 24, 29], "missed_act": [21, 23, 24, 29], "measurement_ok": [21, 23, 24, 29], "delay_viol": [21, 23, 24, 29], "11": [21, 23, 24, 29], "delay_test_1_viol": [21, 23, 24, 29], "12": [21, 23, 24], "delay_test_2_viol": [21, 23, 24, 29], "13": [21, 23, 24], "owd": [21, 23, 24, 27, 29], "14": [21, 23, 24], "max_owd": [21, 23, 24, 29], "15": [21, 23, 24], "tx_rate": [21, 23, 24, 29], "056": [21, 23, 24], "942": 21, "16": [21, 23, 24, 29], "prioriti": [21, 23, 24, 29], "17": [21, 23, 24], "18": [21, 23, 24], "19": [21, 23, 24], "20": [21, 23, 24], "21": [21, 23, 24], "22": [21, 23, 24], "23": [21, 23, 24], "24": [21, 23, 24], "25": [21, 23, 24], "26": [21, 23, 24], "27": [21, 23, 24], "28": [21, 23, 24], "29": [21, 23, 24], "31": [21, 23, 24], "x_loc": [21, 23, 24, 29], "83": [21, 24], "836": 21, "78": [21, 23, 24], "46": [21, 23, 24], "913": 21, "607": 21, "33": [21, 23, 24], "y_loc": [21, 23, 24, 29], "645": [21, 23], "388": 21, "493": 21, "089": 21, "242": 21, "34": [21, 23, 24], "35": [21, 23, 24], "36": [21, 23, 24], "37": [21, 23, 24], "100": [21, 23, 24], "39": [21, 23, 24], "uniform": [21, 23, 24], "2d": [21, 23, 24], "plane": [21, 23, 24], "x": [21, 23, 24, 29], "y": [21, 23, 24, 29], "boundari": [21, 23, 24], "instal": [21, 23, 24, 27], "walk": [21, 23, 24], "mobil": [21, 23, 24], "model": [21, 23, 24, 25], "mx": [22, 23, 24], "ratio": [22, 24, 29], "consid": [22, 23, 24], "link": [22, 23, 24, 27, 29], "dimens": 23, "qos_steer": 23, "randomli": [23, 24], "intellig": 23, "direct": [23, 27, 29], "best": 23, "qualiti": 23, "enable_dynamic_flow_priorit": 23, "fals": 23, "contend": 23, "restrict": 23, "admiss": 23, "subset": 23, "dscp": 23, "mark": [23, 29], "qos_requir": 23, "test_duration_m": 23, "durat": 23, "delay_bound_m": [23, 29], "delay_violation_target": 23, "02": 23, "target": 23, "loss_target": 23, "001": 23, "regard": 23, "packet": [23, 29], "otherwis": [23, 29], "met": 23, "set": 23, "zero": 23, "estim": [23, 24, 29], "channel": [23, 24, 25], "capac": [23, 24, 29], "mechan": 23, "maxim": [23, 24], "lead": 23, "essenc": 23, "optim": [23, 25, 28], "indirectli": 23, "overal": 23, "11000": 23, "091666666666667": 23, "09166666666666666": 23, "016": 23, "54": [23, 24], "983333333333334": 23, "05": 23, "033": 23, "061": 23, "044": 23, "03": 23, "split_ratio": [23, 24, 27, 29], "928": 23, "04": 23, "752": 23, "728": 23, "032": 23, "708": 23, "93": [23, 24], "599": 23, "69": 23, "47": [23, 24], "53": [23, 24], "288": 23, "229": 23, "40": [23, 24], "205": 23, "659": 23, "79": 23, "389": 23, "285": 23, "856": 23, "41": [23, 24], "294": 23, "042": 23, "656": 23, "48": [23, 24], "126": 23, "449": 23, "42": [23, 24], "43": [23, 24], "048": 23, "44": [23, 24], "45": [23, 24], "99": 23, "81": 23, "49": [23, 24], "872": 23, "008": 23, "696": [23, 24], "50": [23, 24], "51": [23, 24], "52": [23, 24], "239": 23, "187": 23, "56": [23, 24], "station": [24, 29], "closest": 24, "handov": 24, "disabl": 24, "transmitt": 24, "f": 24, "6000": [24, 29], "625": 24, "192": 24, "208": 24, "272": 24, "792": 24, "429": 24, "964": 24, "721": 24, "716": 24, "616": [24, 29], "024": [24, 29], "57": [24, 29], "144": [24, 29], "202": 24, "118": 24, "312": 24, "217": 24, "161": 24, "359": 24, "119": 24, "287": 24, "023": 24, "871": 24, "703": 24, "198": 24, "715": 24, "676": 24, "021": 24, "988": 24, "216": 24, "73": 24, "712": 24, "568": 24, "432": 24, "117": 24, "310": 24, "61": 24, "innov": 25, "concept": 25, "seen": 25, "four": 25, "kei": 25, "knowledg": 25, "fulli": 25, "customiz": 25, "host": 25, "cloud": 25, "open": 25, "inject": 25, "flexibl": [25, 28], "languag": 25, "platform": 25, "separ": [25, 29], "freedom": 25, "emploi": 25, "smoothli": 25, "c": 25, "hide": 25, "secur": 25, "oversight": 25, "independ": 25, "deploy": 25, "deploi": 25, "workload": 25, "thei": 25, "purpos": [25, 28, 29], "offlin": [25, 28], "own": [25, 28], "special": [25, 28], "art": [25, 28], "sota": [25, 28], "tutori": [25, 27, 28], "guidanc": 25, "instruct": [25, 27, 28], "variou": 25, "convers": 25, "comprehens": 25, "overview": 25, "focus": 25, "layer": 25, "involv": [25, 27], "abstract": 25, "power": 25, "aspect": 25, "endeavor": 25, "full": 25, "stack": 25, "edg": 25, "fidel": 25, "proprietari": [25, 27], "applic": [25, 29], "xapp": 25, "rapp": 25, "ric": 25, "digit": 25, "twin": [25, 28], "section": [26, 27], "outlin": 26, "showcas": 26, "inclus": 27, "extern": 27, "equip": 27, "ll": 27, "ultim": 27, "merg": 27, "procedur": 27, "setup": 27, "port": 27, "alter": 27, "8087": 27, "session_nam": 27, "session_kei": 27, "network_gym_env": 27, "common_config": 27, "accord": 27, "start_custom_env": 27, "py": 27, "don": 27, "modifi": [27, 28], "grant": 27, "Then": 27, "start_client_demo": 27, "print": [27, 29], "custom_act": 27, "custom_tag": 27, "9269856811": 27, "5833047032": 27, "6873341799": 27, "0909367353": 27, "3403381109": 27, "0730143189": 27, "4166952968": 27, "3126658201": 27, "9090632796": 27, "6596618891": 27, "start_t": 27, "end_t": 27, "cid": 27, "20000": 27, "21000": 27, "phy": 27, "sole": 27, "ve": 27, "re": 27, "readi": 27, "mirror": 27, "uml": 27, "sequenc": 27, "diagram": 27, "locat": 27, "experi": 27, "contact": 27, "integr": [27, 28], "fulfil": 28, "simpli": 28, "popular": 28, "ones": 28, "ppo": 28, "proxim": 28, "ddpg": 28, "deep": 28, "determinist": 28, "gradient": 28, "sac": 28, "soft": 28, "actor": 28, "critic": 28, "td3": 28, "a2c": 28, "moreov": 28, "seamlessli": 28, "person": 29, "match": 29, "experiment": 29, "field": 29, "explan": 29, "uniqu": 29, "wherea": 29, "convent": 29, "prefix": 29, "captur": 29, "downlink": 29, "origin": 29, "numer": 29, "structur": 29, "acce": 29, "count": 29, "uplink": 29, "too": 29, "long": 29, "wait": 29, "therefor": 29, "alwai": 29, "ulmeasurement_ok": 29, "deliveri": 29, "threshold": 29, "delay_test_1_thresh_m": 29, "delay_test_2_thresh_m": 29, "reorder": 29, "order": 29, "70": 29, "90": 29, "sinc": 29, "255": 29, "staion": 29, "rtp": 29, "media": 29, "avoid": 29, "techniqu": 29, "histori": 29, "queu": 29, "aggreg": 29, "ecn": 29, "previou": 29, "expert": 29, "inter": 29, "most": 29, "recent": 29, "grow": 29, "subscript": 29, "row": 29, "iterrow": 29, "df_rate": 29}, "objects": {"network_gym_client": [[2, 0, 1, "", "Adapter"], [7, 0, 1, "", "Env"], [8, 0, 1, "", "NorthBoundClient"]], "network_gym_client.Adapter": [[2, 1, 1, "", "df_to_dict"], [2, 1, 1, "", "fill_empty_feature"], [2, 1, 1, "", "wandb_log"], [2, 1, 1, "", "wandb_log_buffer_append"]], "network_gym_client.Env": [[7, 2, 1, "", "action_space"], [7, 2, 1, "", "adapter"], [7, 2, 1, "", "northbound_interface_client"], [7, 2, 1, "", "observation_space"], [7, 1, 1, "", "reset"], [7, 1, 1, "", "step"]], "network_gym_client.NorthBoundClient": [[8, 1, 1, "", "connect"], [8, 1, 1, "", "process_measurement"], [8, 1, 1, "", "recv"], [8, 1, 1, "", "send"]], "network_gym_client.envs.network_slicing": [[3, 0, 1, "", "Adapter"]], "network_gym_client.envs.network_slicing.Adapter": [[3, 1, 1, "", "get_action_space"], [3, 1, 1, "", "get_observation"], [3, 1, 1, "", "get_observation_space"], [3, 1, 1, "", "get_policy"], [3, 1, 1, "", "get_rbg_size"], [3, 1, 1, "", "get_reward"]], "network_gym_client.envs.nqos_split": [[4, 0, 1, "", "Adapter"]], "network_gym_client.envs.nqos_split.Adapter": [[4, 1, 1, "", "get_action_space"], [4, 1, 1, "", "get_observation"], [4, 1, 1, "", "get_observation_space"], [4, 1, 1, "", "get_policy"], [4, 1, 1, "", "get_reward"], [4, 1, 1, "", "netowrk_util"]], "network_gym_client.envs.qos_steer": [[5, 0, 1, "", "Adapter"]], "network_gym_client.envs.qos_steer.Adapter": [[5, 1, 1, "", "calculate_wifi_qos_user_num"], [5, 1, 1, "", "get_action_space"], [5, 1, 1, "", "get_observation"], [5, 1, 1, "", "get_observation_space"], [5, 1, 1, "", "get_policy"], [5, 1, 1, "", "get_reward"]], "network_gym_client.envs.rmcat": [[6, 0, 1, "", "Adapter"]], "network_gym_client.envs.rmcat.Adapter": [[6, 1, 1, "", "get_action_space"], [6, 1, 1, "", "get_observation"], [6, 1, 1, "", "get_observation_space"], [6, 1, 1, "", "get_policy"], [6, 1, 1, "", "get_reward"]], "network_gym_env": [[10, 0, 1, "", "Configure"], [11, 0, 1, "", "DummySim"]], "network_gym_env.Configure": [[10, 1, 1, "", "run"]], "network_gym_env.DummySim": [[11, 1, 1, "", "generate_dummy_measurement"], [11, 1, 1, "", "run_one_interval"], [11, 1, 1, "", "start_simulation"]], "network_gym_env.southbound_interface": [[12, 1, 1, "", "southbound_connect"]]}, "objtypes": {"0": "py:class", "1": "py:function", "2": "py:attribute"}, "objnames": {"0": ["py", "class", "Python class"], "1": ["py", "function", "Python function"], "2": ["py", "attribute", "Python attribute"]}, "titleterms": {"build": 0, "document": 0, "websit": 0, "instal": 0, "depend": 0, "us": 0, "tip": 0, "auto": 0, "gener": 0, "rst": 0, "file": 0, "add": 0, "class": [0, 2], "function": 0, "md": 0, "network_gym_cli": [1, 7, 8], "adapt": [2, 3, 4, 5, 6], "The": 2, "base": 2, "method": [2, 3, 4, 5, 6, 7, 8, 10, 11, 12], "network_sl": 3, "reward": [3, 4, 5, 6, 19, 21, 23, 24], "addit": [3, 4, 5, 6, 8], "nqos_split": 4, "qos_steer": 5, "rmcat": [6, 19], "env": 7, "attribut": 7, "northbound": 8, "network_gym_env": [9, 10, 11, 12], "networkgym": [9, 15, 16, 17, 25], "uml": 9, "sequenc": 9, "diagram": 9, "configur": 10, "dummi": [11, 27], "simul": [11, 17, 27], "dummysim": 11, "southbound": 12, "network_gym_serv": 13, "contact": 14, "motiv": 15, "network": [15, 20, 21, 25], "ai": [15, 25], "model": 15, "algorithm": [15, 28], "develop": 15, "cycl": 15, "challeng": 15, "face": 15, "": 15, "approach": 15, "address": 15, "thi": 15, "overview": 16, "compon": 16, "interfac": 16, "client": [16, 17], "api": [16, 25], "server": 16, "environ": [16, 17, 27], "quickstart": 17, "access": [17, 23, 24], "servic": 17, "via": [17, 25], "vlab": 17, "basic": 17, "usag": 17, "upon": 17, "start": [17, 21, 23, 24], "follow": 17, "seri": 17, "step": 17, "occur": 17, "dure": 17, "process": 17, "repeat": 17, "when": 17, "conclud": 17, "initi": [17, 27], "interact": 17, "explain": 17, "code": [17, 26], "action": [17, 19, 21, 23, 24], "observ": [17, 19, 21, 23, 24], "space": [17, 19, 21, 23, 24], "modifi": 17, "congest": 18, "control": 18, "descript": [19, 21, 23, 24], "prerequisit": [19, 21, 23, 24], "custom": [19, 21, 23, 24, 27, 28], "argument": [19, 21, 23, 24], "episod": [19, 21, 23, 24], "end": [19, 21, 23, 24], "slice": [20, 21], "cellular": 21, "state": [21, 23, 24], "traffic": [22, 23, 24], "manag": 22, "multi": [23, 24], "qo": 23, "steer": 23, "split": 24, "transit": 24, "dynam": 24, "democrat": 25, "sim": 25, "aa": 25, "object": 25, "agent": [25, 28], "gymnasium": 25, "scope": 25, "limit": [25, 26], "handl": 26, "time": 26, "truncat": 26, "termin": 26, "sequenti": 26, "train": [26, 28], "exampl": 26, "python": 26, "implement": 27, "your": 27, "connect": 27, "substitut": 27, "releas": 27, "system": 28, "default": 28, "stabl": 28, "baselines3": 28, "cleanrl": 28, "work": 29, "measur": 29, "column": 29, "support": 29, "per": 29, "sourc": 29, "retriev": 29}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.viewcode": 1, "sphinx": 57}, "alltitles": {"Build Documentation Website": [[0, "build-documentation-website"]], "Install Dependencies": [[0, "install-dependencies"]], "Build Website": [[0, "build-website"]], "Useful Tips": [[0, "useful-tips"]], "Auto Generate rst Files": [[0, "auto-generate-rst-files"]], "Add Classes and Functions to md Files": [[0, "add-classes-and-functions-to-md-files"]], "network_gym_client": [[1, "network-gym-client"]], "Adapter": [[2, "adapter"]], "The Base Class": [[2, "the-base-class"]], "Methods": [[2, "methods"], [3, "methods"], [4, "methods"], [5, "methods"], [6, "methods"], [7, "methods"], [8, "methods"], [10, "methods"], [11, "methods"], [12, "methods"]], "network_slicing Adapter": [[3, "network-slicing-adapter"]], "": [[3, "id1"], [4, "id1"], [5, "id1"], [6, "id1"]], "Reward Methods": [[3, "reward-methods"], [5, "reward-methods"], [6, "reward-methods"]], "Additional Methods": [[3, "additional-methods"], [4, "additional-methods"], [5, "additional-methods"], [6, "additional-methods"], [8, "additional-methods"]], "nqos_split Adapter": [[4, "nqos-split-adapter"]], "Reward Method": [[4, "reward-method"]], "qos_steer Adapter": [[5, "qos-steer-adapter"]], "rmcat Adapter": [[6, "rmcat-adapter"]], "Env": [[7, "env"]], "network_gym_client.Env": [[7, "network-gym-client-env"]], "Attributes": [[7, "attributes"]], "NorthBound": [[8, "northbound"]], "network_gym_client.NorthBound": [[8, "network-gym-client-northbound"]], "network_gym_env": [[9, "network-gym-env"]], "NetworkGym UML Sequence Diagram": [[9, "networkgym-uml-sequence-diagram"]], "Configure": [[10, "configure"]], "network_gym_env.Configure": [[10, "network-gym-env-configure"]], "Dummy Simulator": [[11, "dummy-simulator"]], "network_gym_env.DummySim": [[11, "network-gym-env-dummysim"]], "SouthBound": [[12, "southbound"]], "network_gym_env.SouthBound": [[12, "network-gym-env-southbound"]], "network_gym_server": [[13, "network-gym-server"]], "Contact": [[14, "contact"]], "Motivation": [[15, "motivation"]], "Network AI Models/Algorithms Development Cycle": [[15, "network-ai-models-algorithms-development-cycle"]], "Challenges Faced by Network AI Developers": [[15, "challenges-faced-by-network-ai-developers"]], "NetworkGym\u2019s Approach to Addressing this Challenge": [[15, null], [15, null], [15, null]], "Overview": [[16, "overview"]], "NetworkGym Components and Interfaces": [[16, "networkgym-components-and-interfaces"]], "Client API": [[16, "client-api"]], "Server API": [[16, "server-api"]], "Environment API": [[16, "environment-api"]], "Quickstart": [[17, "quickstart"]], "Accessing the NetworkGym Service via vLab": [[17, "accessing-the-networkgym-service-via-vlab"]], "Basic Usage": [[17, "basic-usage"]], "\u25b6\ufe0f Upon starting the Client, the following series of steps occur:": [[17, null]], "\ud83d\udd01 During the simulation, the process repeats as follows:": [[17, null]], "\u23f9\ufe0f When the NetworkGym or the simulation concludes:": [[17, null]], "Initializing Environments": [[17, "initializing-environments"]], "Interacting with the Environment": [[17, "interacting-with-the-environment"]], "Explaining the code": [[17, "explaining-the-code"]], "Action and observation spaces": [[17, "action-and-observation-spaces"]], "Modifying the environment": [[17, "modifying-the-environment"]], "Congestion Control": [[18, "congestion-control"]], "RMCAT": [[19, "rmcat"]], "Description": [[19, "description"], [21, "description"], [23, "description"], [24, "description"]], "Prerequisite": [[19, "prerequisite"], [21, "prerequisite"], [23, "prerequisite"], [24, "prerequisite"]], "Observation Space": [[19, "observation-space"], [21, "observation-space"], [23, "observation-space"], [24, "observation-space"]], "Action Space": [[19, "action-space"], [21, "action-space"], [23, "action-space"], [24, "action-space"]], "Reward": [[19, "reward"], [21, "reward"], [23, "reward"], [24, "reward"]], "Custom Observation Space and Reward": [[19, "custom-observation-space-and-reward"], [21, "custom-observation-space-and-reward"], [23, "custom-observation-space-and-reward"], [24, "custom-observation-space-and-reward"]], "Arguments": [[19, "arguments"], [21, "arguments"], [23, "arguments"], [24, "arguments"]], "Episode End": [[19, "episode-end"], [21, "episode-end"], [23, "episode-end"], [24, "episode-end"]], "Network Slicing": [[20, "network-slicing"]], "Cellular Network Slicing": [[21, "cellular-network-slicing"]], "Starting State": [[21, "starting-state"], [23, "starting-state"], [24, "starting-state"]], "Traffic Management": [[22, "traffic-management"]], "Multi-Access QoS Traffic Steering": [[23, "multi-access-qos-traffic-steering"]], "Multi-Access Traffic Splitting": [[24, "multi-access-traffic-splitting"]], "Transition Dynamics": [[24, "transition-dynamics"]], "NetworkGym: Democratizing Network AI via Sim-aaS.": [[25, "networkgym-democratizing-network-ai-via-sim-aas"]], "Objectives": [[25, "objectives"]], "Agent": [[25, "agent"]], "Gymnasium API": [[25, "gymnasium-api"]], "NetworkGym API": [[25, "networkgym-api"]], "NetworkGym Scope and Limitations": [[25, "networkgym-scope-and-limitations"]], "Handling Time Limits": [[26, "handling-time-limits"]], "Truncation": [[26, "truncation"]], "Termination": [[26, "termination"]], "Sequential Training Example": [[26, "sequential-training-example"]], "Python code:": [[26, "python-code"]], "Implementing Custom Environment": [[27, "implementing-custom-environment"]], "Initiating your Custom Environment": [[27, "initiating-your-custom-environment"]], "Connecting to Your Custom Environment": [[27, "connecting-to-your-custom-environment"]], "Substituting the Dummy Simulator": [[27, "substituting-the-dummy-simulator"]], "Releasing Your Custom Environment": [[27, "releasing-your-custom-environment"]], "Training Agents": [[28, "training-agents"]], "System Default Agent": [[28, "system-default-agent"]], "Custom Algorithm Agent": [[28, "custom-algorithm-agent"]], "Stable-Baselines3 Agent": [[28, "stable-baselines3-agent"]], "CleanRL Agent": [[28, "cleanrl-agent"]], "Working with Measurements": [[29, "working-with-measurements"]], "Measurement Columns": [[29, "measurement-columns"]], "Supported Measurements per Source": [[29, "supported-measurements-per-source"]], "Retrieving a Measurement": [[29, "retrieving-a-measurement"]]}, "indexentries": {"adapter (class in network_gym_client)": [[2, "network_gym_client.Adapter"]], "df_to_dict() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.df_to_dict"]], "fill_empty_feature() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.fill_empty_feature"]], "wandb_log() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.wandb_log"]], "wandb_log_buffer_append() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.wandb_log_buffer_append"]], "adapter (class in network_gym_client.envs.network_slicing)": [[3, "network_gym_client.envs.network_slicing.Adapter"]], "get_action_space() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_policy"]], "get_rbg_size() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_rbg_size"]], "get_reward() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_reward"]], "adapter (class in network_gym_client.envs.nqos_split)": [[4, "network_gym_client.envs.nqos_split.Adapter"]], "get_action_space() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_reward"]], "netowrk_util() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.netowrk_util"]], "adapter (class in network_gym_client.envs.qos_steer)": [[5, "network_gym_client.envs.qos_steer.Adapter"]], "calculate_wifi_qos_user_num() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num"]], "get_action_space() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_reward"]], "adapter (class in network_gym_client.envs.rmcat)": [[6, "network_gym_client.envs.rmcat.Adapter"]], "get_action_space() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_reward"]], "env (class in network_gym_client)": [[7, "network_gym_client.Env"]], "action_space (network_gym_client.env attribute)": [[7, "network_gym_client.Env.action_space"]], "adapter (network_gym_client.env attribute)": [[7, "network_gym_client.Env.adapter"]], "northbound_interface_client (network_gym_client.env attribute)": [[7, "network_gym_client.Env.northbound_interface_client"]], "observation_space (network_gym_client.env attribute)": [[7, "network_gym_client.Env.observation_space"]], "reset() (in module network_gym_client.env)": [[7, "network_gym_client.Env.reset"]], "step() (in module network_gym_client.env)": [[7, "network_gym_client.Env.step"]], "northboundclient (class in network_gym_client)": [[8, "network_gym_client.NorthBoundClient"]], "connect() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.connect"]], "process_measurement() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.process_measurement"]], "recv() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.recv"]], "send() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.send"]], "configure (class in network_gym_env)": [[10, "network_gym_env.Configure"]], "run() (in module network_gym_env.configure)": [[10, "network_gym_env.Configure.run"]], "dummysim (class in network_gym_env)": [[11, "network_gym_env.DummySim"]], "generate_dummy_measurement() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.generate_dummy_measurement"]], "run_one_interval() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.run_one_interval"]], "start_simulation() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.start_simulation"]], "southbound_connect() (in module network_gym_env.southbound_interface)": [[12, "network_gym_env.southbound_interface.southbound_connect"]]}}) \ No newline at end of file +Search.setIndex({"docnames": ["README", "api/network_gym_client", "api/network_gym_client/adapter", "api/network_gym_client/adapter/network_slicing", "api/network_gym_client/adapter/nqos_split", "api/network_gym_client/adapter/qos_steer", "api/network_gym_client/adapter/rmcat", "api/network_gym_client/env", "api/network_gym_client/northbound_interface", "api/network_gym_env", "api/network_gym_env/configure", "api/network_gym_env/simulator", "api/network_gym_env/southbound_interface", "api/network_gym_server", "contact", "content/motivation", "content/overview", "content/quickstart", "environments/congestion_control", "environments/congestion_control/rmcat", "environments/mx_network_slicing", "environments/mx_network_slicing/cellular_network_slicing", "environments/mx_traffic_management", "environments/mx_traffic_management/mx_qos_traffic_steering", "environments/mx_traffic_management/mx_traffic_splitting", "index", "tutorials/handling_time_limits", "tutorials/implementing_custom_environment", "tutorials/training_agents", "tutorials/working_with_measurements"], "filenames": ["README.md", "api/network_gym_client.md", "api/network_gym_client/adapter.md", "api/network_gym_client/adapter/network_slicing.md", "api/network_gym_client/adapter/nqos_split.md", "api/network_gym_client/adapter/qos_steer.md", "api/network_gym_client/adapter/rmcat.md", "api/network_gym_client/env.md", "api/network_gym_client/northbound_interface.md", "api/network_gym_env.md", "api/network_gym_env/configure.md", "api/network_gym_env/simulator.md", "api/network_gym_env/southbound_interface.md", "api/network_gym_server.md", "contact.md", "content/motivation.md", "content/overview.md", "content/quickstart.md", "environments/congestion_control.md", "environments/congestion_control/rmcat.md", "environments/mx_network_slicing.md", "environments/mx_network_slicing/cellular_network_slicing.md", "environments/mx_traffic_management.md", "environments/mx_traffic_management/mx_qos_traffic_steering.md", "environments/mx_traffic_management/mx_traffic_splitting.md", "index.md", "tutorials/handling_time_limits.md", "tutorials/implementing_custom_environment.md", "tutorials/training_agents.md", "tutorials/working_with_measurements.md"], "titles": ["Build Documentation Website", "network_gym_client", "Adapter", "network_slicing Adapter", "nqos_split Adapter", "qos_steer Adapter", "rmcat Adapter", "Env", "NorthBound", "network_gym_env", "Configure", "Dummy Simulator", "SouthBound", "network_gym_server", "Contact", "Motivation", "Overview", "Quickstart", "Congestion Control", "RMCAT", "Network Slicing", "Cellular Network Slicing", "Traffic Management", "Multi-Access QoS Traffic Steering", "Multi-Access Traffic Splitting", "NetworkGym: Democratizing Network AI via Sim-aaS.", "Handling Time Limits", "Implementing Custom Environment", "Training Agents", "Working with Measurements"], "terms": {"option": [0, 1, 7, 17, 19, 21, 23, 24, 25, 26], "creat": [0, 11, 17, 25, 27, 28], "new": [0, 11, 17, 29], "virtual": 0, "python": [0, 17, 25], "environ": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29], "python3": [0, 27], "m": [0, 19, 21, 27, 29], "venv": 0, "sphinx_venv": 0, "sourc": [0, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 19, 21, 23, 24, 25, 28], "bin": 0, "activ": [0, 9, 16, 17, 28], "requir": [0, 15, 19, 21, 23, 24, 29], "librari": 0, "pip": 0, "r": [0, 21, 24], "txt": 0, "In": [0, 9, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27], "doc": 0, "folder": 0, "run": [0, 7, 10, 11, 17, 19, 21, 23, 24, 26], "follow": [0, 1, 7, 10, 19, 21, 23, 24, 25, 27, 29], "command": [0, 27], "export": 0, "static": 0, "_build": 0, "html": 0, "make": [0, 17], "clean": 0, "network_gym_cli": [0, 2, 3, 4, 5, 6, 17, 19, 21, 23, 24, 26], "The": [0, 1, 7, 9, 10, 11, 13, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "contain": [0, 7, 17, 21, 29], "modul": [0, 9], "rm": 0, "sphinx": 0, "apidoc": 0, "o": [0, 17, 25], "d": 0, "2": [0, 7, 19, 21, 23, 24, 25, 27, 29], "markdown": 0, "e": [0, 1, 7, 10, 11, 15, 19, 21, 23, 24, 25, 26, 29], "g": [0, 1, 7, 10, 11, 15, 23, 24, 25, 29], "eval": 0, "autoclass": 0, "env": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 17, 19, 21, 23, 24, 26, 27, 28], "networkgym": [1, 7, 8, 13, 19, 21, 23, 24, 26, 27, 28], "client": [1, 7, 8, 9, 10, 11, 13, 19, 21, 23, 24, 25, 27], "includ": [1, 13, 17, 19, 21, 23, 24, 25, 28, 29], "three": [1, 16, 19, 21, 23, 24, 25, 26], "compon": [1, 9, 10, 13, 25], "custom": [1, 7, 10, 16, 17, 25, 26], "gymnasium": [1, 2, 7, 15, 16, 17, 28], "adapt": [1, 7, 16, 19, 21, 23, 24, 27, 29], "northbound": [1, 7, 13, 16, 17, 25], "interfac": [1, 7, 8, 9, 12, 13, 17, 25, 27], "inheret": 1, "class": [1, 3, 4, 5, 6, 7, 8, 10, 11, 17, 27, 29], "commun": [1, 7, 9, 16, 17], "agent": [1, 3, 4, 5, 6, 7, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29], "us": [1, 7, 10, 11, 15, 17, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29], "standard": [1, 15, 25, 27], "exchang": [1, 9, 10, 17], "ob": [1, 2, 7, 17, 26], "reward": [1, 2, 7, 17, 25, 26, 29], "action": [1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 18, 20, 22, 25, 26, 27, 28, 29], "reset": [1, 7, 17, 26], "step": [1, 7, 19, 21, 23, 24, 26, 27], "function": [1, 2, 3, 4, 5, 6, 17, 19, 21, 23, 24, 25, 29], "import": [1, 17, 19, 21, 23, 24, 26], "gym": [1, 7, 8, 25], "def": 1, "__init__": 1, "self": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11], "arg1": 1, "arg2": 1, "super": 1, "return": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 17], "observ": [1, 3, 4, 5, 6, 7, 25, 28, 29], "termin": [1, 7, 9, 10, 11, 17, 19, 21, 23, 24, 27], "truncat": [1, 7, 17, 19, 21, 23, 24], "info": [1, 2, 7, 17, 26, 27, 28], "seed": [1, 7], "none": [1, 7], "transform": [1, 2, 16], "data": [1, 2, 5, 9, 16, 17, 19, 21, 23, 24, 25, 27, 28, 29], "format": [1, 2, 16, 17, 25, 27], "from": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 17, 19, 21, 23, 24, 25, 26, 28, 29], "network_gym": [1, 2, 7], "other": [1, 10, 26], "wai": [1, 7, 24, 29], "around": [1, 25], "network": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 17, 19, 23, 24, 27, 28, 29], "stat": [1, 2, 3, 4, 5, 6, 8, 11, 25], "chang": [1, 26], "polici": [1, 2, 3, 4, 5, 6, 8, 9, 17, 28, 29], "connect": [1, 7, 8, 9, 10, 11, 12, 13, 16, 17, 22, 23, 24, 25], "server": [1, 7, 8, 9, 10, 11, 12, 13, 17, 19, 21, 23, 24, 25, 27], "configur": [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 17, 19, 21, 23, 24, 25, 26, 29], "paramet": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 19, 21, 23, 24, 26], "between": [1, 2, 7, 9, 10, 16, 24, 25, 27], "envrion": [1, 13], "config_json": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26], "thi": [2, 3, 4, 5, 6, 7, 9, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 29], "i": [2, 3, 4, 7, 8, 10, 11, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "an": [2, 7, 9, 10, 13, 15, 16, 17, 19, 21, 23, 24, 25, 26, 28, 29], "It": [2, 7, 9, 16, 17, 19, 21, 23, 24, 25, 26], "measur": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 17, 19, 21, 23, 24, 25, 27], "json": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26, 27, 29], "space": [2, 3, 4, 5, 6, 7], "also": [2, 7, 15, 17, 21, 23, 24, 25, 26, 27], "can": [2, 7, 9, 10, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "appli": [2, 29], "initi": [2, 3, 4, 5, 6, 7, 8, 9, 10, 21], "file": [2, 3, 4, 5, 6, 7, 8, 11, 12, 17, 19, 21, 23, 24, 26, 27, 29], "wandb_log_buffer_append": 2, "add": [2, 13, 17, 19, 21, 23, 24, 29], "wandb": [2, 17], "log": [2, 7, 24], "buffer": 2, "send": [2, 7, 8, 9, 11, 17, 18, 19, 25, 27, 28, 29], "later": 2, "wandb_log": 2, "dict": [2, 7, 11, 17], "inform": [2, 7, 17, 19, 23, 24, 25], "append": [2, 19, 21, 23, 24, 29], "df_to_dict": 2, "df": [2, 3, 4, 5, 6, 29], "id_nam": 2, "id": [2, 7, 8, 10, 19, 21, 23, 24, 29], "datatyp": 2, "panda": [2, 5], "datafram": [2, 3, 4, 5, 6, 8], "dictionari": [2, 7, 17], "object": [2, 7, 21, 29], "descript": [2, 29], "string": 2, "descritpt": 2, "convert": 2, "fill_empty_featur": 2, "featur": [2, 3, 4, 5, 6, 19, 21, 23, 24, 25], "valu": [2, 4, 7, 17, 19, 21, 23, 24, 27, 29], "fill": 2, "miss": [2, 29], "input": [2, 17, 29], "pd": [2, 3, 4, 5, 6, 8], "int": [2, 3, 7, 8, 10, 11], "list": [2, 7, 10, 11, 17, 19, 21, 23, 24, 29], "result": [2, 7, 11, 17, 19, 21, 23, 24, 26], "after": [2, 7, 17, 19, 21, 23, 24, 26, 29], "replac": 2, "base": [3, 4, 5, 6, 9, 16, 17, 19, 24, 25, 29], "get_observ": [3, 4, 5, 6, 19, 21, 23, 24, 29], "prepar": [3, 4, 5, 6], "should": [3, 4, 5, 6, 7, 17, 21, 27, 29], "same": [3, 4, 5, 6, 10, 21, 27], "number": [3, 4, 5, 6, 7, 11, 17, 19, 21, 23, 24, 26, 27], "defin": [3, 4, 5, 6, 7, 17, 19, 21, 23, 24, 25, 28], "get_observation_spac": [3, 4, 5, 6], "get_reward": [3, 4, 5, 6, 19, 21, 23, 24, 29], "get_polici": [3, 4, 5, 6], "rl": [3, 4, 5, 6, 7, 17, 25, 28, 29], "No": [3, 17, 25, 28], "yet": [3, 19], "get_action_spac": [3, 4, 5, 6], "get": [3, 4, 5, 6, 7], "get_rbg_siz": 3, "bandwidth": [3, 23, 29], "comput": [3, 17, 19, 21, 23, 24, 25, 29], "resourc": [3, 20, 21, 23, 24, 29], "block": [3, 21, 29], "group": [3, 21, 27], "size": [3, 29], "bandwith": 3, "rb": [3, 21], "code": [3, 10, 28, 29], "coppi": 3, "ns3": [3, 19], "pf": 3, "type": [3, 7, 17, 25, 27], "0": [3, 4, 7, 17, 19, 21, 23, 24, 26, 27, 29], "alloc": [3, 20, 21], "rbg": 3, "resouc": 3, "netowrk_util": [4, 24], "throughput": [4, 19, 21, 23, 24, 29], "delai": [4, 7, 21, 23, 24, 28, 29], "alpha": 4, "5": [4, 19, 21, 23, 24, 27, 29], "calcul": [4, 5, 23, 29], "util": [4, 10, 16, 21, 23, 24, 25, 27, 29], "specifi": [4, 7, 17, 19, 21, 23, 24, 26], "balanc": 4, "default": [4, 5, 7, 17, 19, 21, 25, 29], "float": 4, "repres": [4, 17, 19, 21, 23, 24, 29], "bit": 4, "per": [4, 5, 19, 21, 23, 24, 26], "second": [4, 17, 19, 23, 24, 26], "metric": [4, 17], "calculate_wifi_qos_user_num": [5, 23], "qos_rat": [5, 21, 23, 24, 27, 29], "qo": [5, 15, 17, 22, 29], "user": [5, 7, 11, 17, 21, 22, 23, 24, 25, 27, 28, 29], "over": [5, 22, 23, 24, 29], "wi": [5, 22, 23, 24, 27], "fi": [5, 22, 23, 24, 27], "rate": [5, 18, 19, 21, 23, 24, 27, 29], "doubl": 5, "initil": [7, 11], "intern": [7, 17], "state": [7, 17, 25, 28], "": [7, 9, 10, 17, 19, 21, 23, 24, 26, 27, 28, 29], "prng": 7, "np_random": 7, "If": [7, 17, 19, 23, 26, 29], "doe": [7, 25], "alreadi": [7, 23], "have": [7, 17, 19, 21, 23, 24, 25, 26, 28], "pass": 7, "chosen": 7, "some": [7, 17, 29], "entropi": 7, "timestamp": [7, 11, 29], "dev": 7, "urandom": 7, "howev": [7, 23, 29], "ha": [7, 17, 19, 21, 23, 24, 26], "you": [7, 16, 17, 19, 21, 23, 24, 26, 27, 28, 29], "integ": 7, "even": 7, "exist": [7, 29], "usual": 7, "want": 7, "right": [7, 21], "been": [7, 17, 28], "never": 7, "again": 7, "pleas": [7, 25, 27], "refer": [7, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29], "minim": [7, 19, 21, 23, 24], "exampl": [7, 17, 19, 21, 23, 24, 29], "abov": [7, 25], "see": [7, 19, 21, 23, 24], "paradigm": 7, "addit": [7, 17, 25], "how": [7, 16, 17, 25, 26, 27], "depend": [7, 27], "specif": [7, 15, 16, 19, 21, 23, 24, 25, 26, 28, 29], "obstyp": 7, "auxiliari": 7, "complement": 7, "analog": 7, "one": [7, 9, 11, 17, 21, 24, 26, 29], "timestep": [7, 17], "dynam": [7, 23, 29], "gamsim": 7, "check": 7, "last": [7, 26, 29], "episod": [7, 17, 26, 27], "done": [7, 17], "acttyp": 7, "provid": [7, 10, 16, 17, 19, 23, 24, 25, 26, 27, 28, 29], "updat": [7, 17, 22, 24], "element": [7, 16, 17, 29], "observation_spac": [7, 17], "next": [7, 17], "due": 7, "supportsfloat": 7, "take": [7, 17, 21, 23, 29], "bool": 7, "whether": 7, "reach": [7, 17], "under": 7, "mdp": 7, "task": [7, 27], "which": [7, 16, 17, 21, 23], "posit": [7, 21, 23, 24], "neg": 7, "goal": [7, 19, 23, 24, 29], "move": 7, "lava": 7, "sutton": 7, "barton": 7, "gridworld": 7, "true": [7, 17, 23], "need": [7, 15, 17, 21], "call": [7, 17], "condit": 7, "outsid": 7, "scope": 7, "satisfi": 7, "typic": 7, "timelimit": [7, 17], "could": [7, 17, 29], "indic": [7, 21, 23, 29], "physic": [7, 25], "go": 7, "out": [7, 17, 25, 29], "bound": [7, 17, 29], "end": [7, 9, 17, 25, 26], "prematur": 7, "befor": 7, "diagnost": 7, "help": [7, 17], "debug": 7, "learn": [7, 15, 25, 27, 28], "raw": 7, "flag": 7, "deprec": 7, "A": [7, 11, 17, 19, 21, 23, 24, 25, 26, 28, 29], "boolean": 7, "case": [7, 15, 17, 21, 25, 29], "further": [7, 17, 19, 21, 23, 24], "undefin": 7, "wa": [7, 29], "remov": 7, "openai": 7, "v26": 7, "favor": 7, "signal": [7, 9, 10, 17, 24, 29], "mai": [7, 17, 26], "emit": 7, "differ": [7, 17, 25, 26], "reason": 7, "mayb": 7, "underli": [7, 10], "solv": 7, "successfulli": [7, 27], "certain": [7, 19, 21, 23, 24, 26], "exceed": [7, 17], "simul": [7, 9, 10, 15, 16, 25, 29], "enter": 7, "invalid": 7, "action_spac": [7, 17, 26, 28], "correspond": [7, 16, 29], "valid": [7, 17, 29], "all": [7, 10, 17, 19, 20, 21, 23, 24, 27, 29], "For": [7, 17, 19, 21, 23, 24, 25, 27, 29], "discret": [7, 17], "give": 7, "mean": 7, "ar": [7, 9, 10, 17, 19, 21, 23, 24, 27], "two": [7, 9, 13, 16, 25, 26], "1": [7, 17, 19, 21, 23, 24, 26, 27, 29], "box": [7, 16, 17, 19, 21, 23, 24], "3": [7, 9, 10, 15, 16, 17, 19, 21, 23, 24, 25, 26, 27, 29], "4028234663852886e": 7, "38": [7, 21, 23, 24], "4": [7, 19, 21, 23, 24, 27, 29], "float32": [7, 19, 21, 23, 24], "shape": [7, 17, 19, 21, 23, 24], "denot": [7, 19, 21, 23, 24, 26, 29], "arrai": [7, 17, 21, 28], "we": [7, 13, 17, 21, 23, 24, 25, 26], "well": 7, "high": [7, 19, 23, 24, 25, 29], "8000002e": 7, "00": 7, "4028235e": 7, "1887903e": 7, "01": 7, "dtype": 7, "low": [7, 19, 24, 29], "northbound_interface_cli": 7, "traslat": 7, "dataformat": 7, "northboundcli": 8, "sever": 8, "repli": 8, "start": [8, 9, 10, 11, 19, 26, 27], "request": [8, 9], "where": [8, 17, 21, 24, 29], "load": [8, 17, 19, 21, 23, 24, 29], "recv": 8, "receiv": [8, 9, 11, 17, 19, 21, 23, 24, 27, 29], "messag": [8, 9, 10], "process_measur": 8, "reply_json": 8, "process": [8, 9, 26, 27], "compris": [9, 16], "core": 9, "These": [9, 28], "establish": [9, 10, 16, 17, 27], "via": [9, 12, 27], "design": [9, 21, 25, 28], "southbound": [9, 11, 13, 16, 17, 25, 27], "serv": [9, 21, 25], "bridg": 9, "facilit": [9, 10, 16, 17, 25], "statist": [9, 29], "keep": [9, 16], "aliv": 9, "period": [9, 10, 18, 19, 20, 22, 23, 24], "dispatch": [9, 10], "hello": [9, 10, 27], "upon": [9, 10], "disengag": 9, "n": [9, 10, 15, 16, 17, 21, 23, 24, 25], "smulat": 9, "instig": 9, "plan": [9, 13, 17], "incorpor": 9, "emul": [9, 16], "testb": [9, 16, 27, 29], "altern": 9, "earmark": 9, "futur": 9, "releas": [9, 13], "depict": [9, 16], "flow": [9, 18, 19, 23, 29], "complet": [9, 27], "cycl": 9, "within": [9, 16, 17, 21, 27, 29], "session": [9, 16, 17, 19, 21, 23, 24, 26], "launch": [9, 10, 17, 27], "env_config": [9, 10, 11, 26, 27], "trigger": [9, 28], "gener": [9, 11, 19, 21, 23, 24, 26, 27, 29], "incom": 9, "onc": [9, 17, 19, 21, 23, 24, 26, 27], "conclud": 9, "map": [9, 13, 16, 17], "onli": [9, 10, 19, 21, 23, 24, 27, 29], "env_sim": [9, 10, 11, 27], "ani": [9, 17, 23, 25], "given": [9, 19, 21, 23, 24], "moment": [9, 21], "cannot": [9, 19, 21, 23, 24, 26], "occur": 9, "simultan": 9, "standpoint": 9, "treat": 9, "both": [9, 17, 19, 21, 22, 23, 24, 26, 29], "equival": 9, "identifi": [9, 29], "them": [9, 17, 21, 25], "entiti": [9, 10, 25, 27, 29], "networkgymsim": 10, "env_list": [10, 27], "recept": 10, "its": [10, 16, 19, 21, 23, 24, 29], "socket": [10, 11, 12, 27], "ident": [10, 11, 12], "identif": 10, "subsequ": [10, 19, 21, 23, 24, 26], "commenc": 10, "conclus": 10, "close": 10, "reconnect": 10, "point": [10, 19, 21, 23, 24, 26, 29], "perceiv": 10, "divis": 10, "advantag": [10, 28], "straightforward": [10, 17, 27], "expans": 10, "test": [10, 15, 23, 27, 29], "while": [10, 20, 21, 26], "str": [10, 11, 12], "support": [10, 16, 17, 23, 27], "non": [10, 17], "offic": 10, "account": [10, 21, 29], "name": [10, 11, 17, 19, 21, 23, 24, 27, 29], "env_ident": 11, "client_ident": 11, "msg_json": 11, "random": [11, 17, 21, 23, 24], "sampl": [11, 17, 26, 28], "when": [11, 13, 19, 21, 23, 24, 25, 27, 28, 29], "resum": 11, "env_idennt": 11, "env_port": 11, "first": [11, 17, 19, 23, 24, 26], "who": 11, "msg": [11, 27], "cliet": 11, "generate_dummy_measur": 11, "tag": [11, 27, 29], "num_us": 11, "ad": 11, "start_simul": 11, "api": [11, 15, 28], "report": 11, "run_one_interv": 11, "interv": [11, 17, 29], "southbound_interfac": 12, "southbound_connect": 12, "indent": 12, "zmq": 12, "manag": [13, 17, 23, 24, 25], "select": [13, 16, 17, 19, 21, 22, 23, 24, 27], "idl": 13, "instanc": [13, 17, 25, 27, 29], "rout": [13, 16], "tabl": 13, "At": [13, 16, 19, 21, 23, 24, 26], "present": [13, 16, 17, 23, 27], "do": 13, "softwar": [13, 25], "email": 14, "github": [14, 19, 21, 23, 24, 27], "slack": 14, "real": [15, 18, 19], "world": [15, 25], "dataset": 15, "control": [15, 19, 23, 25], "oper": [15, 19, 21, 23, 24, 25, 26], "difficult": 15, "acquir": 15, "align": [15, 27], "usag": [15, 21, 29], "itself": 15, "enough": 15, "train": [15, 17, 19, 21, 23, 24, 25], "reinforc": [15, 25, 28], "etc": [15, 25], "current": [15, 29], "enabl": [15, 16, 17, 19, 21, 23, 24, 25, 28, 29], "multi": [15, 29], "access": [15, 19, 21, 25, 27, 29], "traffic": [15, 18, 19, 29], "split": [15, 22, 29], "awar": 15, "steer": [15, 22], "cellular": [15, 20, 23, 24], "ran": [15, 17, 25], "slice": [15, 23, 24, 29], "tool": [15, 25], "often": 15, "veri": 15, "complex": [15, 29], "especi": 15, "research": [15, 16, 17, 25], "without": [15, 23, 25], "expertis": [15, 25], "lack": 15, "common": 15, "simpl": [15, 17], "evalu": [15, 23, 27], "benchmark": 15, "adher": [15, 21], "addition": [15, 16, 17, 27], "offer": [15, 16, 25, 28], "framework": [16, 25], "consist": [16, 17, 25], "essenti": [16, 17, 25, 27], "each": [16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29], "plai": [16, 25], "crucial": [16, 25], "role": [16, 25, 28], "system": [16, 17, 21, 25, 29], "encompass": 16, "seamless": 16, "interact": [16, 25, 26, 27, 28], "displai": 16, "below": [16, 17, 26, 28, 29], "graphic": 16, "architectur": [16, 25], "visual": [16, 17], "aid": 16, "divers": [16, 27], "collabor": [16, 17], "harmoni": 16, "form": 16, "streamlin": [16, 25, 27], "effect": [16, 17, 25], "servic": [16, 20, 21, 23, 25], "tailor": [16, 21, 25], "advanc": 16, "ai": [16, 17], "develop": [16, 17, 25], "By": [16, 21, 25, 29], "click": [16, 19, 21, 23, 24, 29], "blue": 16, "navig": 16, "directli": 16, "page": [16, 25], "respect": [16, 17, 21, 25], "main": 16, "respons": 16, "allow": [16, 17, 25, 28], "compat": 16, "like": [16, 21, 25], "stabl": [16, 25], "baselines3": [16, 25], "cleanrl": [16, 25], "desir": [16, 23], "central": 16, "maintain": [16, 25], "track": 16, "assign": [16, 21, 23, 24, 27], "dure": [16, 23], "either": [16, 17, 19, 21, 23, 24], "through": [16, 17, 19, 21, 23, 24, 25, 26, 27], "distinct": [16, 21], "machin": [17, 19, 21, 23, 24, 25], "requst": 17, "mlwin": 17, "ring": 17, "univers": 17, "public": 17, "internet": 17, "intel": 17, "issu": [17, 29], "feel": 17, "free": 17, "u": [17, 27], "netaigym": [17, 27], "gmail": [17, 27], "com": [17, 27], "sim": 17, "aa": 17, "devcloud": 17, "avail": [17, 19, 21, 23, 24, 25, 28, 29], "meanwhil": [17, 21], "enhanc": 17, "capabl": 17, "5g": 17, "distribut": [17, 21, 23, 24], "energi": 17, "save": 17, "predict": 17, "more": [17, 19, 21, 23, 24, 25, 29], "gain": 17, "begin": 17, "download": [17, 19, 21, 23, 24], "your": [17, 25, 29], "prompt": 17, "collect": [17, 25, 28], "forward": [17, 27], "norhtbound": 17, "nb": 17, "algorithm": [17, 19, 21, 25, 29], "store": [17, 29], "relev": 17, "transmit": [17, 19, 21, 23, 24, 27], "sb": [17, 27], "continu": [17, 19, 21, 23, 24, 26], "conveni": 17, "websit": 17, "demonstr": [17, 26, 27], "packag": 17, "networkgymenv": [17, 19, 21, 23, 24, 26], "client_id": [17, 19, 21, 23, 24, 26], "argument": 17, "classic": 17, "loop": [17, 25], "implement": [17, 19, 21, 23, 25], "load_config_fil": [17, 19, 21, 23, 24, 26], "env_nam": [17, 26, 27], "nqos_split": [17, 24, 26], "num_step": [17, 26], "1000": [17, 26], "rang": [17, 21, 26, 28], "exit": [17, 19, 21, 23, 24, 26], "break": [17, 26], "epsiod": [17, 26], "up": [17, 26], "still": [17, 26], "anoth": [17, 26], "keyword": 17, "associ": 17, "dedic": [17, 20, 21], "worker": 17, "until": 17, "parallel": 17, "obtain": 17, "perform": [17, 18, 19, 20, 22, 23, 24, 25, 26], "As": 17, "along": 17, "One": 17, "multipl": [17, 21, 23, 24, 29], "fix": 17, "restart": 17, "scenario": [17, 23], "sequenti": 17, "success": 17, "insight": 17, "time": [17, 18, 19, 21, 23, 24, 29], "constraint": [17, 21], "found": 17, "handl": [17, 19, 21, 23, 24], "limit": [17, 19, 21, 23, 24, 27], "illustr": [17, 26, 29], "everi": [17, 21, 23, 24], "attribut": 17, "understand": 17, "expect": [17, 27], "output": [17, 27, 29], "instead": 17, "prefer": 17, "empti": [17, 25, 28], "shown": [17, 27, 28], "np": [17, 28], "inherit": 17, "major": 17, "possibl": [17, 23], "might": [17, 29], "describ": 17, "dimension": 17, "upper": 17, "lower": 17, "our": [17, 21, 27], "shift": 17, "tupl": 17, "multibinari": 17, "binari": 17, "multidiscret": [17, 23], "wrapper": 17, "maximum": [17, 21, 23, 29], "clipact": 17, "clip": 17, "li": 17, "rescaleact": 17, "rescal": 17, "lie": 17, "timeawareobserv": 17, "about": [17, 19, 21, 23, 24, 25], "index": [17, 29], "ensur": [17, 19, 21, 23, 24], "transit": 17, "markov": 17, "flattenobserv": 17, "flatten": 17, "normalizeobserv": 17, "normal": 17, "t": [17, 19, 21, 23, 24, 27, 29], "coordin": [17, 29], "center": [17, 25], "unit": [17, 27], "varianc": 17, "wrap": 17, "arg": 17, "normal_obs_env": 17, "rmcat": [18, 29], "adjust": [18, 19, 21, 23, 27], "rtp": [18, 19, 29], "media": [18, 19, 29], "avoid": [18, 19, 29], "techniqu": [18, 19, 29], "part": [19, 23, 24], "congest": [19, 29], "repo": [19, 21, 23, 24, 27], "cisco": 19, "inf": [19, 21, 23, 24], "150000": 19, "1500000": 19, "config": [19, 21, 23, 24], "problem": [19, 24], "strateg": [19, 21, 24], "aim": [19, 23, 24], "achiev": [19, 23, 24, 25, 29], "latenc": [19, 24], "loss": [19, 23, 29], "nada": [19, 29], "vlab": [19, 21, 23, 24, 27], "ndarrai": [19, 21, 23, 24], "view": [19, 21, 23, 24, 29], "work": [19, 21, 23, 24], "elabor": [19, 21, 23, 24], "loglen": [19, 29], "100710": 19, "58": 19, "qdel": [19, 29], "30": [19, 21, 23, 24], "rtt": [19, 29], "55": [19, 21, 23, 24], "ploss": [19, 29], "plr": [19, 29], "xcurr": [19, 29], "17099952697754": 19, "6": [19, 21, 23, 24, 27], "rrate": [19, 29], "774531": 19, "375": 19, "7": [19, 21, 23, 24, 27], "srate": [19, 29], "585416": 19, "8": [19, 21, 23, 24, 27], "nada_sr": [19, 29], "557536": 19, "1875": 19, "9": [19, 21, 23, 24, 27], "avgint": [19, 29], "10": [19, 21, 23, 24, 27], "curint": [19, 29], "round": [19, 29], "trip": [19, 29], "represet": 19, "level": [19, 20, 21, 23], "third": [19, 23, 24], "min": [19, 21, 23, 24], "max": [19, 21, 23, 24], "500": [19, 23, 29], "bp": [19, 29], "Not": 19, "edit": [19, 21, 23, 24], "subscribed_network_stat": [19, 21, 23, 24, 29], "necessari": [19, 21, 23, 24], "transmiss": [19, 21, 23, 24], "unus": [19, 21, 23, 24], "reduc": [19, 21, 23, 24], "overhead": [19, 21, 23, 24], "To": [19, 21, 23, 24, 28, 29], "subscrib": [19, 21, 23, 24, 29], "combin": [19, 21, 23, 24, 29], "would": [19, 21, 23, 24, 27, 29], "detail": [19, 21, 23, 24, 25, 28], "episodes_per_sess": [19, 21, 23, 24, 26], "steps_per_episod": [19, 21, 23, 24, 26], "l": [19, 21, 23, 24, 26], "happen": [19, 21, 23, 24], "length": [19, 21, 23, 24, 26], "note": [19, 21, 23, 24, 26, 29], "reconfigur": [19, 21, 23, 24, 26], "program": [19, 21, 23, 24, 25, 26], "entir": [19, 21, 23, 24, 26], "among": [20, 21], "priorit": [20, 21, 23, 29], "share": [20, 21, 27], "strive": [20, 21], "meet": [20, 21, 23, 29], "agreement": [20, 21], "partit": 21, "percept": 21, "possess": 21, "abil": 21, "great": 21, "effici": [21, 23], "belong": 21, "mac": 21, "schedul": 21, "proport": 21, "fair": 21, "guarante": 21, "equit": 21, "imparti": 21, "network_sl": 21, "pose": 21, "challeng": [21, 23], "compet": 21, "finit": 21, "primari": [21, 23], "multidimension": 21, "five": 21, "dl": [21, 23, 24, 27, 29], "max_rat": [21, 23, 24, 27, 29], "lte": [21, 22, 23, 24, 27, 29], "3000": 21, "75": 21, "cell_id": [21, 23, 24, 29], "slice_id": [21, 23, 24, 29], "rb_usag": [21, 23, 24, 29], "88": 21, "08": 21, "32": [21, 23, 24], "cell": [21, 23, 24, 29], "65": [21, 23, 24], "68": 21, "33333333": 21, "96": 21, "ul": [21, 23, 24, 29], "missed_act": [21, 23, 24, 29], "gma": [21, 23, 24, 27, 29], "measurement_ok": [21, 23, 24, 29], "delay_viol": [21, 23, 24, 29], "11": [21, 23, 24, 29], "delay_test_1_viol": [21, 23, 24, 29], "12": [21, 23, 24], "delay_test_2_viol": [21, 23, 24, 29], "13": [21, 23, 24], "owd": [21, 23, 24, 27, 29], "14": [21, 23, 24], "max_owd": [21, 23, 24, 29], "15": [21, 23, 24], "tx_rate": [21, 23, 24, 29], "056": [21, 23, 24], "942": 21, "16": [21, 23, 24, 29], "prioriti": [21, 23, 24, 29], "17": [21, 23, 24], "18": [21, 23, 24], "19": [21, 23, 24], "traffic_ratio": [21, 23, 24, 29], "20": [21, 23, 24], "21": [21, 23, 24], "22": [21, 23, 24], "23": [21, 23, 24], "24": [21, 23, 24], "25": [21, 23, 24], "26": [21, 23, 24], "27": [21, 23, 24], "28": [21, 23, 24], "29": [21, 23, 24], "31": [21, 23, 24], "x_loc": [21, 23, 24, 29], "83": [21, 24], "836": 21, "78": [21, 23, 24], "46": [21, 23, 24], "913": 21, "607": 21, "33": [21, 23, 24], "y_loc": [21, 23, 24, 29], "645": [21, 23], "388": 21, "493": 21, "089": 21, "242": 21, "34": [21, 23, 24], "35": [21, 23, 24], "36": [21, 23, 24], "37": [21, 23, 24], "100": [21, 23, 24], "39": [21, 23, 24], "total": [21, 23], "mbp": [21, 23, 24, 27], "fraction": 21, "express": 21, "percentag": [21, 29], "sum": [21, 29], "across": [21, 23], "equal": [21, 24, 29], "averag": [21, 29], "violat": [21, 23, 29], "wise": 21, "record": [21, 29], "experienc": 21, "millisecond": 21, "exce": 21, "surpass": 21, "appropri": 21, "scale": 21, "down": 21, "softmax": 21, "mathemat": 21, "sum_": 21, "a_i": 21, "quad": 21, "frac": 21, "j": 21, "a_j": 21, "num": [21, 24], "factor": 21, "cost": 21, "formul": 21, "left": 21, "text": 21, "_i": 21, "lambda": 21, "cdot": 21, "delay_violation_r": 21, "gamma": 21, "rb_usage_r": 21, "emphasi": 21, "consider": 21, "uniform": [21, 23, 24], "2d": [21, 23, 24], "plane": [21, 23, 24], "x": [21, 23, 24, 29], "y": [21, 23, 24, 29], "boundari": [21, 23, 24], "instal": [21, 23, 24, 27], "walk": [21, 23, 24], "mobil": [21, 23, 24], "model": [21, 23, 24, 25], "mx": [22, 23, 24], "ratio": [22, 24, 29], "consid": [22, 23, 24], "link": [22, 23, 24, 27, 29], "dimens": 23, "qos_steer": 23, "randomli": [23, 24], "intellig": 23, "direct": [23, 27, 29], "best": 23, "qualiti": 23, "enable_dynamic_flow_priorit": 23, "fals": 23, "contend": 23, "wifi": [23, 24, 29], "restrict": 23, "admiss": 23, "subset": 23, "dscp": 23, "mark": [23, 29], "qos_requir": 23, "test_duration_m": 23, "durat": 23, "delay_bound_m": [23, 29], "delay_violation_target": 23, "02": 23, "target": 23, "loss_target": 23, "001": 23, "regard": 23, "packet": [23, 29], "otherwis": [23, 29], "met": 23, "set": 23, "zero": 23, "11000": 23, "091666666666667": 23, "09166666666666666": 23, "016": 23, "54": [23, 24], "983333333333334": 23, "05": 23, "033": 23, "061": 23, "044": 23, "03": 23, "split_ratio": [23, 24, 27, 29], "928": 23, "04": 23, "752": 23, "728": 23, "032": 23, "708": 23, "93": [23, 24], "599": 23, "69": 23, "47": [23, 24], "53": [23, 24], "288": 23, "229": 23, "40": [23, 24], "205": 23, "659": 23, "79": 23, "389": 23, "285": 23, "856": 23, "41": [23, 24], "294": 23, "042": 23, "656": 23, "48": [23, 24], "126": 23, "449": 23, "42": [23, 24], "43": [23, 24], "048": 23, "44": [23, 24], "45": [23, 24], "99": 23, "81": 23, "49": [23, 24], "872": 23, "008": 23, "696": [23, 24], "50": [23, 24], "51": [23, 24], "52": [23, 24], "239": 23, "187": 23, "56": [23, 24], "estim": [23, 24, 29], "channel": [23, 24, 25], "capac": [23, 24, 29], "mechan": 23, "maxim": [23, 24], "lead": 23, "essenc": 23, "optim": [23, 25, 28], "indirectli": 23, "overal": 23, "station": [24, 29], "closest": 24, "handov": 24, "disabl": 24, "6000": [24, 29], "625": 24, "192": 24, "208": 24, "272": 24, "792": 24, "429": 24, "964": 24, "721": 24, "716": 24, "616": [24, 29], "024": [24, 29], "57": [24, 29], "144": [24, 29], "202": 24, "118": 24, "312": 24, "217": 24, "161": 24, "359": 24, "119": 24, "287": 24, "023": 24, "871": 24, "703": 24, "198": 24, "715": 24, "676": 24, "021": 24, "988": 24, "216": 24, "73": 24, "712": 24, "568": 24, "432": 24, "117": 24, "310": 24, "61": 24, "transmitt": 24, "f": 24, "innov": 25, "concept": 25, "seen": 25, "four": 25, "kei": 25, "knowledg": 25, "fulli": 25, "customiz": 25, "host": 25, "cloud": 25, "open": 25, "inject": 25, "flexibl": [25, 28], "languag": 25, "platform": 25, "separ": [25, 29], "freedom": 25, "emploi": 25, "smoothli": 25, "c": 25, "hide": 25, "secur": 25, "oversight": 25, "independ": 25, "deploy": 25, "deploi": 25, "workload": 25, "thei": 25, "purpos": [25, 28, 29], "offlin": [25, 28], "own": [25, 28], "special": [25, 28], "art": [25, 28], "sota": [25, 28], "tutori": [25, 27, 28], "guidanc": 25, "instruct": [25, 27, 28], "variou": 25, "convers": 25, "comprehens": 25, "overview": 25, "focus": 25, "layer": 25, "involv": [25, 27], "abstract": 25, "power": 25, "aspect": 25, "endeavor": 25, "full": 25, "stack": 25, "edg": 25, "fidel": 25, "proprietari": [25, 27], "applic": [25, 29], "xapp": 25, "rapp": 25, "ric": 25, "digit": 25, "twin": [25, 28], "section": [26, 27], "outlin": 26, "showcas": 26, "inclus": 27, "extern": 27, "equip": 27, "ll": 27, "ultim": 27, "merg": 27, "procedur": 27, "setup": 27, "port": 27, "alter": 27, "8087": 27, "session_nam": 27, "session_kei": 27, "network_gym_env": 27, "common_config": 27, "accord": 27, "start_custom_env": 27, "py": 27, "don": 27, "modifi": [27, 28], "grant": 27, "demo": [27, 28], "Then": 27, "start_client_demo": 27, "print": [27, 29], "custom_act": 27, "custom_tag": 27, "9269856811": 27, "5833047032": 27, "6873341799": 27, "0909367353": 27, "3403381109": 27, "0730143189": 27, "4166952968": 27, "3126658201": 27, "9090632796": 27, "6596618891": 27, "start_t": 27, "end_t": 27, "cid": 27, "20000": 27, "21000": 27, "phy": 27, "sole": 27, "ve": 27, "re": 27, "readi": 27, "mirror": 27, "uml": 27, "sequenc": 27, "diagram": 27, "locat": 27, "experi": 27, "contact": 27, "integr": [27, 28], "fulfil": 28, "simpli": 28, "popular": 28, "ones": 28, "ppo": 28, "proxim": 28, "ddpg": 28, "deep": 28, "determinist": 28, "gradient": 28, "sac": 28, "soft": 28, "actor": 28, "critic": 28, "td3": 28, "a2c": 28, "moreov": 28, "seamlessli": 28, "person": 29, "match": 29, "experiment": 29, "field": 29, "explan": 29, "uniqu": 29, "wherea": 29, "convent": 29, "prefix": 29, "captur": 29, "downlink": 29, "origin": 29, "numer": 29, "structur": 29, "acce": 29, "count": 29, "uplink": 29, "too": 29, "long": 29, "wait": 29, "therefor": 29, "alwai": 29, "ulmeasurement_ok": 29, "deliveri": 29, "threshold": 29, "delay_test_1_thresh_m": 29, "delay_test_2_thresh_m": 29, "reorder": 29, "order": 29, "70": 29, "90": 29, "sinc": 29, "255": 29, "staion": 29, "histori": 29, "queu": 29, "aggreg": 29, "ecn": 29, "previou": 29, "expert": 29, "inter": 29, "most": 29, "recent": 29, "grow": 29, "subscript": 29, "row": 29, "iterrow": 29, "df_rate": 29}, "objects": {"network_gym_client": [[2, 0, 1, "", "Adapter"], [7, 0, 1, "", "Env"], [8, 0, 1, "", "NorthBoundClient"]], "network_gym_client.Adapter": [[2, 1, 1, "", "df_to_dict"], [2, 1, 1, "", "fill_empty_feature"], [2, 1, 1, "", "wandb_log"], [2, 1, 1, "", "wandb_log_buffer_append"]], "network_gym_client.Env": [[7, 2, 1, "", "action_space"], [7, 2, 1, "", "adapter"], [7, 2, 1, "", "northbound_interface_client"], [7, 2, 1, "", "observation_space"], [7, 1, 1, "", "reset"], [7, 1, 1, "", "step"]], "network_gym_client.NorthBoundClient": [[8, 1, 1, "", "connect"], [8, 1, 1, "", "process_measurement"], [8, 1, 1, "", "recv"], [8, 1, 1, "", "send"]], "network_gym_client.envs.network_slicing": [[3, 0, 1, "", "Adapter"]], "network_gym_client.envs.network_slicing.Adapter": [[3, 1, 1, "", "get_action_space"], [3, 1, 1, "", "get_observation"], [3, 1, 1, "", "get_observation_space"], [3, 1, 1, "", "get_policy"], [3, 1, 1, "", "get_rbg_size"], [3, 1, 1, "", "get_reward"]], "network_gym_client.envs.nqos_split": [[4, 0, 1, "", "Adapter"]], "network_gym_client.envs.nqos_split.Adapter": [[4, 1, 1, "", "get_action_space"], [4, 1, 1, "", "get_observation"], [4, 1, 1, "", "get_observation_space"], [4, 1, 1, "", "get_policy"], [4, 1, 1, "", "get_reward"], [4, 1, 1, "", "netowrk_util"]], "network_gym_client.envs.qos_steer": [[5, 0, 1, "", "Adapter"]], "network_gym_client.envs.qos_steer.Adapter": [[5, 1, 1, "", "calculate_wifi_qos_user_num"], [5, 1, 1, "", "get_action_space"], [5, 1, 1, "", "get_observation"], [5, 1, 1, "", "get_observation_space"], [5, 1, 1, "", "get_policy"], [5, 1, 1, "", "get_reward"]], "network_gym_client.envs.rmcat": [[6, 0, 1, "", "Adapter"]], "network_gym_client.envs.rmcat.Adapter": [[6, 1, 1, "", "get_action_space"], [6, 1, 1, "", "get_observation"], [6, 1, 1, "", "get_observation_space"], [6, 1, 1, "", "get_policy"], [6, 1, 1, "", "get_reward"]], "network_gym_env": [[10, 0, 1, "", "Configure"], [11, 0, 1, "", "DummySim"]], "network_gym_env.Configure": [[10, 1, 1, "", "run"]], "network_gym_env.DummySim": [[11, 1, 1, "", "generate_dummy_measurement"], [11, 1, 1, "", "run_one_interval"], [11, 1, 1, "", "start_simulation"]], "network_gym_env.southbound_interface": [[12, 1, 1, "", "southbound_connect"]]}, "objtypes": {"0": "py:class", "1": "py:function", "2": "py:attribute"}, "objnames": {"0": ["py", "class", "Python class"], "1": ["py", "function", "Python function"], "2": ["py", "attribute", "Python attribute"]}, "titleterms": {"build": 0, "document": 0, "websit": 0, "instal": 0, "depend": 0, "us": 0, "tip": 0, "auto": 0, "gener": 0, "rst": 0, "file": 0, "add": 0, "class": [0, 2], "function": 0, "md": 0, "network_gym_cli": [1, 7, 8], "adapt": [2, 3, 4, 5, 6], "The": 2, "base": 2, "method": [2, 3, 4, 5, 6, 7, 8, 10, 11, 12], "network_sl": 3, "reward": [3, 4, 5, 6, 19, 21, 23, 24], "addit": [3, 4, 5, 6, 8], "nqos_split": 4, "qos_steer": 5, "rmcat": [6, 19], "env": 7, "attribut": 7, "northbound": 8, "network_gym_env": [9, 10, 11, 12], "networkgym": [9, 15, 16, 17, 25], "uml": 9, "sequenc": 9, "diagram": 9, "configur": 10, "dummi": [11, 27], "simul": [11, 17, 27], "dummysim": 11, "southbound": 12, "network_gym_serv": 13, "contact": 14, "motiv": 15, "network": [15, 20, 21, 25], "ai": [15, 25], "model": 15, "algorithm": [15, 28], "develop": 15, "cycl": 15, "challeng": 15, "face": 15, "": 15, "approach": 15, "address": 15, "thi": 15, "overview": 16, "compon": 16, "interfac": 16, "client": [16, 17], "api": [16, 25], "server": 16, "environ": [16, 17, 27], "quickstart": 17, "access": [17, 23, 24], "servic": 17, "via": [17, 25], "vlab": 17, "basic": 17, "usag": 17, "upon": 17, "start": [17, 21, 23, 24], "follow": 17, "seri": 17, "step": 17, "occur": 17, "dure": 17, "process": 17, "repeat": 17, "when": 17, "conclud": 17, "initi": [17, 27], "interact": 17, "explain": 17, "code": [17, 26], "action": [17, 19, 21, 23, 24], "observ": [17, 19, 21, 23, 24], "space": [17, 19, 21, 23, 24], "modifi": 17, "congest": 18, "control": 18, "descript": [19, 21, 23, 24], "prerequisit": [19, 21, 23, 24], "custom": [19, 21, 23, 24, 27, 28], "argument": [19, 21, 23, 24], "episod": [19, 21, 23, 24], "end": [19, 21, 23, 24], "slice": [20, 21], "cellular": 21, "state": [21, 23, 24], "traffic": [22, 23, 24], "manag": 22, "multi": [23, 24], "qo": 23, "steer": 23, "split": 24, "transit": 24, "dynam": 24, "democrat": 25, "sim": 25, "aa": 25, "object": 25, "agent": [25, 28], "gymnasium": 25, "scope": 25, "limit": [25, 26], "handl": 26, "time": 26, "truncat": 26, "termin": 26, "sequenti": 26, "train": [26, 28], "exampl": 26, "python": 26, "implement": 27, "your": 27, "connect": 27, "substitut": 27, "releas": 27, "system": 28, "default": 28, "stabl": 28, "baselines3": 28, "cleanrl": 28, "work": 29, "measur": 29, "column": 29, "support": 29, "per": 29, "sourc": 29, "retriev": 29}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.viewcode": 1, "sphinx": 57}, "alltitles": {"Build Documentation Website": [[0, "build-documentation-website"]], "Install Dependencies": [[0, "install-dependencies"]], "Build Website": [[0, "build-website"]], "Useful Tips": [[0, "useful-tips"]], "Auto Generate rst Files": [[0, "auto-generate-rst-files"]], "Add Classes and Functions to md Files": [[0, "add-classes-and-functions-to-md-files"]], "network_gym_client": [[1, "network-gym-client"]], "Adapter": [[2, "adapter"]], "The Base Class": [[2, "the-base-class"]], "Methods": [[2, "methods"], [3, "methods"], [4, "methods"], [5, "methods"], [6, "methods"], [7, "methods"], [8, "methods"], [10, "methods"], [11, "methods"], [12, "methods"]], "network_slicing Adapter": [[3, "network-slicing-adapter"]], "": [[3, "id1"], [4, "id1"], [5, "id1"], [6, "id1"]], "Reward Methods": [[3, "reward-methods"], [5, "reward-methods"], [6, "reward-methods"]], "Additional Methods": [[3, "additional-methods"], [4, "additional-methods"], [5, "additional-methods"], [6, "additional-methods"], [8, "additional-methods"]], "nqos_split Adapter": [[4, "nqos-split-adapter"]], "Reward Method": [[4, "reward-method"]], "qos_steer Adapter": [[5, "qos-steer-adapter"]], "rmcat Adapter": [[6, "rmcat-adapter"]], "Env": [[7, "env"]], "network_gym_client.Env": [[7, "network-gym-client-env"]], "Attributes": [[7, "attributes"]], "NorthBound": [[8, "northbound"]], "network_gym_client.NorthBound": [[8, "network-gym-client-northbound"]], "network_gym_env": [[9, "network-gym-env"]], "NetworkGym UML Sequence Diagram": [[9, "networkgym-uml-sequence-diagram"]], "Configure": [[10, "configure"]], "network_gym_env.Configure": [[10, "network-gym-env-configure"]], "Dummy Simulator": [[11, "dummy-simulator"]], "network_gym_env.DummySim": [[11, "network-gym-env-dummysim"]], "SouthBound": [[12, "southbound"]], "network_gym_env.SouthBound": [[12, "network-gym-env-southbound"]], "network_gym_server": [[13, "network-gym-server"]], "Contact": [[14, "contact"]], "Motivation": [[15, "motivation"]], "Network AI Models/Algorithms Development Cycle": [[15, "network-ai-models-algorithms-development-cycle"]], "Challenges Faced by Network AI Developers": [[15, "challenges-faced-by-network-ai-developers"]], "NetworkGym\u2019s Approach to Addressing this Challenge": [[15, null], [15, null], [15, null]], "Overview": [[16, "overview"]], "NetworkGym Components and Interfaces": [[16, "networkgym-components-and-interfaces"]], "Client API": [[16, "client-api"]], "Server API": [[16, "server-api"]], "Environment API": [[16, "environment-api"]], "Quickstart": [[17, "quickstart"]], "Accessing the NetworkGym Service via vLab": [[17, "accessing-the-networkgym-service-via-vlab"]], "Basic Usage": [[17, "basic-usage"]], "\u25b6\ufe0f Upon starting the Client, the following series of steps occur:": [[17, null]], "\ud83d\udd01 During the simulation, the process repeats as follows:": [[17, null]], "\u23f9\ufe0f When the NetworkGym or the simulation concludes:": [[17, null]], "Initializing Environments": [[17, "initializing-environments"]], "Interacting with the Environment": [[17, "interacting-with-the-environment"]], "Explaining the code": [[17, "explaining-the-code"]], "Action and observation spaces": [[17, "action-and-observation-spaces"]], "Modifying the environment": [[17, "modifying-the-environment"]], "Congestion Control": [[18, "congestion-control"]], "RMCAT": [[19, "rmcat"]], "Description": [[19, "description"], [21, "description"], [23, "description"], [24, "description"]], "Prerequisite": [[19, "prerequisite"], [21, "prerequisite"], [23, "prerequisite"], [24, "prerequisite"]], "Observation Space": [[19, "observation-space"], [21, "observation-space"], [23, "observation-space"], [24, "observation-space"]], "Action Space": [[19, "action-space"], [21, "action-space"], [23, "action-space"], [24, "action-space"]], "Reward": [[19, "reward"], [21, "reward"], [23, "reward"], [24, "reward"]], "Customize Observation Space and Reward": [[19, "customize-observation-space-and-reward"]], "Arguments": [[19, "arguments"], [21, "arguments"], [23, "arguments"], [24, "arguments"]], "Episode End": [[19, "episode-end"], [21, "episode-end"], [23, "episode-end"], [24, "episode-end"]], "Network Slicing": [[20, "network-slicing"]], "Cellular Network Slicing": [[21, "cellular-network-slicing"]], "Custom Observation Space and Reward": [[21, "custom-observation-space-and-reward"], [23, "custom-observation-space-and-reward"], [24, "custom-observation-space-and-reward"]], "Starting State": [[21, "starting-state"], [23, "starting-state"], [24, "starting-state"]], "Traffic Management": [[22, "traffic-management"]], "Multi-Access QoS Traffic Steering": [[23, "multi-access-qos-traffic-steering"]], "Multi-Access Traffic Splitting": [[24, "multi-access-traffic-splitting"]], "Transition Dynamics": [[24, "transition-dynamics"]], "NetworkGym: Democratizing Network AI via Sim-aaS.": [[25, "networkgym-democratizing-network-ai-via-sim-aas"]], "Objectives": [[25, "objectives"]], "Agent": [[25, "agent"]], "Gymnasium API": [[25, "gymnasium-api"]], "NetworkGym API": [[25, "networkgym-api"]], "NetworkGym Scope and Limitations": [[25, "networkgym-scope-and-limitations"]], "Handling Time Limits": [[26, "handling-time-limits"]], "Truncation": [[26, "truncation"]], "Termination": [[26, "termination"]], "Sequential Training Example": [[26, "sequential-training-example"]], "Python code:": [[26, "python-code"]], "Implementing Custom Environment": [[27, "implementing-custom-environment"]], "Initiating your Custom Environment": [[27, "initiating-your-custom-environment"]], "Connecting to Your Custom Environment": [[27, "connecting-to-your-custom-environment"]], "Substituting the Dummy Simulator": [[27, "substituting-the-dummy-simulator"]], "Releasing Your Custom Environment": [[27, "releasing-your-custom-environment"]], "Training Agents": [[28, "training-agents"]], "System Default Agent": [[28, "system-default-agent"]], "Custom Algorithm Agent": [[28, "custom-algorithm-agent"]], "Stable-Baselines3 Agent": [[28, "stable-baselines3-agent"]], "CleanRL Agent": [[28, "cleanrl-agent"]], "Working with Measurements": [[29, "working-with-measurements"]], "Measurement Columns": [[29, "measurement-columns"]], "Supported Measurements per Source": [[29, "supported-measurements-per-source"]], "Retrieving a Measurement": [[29, "retrieving-a-measurement"]]}, "indexentries": {"adapter (class in network_gym_client)": [[2, "network_gym_client.Adapter"]], "df_to_dict() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.df_to_dict"]], "fill_empty_feature() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.fill_empty_feature"]], "wandb_log() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.wandb_log"]], "wandb_log_buffer_append() (in module network_gym_client.adapter)": [[2, "network_gym_client.Adapter.wandb_log_buffer_append"]], "adapter (class in network_gym_client.envs.network_slicing)": [[3, "network_gym_client.envs.network_slicing.Adapter"]], "get_action_space() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_policy"]], "get_rbg_size() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_rbg_size"]], "get_reward() (in module network_gym_client.envs.network_slicing.adapter)": [[3, "network_gym_client.envs.network_slicing.Adapter.get_reward"]], "adapter (class in network_gym_client.envs.nqos_split)": [[4, "network_gym_client.envs.nqos_split.Adapter"]], "get_action_space() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.get_reward"]], "netowrk_util() (in module network_gym_client.envs.nqos_split.adapter)": [[4, "network_gym_client.envs.nqos_split.Adapter.netowrk_util"]], "adapter (class in network_gym_client.envs.qos_steer)": [[5, "network_gym_client.envs.qos_steer.Adapter"]], "calculate_wifi_qos_user_num() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.calculate_wifi_qos_user_num"]], "get_action_space() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.qos_steer.adapter)": [[5, "network_gym_client.envs.qos_steer.Adapter.get_reward"]], "adapter (class in network_gym_client.envs.rmcat)": [[6, "network_gym_client.envs.rmcat.Adapter"]], "get_action_space() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_action_space"]], "get_observation() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_observation"]], "get_observation_space() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_observation_space"]], "get_policy() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_policy"]], "get_reward() (in module network_gym_client.envs.rmcat.adapter)": [[6, "network_gym_client.envs.rmcat.Adapter.get_reward"]], "env (class in network_gym_client)": [[7, "network_gym_client.Env"]], "action_space (network_gym_client.env attribute)": [[7, "network_gym_client.Env.action_space"]], "adapter (network_gym_client.env attribute)": [[7, "network_gym_client.Env.adapter"]], "northbound_interface_client (network_gym_client.env attribute)": [[7, "network_gym_client.Env.northbound_interface_client"]], "observation_space (network_gym_client.env attribute)": [[7, "network_gym_client.Env.observation_space"]], "reset() (in module network_gym_client.env)": [[7, "network_gym_client.Env.reset"]], "step() (in module network_gym_client.env)": [[7, "network_gym_client.Env.step"]], "northboundclient (class in network_gym_client)": [[8, "network_gym_client.NorthBoundClient"]], "connect() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.connect"]], "process_measurement() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.process_measurement"]], "recv() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.recv"]], "send() (in module network_gym_client.northboundclient)": [[8, "network_gym_client.NorthBoundClient.send"]], "configure (class in network_gym_env)": [[10, "network_gym_env.Configure"]], "run() (in module network_gym_env.configure)": [[10, "network_gym_env.Configure.run"]], "dummysim (class in network_gym_env)": [[11, "network_gym_env.DummySim"]], "generate_dummy_measurement() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.generate_dummy_measurement"]], "run_one_interval() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.run_one_interval"]], "start_simulation() (in module network_gym_env.dummysim)": [[11, "network_gym_env.DummySim.start_simulation"]], "southbound_connect() (in module network_gym_env.southbound_interface)": [[12, "network_gym_env.southbound_interface.southbound_connect"]]}}) \ No newline at end of file