This repo contains demos and datasets of ACM Multimedia 2021 Grand Challenge: Meet Deadline Requirements. You can see more information about this challenge on our website. Any registration and participation are welcome.
Basic requirements: You can install our emulator platform(simple-emulator) and basic packages needed by the emulator using:
pip3 install -r requirements.txtOther requirements: If you want to use a learning-based solution, you need to install some other packages. For example, you need to install torch to run RL-demo-using-torch.
(*The packages supported on our submission evaluating system can be found in import-package.)
After basic requirements installed, you can run run_this.py to evaluate the reno solution demo:
python3 run_this.pyNow you have successfully run the demo Reno, which can be used directly in the submission.
If you want to change the evaluation process, you can modify run_this.py or create your own evaluating program. Solution Evaluation will show you more details on how to achieve this.
Participants need to implement a Scheduler. The Scheduler consists of two parts: Blocks Scheduler and Bandwidth Estimator. We have provided some demos of solution.
Blocks Scheduler selects which block in block_queue should be sent at the time cur_time. You need to implement your scheduler algorithm in method select_block.
The emulator will call select_block every time sender tries to select a block to send. Your algorithm needs to select a block and return its id.
Input: select_block will get cur_time and block_queue:
-
cur_time
The time when this method is called.
-
block_queue
This parameter is a list object composed of currently sendable blocks. It is recommended to read about its detailed description: block_queue
Output: Returns the index of the block to be sent. For example, returning 0 means the first block in block_queue will be sent.
Bandwidth Estimator updates the sending rate of sender. You need to implement your bandwidth estimator algorithm in method cc_trigger.
The emulator will call cc_trigger every time packet acknowledged or packet dropped event happens. You can update the sending rate and states of your algorithm when cc_trigger is called.
Input: cc_trigger will get cur_time and a two-tuple event_info:
-
cur_time
The time when this method is called.
-
event_info
-
event_type
There are three categories for the packet involved in the event: PACKET_TYPE_FINISHED, PACKET_TYPE_TEMP, PACKET_TYPE_DROP.
PACKET_TYPE_FINISHED: The acknowledge packet that successfully reached the sender;
PACKET_TYPE_TEMP: The packet that has not yet reached the sender;
PACKET_TYPE_DROP: The packet that is used to inform the sender of packet loss.
-
packet_information_dict
A dictionary composed of the brief information of the packet. We recommend you to get more information at packet_information_dict.
-
Output: In addition to controlling by the estimated bandwidth, we also support solutions based on congestion window. The return of cc_trigger consists of cwnd and send_rate. E.g
{
"cwnd" : 10,
"send_rate" : 10
}You can set or update the states of your algorithm in methods __init__ and on_packet_sent.
Moreover, you can also update the send_rate or cwnd when returning from on_packet_sent. Emulator calls on_packet_sent every time sender tries to send a packet.
# Instantiate the solution
my_solution = solution.MySolution()
# Create emulator
emulator = Emulator(
block_file=["datasets/application_traces/data_video.csv", "datasets/application_traces/data_audio.csv"],
trace_file="datasets/network_traces/trace.txt",
solution=my_solution,
# The system will choose a number as the link buffer from the range MIN_QUEUE to MAX_QUEUE
MIN_QUEUE=55,
MAX_QUEUE=55,
# enable logging packet. You can train faster if USE_CWND=False
ENABLE_LOG=True
)The block_file, trace_file, solution are necessary to construct an Emulator. And the function also provides some optional parameters to help diversified experiments. All the parameters and their descriptions are below:
| Parameters | Default s | Description |
|---|---|---|
| block_file | \ | The list contains one or more application trace files |
| trace_file | \ | The network trace file which describe the bandwidth, loss rate, etc. |
| solution | \ | The object initialized for the algorithm to be tested |
| USE_CWND | False | Whether the algorithm is based on the congestion window |
| ENABLE_DEBUG | False | Output DEBUG information |
| MIN_QUEUE | 55 | The minimum link buffer size |
| MAX_QUEUE | 55 | The maximum link buffer size |
| ENABLE_LOG | True | Output packet-level LOG |
| MAX_PACKET_LOG_ROWS | 4000 | The maximum number of records in a packet-level log file, if it exceeds the maximum number of records, a new file will be created |
| SEED | - | Emulator random seed |
| RUN_DIR | - | Emulator run root directory |
emulator.run_for_dur(15)The optional parameter indicates the cut-off time (in seconds) of the emulation. The default is infinite, that is, the emulation will not end until application trace ends.
print("Qoe : %d" % (cal_qoe()) )Calculate the QoE score based on the output/block.log in the running result.
In order to help participants better optimize the solution, we also provide some functional tools in the emulator.
emulator.print_debug()
analyze_emulator(log_file="output/packet_log/packet-0.log", file_range="all")
plot_rate(log_file="output/packet_log/packet-0.log", trace_file="datasets/network_traces/trace.txt", file_range="all", sender=[1])Show a brief analysis result, including delay, the number of packet lost, etc. The output example and corresponding explanations are as follows.
---Link Debug---
Link: 1 # Link ID
Bandwidth: 1333.333333 # Link bandwidth
Delay: 0.001000 # Link propagation delay
Queue Delay: 0.003000 # Queuing delay of the last packet in this link
Max Queue Delay: 0.041250 # Maximum queuing delay in this link
One Packet Queue Delay: 0.000750 # Queuing delay of a single packet in this link
Link: 2
Bandwidth: inf
Delay: 0.001000
Queue Delay: 0.000000
Max Queue Delay: 0.000000
One Packet Queue Delay: 0.000000
---Sender Debug---
Sender: 1 # Sender ID
Rate: inf # Sender's sending rate
Sent: 18247 # Number of packet sent
Acked: 18184 # Number of packet acked
Lost: 61 # Number of packet lost
Min Latency: 0.002 # Minimum queuing delay in acked packets
The delay curve of the tested algorithm during the evaluation process.
The required parameter is log_file. And the function also provides some optional parameters to help diversified analysis. All the parameters and their descriptions are below:
| Parameters | Defaults | Description |
|---|---|---|
| log_file | \ | The packet-level log file that needs to be analyzed |
| rows | None | Limit the number of log lines that will be read |
| trace_file | None | The path of network trace file. Use this option to draw bandwidth and statistical throughput in the same picture |
| time_range | None | Log file time range used |
| scatter | False | Whether to switch to scatter chart (default is polyline) |
| file_range | None | Log file range. For multiple file logs, you can use this item to specify the range (for example: 'file_range=[1,3]' means files from 1 to 3 (left closed and right open interval), "all" means all files) |
| sender | None | Filter log file by sender ID |
The transmission speed change curve during the evaluation progress.
The required parameter is log_file. And the function also provides some optional parameters to help diversified analysis. All the parameters and their descriptions are below:
| Parameters | Defaults | Description |
|---|---|---|
| log_file | \ | The packet-level log file that needs to be analyzed |
| rows | None | Limit the number of log lines that will be read |
| trace_file | None | The path of network trace file. Use this option to draw bandwidth and statistical throughput in the same picture |
| time_range | None | Log file time range used |
| scatter | False | Whether to switch to scatter chart (default is polyline) |
| file_range | None | Log file range. For multiple file logs, you can use this item to specify the range (for example: 'file_range=[1,3]' means files from 1 to 3 (left closed and right open interval), "all" means all files) |
| sender | None | Filter log file by sender ID |
| size | 1 | Draw a data point every size seconds |