This package contains the implementation of a UDP server. It supports concurrent processing and validating digital signature and repeating key XOR'd cyclic checksum CRC32 DWORDS of incoming UDP packet. The user can also specify a delay (in seconds) to the logging of validation failures to file.
This UDP server and its implementation sports the following features.
This server spawns worker threads to handle validation of digital signature and
cyclic checksums of packets. Therefore, the server will be able to handle
multiple packets of different packet_id at the same time, allowing for more
packets to be received and processed.
This server stores previous cylic checksum iteration in FileChecksums object,
allowing for a swift calculation of CRC32 of an entire binary file of
a large number of cyclic iteration.
Utilizing a separate logger thread for each packet_id, the server will be
able to handle delayed logging without blocking cylic checksums calcullation and
validation. The details of this feature is specified in Logger thread
of the Design section.
This package was implemented and tested extensively in Python 3.11.8.
This package is run through the command line.
- Change directory to the root directory of this project
cd <project's directory>
- Install packages
pip install -r requirements.txt
- Command line arguments
The main entry point for this server is
server.py.
--keys: a dictionary of{packet_id: key_file_path}mappings--binaries: a dictionary of{packet_id: binary_path}mappings--delay (-d): delay, (in seconds) for writing to log files--port (-p): port, to receive packets on
Example run:
python server.py --keys '{"0x42": "key.bin"}' --binaries '{"0x42": "cat.jpg"}' -d '0' -p '1337'
====================================================================
( 4 bytes ) Unique Packet ID for the checksummed binary
====================================================================
( 4 bytes ) Packet Sequence # (Total Checksums Processed)
====================================================================
( 2 bytes ) Multibyte Repeating XOR Key | ( 2 bytes ) # of Checksums
====================================================================
( Variable ) Repeating key XOR'd Cyclic Checksum CRC32 DWORDs
....
....
....
====================================================================
( 64 bytes ) RSA 512 SHA-256 Digital Signature (for above fields)
====================================================================
Here is the design diagram of the UDP server:
Upon starting, the thread will preprocess the given public keys and binaries.
It will also create two logging.Logger objects, one for verification_failures.log
and one for checksum_failures.log, to assist with logging to file later.
Everything will be packed in a ServerConfig object and the server will start
up on the given address and port ready for service.
NOTICE: Due to the limited implementation and time, the server will only take
in a packet with size at most 4096 bytes.
Upon receing a packet, the server will check for its content and verify against
the packet specification. Once everything is valid, the server will be able to
extract its packet_id, packet sequence number, multibyte repeating XOR key,
number of checksums, its checksum DWORDS, and its digital signature. With the
packet_id, the server will assign that packet to the work queue of the thread
handling the other two verifications. If the thread handling that has not been
spawned yet, the server will do it upon seeing a new packet_id.
Upon spawning, the worker will initialize its companion logger thread that will help the worker log to file when needed. This need for a logger thread arises due to the log delaying feature, so the worker can continue working when the logger handles the delaying.
Each worker will be responsible for processing the signature and checksums
validation for packets containing the packet_id that it was assigned to.
The worker will receive its work (packet) from its asigned work queue. If the
queue is empty, the queue will block the execution until a packet comes in for
processing.
The design for this logger thread is an interesting one. We know that logs will come in sequentially, which means that the order of the log request in the queue will be chronological. This is because this logger is only servering one worker thread, and because each worker thread processes only one packet at a given time, it is certain that the log requests will be chronological. Using this property, we can just take one log request from the queue, wait out the delay (if any), and continue with another logging request.
The author recognizes the following limitations of this UDP server design and implementation.
Due to the nature of UDP packets, the entire size of the packet is not known,
or at least is not supported in Python's server module. There is one way,
however, that we can retrieve the entire packet. Since the number of checksums
are given, we can use that to calculate how big the entire packet is overall.
Upon receiving the first 4096 bytes of the packet, we can calculate the size
of the incoming packet, and continue requesting more if necessary. However,
due to time constraint, the author has not been able to implement this yet.
In this implementation, cyclic checksums iterations are calculated by storing
previous iterations in a list of the FileChecksums object. This means that
this server can only support verifying checksums of cyclic iteration of up to
however big the Python's List can store. There are ways of mitigating this
storage, for example, storing only a limited number of checksums iteration
(e.g. every 1000 iteration), and calculating from the closest point to the
desired iteration.
The packet sequence number has only 4 bytes, which means the server can support a sequence number of up to 4,294,967,295.
Due to Python's Global Interpreter Lock, which prevents multiple threads
from executing Python bytecode at the same time. This means that Python does not
have true multithreading.This prevents this server from supporting a lot of
unique packet_ids, since more packet_ids means more threads, and since
Python does not support true multithreading, this will create performance
problems in the long run.
The author admits having limited knowledge in concurrency and distributed systems. (I have only taken Computer Organizations and Parallel & Disributed Systems in college, and limited practical experience, aside from this project, of course). There might be other problems related to multiprocessing or multithreading that this server does not support and the author admits limitations to this.