Akademy is a module containing composable object classes for developing reinforcement learning algorithms focused on quantitative trading and time-series forecasting. This module is a work-in-progress and should, at no time, be assumed to be designed well or be free of bugs.
Akademy is designed using an Agent-Environment model such that Agent-class
objects ingest information from Environment-class objects (Env), produce
an Action, which is then applied to the Environment which results in a
change in State and possible reward to offer feedback to the agent.
Note: this module does not provide any training routines -- only the object class that can be used to support the implementation of custom training routines.
To install akademy use the following command in the desired Python 3.7+
environment:
pip install akademy
Once installed, developers will have access to Agent, TradeEnv, and Network
class objects in which to design Reinforcement Learning algorithms to train models.
Sample training routine:
from akademy.models.envs import TradeEnv
from akademy.models.agents import DQNAgent
from akademy.common.utils import load_spy_daily
# loads the dataset used during training
data = load_spy_daily(count=2500)
# load the Trading Environment
env = TradeEnv(
data=data,
window=50,
asset="spy",
)
# load the agent to train
agent = DQNAgent(
action_count=env.action_space.n,
state_shape=env.observation_space.shape
)
# load user-defined training routine
training_routine(
agent=agent,
env=env
)Unit testing can be run via the following command:
python -m unittest
For detailed information the --verbose flag can be used. For more detailed
usage consult the unittest module documentation.
This module comes with minimal data for Agents and Environments to train on. The current data available is listed below, along with sources for the most up-to-date versions as well:
Location: /data/SPY.CSV
Start: 1993-01-29
End: 2023-01-23
Total Rows: 7,454 (excludes header)
Header: Date,Open,High,Low,Close,Adj Close,Volume
Source: https://finance.yahoo.com/quote/SPY/history?p=SPY
note: Any data can be used easily enough via conversion into a Pandas DataFrame
object, but must contain information for date and pricing data for
open, high, low, and close as well as volume such that each row has
at least those 6 features or the latter 5 and an index representative of date.
The Gym project by OpenAI has been sunset and now maintained as Gymnasium
by the Farama-Foundation. The
Env classes present here make use of the newer Gymnasium package which, among
other differences, produces an extra item in the step method indicating whether
an environment has been truncated. See here
PyTorch requires some additional consideration for setup depending on use-case. Akademy uses an approach whereby CPU-based training and inferences are possible via parameterized function calls. However, GPU use (e.g. CUDA) requires local considerations. [See here] (https://pytorch.org/get-started/locally/) for a more in-depth discussion and guide.
This module currently uses the 1.* version, though a 2.* version release is imminent and an upgrade to that version is planned.