diff --git a/docs/src/esn_tutorials/deep_esn.md b/docs/src/esn_tutorials/deep_esn.md index 55f7def9..08f4e9f1 100644 --- a/docs/src/esn_tutorials/deep_esn.md +++ b/docs/src/esn_tutorials/deep_esn.md @@ -41,9 +41,9 @@ The construction of the ESN is also really similar. The only difference is that ```@example deep_lorenz using ReservoirComputing -reservoirs = [rand_sparse(;radius = 1.1, sparsity = 0.1), - rand_sparse(;radius = 1.2, sparsity = 0.1), - rand_sparse(;radius = 1.4, sparsity = 0.1)] +reservoirs = [rand_sparse(; radius = 1.1, sparsity = 0.1), + rand_sparse(; radius = 1.2, sparsity = 0.1), + rand_sparse(; radius = 1.4, sparsity = 0.1)] esn = DeepESN(input_data, 3, 200; reservoir = reservoirs, diff --git a/docs/src/esn_tutorials/different_drivers.md b/docs/src/esn_tutorials/different_drivers.md index bdcbbb73..5a6fc983 100644 --- a/docs/src/esn_tutorials/different_drivers.md +++ b/docs/src/esn_tutorials/different_drivers.md @@ -73,8 +73,8 @@ case5 = MRNN([tanh, f4], 0.9, [0.43, 0.13]) test_cases = [base_case, case3, case4, case5] for case in test_cases esn = ESN(training_input, 1, 100, - input_layer = weighted_init(;scaling = 0.3), - reservoir = rand_sparse(;radius = 0.4), + input_layer = weighted_init(; scaling = 0.3), + reservoir = rand_sparse(; radius = 0.4), reservoir_driver = case, states_type = ExtendedStates()) wout = train(esn, training_target, StandardRidge(10e-6)) diff --git a/src/esn/deepesn.jl b/src/esn/deepesn.jl index bf680e1d..478a31b3 100644 --- a/src/esn/deepesn.jl +++ b/src/esn/deepesn.jl @@ -23,52 +23,57 @@ suited for complex sequential tasks where depth can help capture hierarchical temporal features. # Parameters -- `train_data`: The training dataset used for the ESN. -This should be structured as sequential data where applicable. -- `in_size`: The size of the input layer, i.e., the number of -input units to the ESN. -- `res_size`: The size of each reservoir, i.e., the number of neurons -in each hidden layer of the ESN. + + - `train_data`: The training dataset used for the ESN. + This should be structured as sequential data where applicable. + - `in_size`: The size of the input layer, i.e., the number of + input units to the ESN. + - `res_size`: The size of each reservoir, i.e., the number of neurons + in each hidden layer of the ESN. # Optional Keyword Arguments -- `depth`: The number of reservoir layers in the Deep ESN. Default is 2. -- `input_layer`: A function or an array of functions to initialize the input -matrices for each layer. Default is `scaled_rand` for each layer. -- `bias`: A function or an array of functions to initialize the bias vectors -for each layer. Default is `zeros64` for each layer. -- `reservoir`: A function or an array of functions to initialize the reservoir -matrices for each layer. Default is `rand_sparse` for each layer. -- `reservoir_driver`: The driving system for the reservoir. -Default is an RNN model. -- `nla_type`: The type of non-linear activation used in the reservoir. -Default is `NLADefault()`. -- `states_type`: Defines the type of states used in the ESN (e.g., standard states). -Default is `StandardStates()`. -- `washout`: The number of initial timesteps to be discarded in the ESN's training phase. -Default is 0. -- `rng`: Random number generator used for initializing weights. Default is the package's -default random number generator. -- `T`: The data type for the matrices (e.g., `Float64`). Influences computational -efficiency and precision. -- `matrix_type`: The type of matrix used for storing the training data. -Default is inferred from `train_data`. + + - `depth`: The number of reservoir layers in the Deep ESN. Default is 2. + - `input_layer`: A function or an array of functions to initialize the input + matrices for each layer. Default is `scaled_rand` for each layer. + - `bias`: A function or an array of functions to initialize the bias vectors + for each layer. Default is `zeros64` for each layer. + - `reservoir`: A function or an array of functions to initialize the reservoir + matrices for each layer. Default is `rand_sparse` for each layer. + - `reservoir_driver`: The driving system for the reservoir. + Default is an RNN model. + - `nla_type`: The type of non-linear activation used in the reservoir. + Default is `NLADefault()`. + - `states_type`: Defines the type of states used in the ESN (e.g., standard states). + Default is `StandardStates()`. + - `washout`: The number of initial timesteps to be discarded in the ESN's training phase. + Default is 0. + - `rng`: Random number generator used for initializing weights. Default is the package's + default random number generator. + - `T`: The data type for the matrices (e.g., `Float64`). Influences computational + efficiency and precision. + - `matrix_type`: The type of matrix used for storing the training data. + Default is inferred from `train_data`. # Returns -- A `DeepESN` instance configured according to the provided parameters -and suitable for further training and prediction tasks. + + - A `DeepESN` instance configured according to the provided parameters + and suitable for further training and prediction tasks. # Example + ```julia # Prepare your training data train_data = [your_training_data_here] # Create a DeepESN with specific parameters -deepESN = DeepESN(train_data, 10, 100, depth=3, washout=100) +deepESN = DeepESN(train_data, 10, 100, depth = 3, washout = 100) # Proceed with training and prediction (pseudocode) train(deepESN, target_data) prediction = predict(deepESN, new_data) ``` + The DeepESN model is ideal for tasks requiring the processing of sequences with complex temporal dependencies, benefiting from the multiple reservoirs to capture different levels of abstraction and temporal dynamics. diff --git a/src/esn/hybridesn.jl b/src/esn/hybridesn.jl index 2838d53b..ad134a9e 100644 --- a/src/esn/hybridesn.jl +++ b/src/esn/hybridesn.jl @@ -66,45 +66,49 @@ specifying the reservoir size, input size, and various other parameters that influence the network's behavior and learning capacity. # Parameters -- `model`: A `KnowledgeModel` instance representing the knowledge-based model -to be integrated with the ESN. -- `train_data`: The training dataset used for the ESN. This data can be -preprocessed or raw data depending on the nature of the problem and the -preprocessing steps considered. -- `in_size`: The size of the input layer, i.e., the number of input units -to the ESN. -- `res_size`: The size of the reservoir, i.e., the number of neurons in -the hidden layer of the ESN. + + - `model`: A `KnowledgeModel` instance representing the knowledge-based model + to be integrated with the ESN. + - `train_data`: The training dataset used for the ESN. This data can be + preprocessed or raw data depending on the nature of the problem and the + preprocessing steps considered. + - `in_size`: The size of the input layer, i.e., the number of input units + to the ESN. + - `res_size`: The size of the reservoir, i.e., the number of neurons in + the hidden layer of the ESN. # Optional Keyword Arguments -- `input_layer`: A function to initialize the input matrix. Default is `scaled_rand`. -- `reservoir`: A function to initialize the reservoir matrix. Default is `rand_sparse`. -- `bias`: A function to initialize the bias vector. Default is `zeros64`. -- `reservoir_driver`: The driving system for the reservoir. Default is an RNN model. -- `nla_type`: The type of non-linear activation used in the reservoir. -Default is `NLADefault()`. -- `states_type`: Defines the type of states used in the ESN (e.g., standard states). -Default is `StandardStates()`. -- `washout`: The number of initial timesteps to be discarded in the ESN's training phase. -Default is 0. -- `rng`: Random number generator used for initializing weights. Default is the package's -default random number generator. -- `T`: The data type for the matrices (e.g., `Float32`). Influences computational -efficiency and precision. -- `matrix_type`: The type of matrix used for storing the training data. Default is -inferred from `train_data`. + + - `input_layer`: A function to initialize the input matrix. Default is `scaled_rand`. + - `reservoir`: A function to initialize the reservoir matrix. Default is `rand_sparse`. + - `bias`: A function to initialize the bias vector. Default is `zeros64`. + - `reservoir_driver`: The driving system for the reservoir. Default is an RNN model. + - `nla_type`: The type of non-linear activation used in the reservoir. + Default is `NLADefault()`. + - `states_type`: Defines the type of states used in the ESN (e.g., standard states). + Default is `StandardStates()`. + - `washout`: The number of initial timesteps to be discarded in the ESN's training phase. + Default is 0. + - `rng`: Random number generator used for initializing weights. Default is the package's + default random number generator. + - `T`: The data type for the matrices (e.g., `Float32`). Influences computational + efficiency and precision. + - `matrix_type`: The type of matrix used for storing the training data. Default is + inferred from `train_data`. # Returns -- A `HybridESN` instance configured according to the provided parameters and -suitable for further training and prediction tasks. + + - A `HybridESN` instance configured according to the provided parameters and + suitable for further training and prediction tasks. # Example + ```julia # Define a KnowledgeModel km = KnowledgeModel(prior_model_function, u0, (0, 100), 1000) # Create a HybridESN -hesn = HybridESN(km, train_data, 10, 100; washout=100) +hesn = HybridESN(km, train_data, 10, 100; washout = 100) # Train and predict train(hesn, target_data)