Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyMC/PyTensor Implementation of Pathfinder VI #387

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Commits on Oct 19, 2024

  1. Configuration menu
    Copy the full SHA
    4540b84 View commit details
    Browse the repository at this point in the history
  2. Minor changes made to the fit_pathfinder function and added test

    `fit_pathfinder`
    - Edited `fit_pathfinder` to produce `pathfinder_state`, `pathfinder_info`, `pathfinder_samples` and `pathfinder_idata` for closer examination of the outputs.
    - Changed the `num_samples` argument name to `num_draws` to avoid `TypeError` got multiple values for keyword argument 'num_samples'.
    - Initial points are automatically set to jitter as jitter is required for pathfinder.
    
    Extras
    - New function 'get_jaxified_logp_ravel_inputs' to simplify previous code structure in fit_pathfinder.
    
    Tests
    - Added extra test for pathfinder to test pathfinder_info variables and pathfinder_idata  are consistent for a given random seed.
    aphc14 committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    0c880d2 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2024

  1. Configuration menu
    Copy the full SHA
    8835cd5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    663a60a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    05aeeaf View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. feat(pathfinder): add PyMC-based Pathfinder VI implementation

    Add a new PyMC-based implementation of Pathfinder VI that uses PyTensor operations which provides support for both PyMC and BlackJAX backends in fit_pathfinder.
    aphc14 committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    0db91fe View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Multipath Pathfinder VI implementation in pymc-experimental

    - Implemented  in  to support running multiple Pathfinder instances in parallel.
    - Implemented  function in  for Pareto Smoothed Importance Resampling (PSIR).
    - Moved relevant pathfinder files into the  directory.
    - Updated tests to reflect changes in the Pathfinder implementation and added tests for new functionalities.
    aphc14 committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    cb4436c View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. Configuration menu
    Copy the full SHA
    2efb511 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fdc3f38 View commit details
    Browse the repository at this point in the history
  3. Added placeholder/reminder to remove jax dependency when converting t…

    …race data to InferenceData
    aphc14 committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    1fd7a11 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ef2956f View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. Reduced size of compute graph with pathfinder_body_fn

    Summaryh of changes:
    - Remove multiprocessing code in favour of reusing compiled  for each path
    -  takes only random_seed as argument for each path
    - Compute graph significantly smaller by using pure pytensor op and symoblic variables
    - Added LBFGSOp to compile with pytensor.function
    - Cleaned up codes using pytensor variables
    aphc14 committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    8b134b7 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. - Added TODO comments for implementing Taylor approximation methods: …

    …and .
    
    - Corrected the dimensions in comments for matrices Q and R in the  function.
    - Uumerical stability in the  calculation by changing from  to .
    aphc14 committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6484b3d View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. fix: correct posterior approximations in Pathfinder VI

    Fixed incorrect and inconsistent posterior approximations in the Pathfinder VI
    algorithm by:
    
    1. Adding missing parentheses in the phi calculation to ensure proper order
       of operations in matrix multiplications
    2. Changing the sign in mu calculation from 'x +' to 'x -' to match Stan's
       implementation (which differs from the original paper)
    
    The resulting changes now make the posterior approximations more reliable.
    aphc14 committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    aa765fb View commit details
    Browse the repository at this point in the history
  2. feat: Add dense BFGS sampling for Pathfinder VI

    Implements both sparse and dense BFGS sampling approaches for Pathfinder VI:
    - Adds bfgs_sample_dense for cases where 2*maxcor >= num_params.
    - Moved existing  and  computations to bfgs_sample_sparse, making the sparse use cases more explicit.
    
    Other changes:
    - Sets default maxcor=5 instead of dynamic sizing based on parameters
    
    Dense approximations are recommended when the target distribution has higher dependencies among the parameters.
    aphc14 committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    4299a58 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2024

  1. feat: improve Pathfinder performance and compatibility

    Bigger changes:
    - Made pmx.fit compatible with method='pathfinder'
    - Remove JAX dependency when inference_backend='pymc' to support Windows users
    - Improve runtime performance by setting trust_input=True for compiled functions
    
    Minor changes:
    - Change default num_paths from 1 to 4 for stable and reliable approximations
    - Change LBFGS code using dataclasses
    - Update tests to handle both PyMC and BlackJAX backends
    aphc14 committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    f1a54c6 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2024

  1. minor: improve error handling in Pathfinder VI

    - Add LBFGSInitFailed exception for failed LBFGS initialisation
    - Skip failed paths in multipath_pathfinder and track number of failures
    - Handle NaN values from Cholesky decompsition in bfgs_sample
    - Add checks for numericl stabilty in matrix operations
    
    Slight performance improvements:
    - Set allow_gc=False in scan ops
    - Use FAST_RUN mode consistently
    aphc14 committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    ea802fc View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2024

  1. Progress bar and other minor changes

    Major:
      - Added progress bar support.
    
    Minor
      - Added  exception for non-finite log prob values
      - Removed .
      - Allowed maxcor argument to be None, and dynamically set based on the number of model parameters.
      - Improved logging to inform users about failed paths and lbfgs initialisation.
    aphc14 committed Nov 27, 2024
    Configuration menu
    Copy the full SHA
    a77f2c8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9faaa72 View commit details
    Browse the repository at this point in the history