Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API changes to constructing DenseSkOps and populating their buffers, set next_state on construction, add web documentation, more stat tests #104

Merged
merged 38 commits into from
Aug 22, 2024

Conversation

rileyjmurray
Copy link
Contributor

@rileyjmurray rileyjmurray commented Jul 27, 2024

This PR is huge. I'm going to merge it even though there are outstanding documentation TODOs, as indicated in Issue #107.

Changes in RandBLAS proper

Major new identifiers

  • SketchingDistribution and SketchingOperator C++20 concepts.
  • isometry_scale_factor (overloaded for DenseSkOp and SparseSkOp), returns the scale that should canonically be applied to the full sketching operator assuming its entries are sampled in RandBLAS' standard way (uniform $\sqrt{3}[-1,1]$ or Gaussian(0, 1) for dense, or length-vec_nnz vectors of iid Rademachers for sparse).
  • dense::compute_next_state and sparse::compute_next_state, for computing the appropriate value of S.next_state in O(1) time. This enables defining sequences of sketching operators with the same key without having to explicitly generate the operators.

Minor new identifiers

  • For dense sketching operators, submatrix_as_blackbox accepts a sketching operator, row and column counts, and row and column offsets, and returns a DenseSkOp with a BlackBox distribution whose entries match those of the appropriate submatrix of the given sketching operator.
  • safe_int_product (in RandBLAS/base.hh) checks for overflows. This is useful when computing potentially-large index offsets for implicitly defined matrices.
  • util::omatcopy for general stride changes
  • util::overwrite_triangle, for preparing a triangular matrix to be treated as a general matrix in gemm or trmm.
  • A new value MajorAxis::Unknown in the MajorAxis enum, for use with BlackBox distribution families.
  • An overload of repeated_fisher_yates that skips writes to vals and idxs_minor and that has more intuitive argument names.

Other changes

  • The default scalar distribution for DenseSkOp objects is now Gaussian instead of uniform.
  • The default scaling of uniform sketching operators is $[-\sqrt{3},\sqrt{3}]$, since this choice provides for unit variance.
  • rewrite of dense::fill_dense_submat_impl, needed in order to efficiently implement dense::compute_next_state.
  • In SparseSkOp, the typename RNG_t has been removed, a typename state_t has been added (equal to RNGState<RNG>) and the typename T_t typename has been renamed to scalar_t. Similar changes have been made to DenseSkOp for consistency across the two APIs.
  • Changed the verbose overload offill_dense for writing an implicit submatrix to a given buffer. This function now requires a layout parameter as its first argument. It performs an out-of-place change of layout if the requested layout is different from the natural layout implied by the distribution object.
  • Moved several function definitions for DenseSkOp and SparseSkOp into the struct definitions, rather than only declaring signatures in the struct definition and implementing the functions outside the struct.

Changes in documentation

  • Added sample_indices_iid_uniform, sample_indicies_iid, and weights_to_cdf to a new page in the API reference.
  • Added a section to the tutorial on updating sketches.

Changes in tests

New / significantly changed tests

  • test_continuous.cc: Kolmogorov-Smirnov tests for distributional correctness of uniform [-sqrt(3),sqrt(3)] and Gaussian sampling. This includes a helper function for exactly computing the KS test statistic for distributions whose CDF is continuous, which has some surprising subtleties.
  • test_r123.cc: a significant clean up of (and slight expansion of) test_r123_kat.cc.
  • test_distortion.cc: statistical tests for subspace embedding distortion of dense sketching operators. This required a bunch of new infrastructure, described below.

handrolled_lapack.hh implements algorithms that would generally be considered "LAPACK-level," although the algorithms won't be found in LAPACK itself.

  • cholesky (sequential and blocked)
  • cholesky qr (with a flag to run a second pass)
  • QR based on block classic Gram-Schmidt, where blocks are handled by cholesky qr2.
  • QR with a two-pass version of blocked classic Gram-Schmidt. (Not equivalent to modified Gram-Schmidt.)
  • A function to check if the outer-most Gershgorin discs for a positive definite matrix have converged to within some specified relative radius.
  • cholesky iteration to compute all eigenvalues of a positive definite matrix (with termination criteria based only on extremal eigenvalues). This is actually prohibitively slow except when the matrix is very small. But it works, and it's tested, so might as well keep it around.
  • power method to compute the extremal eigenvalues of a positive definite matrix. The number of iterations is chosen based on a well-known convergence result for the power method in the absence of a spectral gap. The smallest eigenvalue is obtained by Cholesky decomposing the matrix, explicitly forming its inverse, and then running the standard power method. This leads to faster performance for matrix dimensions and tolerances that we need in statistical tests.

…menting random123 Nx32 arrays by 64 bit integers.
…exing into an implicit array by a potentially very-large number). Moved SparseSkOp and DenseSkOp constructor and destructor implementations into class definitions, rather than only declaring function signatures in the class definitions. Changed next_state for DenseSkOp and SparseSkOp to be const. Add shell/unimplemented compute_next_state functions for DenseSkOp and SparseSkOp for computing the correct value for next_state when a sketching operator is constructed. Add a fill_dense overload that lets you specify a required buffer layout (this function allocates workspace for an out-of-place transpose if needed). Add an omatcopy function for out-of-place stride changes that doesnt do any bound checking.
…eSkOp. Breaking API change that removes fill_dense(dist, n_rows, n_cols, ro_s, co_s, buff, seed) --- which returned a tuple of Layout and RNGState --- and replaces it with fill_dense(layout, dist, n_rows, n_cols, ro_s, co_s, buff, seed) --- which returns an RNGState.
@rileyjmurray rileyjmurray changed the title WIP : Ability to specify layout for DenseSkOp.buff, set next_state on construction, more stat tests WIP : API changes to constructing DenseSkOps and populating their buffers, set next_state on construction, add web documentation, more stat tests Aug 2, 2024
@rileyjmurray rileyjmurray marked this pull request as ready for review August 19, 2024 17:51
@rileyjmurray rileyjmurray changed the title WIP : API changes to constructing DenseSkOps and populating their buffers, set next_state on construction, add web documentation, more stat tests API changes to constructing DenseSkOps and populating their buffers, set next_state on construction, add web documentation, more stat tests Aug 21, 2024
…C++20 concepts. Bring DenseSkOp and SparseSkOp more in alignment with one another. Simplify some unnecessarily verbose templating for functions that accepted sketching operators or sparse matrices as inputs.
…e meta-tests for power iteration implementation cheaper (by virtue of testing against the same tolerance but running fewer iterations, so the test is now more adversarial).
@rileyjmurray rileyjmurray merged commit 614c1c2 into main Aug 22, 2024
5 checks passed
@rileyjmurray rileyjmurray deleted the more-stat-tests branch August 22, 2024 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant