PoST package structure for v1.0

[fasmat]

The package structure of PoST is quite confusing and could be improved for the next major release. I propose the following layout for clearer naming of types and generally reducing the number of individual packages a user of the library has to interact with:

• post: The root package containing the module definition, the initializer, all types and functions associated with it and everything that is currently in config and shared and doesn't fit and one of the following packages
• post/internal: Internal functionality not to be used directly by users of this library. This includes the bridges to the C and rust code and possibly other internal types and functions not intended to be available outside of the module
• post/internal/oracle: The oracle should be an internal package since its use outside of generating and verifying proofs is limited and users of the library should call those functions over the oracle directly.
• post/proof: Contains the Generate and Verify functions, as well as the Proof and Metadata types from shared.

With this structure it is unlikely that a user of the library has to import more than one package at a time, while at the moment proving and verifying also always require to also import config and shared both of which collide with packages with the same name in other modules. post.Config is more meaningful than config.Config, proving.Generate/ verifying.Verify are less clear compared to proof.Generate / proof.Verify.

Also there is a clear dependency chain between packages of these structure:

post/proof -> post -> post/internal

The former packages only import the later and not vice versa.

[fasmat]

From architecture discussion:

• Package post will have one type: Data that is used to interact with PoST files on disk
  • post.Open will return a post.Data object after doing some basic checks (i.e. parsing metadata and checking correct number of files and sizes). If these basic checks fail it will return an error indicating the issue. Opening a post data dir does not require to instantiate post-rs so no opencl provider is needed at all. Additionally the directory will be locked with a lockfile to avoid 2 processes (e.g. go-sm and postcli) from accessing the data directory simultaneously (it will still be possible to resize while smeshing)
  • post.Init will be used to initialize PoST in a directory with the given data and is just a function that receives a context that allows cancellation, Initialization can be resumed when cancelled
  • post.Resize can be used to resize an existing PoST. Most of the process can be done while the directory is locked for access (new files will be created in a sub directory). It can also be canceled. It will error when merging new files with existing files (moving data files to the right place and updating postdata_metadata.json ) if the post data dir is locked, but can be resumed from that step when no other process is locking the directory
• post/proof is a new package that merges verify and proving packages
  • Generating a proof requires the post.Data object (to ensure directory is locked) and can be cancelled (again as far as possible from the go-side; for rust we need to think about how to make cancellation happen)
  • Verifying a proof does not require a post.Data object and will work pretty much identically as of now
• config, shared, oracle, persistence, initialization packages will all be dropped
  their contents will partially be moved to the two packages above, deleted or moved to internal to not be accessed outside functions of the module