diff --git a/OPEN_SOURCE_PLAN.md b/OPEN_SOURCE_PLAN.md deleted file mode 100644 index 7892fad5c..000000000 --- a/OPEN_SOURCE_PLAN.md +++ /dev/null @@ -1,83 +0,0 @@ - - - - -# Open Source Execution Plan for SPK et al. - -This is our working plan for how to turn this into a fully public, -community-developed and widely used project. - -Contributors should propose edits to this, add milestones or tasks I have -forgotten, and check off items as they are completed. We'll only keep this -document alive as long as it is useful to us. - -## Before any sharing - -There are some housekeeping tasks to perform to the code base before anyone -outside Imageworks can have access to it: - -- [x] Set up external (but private) repos, add internal developers. -- [x] Set up email list, add internal developers. -- [x] The code needs a LICENSE file attached, and copyright notices affixed - to all the code modules. -- [x] README, CONTRIBUTING guide, basic procedures, etc. -- the set of - things expected for any open source project. -- [x] Some work to make it buildable and testable, so that the initial - co-developers will actually be able to kick the tires. -- [x] If possible, set up initial continuous integration so we know it - builds outside of our environment and can pass basic tests. -- [x] Get **final** business clearance to open source. -- [x] Add external developers to mail list, GitHub repos. - - -## Initial private share to work toward minimal viable product - -The code is not currently ready to let just anybody use it -- it lacks a -simple way to build it outside our environment, needs documentation, comes -with no build/packaging recipes, etc. We don't want to go "public" until -it's in shape that it will make a good first impression and be useful for -people to start using. - -We propose to host it on GitHub in the Imageworks area, but as private -repositories that are only visible with per-user permissions. We will -initially invite the other people who have expressed interest in helping us -to co-develop the software moving forward. As we start to talk more publicly -about what we're working on, we can also invite anyone else who subsequently -expresses an interest in helping with the process. - -The purpose of this privately shared period is get to a "MVP" state, which -involves at least the following: - -- [x] Let the other presumed contributors see the code for the first time, try it out, evaluate it, figure out which parts need alteration to suit their needs, and which parts of the project moving forward they would like to help with. -- [x] Set up a regular stakeholder meeting cadence. -- [ ] Combine the (currently) three repositories (for the file system layer, the packager, and the application launcher) into a single project/repository. -- [x] Make sure that it's easily buildable and deployable outside the SPI environment, including build/deploy documentation. -- [x] Set up a continuous integration system that can build and test the system, using GitHub Actions. -- [ ] Figure out how to organize package recipes within the repository, and seed it with package recipes for all the common VFX packages so that it will be a useful working system "out of the box" when it goes truly public. -- [ ] Set up web site and online documentation. -- [ ] Need logo and other basic graphic design assets. -- [ ] Achieve consensus on project governance -- who chairs meetings, how do we reach decisions, is there a chief architect with final call on technical disputes, how are people added to steering committee, etc. -- [ ] Anything else, technical or organizational, that comes up or is identified by the co-developers, that we think needs to be fixed before it's ready to be public. - - -## Public share - -When the stakeholders think that the project is ready to launch publicly, we can just switch the existing repository from "private" to "public." We'll probably want some announcement or PR at that time. - - -## Possible ASWF future - -It certainly seems like an ideal project to turn over to the ASWF. It is hard to determine the best timeline for this at this point, particularly before the other co-developers have been fully identified and before any of them have had a chance to see the project in action. - -It may be that the right approach is to continue hosting it in the Imageworks GitHub account until it reaches a maturity level that the stakeholders decide they want to apply to turn it over to the ASWF. It is also possible that soon after the stakeholders see it and start working, they will desire to have it hosted in a truly neutral place like the ASWF as soon as possible, as a way to provide an even better foundation upon which many parties can collaborate. We will see how it goes and collectively decide when the time is right to try to have it hosted at ASWF. - - -## Proposed Timeline - -This is all subject to change; this is just a stake in the ground to -document a possible future. - -- Immediately: Set up private repos (without sharing with anyone outside), start working on the "pre-share" tasks listed above. -- By the end of next week (Apr 23), if possible: invite the outside stakeholders to have read permissions on the private repository. -- SIGGRAPH might be a good goal for a "go public" date and formal announcement about the project. The actual date for switching the repo to public visibility might be somewhat before or somewhat after, depending on how quickly we can accomplish the "private share" tasks. -- TBD when the right time is to apply for ASWF governance of the project, we'll have to see how the other steps go. diff --git a/README.md b/README.md index 8a0fd3595..42b95be46 100644 --- a/README.md +++ b/README.md @@ -26,10 +26,10 @@ See the main [docs](docs/) for details on using spk, starting with the [index](d ## License -SPK/SPFS/spawn are Copyright (c) Contributors to the SPK project. +SPK/SPFS are Copyright (c) Contributors to the SPK project. All Rights Reserved. -SPK/SPFS/spawn are distributed using the [Apache-2.0 license](LICENSE.txt). +SPK/SPFS are distributed using the [Apache-2.0 license](LICENSE.txt). ## Structure of this project @@ -37,26 +37,13 @@ SPK/SPFS/spawn are distributed using the [Apache-2.0 license](LICENSE.txt). `spk` is the software packaging system built on top of SPFS. -`spawn` is the application launcher for spk packages. - -These are spread over two code bases at the moment, but may -be merged into a single project, [spk](https://github.com/spkenv/spk). -Please refer to [spk](https://github.com/spkenv/spk) for almost all -information about staging the open source project, that's where the -developer documentation and communication will live, including -[Contributing to SPK](https://github.com/spkenv/spk/CONTRIBUTING.md). - ## Contributing Please read [Contributing to SPK](https://github.com/spkenv/spk/CONTRIBUTING.md). -## Development plan - -Please read [SPK open source development plan](https://github.com/spkenv/spk/OPEN_SOURCE_PLAN.md). - ## Development -Both spk and spfs are written in Rust and use cargo. The best way to get started with rust development is to install the latest stable rust toolchain using [rustup](https://rustup.sh). +Both SPK and SPFS are written in Rust and use cargo. The best way to get started with Rust development is to install the latest stable Rust toolchain using [rustup](https://rustup.sh). For details on architecture and design of the codebase, see the [developer docs](docs/develop). @@ -83,7 +70,7 @@ make install ### RPM Package -The codebase is setup to produce a centos7-compatible rpm package for both spfs and spk by building them in a docker container. To create the rpm package, you will need docker installed. These packages are also built and made available in this repository's CI. +The codebase is set up to produce a centos7-compatible rpm package for both spfs and spk by building them in a docker container. To create the rpm package, you will need docker installed. These packages are also built and made available in this repository's CI. ```sh # build the rpm package via docker and copy into ./dist/rpm @@ -92,7 +79,7 @@ make rpms ### Testing -Both projecs have a number of unit and integration tests as well as testable examples that can all be executed with `make test`. The tests for spk need to be executed under an spfs runtime in order to properly execute. We also have configured linting rules that must pass for all contributions. +Both projects have a number of unit and integration tests as well as testable examples that can all be executed with `make test`. The tests for spk need to be executed under an SPFS runtime in order to properly execute. We also have configured linting rules that must pass for all contributions. Our repository is broken down into a number of smaller crates for easier development, and they can be individually targeted in the makefile which can greatly reduce the time it takes for testing and linting. @@ -108,6 +95,10 @@ make lint test CRATES=spfs-encoding,spfs-cli-common ### Bootstrapping +> [!WARNING] +> The existing bootstrapping setup was developed for CentOS 7 and may need +> updates as we transition to el9 distributions. + In a new environment, it can be helpful to build all of the core packages whose recipes ship with SPK. A script has been provided which runs through all of the builds for these packages in the right order. ```sh @@ -123,6 +114,10 @@ Some of these package specs have not yet been used or tested fully or ironed out #### Using Docker +> [!WARNING] +> The this process was setup to create packages in CentOS 7 and may need +> updates as we transition to el9 distributions. + Currently, this process can only be run on an rpm-based system, as it relies on some rpm packages being installed on the host in order to bootstrap the build process. If you are not running on an rpm-based system, you can run the process in a container instead: ```sh @@ -137,7 +132,7 @@ make packages.import #### Conversion Packages -Spk has logic to automatically convert pip packages to spk packages for easy python environment creation. This logic lives and runs inside of it's own spk package/environment. If you have python3 already installed, you can generate this package locally like so: +Spk has logic to automatically convert pip packages to spk packages for easy Python environment creation. This logic lives and runs inside of its own spk package/environment. If you have python3 already installed, you can generate this package locally like so: ```sh make converters @@ -153,10 +148,10 @@ spk convert pip --help #### Other Notes -- The make `packages.python2` and `packages.python3` targets can be used to bootstrap just enough to be able to build python for spk. The python recipes will build multiple python versions for each gcc48 and 63 as well as for the different python abi's +- The `make packages.python3` target can be used to bootstrap just enough to be able to build Python for SPK. The Python recipes will build multiple Python versions for each gcc48 and 63 as well as for the different Python ABIs - The make `packages.gnu` target can be used to bootstrap just enough to get "native" spk packages for gcc48 and gcc63 -Of course, the packages themselves can also be build with the `spk build ` command directly, though you may find that some required build dependencies need to be generated with the `make packages.bootstrap.full` command first. +Of course, the packages themselves can also be built with the `spk build ` command directly, though you may find that some required build dependencies need to be generated with the `make packages.bootstrap.full` command first. The following RPM packages must be installed in order to create the bootstrap packages. @@ -184,13 +179,14 @@ sudo yum install -y \ zip \ zlib ``` -SPFS has a number of unit tests written in rust that can be run using the `cargo` command. + +SPFS has a number of unit tests written in Rust that can be run using the `cargo` command or via our make target. ```sh -cargo test +make test ``` -Additionally, there are a number of integration tests that validate the fully installed state of spfs. These are generally a series of spfs command line calls that validate the creation and usage of the `/spfs` filesystem. +Additionally, there are a number of integration tests that validate the fully installed state of spfs. These are generally a series of SPFS command line calls that validate the creation and usage of the `/spfs` filesystem. ```sh cargo build @@ -200,9 +196,9 @@ tests/integration/run_all.sh ### Windows -To build and run on windows, you need a couple of dependencies that are easiest to install via chocolatey. +To build and run on Windows, you need a couple of dependencies that are easiest to install via Chocolatey. -``` +```sh choco install protoc llvm winfsp ``` @@ -214,7 +210,7 @@ Benchmark tests can be found in `benches/`. All benchmark tests can be run with cargo bench --bench spfs_bench ``` -A common workflow as described [here](https://bheisler.github.io/criterion.rs/book/user_guide/command_line_options.html#baselines) is to record a baseline measurement to use as a reference to compare future measurements to. +A common workflow as described [here](https://bheisler.github.io/criterion.rs/book/user_guide/command_line_options.html#baselines) is to record a baseline measurement to use as a reference to compare future measurements against. ```sh git checkout main