Common Requirements Enumeration Application

This is work in progress. See the application working at https://www.opencre.org

This python web and cli application handles adding and presenting CREs.

Installing

To install this application you need python3, yarn and virtualenv. Clone the repository:

git clone https://github.com/OWASP/common-requirement-enumeration 

Copy sqlite database to required location

cp cres/db.sqlite standards_cache.sqlite

Install dependencies

 make install 

Running

To run the CLI application, you can run

python cre.py --help

To download a remote cre spreadsheet locally you can run

python cre.py --review --from_spreadsheet < google sheets url>

To add a remote spreadsheet to your local database you can run

python cre.py --add --from_spreadsheet < google sheets url>

To run the web application for development you can run

make dev-run

Alternatively, you can use the dockerfile with

make docker && make docker-run

To run the web application for production you need gunicorn and you can run from within the cre_sync dir

make prod-run

Developing

You can run backend tests with

make test

You can run get a coverage report with

make cover

Try to keep the coverage above 70%

Repo Moved here from https://github.com/northdpole/www-project-integration-standards

Contributing

Please see Contributing for contributing instructions

Development Notes

• add tests

• defs

• db

• parsers

• mapping_add ( done for important methods ) argparse logic only remains

• spreadsheet_utils

• frontend

• add parse from export format

• add parse from export format where the root doc is a standard and it links to cres or groups

• add parse from spreadsheet with unknown standards (for key,val in items add_standard)

• merge spreadsheet to yaml and mapping add, they do the same thing

• add the ability for standards to link other standards, then you can handle assigning CREs yourself

• support importing yaml export files of more than 1 levels deep

• add export for Standards unmapped to CREs as lone standards (useful for visibility)

• add sparse_spreadsheet_export functionality one level of mapping per row, either everything that maps to standard X or everything that maps to CRE x

• add parse from export format

• add github actions ci

• make into flask rest api

• > refer use case (search by cre)

• > search by standard

• add the ability for a mapping document to have multiple yamls in it

• add db integration of tags

• add tags in db (search by tag, export with tags etc)

• add parser integration of tags (parse the new new new spreadsheet template which incorporates tags)

• add search by tag in rest

• add dockerfile

• add conditional export (select the standards you want exported get mappings between them) (gap analysis use case) ~ -- Done

• add flask cover command from here https://github.com/miguelgrinberg/flasky/blob/master/flasky.py#L33

• Make Standards versioned ~ -- Done

• write frontend

• make results per page a config item from env

• migrate to new repo

• add black autoformater

• merge frontend changes to master

• Typed Python?

= Future Considerations =

• improve test coverage -- we are at 73%, let's increase to 80%

• Make frontend show gap analysis

• Make frontend export search results and gap analysis to spreadsheet (supply backend with an "export=True" arg)

• Make frontned able to import from spreadsheet template.

• Make frontend able to import from files

• Make frontend able to import by filing in a form.

• make pagination also for tag results and gap analysis

• make library out of file format and spreadsheet template parsers

• add more linkTypes, Child, Controls, Tests, others.

• Add more Document types, Tool, Library

• Figure a way to dynamically register new Custom Resource Definitions and register custom logic on what to do on import/export and search.

• write docs and record usage gif