Merge pull request #365 from SDXorg/pysd37

pysd 3.7.0

.github/workflows/build-docs.yml

@@ -0,0 +1,27 @@
+
+# Build html docs. Any warning building the docs will produce an error.
+
+name: Build docs
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          submodules: recursive
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: '3.7'
+      - name: Install dependencies
+        run: |
+          pip install .
+          pip install -r docs/requirements.txt
+      - name: Test build html
+        run: |
+          cd docs
+          make html SPHINXOPTS="-W"

.github/workflows/link-check.yml

@@ -0,0 +1,18 @@
+name: Link check
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  linkChecker:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Link Checker
+        uses: lycheeverse/[email protected]
+        with:
+          fail: true
+        env:
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2013-2017 James Houghton
+Copyright (c) 2013-2022 PySD contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

MANIFEST.in

@@ -1,5 +1,5 @@
 include requirements.txt
 include README.md
-include docs/images/PySD_Logo*
 include LICENSE
+include docs/images/PySD_Logo*
 graft pysd/translators/*/parsing_grammars

README.md

@@ -1,17 +1,18 @@
 PySD
 ====
-
-[![Coverage Status](https://coveralls.io/repos/github/JamesPHoughton/pysd/badge.svg?branch=master)](https://coveralls.io/github/JamesPHoughton/pysd?branch=master)
+[![Maintained](https://img.shields.io/badge/Maintained-Yes-brightgreen.svg)](https://github.com/SDXorg/pysd/pulse)
+[![Coverage Status](https://coveralls.io/repos/github/SDXorg/pysd/badge.svg?branch=master)](https://coveralls.io/github/SDXorg/pysd?branch=master)
 [![Anaconda-Server Badge](https://anaconda.org/conda-forge/pysd/badges/version.svg)](https://anaconda.org/conda-forge/pysd)
 [![PyPI version](https://badge.fury.io/py/pysd.svg)](https://badge.fury.io/py/pysd)
 [![PyPI status](https://img.shields.io/pypi/status/pysd.svg)](https://pypi.python.org/pypi/pysd/)
 [![Py version](https://img.shields.io/pypi/pyversions/pysd.svg)](https://pypi.python.org/pypi/pysd/)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5654824.svg)](https://doi.org/10.5281/zenodo.5654824)
+[![Contributions](https://img.shields.io/badge/contributions-welcome-blue.svg)](https://pysd.readthedocs.io/en/latest/development/development_index.html)
 [![Docs](https://readthedocs.org/projects/pysd/badge/?version=latest)](https://pysd.readthedocs.io/en/latest/?badge=latest)
 
-![PySD Logo](https://raw.githubusercontent.com/JamesPHoughton/pysd/5cc4fe5dc65e6b5140a00e87a1be9d261570ee8d/docs/images/PySD_Logo_letters.svg?style=centerme)
+![PySD Logo](https://raw.githubusercontent.com/SDXorg/pysd/5cc4fe5dc65e6b5140a00e87a1be9d261570ee8d/docs/images/PySD_Logo_letters.svg?style=centerme)
 
-This project is a simple library for running [System Dynamics](http://en.wikipedia.org/wiki/System_dynamics) models in python, with the purpose of improving integration of *Big Data* and *Machine Learning* into the SD workflow.
+This project is a library for running [System Dynamics](http://en.wikipedia.org/wiki/System_dynamics) models in Python, with the purpose of improving integration of *Big Data* and *Machine Learning* into the SD workflow.
 
 **The current version needs to run at least Python 3.7.**
 
@@ -22,13 +23,13 @@ See the [project documentation](http://pysd.readthedocs.org/) for information ab
 - [Installation](http://pysd.readthedocs.org/en/latest/installation.html)
 - [Getting Started](http://pysd.readthedocs.org/en/latest/getting_started.html)
 
-For standard methods for data analysis with SD models, see the  [PySD Cookbook](https://github.com/JamesPHoughton/PySD-Cookbook), containing (for example):
+For standard methods for data analysis with SD models, see the  [PySD Cookbook](https://github.com/SDXorg/PySD-Cookbook), containing (for example):
 
-- [Model Fitting](http://nbviewer.ipython.org/github/JamesPHoughton/PySD-Cookbook/blob/master/2_1_Fitting_with_Optimization.ipynb)
-- [Surrogating model components with machine learning regressions](http://nbviewer.ipython.org/github/JamesPHoughton/PySD-Cookbook/blob/master/6_1_Surrogating_with_regression.ipynb)
-- [Multi-Scale geographic comparison of model predictions](http://nbviewer.ipython.org/github/JamesPHoughton/PySD-Cookbook/blob/master/Exploring%20models%20across%20geographic%20scales.ipynb)
+- [Model Fitting](http://nbviewer.ipython.org/github/SDXorg/PySD-Cookbook/blob/master/source/analyses/fitting/Fitting_with_Optimization.ipynb)
+- [Surrogating model components with machine learning regressions](http://nbviewer.ipython.org/github/SDXorg/PySD-Cookbook/blob/master/source/analyses/surrogating_functions/Surrogating_with_regression.ipynb)
+- [Multi-Scale geographic comparison of model predictions](http://nbviewer.ipython.org/github/SDXorg/PySD-Cookbook/blob/master/source/analyses/geo/Exploring_models_across_geographic_scales.ipynb)
 
-If you use PySD in any published work, consider citing the [PySD Introductory Paper](https://github.com/JamesPHoughton/pysd/blob/master/docs/PySD%20Intro%20Paper%20Preprint.pdf):
+If you use PySD in any published work, consider citing the [PySD Introductory Paper](https://github.com/SDXorg/pysd/blob/master/docs/PySD%20Intro%20Paper%20Preprint.pdf):
 
 >Houghton, James; Siegel, Michael. "Advanced data analytics for system dynamics models using PySD." *Proceedings of the 33rd International Conference of the System Dynamics Society.* 2015.
 
@@ -50,19 +51,17 @@ If you'd like to work with this repository directly, you'll need to use a recurs
 
 The command should be something like:
 ```shell
-git clone --recursive https://github.com/JamesPHoughton/pysd.git
+git clone --recursive https://github.com/SDXorg/pysd.git
 ```
 
 ### Extensions
 
 You can use PySD in [R](https://www.r-project.org/) via the [PySD2R](https://github.com/JimDuggan/pysd2r) package, also available on [cran](https://CRAN.R-project.org/package=pysd2r).
 
-### Contributors
+### Contributing
 
-Many people have contributed to developing this project - by
-[submitting code](https://github.com/JamesPHoughton/pysd/graphs/contributors), bug reports, and advice.
+PySD is currently a community-maintained project, any contribution is welcome.
 
-Special thanks to the [sdCloud.io](http://sdcloud.io) development team, who have
-made great contributions to XMILE support, and for integrating PySD into their cloud-based model simulation environment.
+Many people have contributed to developing this project - by [submitting code](https://github.com/SDXorg/pysd/graphs/contributors), bug reports, and advice. Main historic changes in PySD are described in the [About PySD section](https://pysd.readthedocs.io/en/latest/about.html). The [Developer Documentation](https://pysd.readthedocs.io/en/latest/development/development_index.html) could help new developers.
 
-Extra special thanks to [@enekomartinmartinez](https://github.com/enekomartinmartinez) for dramatically pushing forward subscript capabilities (and many other attributes).
+The code for this package is available at: https://github.com/SDXorg/pysd

docs/about.rst

@@ -1,6 +1,21 @@
 About the Project
 =================
 
+PySD was created in 2014 by `James P Houghton <https://github.com/JamesPHoughton>`_ to translate Vensim models to Python. The original goal for translating SD models into Python was to be able to take advantage of all the tools available in Python and thus to extent what is possible using Vensim.
+
+Since the creation of the library, many people have contributed to the project by reporting and fixing bugs and adding new features. These contributions are listed in the `contributions section of the GitHub repository <https://github.com/SDXorg/pysd/graphs/contributors>`_.
+
+Some of the big changes that have allowed PySD to get to its current state are the development of an XMILE to Python translator in 2017 by `Alex Prey <https://github.com/alexprey>`_ and the restructuring of the translation and model building through an Abstract Syntax by `Eneko Martin-Martinez <https://github.com/enekomartinmartinez>`_ in 2022.
+
+Some other contributions until release 3.0.0 were:
+
+- `Julien Malard-Adam <https://github.com/julienmalard>`_ added unicode support for the Vensim parser.
+- `sdCloud.io <http://sdcloud.io>`_ development team made great contributions to improve XMILE support and integrated PySD into their cloud-based model simulation environment.
+- `Eneko Martin-Martinez <https://github.com/enekomartinmartinez>`_ pushed forward the subscripts capabilities for both Vensim and XMILE and included support for several Vensim functions and improved the performance.
+- `Roger Samsó <https://github.com/rogersamso>`_ included a parser for the Vensim sketch and added the option to split a Vensim model per view based on the sketch information.
+
+The changes made since release 3.0.0 are tracked in the :doc:`whats_new` section.
+
 
 Motivation: The (coming of) age of Big Data
 -------------------------------------------
@@ -28,5 +43,3 @@ A third category of tools imports the models created by traditional tools to per
 The central paradigm of PySD is that it is more efficient to bring the mature capabilities of system dynamics into an environment in use for active development in data science, than to attempt to bring each new development in inference and machine learning into the system dynamics enclave.
 
 PySD reads a model file – the product of a modeling program such as Vensim or Stella/iThink – and cross compiles it into Python, providing a simulation engine that can run these models natively in the Python environment. It is not a substitute for these tools, and cannot be used to replace a visual model construction environment.
-
-

docs/command_line_usage.rst

@@ -26,11 +26,11 @@ In order to set the output file path, the *-o/--output-file* argument can be use
     python -m pysd -o my_output_file.csv Teacup.mdl
 
 .. note::
-    The output file can be a *.csv* or *.tab*.
+    The output file format may be *.csv*, *.tab* or *.nc*.
 
 .. note::
-    If *-o/--output-file* is not given, the output will be saved in a file
-    that starts with the model file name followed by a time stamp to avoid
+    If *-o/--output-file* is not given, the output will be saved in a *.tab*
+    file that starts with the model file name followed by a time stamp to avoid
     overwritting files.
 
 Activate progress bar

docs/conf.py

@@ -58,8 +58,8 @@
 ]
 
 extlinks = {
-    "issue": ("https://github.com/JamesPHoughton/pysd/issues/%s", "issue #%s"),
-    "pull": ("https://github.com/JamesPHoughton/pysd/pull/%s", "PR #%s"),
+    "issue": ("https://github.com/SDXorg/pysd/issues/%s", "issue #%s"),
+    "pull": ("https://github.com/SDXorg/pysd/pull/%s", "PR #%s"),
 }
 
 # Add any paths that contain templates here, relative to this directory.
@@ -74,8 +74,8 @@
 
 # General information about the project.
 project = 'PySD'
-copyright = '2016, James Houghton'
-author = 'James Houghton'
+copyright = '2022, PySD contributors'
+author = 'PySD contributors'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -128,7 +128,7 @@
 # author, documentclass [howto, manual, or own class]).
 latex_documents = [
   (master_doc, 'PySD.tex', 'PySD Documentation',
-   'James Houghton', 'manual'),
+   'PySD contributors', 'manual'),
 ]
 
 # -- Options for manual page output ---------------------------------------

docs/development/guidelines.rst

@@ -25,7 +25,7 @@ of the `/tests/` directory.
 
 In order to run all the tests :py:mod:`pytest` should be used.
 A `Makefile` is given to run easier the tests with :py:mod:`pytest`, check
-`tests/README <https://github.com/JamesPHoughton/pysd/tree/master/tests/README.md>`_
+`tests/README <https://github.com/SDXorg/pysd/tree/master/tests/README.md>`_
 for more information.
 
 These tests run quickly and should be executed when any changes are made to ensure

docs/development/pathway.rst

@@ -2,7 +2,7 @@ PySD Development Pathway
 ========================
 
 High priority features, bugs, and other elements of active effort are listed on the `github issue
-tracker. <https://github.com/JamesPHoughton/pysd/issues>`_ To get involved see :doc:`guidelines`.
+tracker. <https://github.com/SDXorg/pysd/issues>`_ To get involved see :doc:`guidelines`.
 
 
 High Priority

docs/generate_tables.py

@@ -1,6 +1,7 @@
-import pandas as pd
 from pathlib import Path
 
+import pandas as pd
+
 
 def generate(table, columns, output):
     """Generate markdown table."""

docs/getting_started.rst

@@ -1,6 +1,14 @@
 Getting Started
 ===============
 
+.. note::
+   A cookbook of simple recipes for advanced data analytics using PySD is available at:
+   http://pysd-cookbook.readthedocs.org/
+
+   The cookbook includes models, sample data, and code in the form of iPython notebooks that demonstrate a variety of data integration and analysis tasks.
+   These models can be executed on your local machine, and modified to suit your particular analysis requirements.
+
+
 Importing a model and getting started
 -------------------------------------
 To begin, we must first load the PySD module, and use it to import a model file::
@@ -156,6 +164,21 @@ The subscripted variables, in general, will be returned as :py:class:`xarray.Dat
 
    >>> model.run(flatten_output=True)
 
+
+Storing simulation results on a file
+------------------------------------
+Simulation results can be stored as *.csv*, *.tab* or *.nc* (netCDF4) files by defining the desired output file path in the `output_file` argument, when calling the :py:meth:`.run` method::
+
+   >>> model.run(output_file="results.tab")
+
+If the `output_file` is not set, the :py:meth:`.run` method will return a :py:class:`pandas.DataFrame`.
+
+For most cases, the *.tab* file format is the safest choice. It is preferable over the *.csv* format when the model includes subscripted variables. The *.nc* format is recommended for large models, and when the user wants to keep metadata such as variable units and description.
+
+.. warning::
+   *.nc* files require :py:mod:`netcdf4` library which is an optional requirement and thus not installed automatically with the package. We recommend using :py:mod:`netcdf4` 1.6.0 or above, however, it will also work with :py:mod:`netcdf4` 1.5.0 or above.
+
+
 Setting parameter values
 ------------------------
 In some situations we may want to modify the parameters of the model to investigate its behavior under different assumptions. There are several ways to do this in PySD, but the :py:meth:`.run` method gives us a convenient method in the `params` keyword argument.
@@ -174,7 +197,7 @@ If the parameter value to change is a subscripted variable (vector, matrix...),
 
    >>> model.run(params={'Subscripted var': 0})
 
-A partial :py:class:`xarray.DataArray` can be used. For example a new variable with ‘dim2’ but not ‘dim2’. In that case, the result will be repeated in the remaining dimensions::
+A partial :py:class:`xarray.DataArray` can be used. For example a new variable with ‘dim2’ but not ‘dim1’. In that case, the result will be repeated in the remaining dimensions::
 
    >>> import xarray as xr
    >>> new_value = xr.DataArray([1, 5], {'dim2': [1, 2]}, ['dim2'])

docs/index.rst

@@ -4,20 +4,25 @@ PySD
 
 |made-with-sphinx-doc|
 |DOI|
+|Maintained|
 |PyPI license|
 |conda package|
 |PyPI package|
 |PyPI status|
 |PyPI pyversions|
+|Contributions|
 
 .. |made-with-sphinx-doc| image:: https://img.shields.io/badge/Made%20with-Sphinx-1f425f.svg
    :target: https://www.sphinx-doc.org/
 
+.. |Maintained| image:: https://img.shields.io/badge/Maintained-Yes-brightgreen.svg
+   :target: https://github.com/SDXorg/pysd/pulse
+
 .. |docs| image:: https://readthedocs.org/projects/pysd/badge/?version=latest
    :target: https://pysd.readthedocs.io/en/latest/?badge=latest
 
 .. |PyPI license| image:: https://img.shields.io/pypi/l/sdqc.svg
-   :target: https://github.com/JamesPHoughton/pysd/blob/master/LICENSE
+   :target: https://github.com/SDXorg/pysd/blob/master/LICENSE
 
 .. |PyPI package| image:: https://badge.fury.io/py/pysd.svg
     :target: https://badge.fury.io/py/pysd
@@ -34,6 +39,9 @@ PySD
 .. |DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.5654824.svg
    :target: https://doi.org/10.5281/zenodo.5654824
 
+.. |Contributions| image:: https://img.shields.io/badge/contributions-welcome-blue.svg
+   :target: https://pysd.readthedocs.io/en/latest/development/development_index.html
+
 This project is a simple library for running System Dynamics models in Python, with the purpose of improving integration of Big Data and Machine Learning into the SD workflow.
 
 PySD translates :doc:`Vensim <structure/vensim_translation>` or
@@ -63,16 +71,19 @@ The cookbook includes models, sample data, and code in the form of iPython noteb
 
 Contributing
 ^^^^^^^^^^^^
-The code for this package is available at: https://github.com/JamesPHoughton/pysd
+|Contributions|
+
+PySD is currently a community-maintained project, any contribution is welcome.
+
+The code for this package is available at: https://github.com/SDXorg/pysd
 
-If you find a bug, or are interested in a particular feature, see :doc:`reporting bugs <../reporting_bugs>`.
+If you find any bug, or are interested in a particular feature, see :doc:`reporting bugs <../reporting_bugs>`.
 
-If you are interested in contributing to the development of PySD, see the :doc:`developer documentation <../development/development_index>`
-listed above.
+If you are interested in contributing to the development of PySD, see the :doc:`developer documentation <../development/development_index>` listed above.
 
 Citing
 ^^^^^^
-If you use PySD in any published work, consider citing the `PySD Introductory Paper <https://github.com/JamesPHoughton/pysd/blob/master/docs/PySD%20Intro%20Paper%20Preprint.pdf>`_::
+If you use PySD in any published work, consider citing the `PySD Introductory Paper <https://github.com/SDXorg/pysd/blob/master/docs/PySD%20Intro%20Paper%20Preprint.pdf>`_::
 
    Houghton, James; Siegel, Michael. "Advanced data analytics for system dynamics models using PySD." *Proceedings of the 33rd International Conference of the System Dynamics Society.* 2015.
 

docs/installation.rst

@@ -23,9 +23,9 @@ To install from source, clone the project with git:
 
 .. code-block:: bash
 
-   git clone https://github.com/JamesPHoughton/pysd.git
+   git clone https://github.com/SDXorg/pysd.git
 
-or download the latest version from the project repository: https://github.com/JamesPHoughton/pysd
+or download the latest version from the project repository: https://github.com/SDXorg/pysd
 
 In the source directory use the command:
 
@@ -66,6 +66,10 @@ In order to plot model outputs as shown in :doc:`Getting started <../getting_sta
 
 * Matplotlib
 
+In order to be able to export data to netCDF (*.nc*) files:
+
+* netCDF4
+
 
 These Python libraries bring additional data analytics capabilities to the analysis of SD models:
 
@@ -82,5 +86,5 @@ These modules can be installed using pip with a syntax similar to the above.
 
 Additional Resources
 --------------------
-The `PySD Cookbook <https://github.com/JamesPHoughton/PySD-Cookbook>`_ contains recipes that can help you get set up with PySD.
+The `PySD Cookbook <https://github.com/SDXorg/PySD-Cookbook>`_ contains recipes that can help you get set up with PySD.
 

docs/reporting_bugs.rst

@@ -3,7 +3,7 @@ Reporting bugs
 
 Before reporting any bug, please make sure that you are using the latest version of PySD. You can get the version number by running `python -m pysd -v` on the command line.
 
-All bugs must be reported in the project's `issue tracker on github <https://github.com/JamesPHoughton/pysd/issues>`_.
+All bugs must be reported in the project's `issue tracker on github <https://github.com/SDXorg/pysd/issues>`_.
 
 .. note::
   Not all the features and functions are implemented. If you are in trouble while translating or running a Vensim or Xmile model check the :ref:`Vensim supported functions <Vensim supported functions>` or :ref:`Xmile supported functions <Xmile supported functions>` and consider that when openning a new issue.