diff --git a/.coveragerc b/.coveragerc index 5413503a..5ee79611 100644 --- a/.coveragerc +++ b/.coveragerc @@ -3,8 +3,8 @@ branch = True omit = */pycm/__main__.py */pycm/__init__.py - */pycm/pycm_profile.py - */pycm/pycm_test.py + */pycm/profile.py + */pycm/basic_test.py [report] # Regexes for lines to exclude from consideration exclude_lines = diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 3efe6d3a..8942bee4 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -1,4 +1,6 @@ -# Contribution +# Contribution + +**Last Update: 2024-10-08** Changes and improvements are more than welcome! ❤️ Feel free to fork and open a pull request. @@ -26,18 +28,18 @@ Please consider the following : ## Class statistic -1. Add new functions to `pycm_class_func.py` -2. Update `CLASS_PARAMS` dictionary in `pycm_param.py` -3. Update `class_statistics` function in `pycm_class_func.py` +1. Add new functions to `class_funcs.py` +2. Update `CLASS_PARAMS` dictionary in `params.py` +3. Update `class_statistics` function in `class_funcs.py` - Call statistic function and store result in `result` dictionary -4. Update `PARAMS_DESCRIPTION` dictionary in `pycm_param.py` by a short description - - If you don't want capitalization, update `CAPITALIZE_FILTER` list in `pycm_param.py` (*Optional*) +4. Update `PARAMS_DESCRIPTION` dictionary in `params.py` by a short description + - If you don't want capitalization, update `CAPITALIZE_FILTER` list in `params.py` (*Optional*) 5. Update `References` section in `Document.ipynb` (`IEEE` format) 6. Add description to `Class Statistics` section in `Document.ipynb` - Cite reference - Update table of contents - Use `LaTeX` for formula -7. Update `PARAMS_LINK` dictionary in `pycm_param.py` by document tag (without `#`) +7. Update `PARAMS_LINK` dictionary in `params.py` by document tag (without `#`) 8. Add tests to `overall_test.py` and `function_test.py` in `TEST` folder - If you have any verified test add them to `verified_test.py` 9. Run `autopep8.bat`/`autopep8.sh` (*Optional*, need to install latest version of `autopep8` package) @@ -46,9 +48,9 @@ Please consider the following : ## Overall statistic -1. Add new functions to `pycm_overall_func.py` -2. Update `OVERALL_PARAMS` dictionary in `pycm_param.py` -3. Update `overall_statistics` function in `pycm_class_func.py` +1. Add new functions to `overall_funcs.py` +2. Update `OVERALL_PARAMS` dictionary in `params.py` +3. Update `overall_statistics` function in `class_funcs.py` - Call statistic function and store result in a variable - Add this variable to output 4. Update `References` section in `Document.ipynb` (`IEEE` format) @@ -56,7 +58,7 @@ Please consider the following : - Cite reference - Update table of contents - Use `LaTeX` for formula -6. Update `PARAMS_LINK` dictionary in `pycm_param.py` by document tag (without `#`) +6. Update `PARAMS_LINK` dictionary in `params.py` by document tag (without `#`) 7. Add tests to `overall_test.py` and `function_test.py` in `TEST` folder - If you have any verified test add them to `verified_test.py` 8. Run `autopep8.bat`/`autopep8.sh` (*Optional*, need to install latest version of `autopep8` package) @@ -64,18 +66,18 @@ Please consider the following : ## Interpretation -1. Add new interpretation table as a function to `pycm_interpret.py` -2. Add a score dictionary to `pycm_param.py` +1. Add new interpretation table as a function to `interpret.py` +2. Add a score dictionary to `params.py` - Example : ```PLRI_SCORE = {"Good": 4, "Fair": 3, "Poor": 2, "Negligible": 1, "None": "None"}``` -3. Add a color dictionary to `BENCHMARK_COLOR` in `pycm_param.py` +3. Add a color dictionary to `BENCHMARK_COLOR` in `params.py` - Example : ```"PLRI": {"Negligible": "Red","Poor": "Orange","Fair": "Yellow","Good": "Green","None": "White"}``` 4. If interpretation table is for a class statistic: - Step 2-7 [class statistic](#class-statistic) - - Update `CLASS_BENCHMARK_SCORE_DICT` in `pycm_param.py` + - Update `CLASS_BENCHMARK_SCORE_DICT` in `params.py` 5. If interpretation table is for a overall statistic: - Step 2-6 [overall statistic](#overall-statistic) - - Update `OVERALL_BENCHMARK_SCORE_DICT` in `pycm_param.py` + - Update `OVERALL_BENCHMARK_SCORE_DICT` in `params.py` 6. Add tests to `compare_test.py`, `overall_test.py` and `function_test.py` in `TEST` folder - If you have any verified test add them to `verified_test.py` 7. Run `autopep8.bat`/`autopep8.sh` (*Optional*, need to install latest version of `autopep8` package) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 29a6afa6..27a45a0d 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -68,4 +68,4 @@ jobs: if: matrix.python-version == env.TEST_PYTHON_VERSION && matrix.os == env.TEST_OS - name: cProfile run: | - python -m cProfile -s cumtime pycm/pycm_profile.py + python -m cProfile -s cumtime pycm/profile.py diff --git a/CHANGELOG.md b/CHANGELOG.md index dd27bb8d..1fd0ed08 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,22 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. - Test system modified - `Python 3.12` added to `test.yml` - Warning and error messages updated +- `pycm_util.py` renamed to `utils.py` +- `pycm_test.py` renamed to `basic_test.py` +- `pycm_profile.py` renamed to `profile.py` +- `pycm_param.py` renamed to `params.py` +- `pycm_overall_func.py` renamed to `overall_funcs.py` +- `pycm_output.py` renamed to `output.py` +- `pycm_obj.py` renamed to `cm.py` +- `pycm_multilabel_cm.py` renamed to `multilabel_cm.py` +- `pycm_interpret.py` renamed to `interpret.py` +- `pycm_handler.py` renamed to `handlers.py` +- `pycm_error.py` renamed to `errors.py` +- `pycm_distance.py` renamed to `distance.py` +- `pycm_curve.py` renamed to `curve.py` +- `pycm_compare.py` renamed to `compare.py` +- `pycm_class_func.py` renamed to `class_funcs.py` +- `pycm_ci.py` renamed to `ci.py` ## [4.0] - 2023-06-07 ### Added - `pycmMultiLabelError` class diff --git a/Otherfiles/RELEASE.md b/Otherfiles/RELEASE.md index f94d34de..f919c337 100644 --- a/Otherfiles/RELEASE.md +++ b/Otherfiles/RELEASE.md @@ -9,7 +9,7 @@ 2. `README.md` 3. `Otherfiles/version_check.py` 4. `Otherfiles/meta.yaml` - 5. `pycm/pycm_param.py` + 5. `pycm/params.py` 6. `Document/Document.ipynb` 3. Update `CHANGELOG.md` 1. Add a new header under `Unreleased` section (Example: `## [0.1] - 2022-08-17`) diff --git a/Otherfiles/version_check.py b/Otherfiles/version_check.py index c87ded80..33072794 100644 --- a/Otherfiles/version_check.py +++ b/Otherfiles/version_check.py @@ -36,7 +36,7 @@ os.path.join("Document", "Example1_files", "cm2.html"): HTML_ITEMS, os.path.join("Document", "Example1_files", "cm3.html"): HTML_ITEMS, os.path.join("Otherfiles", "test.html"): HTML_ITEMS, - os.path.join("pycm", "pycm_param.py"): PARAMS_ITEMS, + os.path.join("pycm", "params.py"): PARAMS_ITEMS, os.path.join(".github", "ISSUE_TEMPLATE", "bug_report.yml"): ISSUE_TEMPLATE_ITEMS, } diff --git a/Test/error_test.py b/Test/error_test.py index 7afd2db9..9cf46483 100644 --- a/Test/error_test.py +++ b/Test/error_test.py @@ -8,310 +8,310 @@ >>> cm_2 = ConfusionMatrix(y_actu, 2) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must be provided as a list or a NumPy array. +pycm.errors.pycmVectorError: Input vectors must be provided as a list or a NumPy array. >>> cm_3 = ConfusionMatrix(y_actu, [1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must have the same length. +pycm.errors.pycmVectorError: Input vectors must have the same length. >>> cm_4 = ConfusionMatrix([], []) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must not be empty. +pycm.errors.pycmVectorError: Input vectors must not be empty. >>> cm_5 = ConfusionMatrix([1, 1, 1, ], [1, 1, 1, 1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must have the same length. +pycm.errors.pycmVectorError: Input vectors must have the same length. >>> cm_6 = ConfusionMatrix(matrix={0: {0: 2, 1: 50, 2: 6}, 1: {0: 5, 1: 50, 2: 3}, 2: {0: 1, 1: 7, 2: 50}}) >>> cm_6.position() Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: This option is only available in vector mode. +pycm.errors.pycmVectorError: This option is only available in vector mode. >>> cm = ConfusionMatrix([1, 2, 3, 4], [1, 2, 3, 4], classes=[1, 2, 2, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: `classes` must contain unique labels with no duplicates. +pycm.errors.pycmVectorError: `classes` must contain unique labels with no duplicates. >>> cm3=ConfusionMatrix(matrix={}) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: Invalid input confusion matrix format. +pycm.errors.pycmMatrixError: Invalid input confusion matrix format. >>> cm_4=ConfusionMatrix(matrix={1: {1: 2, "1": 2}, "1": {1: 2, "1": 3}}) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: All input matrix classes must be of the same type. +pycm.errors.pycmMatrixError: All input matrix classes must be of the same type. >>> cm_5=ConfusionMatrix(matrix={1: {1: 2}}) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: The number of classes must be at least 2. +pycm.errors.pycmMatrixError: The number of classes must be at least 2. >>> cm = ConfusionMatrix([1, 2, 3, 4], [1, 2, 3, 4], classes=[1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: The number of classes must be at least 2. +pycm.errors.pycmMatrixError: The number of classes must be at least 2. >>> cm = ConfusionMatrix([1, 1, 1, 1], [1, 2, 1, 1], classes=[]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: The number of classes must be at least 2. +pycm.errors.pycmMatrixError: The number of classes must be at least 2. >>> y_actu = [2, 0, 2, 2, 0, 1, 1, 2, 2, 0, 1, 2] >>> y_pred = [0, 0, 2, 1, 0, 2, 1, 0, 2, 0, 2, 2] >>> cm = ConfusionMatrix(y_actu, y_pred) >>> cm.distance(metric = 2) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: `metric` type must be DistanceType. +pycm.errors.pycmMatrixError: `metric` type must be DistanceType. >>> cm.relabel([1, 2, 3]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: Invalid mapping format. +pycm.errors.pycmMatrixError: Invalid mapping format. >>> cm.relabel({1: "L1", 2: "L2"}) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: Invalid mapping class names. +pycm.errors.pycmMatrixError: Invalid mapping class names. >>> cm.relabel({0: "L1", 1: "L2", 2: "L2"}) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: Invalid mapping class names. +pycm.errors.pycmMatrixError: Invalid mapping class names. >>> cp = Compare([cm, cm]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: Input must be provided as a dictionary. +pycm.errors.pycmCompareError: Input must be provided as a dictionary. >>> cp = Compare({"cm1": cm}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: At least 2 confusion matrices are required for comparison. +pycm.errors.pycmCompareError: At least 2 confusion matrices are required for comparison. >>> cp = Compare({"cm1": cm, "cm2": []}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: Input must be a dictionary containing pycm.ConfusionMatrix objects. +pycm.errors.pycmCompareError: Input must be a dictionary containing pycm.ConfusionMatrix objects. >>> cm2 = ConfusionMatrix(matrix={"Class1": {"Class1": 9, "Class2": 3, "Class3": 0}, "Class2": {"Class1": 3, "Class2": 5, "Class3": 1}, "Class3": {"Class1": 1, "Class2": 1, "Class3": 4}}) >>> cp = Compare({"cm1": cm, "cm2": cm2}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: All ConfusionMatrix objects must have the same domain (same sample size and number of classes). +pycm.errors.pycmCompareError: All ConfusionMatrix objects must have the same domain (same sample size and number of classes). >>> cm = ConfusionMatrix(matrix={1: {1: 9, 2: 3}, 2: {1: 3, 2: 5}}, classes=[1, 2, 3]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: The specified classes are not a subset of the matrix's classes. +pycm.errors.pycmMatrixError: The specified classes are not a subset of the matrix's classes. >>> y_pred = [0, 0, 2, 1, 0, 2, 1, 0, 2, 0, 2, 1] >>> cm3 = ConfusionMatrix(y_actu, y_pred) >>> cp = Compare({"cm1": cm, "cm2": cm3}, by_class=True, class_weight={1: 1, 2: 1}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: `class_weight` must be a dictionary and specified for all classes. +pycm.errors.pycmCompareError: `class_weight` must be a dictionary and specified for all classes. >>> cp = Compare({"cm1": cm, "cm2": cm3}, by_class=True, class_weight=[]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: `class_weight` must be a dictionary and specified for all classes. +pycm.errors.pycmCompareError: `class_weight` must be a dictionary and specified for all classes. >>> cp = Compare({"cm1": cm, "cm2": cm3}, by_class=True, class_benchmark_weight=[]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: `class_benchmark_weight` must be a dictionary and specified for all class benchmarks. +pycm.errors.pycmCompareError: `class_benchmark_weight` must be a dictionary and specified for all class benchmarks. >>> cp = Compare({"cm1": cm, "cm2": cm3}, by_class=True, overall_benchmark_weight=[]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: `overall_benchmark_weight` must be a dictionary and specified for all overall benchmarks. +pycm.errors.pycmCompareError: `overall_benchmark_weight` must be a dictionary and specified for all overall benchmarks. >>> cm1 = ConfusionMatrix([1, 1, 1, 0], [1, 0, 1, 1], metrics_off=True) >>> cm2 = ConfusionMatrix([1, 1, 1, 0], [1, 0, 1, 1], metrics_off=False) >>> cp = Compare({"cm1":cm1, "cm2":cm2}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCompareError: Comparison cannot be performed when `metrics_off=True` in any matrix. +pycm.errors.pycmCompareError: Comparison cannot be performed when `metrics_off=True` in any matrix. >>> cm.CI("MCC") Traceback (most recent call last): ... -pycm.pycm_error.pycmCIError: Confidence interval calculation for this parameter is not supported in this version of pycm. +pycm.errors.pycmCIError: Confidence interval calculation for this parameter is not supported in this version of pycm. Supported parameters are: TPR, TNR, PPV, NPV, ACC, PLR, NLR, FPR, FNR, AUC, PRE, Kappa, Overall ACC >>> cm.CI(2) Traceback (most recent call last): ... -pycm.pycm_error.pycmCIError: Input must be provided as a string. +pycm.errors.pycmCIError: Input must be provided as a string. >>> cm.average("AUCC") Traceback (most recent call last): ... -pycm.pycm_error.pycmAverageError: Invalid parameter! +pycm.errors.pycmAverageError: Invalid parameter! >>> cm.weighted_average("AUCC") Traceback (most recent call last): ... -pycm.pycm_error.pycmAverageError: Invalid parameter! +pycm.errors.pycmAverageError: Invalid parameter! >>> cm.weighted_average("AUC", weight=2) Traceback (most recent call last): ... -pycm.pycm_error.pycmAverageError: `weight` must be a dictionary and specified for all classes. +pycm.errors.pycmAverageError: `weight` must be a dictionary and specified for all classes. >>> cm.weighted_average("AUC", weight={1: 23}) Traceback (most recent call last): ... -pycm.pycm_error.pycmAverageError: `weight` must be a dictionary and specified for all classes. +pycm.errors.pycmAverageError: `weight` must be a dictionary and specified for all classes. >>> cm.combine(1) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: Input must be an instance of pycm.ConfusionMatrix. +pycm.errors.pycmMatrixError: Input must be an instance of pycm.ConfusionMatrix. >>> cm = ConfusionMatrix([1, 0, 2, 0], [1, 1, 2, 1]) >>> cm.brier_score() Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: This option is only available in binary probability mode. +pycm.errors.pycmVectorError: This option is only available in binary probability mode. >>> cm.log_loss() Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: This option is only available in binary probability mode. +pycm.errors.pycmVectorError: This option is only available in binary probability mode. >>> cm = ConfusionMatrix(["ham", "spam", "ham", "ham"], [0.1, 0.4, 0.25, 1], threshold=lambda x : "ham") >>> cm.brier_score() Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Actual vector contains strings; `pos_class` must be explicitly specified. +pycm.errors.pycmVectorError: Actual vector contains strings; `pos_class` must be explicitly specified. >>> cm.log_loss() Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Actual vector contains strings; `pos_class` must be explicitly specified. +pycm.errors.pycmVectorError: Actual vector contains strings; `pos_class` must be explicitly specified. >>> matrix = [[1, 2, 3], [4, 6, 1], [1, 2, 3]] >>> cm = ConfusionMatrix(matrix=matrix, classes=["L1", "L1", "L3", "L2"]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: `classes` must contain unique labels with no duplicates. +pycm.errors.pycmMatrixError: `classes` must contain unique labels with no duplicates. >>> cm = ConfusionMatrix(matrix=matrix, classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: The length of the classes does not match the length of the array. +pycm.errors.pycmMatrixError: The length of the classes does not match the length of the array. >>> crv = Curve([1, 2, 2, 1], {1, 2, 2, 1}, classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: Input vectors must be provided as a list or a NumPy array. +pycm.errors.pycmCurveError: Input vectors must be provided as a list or a NumPy array. >>> crv = Curve({1, 2, 2, 1}, [1, 2, 2, 1], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: Input vectors must be provided as a list or a NumPy array. +pycm.errors.pycmCurveError: Input vectors must be provided as a list or a NumPy array. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9]], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: Input vectors must have the same length. +pycm.errors.pycmCurveError: Input vectors must have the same length. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, 0.9]], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: The sum of the probability values must equal 1. +pycm.errors.pycmCurveError: The sum of the probability values must equal 1. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.1, 0.9]], classes={1, 2}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: `classes` must be provided as a list. +pycm.errors.pycmCurveError: `classes` must be provided as a list. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.1, 0.9]], classes=[1, 2, 3]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: `classes` does not match the actual vector. +pycm.errors.pycmCurveError: `classes` does not match the actual vector. >>> crv = Curve([1, 1, 1, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.1, 0.9]], classes=[1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: The number of classes must be at least 2. +pycm.errors.pycmCurveError: The number of classes must be at least 2. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, "salam"]], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: Probability vector elements must be numeric. +pycm.errors.pycmCurveError: Probability vector elements must be numeric. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, 0.8]], classes=[1, 2], thresholds={1, 2}) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: `thresholds` must be provided as a list or a NumPy array. +pycm.errors.pycmCurveError: `thresholds` must be provided as a list or a NumPy array. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, 0.8]], classes=[1, 2], thresholds=[0.1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: The number of thresholds must be at least 2. +pycm.errors.pycmCurveError: The number of thresholds must be at least 2. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, 0.8]], classes=[1, 2], thresholds=[0.1, "q"]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: `thresholds` must contain only numeric values. +pycm.errors.pycmCurveError: `thresholds` must contain only numeric values. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.9], [0.2, 0.8]], classes=[1, 1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: `classes` must contain unique labels with no duplicates. +pycm.errors.pycmCurveError: `classes` must contain unique labels with no duplicates. >>> crv = Curve([1, 2, 2, 1], [[0.1, 0.9], [0.1, 0.9], [0.1, 0.8, 0.1], [0.2, 0.8]], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: All elements of the probability vector must have the same length and match the number of classes. +pycm.errors.pycmCurveError: All elements of the probability vector must have the same length and match the number of classes. >>> crv = Curve([1, 2, 2, 1], [[1], [1], [1], [1]], classes=[1, 2]) Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: All elements of the probability vector must have the same length and match the number of classes. +pycm.errors.pycmCurveError: All elements of the probability vector must have the same length and match the number of classes. >>> crv = Curve(actual_vector = np.array([1, 1, 2, 2]), probs = np.array([[0.1, 0.9], [0.4, 0.6], [0.35, 0.65], [0.8, 0.2]]), classes=[2, 1]) >>> crv.area(method="trpz") Traceback (most recent call last): ... -pycm.pycm_error.pycmCurveError: The integral method must be either 'trapezoidal' or 'midpoint'. +pycm.errors.pycmCurveError: The integral method must be either 'trapezoidal' or 'midpoint'. >>> cm = ConfusionMatrix(y_actu, y_pred, metrics_off=True) >>> cm.stat() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.sensitivity_index() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.IBA_alpha(0.2) Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.NB() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.CI("Kappa") Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.average("ACC") Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.weighted_average("ACC") Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.weighted_kappa() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.weighted_alpha() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> cm.aickin_alpha() Traceback (most recent call last): ... -pycm.pycm_error.pycmMatrixError: This method cannot be executed when `metrics_off=True`. +pycm.errors.pycmMatrixError: This method cannot be executed when `metrics_off=True`. >>> mlcm = MultiLabelCM([[0, 1], [1, 1]], [[1, 0], [1, 0]]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. +pycm.errors.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. >>> mlcm = MultiLabelCM([{'dog'}, {'cat', 'dog'}], ['cat', {'cat'}]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. +pycm.errors.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. >>> mlcm = MultiLabelCM(['dog', {'cat', 'dog'}], [{'cat'}, {'cat'}]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. +pycm.errors.pycmVectorError: Failed to extract classes from input. Input vectors should be a list of sets with unified types. >>> mlcm = MultiLabelCM([{'dog'}, {'cat', 'dog'}], [{'cat'}, {'cat'}]) >>> mlcm.get_cm_by_class(1) Traceback (most recent call last): ... -pycm.pycm_error.pycmMultiLabelError: The specified class name is not among the confusion matrix's classes. +pycm.errors.pycmMultiLabelError: The specified class name is not among the confusion matrix's classes. >>> mlcm.get_cm_by_sample(2) Traceback (most recent call last): ... -pycm.pycm_error.pycmMultiLabelError: Index is out of range for the given vector. +pycm.errors.pycmMultiLabelError: Index is out of range for the given vector. >>> mlcm = MultiLabelCM([{'dog'}, {'cat', 'dog'}], [{'cat'}, {'cat', 'bird'}], classes=['dog', 'cat']) >>> mlcm.get_cm_by_class('bird') Traceback (most recent call last): ... -pycm.pycm_error.pycmMultiLabelError: The specified class name is not among the confusion matrix's classes. +pycm.errors.pycmMultiLabelError: The specified class name is not among the confusion matrix's classes. >>> mlcm = MultiLabelCM(2, [[1, 0], [1, 0]]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must be provided as a list or a NumPy array. +pycm.errors.pycmVectorError: Input vectors must be provided as a list or a NumPy array. >>> mlcm = MultiLabelCM([{1, 0}, {1, 0}, {1,1}], [{1, 0}, {1, 0}]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must have the same length. +pycm.errors.pycmVectorError: Input vectors must have the same length. >>> mlcm = MultiLabelCM([], []) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: Input vectors must not be empty. +pycm.errors.pycmVectorError: Input vectors must not be empty. >>> mlcm = MultiLabelCM([{1, 0}, {1, 0}], [{1, 0}, {1, 0}], classes=[1,0,1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmVectorError: `classes` must contain unique labels with no duplicates. +pycm.errors.pycmVectorError: `classes` must contain unique labels with no duplicates. """ diff --git a/Test/function_test.py b/Test/function_test.py index 627bf2be..f0d6574f 100644 --- a/Test/function_test.py +++ b/Test/function_test.py @@ -2,7 +2,7 @@ """ >>> from math import isclose >>> from pycm import * ->>> from pycm.pycm_distance import DISTANCE_MAPPER +>>> from pycm.distance import DISTANCE_MAPPER >>> import os >>> import json >>> import numpy as np @@ -186,12 +186,12 @@ ... >>> online_help(4) ... ->>> from pycm.pycm_output import * ->>> from pycm.pycm_overall_func import * ->>> from pycm.pycm_class_func import * ->>> from pycm.pycm_ci import * ->>> from pycm.pycm_interpret import * ->>> from pycm.pycm_util import * +>>> from pycm.output import * +>>> from pycm.overall_funcs import * +>>> from pycm.class_funcs import * +>>> from pycm.ci import * +>>> from pycm.interpret import * +>>> from pycm.utils import * >>> color_check("red") [255, 0, 0] >>> color_check((255,2,2)) diff --git a/Test/overall_test.py b/Test/overall_test.py index 0908733d..c6ba18e1 100644 --- a/Test/overall_test.py +++ b/Test/overall_test.py @@ -1272,7 +1272,7 @@ True >>> cm.imbalance True ->>> from pycm.pycm_param import IMBALANCED_RECOMMEND, MULTICLASS_RECOMMEND, BINARY_RECOMMEND +>>> from pycm.params import IMBALANCED_RECOMMEND, MULTICLASS_RECOMMEND, BINARY_RECOMMEND >>> set(cm.recommended_list) == set(IMBALANCED_RECOMMEND) True >>> cm = ConfusionMatrix(matrix={1: {1: 60, 2: 9, 3: 1, 4: 0, 5: 0, 6: 0}, 2: {1: 23, 2: 48, 3: 0, 4: 2, 5: 2, 6: 1}, 3: {1: 11, 2: 5, 3: 1, 4: 0, 5: 0, 6: 0}, 4: {1: 0, 2: 2, 3: 0, 4: 7, 5: 1, 6: 3}, 5: {1: 2, 2: 1, 3: 0, 4: 0, 5: 4, 6: 2}, 6: {1: 1, 2: 2, 3: 0, 4: 2, 5: 1, 6: 23}}) # Verified Case diff --git a/Test/plot_error_test.py b/Test/plot_error_test.py index a3c66f95..b5f51822 100644 --- a/Test/plot_error_test.py +++ b/Test/plot_error_test.py @@ -18,14 +18,14 @@ >>> ax = cm.plot() Traceback (most recent call last): ... -pycm.pycm_error.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. +pycm.errors.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. >>> ax = cm.plot(plot_lib='seaborn') Traceback (most recent call last): ... -pycm.pycm_error.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. +pycm.errors.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. >>> crv = Curve(actual_vector = np.array([1, 1, 2, 2]), probs = np.array([[0.1, 0.9], [0.4, 0.6], [0.35, 0.65], [0.8, 0.2]]), classes=[2, 1]) >>> ax = crv.plot(classes=[1]) Traceback (most recent call last): ... -pycm.pycm_error.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. +pycm.errors.pycmPlotError: Failed to import matplotlib module. Please install it using: `pip install matplotlib`. """ diff --git a/Test/plot_test.py b/Test/plot_test.py index 4957482b..284c89a6 100644 --- a/Test/plot_test.py +++ b/Test/plot_test.py @@ -207,11 +207,11 @@ >>> crv.plot(colors=['c']) Traceback (most recent call last): ... -pycm.pycm_error.pycmPlotError: The number of colors does not match the number of classes. +pycm.errors.pycmPlotError: The number of colors does not match the number of classes. >>> crv.plot(markers=['*']) Traceback (most recent call last): ... -pycm.pycm_error.pycmPlotError: The number of markers does not match the number of classes. +pycm.errors.pycmPlotError: The number of markers does not match the number of classes. >>> ax = crv.plot(classes=[1]) >>> ax.get_figure()._suptitle.get_text() 'FPR per TPR' diff --git a/Test/verified_test.py b/Test/verified_test.py index 990660d0..c04e4c75 100644 --- a/Test/verified_test.py +++ b/Test/verified_test.py @@ -3,8 +3,8 @@ >>> import numpy as np >>> from math import isclose >>> from pycm import * ->>> from pycm.pycm_overall_func import NIR_calc ->>> from pycm.pycm_ci import AUC_SE_calc, CI_calc +>>> from pycm.overall_funcs import NIR_calc +>>> from pycm.ci import AUC_SE_calc, CI_calc >>> ABS_TOL = 1e-12 >>> REL_TOL = 0 >>> assert isclose(NIR_calc({'Class2': 804, 'Class1': 196}, 1000), 0.804, abs_tol=ABS_TOL, rel_tol=REL_TOL) # Verified Case - (Caret package) diff --git a/Test/warning_test.py b/Test/warning_test.py index 619bd164..3f38f041 100644 --- a/Test/warning_test.py +++ b/Test/warning_test.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """ >>> from pycm import * ->>> from pycm.pycm_util import deprecated +>>> from pycm.utils import deprecated >>> from pytest import warns >>> large_cm = ConfusionMatrix(list(range(10)) + [2, 3, 5], list(range(10)) + [1, 7, 2]) >>> with warns(RuntimeWarning, match='Confusion matrix is high-dimensional and may not display properly. Consider using the `sparse` flag in printing functions, or save it as a CSV file for better visualization.'): diff --git a/pycm/__init__.py b/pycm/__init__.py index 71e855d0..4d1bcadb 100644 --- a/pycm/__init__.py +++ b/pycm/__init__.py @@ -1,11 +1,11 @@ # -*- coding: utf-8 -*- """PyCM modules.""" -from .pycm_param import PYCM_VERSION, OVERALL_BENCHMARK_LIST, CLASS_BENCHMARK_LIST -from .pycm_error import * -from .pycm_output import pycm_help, online_help -from .pycm_distance import DistanceType -from .pycm_obj import ConfusionMatrix -from .pycm_compare import Compare -from .pycm_multilabel_cm import MultiLabelCM -from .pycm_curve import Curve, ROCCurve, PRCurve +from .params import PYCM_VERSION, OVERALL_BENCHMARK_LIST, CLASS_BENCHMARK_LIST +from .errors import * +from .output import pycm_help, online_help +from .distance import DistanceType +from .cm import ConfusionMatrix +from .compare import Compare +from .multilabel_cm import MultiLabelCM +from .curve import Curve, ROCCurve, PRCurve __version__ = PYCM_VERSION diff --git a/pycm/__main__.py b/pycm/__main__.py index 51b32ae9..1ab1f492 100644 --- a/pycm/__main__.py +++ b/pycm/__main__.py @@ -3,8 +3,8 @@ import doctest import sys -from .pycm_obj import * -from .pycm_output import * +from .cm import * +from .output import * from art import tprint @@ -13,7 +13,7 @@ if len(args) > 1: if args[1].upper() == "TEST": error_flag = doctest.testfile( - "pycm_test.py", + "basic_test.py", optionflags=doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS | doctest.IGNORE_EXCEPTION_DETAIL, verbose=False)[0] diff --git a/pycm/pycm_test.py b/pycm/basic_test.py similarity index 100% rename from pycm/pycm_test.py rename to pycm/basic_test.py diff --git a/pycm/pycm_ci.py b/pycm/ci.py similarity index 100% rename from pycm/pycm_ci.py rename to pycm/ci.py diff --git a/pycm/pycm_class_func.py b/pycm/class_funcs.py similarity index 99% rename from pycm/pycm_class_func.py rename to pycm/class_funcs.py index e1f53449..d7b5fa08 100644 --- a/pycm/pycm_class_func.py +++ b/pycm/class_funcs.py @@ -2,9 +2,9 @@ """Class statistics functions.""" from __future__ import division import math -from .pycm_util import normal_quantile -from .pycm_interpret import * -from .pycm_param import CLASS_PARAMS +from .utils import normal_quantile +from .interpret import * +from .params import CLASS_PARAMS def sensitivity_index_calc(TPR, FPR): diff --git a/pycm/pycm_obj.py b/pycm/cm.py similarity index 98% rename from pycm/pycm_obj.py rename to pycm/cm.py index b2f0d1eb..a82194e4 100644 --- a/pycm/pycm_obj.py +++ b/pycm/cm.py @@ -1,17 +1,17 @@ # -*- coding: utf-8 -*- """ConfusionMatrix module.""" from __future__ import division -from .pycm_error import pycmVectorError, pycmMatrixError, pycmCIError, pycmAverageError, pycmPlotError -from .pycm_handler import __class_stat_init__, __overall_stat_init__ -from .pycm_handler import __obj_assign_handler__, __obj_file_handler__, __obj_matrix_handler__, __obj_vector_handler__, __obj_array_handler__ -from .pycm_handler import __imbalancement_handler__ -from .pycm_class_func import F_calc, IBA_calc, TI_calc, NB_calc, sensitivity_index_calc -from .pycm_overall_func import weighted_kappa_calc, weighted_alpha_calc, alpha2_calc, brier_score_calc, log_loss_calc -from .pycm_distance import DistanceType, DISTANCE_MAPPER -from .pycm_output import * -from .pycm_util import * -from .pycm_param import * -from .pycm_ci import __CI_overall_handler__, __CI_class_handler__ +from .errors import pycmVectorError, pycmMatrixError, pycmCIError, pycmAverageError, pycmPlotError +from .handlers import __class_stat_init__, __overall_stat_init__ +from .handlers import __obj_assign_handler__, __obj_file_handler__, __obj_matrix_handler__, __obj_vector_handler__, __obj_array_handler__ +from .handlers import __imbalancement_handler__ +from .class_funcs import F_calc, IBA_calc, TI_calc, NB_calc, sensitivity_index_calc +from .overall_funcs import weighted_kappa_calc, weighted_alpha_calc, alpha2_calc, brier_score_calc, log_loss_calc +from .distance import DistanceType, DISTANCE_MAPPER +from .output import * +from .utils import * +from .params import * +from .ci import __CI_overall_handler__, __CI_class_handler__ import os import json import numpy diff --git a/pycm/pycm_compare.py b/pycm/compare.py similarity index 98% rename from pycm/pycm_compare.py rename to pycm/compare.py index ba9655ea..2393d57f 100644 --- a/pycm/pycm_compare.py +++ b/pycm/compare.py @@ -1,11 +1,11 @@ # -*- coding: utf-8 -*- """Compare module.""" from __future__ import division -from .pycm_error import pycmCompareError -from .pycm_output import * -from .pycm_util import * -from .pycm_param import * -from .pycm_obj import ConfusionMatrix +from .errors import pycmCompareError +from .output import * +from .utils import * +from .params import * +from .cm import ConfusionMatrix import os import numpy from warnings import warn diff --git a/pycm/pycm_curve.py b/pycm/curve.py similarity index 98% rename from pycm/pycm_curve.py rename to pycm/curve.py index 90689a22..e541b554 100644 --- a/pycm/pycm_curve.py +++ b/pycm/curve.py @@ -1,10 +1,10 @@ # -*- coding: utf-8 -*- """Curve module.""" from __future__ import division -from .pycm_error import pycmCurveError, pycmPlotError -from .pycm_util import threshold_func, thresholds_calc, isfloat -from .pycm_param import * -from .pycm_obj import ConfusionMatrix +from .errors import pycmCurveError, pycmPlotError +from .utils import threshold_func, thresholds_calc, isfloat +from .params import * +from .cm import ConfusionMatrix from warnings import warn import numpy diff --git a/pycm/pycm_distance.py b/pycm/distance.py similarity index 100% rename from pycm/pycm_distance.py rename to pycm/distance.py diff --git a/pycm/pycm_error.py b/pycm/errors.py similarity index 100% rename from pycm/pycm_error.py rename to pycm/errors.py diff --git a/pycm/pycm_handler.py b/pycm/handlers.py similarity index 97% rename from pycm/pycm_handler.py rename to pycm/handlers.py index ffbcd167..d525bb40 100644 --- a/pycm/pycm_handler.py +++ b/pycm/handlers.py @@ -1,11 +1,11 @@ # -*- coding: utf-8 -*- """ConfusionMatrix handlers.""" from __future__ import division -from .pycm_class_func import class_statistics -from .pycm_error import pycmVectorError, pycmMatrixError -from .pycm_overall_func import overall_statistics -from .pycm_util import * -from .pycm_param import * +from .class_funcs import class_statistics +from .errors import pycmVectorError, pycmMatrixError +from .overall_funcs import overall_statistics +from .utils import * +from .params import * import json import types import numpy diff --git a/pycm/pycm_interpret.py b/pycm/interpret.py similarity index 100% rename from pycm/pycm_interpret.py rename to pycm/interpret.py diff --git a/pycm/pycm_multilabel_cm.py b/pycm/multilabel_cm.py similarity index 98% rename from pycm/pycm_multilabel_cm.py rename to pycm/multilabel_cm.py index 461414ea..8626d1c8 100644 --- a/pycm/pycm_multilabel_cm.py +++ b/pycm/multilabel_cm.py @@ -1,9 +1,9 @@ # -*- coding: utf-8 -*- """MultiLabelCM module.""" from __future__ import division -from .pycm_error import pycmVectorError, pycmMultiLabelError -from .pycm_param import * -from .pycm_obj import ConfusionMatrix +from .errors import pycmVectorError, pycmMultiLabelError +from .params import * +from .cm import ConfusionMatrix import numpy diff --git a/pycm/pycm_output.py b/pycm/output.py similarity index 99% rename from pycm/pycm_output.py rename to pycm/output.py index 274f6e2e..ed0e0285 100644 --- a/pycm/pycm_output.py +++ b/pycm/output.py @@ -2,8 +2,8 @@ """Outputs functions.""" from __future__ import division from functools import partial -from .pycm_param import * -from .pycm_util import rounder, sort_char_num +from .params import * +from .utils import rounder, sort_char_num import webbrowser diff --git a/pycm/pycm_overall_func.py b/pycm/overall_funcs.py similarity index 99% rename from pycm/pycm_overall_func.py rename to pycm/overall_funcs.py index 5b4724d4..618fe029 100644 --- a/pycm/pycm_overall_func.py +++ b/pycm/overall_funcs.py @@ -4,9 +4,9 @@ import math import operator as op from functools import reduce -from .pycm_interpret import * -from .pycm_ci import kappa_SE_calc, CI_calc, SE_calc -from .pycm_util import complement +from .interpret import * +from .ci import kappa_SE_calc, CI_calc, SE_calc +from .utils import complement def log_loss_calc( diff --git a/pycm/pycm_param.py b/pycm/params.py similarity index 100% rename from pycm/pycm_param.py rename to pycm/params.py diff --git a/pycm/pycm_profile.py b/pycm/profile.py similarity index 100% rename from pycm/pycm_profile.py rename to pycm/profile.py diff --git a/pycm/pycm_util.py b/pycm/utils.py similarity index 99% rename from pycm/pycm_util.py rename to pycm/utils.py index c46ae290..a54a6115 100644 --- a/pycm/pycm_util.py +++ b/pycm/utils.py @@ -5,8 +5,8 @@ import math import numpy import re -from .pycm_param import * -from .pycm_error import pycmMatrixError +from .params import * +from .errors import pycmMatrixError from warnings import warn from functools import wraps