Release/515 (#697)

* optimized EMR imports for less chance of import errors

* bump versions

* update release notes

* drop deprecated setup.py

* Endpoint tests (#637)

* example databricks serve notebooks and cluster creation

* updated healthcecks

* databricks serve + john snow labs endpoints module

* `clean_cluster` and `write_db_credentials` parameters for db cluster creation

* databricks serve + john snow labs endpoints docs + typo fixes

* databricks serve + john snow labs endpoints docs + typo fixes

* updated tests

* updated tests

* fix block_until_deployed bug

* fix block_until_deployed bug

* support for GPU and nlu.predict params in endpoints

* bump versions

* Docs update

* update notebooks

* endpoint test job generator

* improved list_db_runtime_versions

* multi cluster testing endpoints

* Support for submitting notebook to databricks and various utils & refactor to databricks utils

* db test refactor into  db_info/endpoint/hdfs/submit tests

* get_or_create_test_cluster() and varous more db testing utils, pytest.ini and non-verbose parameterization of tests

* add docs for notebook execution

* bump enterprise nlp version

* Update EMR initialization script (#661)

* update release notes

* update version

* Feature/513/parameterized db submit (#684)

* support for parameterization of NB sumit runs in run_in_databricks()

* tests for parameterization of NB sumit runs in run_in_databricks()

* tests for parameterization of NB sumit runs in run_in_databricks()

* Docs & example notebook for parameterized run_in_databricks

* Feature/513/list of additional db installs (#689)

* support for parameterization of NB sumit runs in run_in_databricks()

* tests for parameterization of NB sumit runs in run_in_databricks()

* tests for parameterization of NB sumit runs in run_in_databricks()

* Docs & example notebook for parameterized run_in_databricks

* Docs & example notebook for parameterized run_in_databricks

* updated noteoboks

* support for extra_pip_installs and pandas==1.5.3 fix on new clusters

* refactor db cluster creation to install_to_databricks and add deprecation warning to nlp.install() for db clusters

* db install  tests with custom libs

* Fix update existing databricks cluster (#680)

* update release notes

* bump versions

* removed pandas >= 2 htofix

* remove debug print

* bump version

---------

Co-authored-by: Kshitiz <[email protected]>

docs/en/jsl/jsl_release_notes.md

@@ -16,6 +16,24 @@ sidebar:
 See [Github Releases](https://github.com/JohnSnowLabs/johnsnowlabs/releases) for detailed information on Release History and Featuresasdas
 
 
+## 5.1.5
+Release date: 9-10-2023
+
+The John Snow Labs 5.1.5 Library released with the following pre-installed and recommended dependencies
+
+
+| Library                                                                         | Version |
+|---------------------------------------------------------------------------------|---------|
+| [Visual NLP](https://nlp.johnsnowlabs.com/docs/en/spark_ocr_versions/ocr_release_notes)        | `5.0.1` |
+| [Enterprise NLP](https://nlp.johnsnowlabs.com/docs/en/licensed_annotators)  | `5.1.1` |
+| [Finance NLP](https://nlp.johnsnowlabs.com/docs/en/financial_release_notes) | `1.X.X` |
+| [Legal NLP](https://nlp.johnsnowlabs.com/docs/en/legal_release_notes)       | `1.X.X` |
+| [NLU](https://github.com/JohnSnowLabs/nlu/releases)                             | `5.0.3` |
+| [Spark-NLP-Display](https://sparknlp.org/docs/en/display)           | `4.4`   |
+| [Spark-NLP](https://github.com/JohnSnowLabs/spark-nlp/releases/)                | `5.1.1` |
+| [Pyspark](https://spark.apache.org/docs/latest/api/python/)                     | `3.1.2` |
+
+
 
 
 ## 5.1.4

johnsnowlabs/auto_install/databricks/install_utils.py

@@ -311,7 +311,6 @@ def uninstall_old_libraries(
                         if fil in path.replace("-", "_"):
                             uninstalls.append({typ: path})
     if uninstalls:
-        print("UNINSTALL ULD STUFF?", uninstalls)
         db.managed_library.uninstall_libraries(
             cluster_id=cluster_id, libraries=uninstalls
         )

johnsnowlabs/settings.py

@@ -12,12 +12,16 @@
 
 
 
-raw_version_jsl_lib = "5.1.4"
+raw_version_jsl_lib = "5.1.5"
+
+
 
 
 raw_version_nlp = "5.1.1"
 
-raw_version_nlu = "5.0.2"
+
+raw_version_nlu = "5.0.3"
+
 
 
 raw_version_pyspark = "3.1.2"