perf(python): upgrade pyarrow version to 15.0.0 (#2056)

<!--
**Thanks for contributing to Fury.**

**If this is your first time opening a PR on fury, you can refer to
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).**

Contribution Checklist

- The **Apache Fury (incubating)** community has restrictions on the
naming of pr titles. You can also find instructions in
[CONTRIBUTING.md](https://github.com/apache/fury/blob/main/CONTRIBUTING.md).

- Fury has a strong focus on performance. If the PR you submit will have
an impact on performance, please benchmark it first and provide the
benchmark result here.
-->

## What does this PR do?

 pip install -v -e .
Using pip 22.0.4 from
D:\code\PycharmProjects\project\venv\lib\site-packages\pip (python 3.9)
Obtaining file:///D:/code/fury/fury/python
  Running command python setup.py egg_info

  A module that was compiled using NumPy 1.x cannot be run in
  NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
  versions of NumPy, modules must be compiled with NumPy 2.0.
  Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

  If you are a user of the module, the easiest solution will be to
  downgrade to 'numpy<2' or try to upgrade the affected module.
  We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\code\fury\fury\python\setup.py", line 32, in <module>
      import pyarrow as pa
File
"D:\code\PycharmProjects\project\venv\lib\site-packages\pyarrow\__init__.py",
line 65, in <module>
      import pyarrow.lib as _lib
  AttributeError: _ARRAY_API not found
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "D:\code\fury\fury\python\setup.py", line 32, in <module>
      import pyarrow as pa
File
"D:\code\PycharmProjects\project\venv\lib\site-packages\pyarrow\__init__.py",
line 65, in <module>
      import pyarrow.lib as _lib
    File "pyarrow\lib.pyx", line 36, in init pyarrow.lib
  ImportError: numpy.core.multiarray failed to import
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a
problem with pip.

<!-- Describe the purpose of this PR. -->

## Related issues

<!--
Is there any related issue? Please attach here.

- #xxxx0
- #xxxx1
- #xxxx2
-->

## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?

## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->

ci/deploy.sh

@@ -164,8 +164,8 @@ install_pyarrow() {
     pip install pyarrow==18.0.0
     pip install numpy
   else
-    pip install pyarrow==14.0.0
-    pip install "numpy<2.0.0"
+    pip install pyarrow==15.0.0
+    # Automatically install numpy
   fi
 }
 

ci/run_ci.py

@@ -27,9 +27,7 @@
 
 BAZEL_VERSION = "6.3.2"
 
-PYARROW_VERSION = "14.0.0"
-
-NUMPY_VERSION = "1.26.4"
+PYARROW_VERSION = "15.0.0"
 
 PROJECT_ROOT_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../")
 
@@ -120,8 +118,8 @@ def _run_js():
 
 def _install_cpp_deps():
     _exec_cmd(f"pip install pyarrow=={PYARROW_VERSION}")
+    # Automatically install numpy
     _exec_cmd("pip install psutil")
-    _exec_cmd(f"pip install numpy=={NUMPY_VERSION}")
     _install_bazel()
 
 

docs/guide/DEVELOPMENT.md

@@ -24,7 +24,13 @@ mvn clean compile -DskipTests
 
 ```bash
 cd python
-pip install pyarrow==14.0.0 Cython wheel numpy pytest
+# Uninstall numpy first so that when we install pyarrow, it will install the correct numpy version automatically.
+# For Python versions less than 3.13, numpy 2 is not currently supported.
+pip uninstall -y numpy
+# Install necessary environment for Python < 3.13.
+pip install pyarrow==15.0.0 Cython wheel pytest
+# For Python 3.13, pyarrow 18.0.0 is available and requires numpy version greater than 2.
+# pip install pyarrow==18.0.0 Cython wheel pytest
 pip install -v -e .
 ```
 
@@ -37,14 +43,14 @@ pip install -v -e .
 Build fury row format：
 
 ```bash
-pip install pyarrow==14.0.0
+pip install pyarrow==15.0.0
 bazel build //cpp/fury/row:fury_row_format
 ```
 
 Build fury row format encoder:
 
 ```bash
-pip install pyarrow==14.0.0
+pip install pyarrow==15.0.0
 bazel build //cpp/fury/encoder:fury_encoder
 ```
 

java/fury-format/pom.xml

@@ -49,7 +49,7 @@
   <properties>
     <maven.compiler.source>8</maven.compiler.source>
     <maven.compiler.target>8</maven.compiler.target>
-    <arrow.version>14.0.0</arrow.version>
+    <arrow.version>15.0.0</arrow.version>
     <jackson-bom.version>2.16.0</jackson-bom.version>
     <fury.java.rootdir>${basedir}/..</fury.java.rootdir>
   </properties>

python/README.md

@@ -6,7 +6,13 @@ Fury is a blazingly-fast multi-language serialization framework powered by just-
 
 ```bash
 cd python
-pip install pyarrow==14.0.0 Cython wheel numpy pytest
+# Uninstall numpy first so that when we install pyarrow, it will install the correct numpy version automatically.
+# For Python versions less than 3.13, numpy 2 is not currently supported.
+pip uninstall -y numpy
+# Install necessary environment for Python < 3.13.
+pip install pyarrow==15.0.0 Cython wheel pytest
+# For Python 3.13, pyarrow 18.0.0 is available and requires numpy version greater than 2.
+# pip install pyarrow==18.0.0 Cython wheel pytest
 pip install -v -e .
 ```
 

python/setup.py

@@ -37,7 +37,7 @@
 except FileExistsError:
     pass
 
-pyarrow_version = "14.0.0" if sys.version_info.minor < 13 else "18.0.0"
+pyarrow_version = "15.0.0" if sys.version_info.minor < 13 else "18.0.0"
 # Check if we're running 64-bit Python
 if not sys.maxsize > 2**32:
     raise RuntimeError("Not supported on 32-bit")