Merge pull request #26 from dcamarmas/master

2.2.1: Minor  improvements

README.md

@@ -1,9 +1,9 @@
-# XPN 2.2
+# XPN 2.3
 
 *Expand Ad-Hoc Parallel File System*
 
 [![License: GPL3](https://img.shields.io/badge/License-GPL3-blue.svg)](https://opensource.org/licenses/GPL-3.0)
-![version](https://img.shields.io/badge/version-2.2-blue)
+![version](https://img.shields.io/badge/version-2.3-blue)
 [![Codacy Badge](https://app.codacy.com/project/badge/Grade/ca0c40db97f64698a2db9992cafdd4ab)](https://www.codacy.com/gh/xpn-arcos/xpn/dashboard?utm_source=github.com&utm_medium=referral&utm_content=xpn-arcos/xpn&utm_campaign=Badge_Grade)
 
 * *Homepage*: <https://xpn-arcos.github.io/xpn-arcos.github.io/>
@@ -15,8 +15,22 @@
 
   The Expand Parallel File System (a.k.a. XPN) can be installed on a cluster with local storage per-node (HDD, SSD or RAM Drive) and a shared home directory.
 
-  The general steps to deploy XPN with Spack are:
+  The general steps to deploy XPN are:
+  ```mermaid
+  flowchart LR
+    A[Start] --> B{Spack?}
+    B -- Yes --> C[1.1 With Spack]
+    B -- No ---> D[install prerequisites]
+    subgraph ide1 [1.2 With autotools]
+    D --> E[download source of XPN and mxml]
+    E --> F[build XPN]
+    F --> G[End]
+    end
+  ```
+
+### 1.1. With Spack
 
+  To deploy XPN with Spack:
   ```
   git clone https://github.com/xpn-arcos/xpn.git
   spack repo add xpn/scripts/spack
@@ -25,39 +39,51 @@
   spack load xpn
   ```
 
-  The traditional steps (without Spack) are:
-
-```mermaid
-stateDiagram-v2
-    direction LR
-    state "1. Install prerequisites"                    as step1
-    state "2. Download the source code of XPN and mxml" as step2
-    state "3. Build Expand"                             as step3
-    [*] --> step1
-    step1 --> step2
-    step2 --> step3
-    step3 --> [*]
-```
+### 1.2. With autotools (configure, make, make install)
 
-### 1.1. Installing prerequisites
+### 1.2.1. Installing prerequisites
 
 XPN needs the typical C development tools (gcc, make, autotools) and a MPI implementation installed.
 
-Some common use cases:
-* If you are administrator of your Linux machine then you need to execute:
-  ```
-  sudo apt-get install -y autoconf automake gcc g++ make libtool libmpich-dev  
-  ```
+* To install the C development tools:
+  * If you are an user of a cluster with modules then you might try to load the compiler module, for example:
+    ```bash
+    module load gcc
+    ```
+    Where "gcc" is the compiler module.
+
+  * If you are administrator of your Linux machine then you need to execute:
+    ```
+    sudo apt-get install -y autoconf automake gcc g++ make libtool build-essential
+    ```
 
-* If you are an user of a cluster with already installed software then you might try the following:
-  ```bash
-  module load icc
-  module load "impi/2017.4"
-  ```
-  Where "icc" is the compiler module and "impi/2017.4" is the MPI module.
-
-
-### 1.2. Download the source code of XPN
+* To install the MPICH implementation of MPI:
+  * If you are an user of a cluster with already installed software then you might try to load the MPI module, for example:
+    ```bash
+    module load "impi/2017.4"
+    ```
+    Where "impi/2017.4" is the MPI module.
+
+  * From source code and with Infiniband (Omni-Path) support:
+    ```
+    wget https://www.mpich.org/static/downloads/4.1.1/mpich-4.1.1.tar.gz
+    tar zxf mpich-4.1.1
+    cd      mpich-4.1.1
+    ./configure --prefix=<path where MPICH is going to be installed> \
+                --enable-threads=multiple \
+                --enable-romio \
+                --with-device=ch4:ofi:psm2 --with-libfabric=<path where your libfabric is installed>
+    make
+    make install
+    ```
+
+  * If you are administrator of your Linux machine then you need to execute:
+    ```
+    sudo apt-get install -y libmpich-dev  
+    ```
+
+
+### 1.2.2. Download the source code of XPN
 
 You need to download the source code of [XPN](https://xpn-arcos.github.io/arcos-xpn.github.io/) and [minixml](http://www.minixml.org).
 
@@ -72,7 +98,7 @@ git clone https://github.com/xpn-arcos/xpn.git
 You must do both 'git clone' requests in the same directory (e.g.: $HOME/src).
 
 
-### 1.3. Building XPN
+### 1.2.3. Building XPN
 
 To build Expand you need to execute:
 ```
@@ -94,7 +120,7 @@ Where:
 * Installation directory will be $HOME/xpn_bin
 
 
-## 4. Executing XPN
+## 2. Executing XPN
 
 First, you need to get familiar with 4 special files:
 * ```<hostfile>``` for MPI, it is a text file with the list of host names (one per line) where XPN servers and XPN client is going to be executed.
@@ -110,7 +136,7 @@ Then, you need to get familiar with 5 special environment variables for XPN clie
 * ```XPN_LOCALITY``` with value 0 for without locality and value 1 for with locality (optional, default: 0).
 
 
-### 4.1 Ad-Hoc Expand (based on MPI)
+### 2.1 Ad-Hoc Expand (based on MPI)
 The typical executions has 3 main steps:
 - First, launch the Expand MPI server (xpn_mpi_server):
 

configure

@@ -5057,6 +5057,16 @@ fi
 
 
 
+#
+# Check __open2 support
+#
+MPICC_INTEL=$(mpicc --version | grep "Intel" | wc -l)
+if test "x$MPICC_INTEL" == "x1"; then
+
+$as_echo "#define HAVE_ICC 1" >>confdefs.h
+
+fi
+
 #
 # Checks for libraries.
 #
@@ -6091,7 +6101,7 @@ LFLAGS+="-L../../../mxml -L../../../xpn/lib "
 
 LIBLINK=$LIBS
 LIBLINK+=" -lmxml"
-LIBS=$LFLAGS" "$LIBS
+LIBS=$LFLAGS" "$LIBS" -lmxml"
 
 if test "$target_cpu" = "AARCH64" -o "$target_cpu" = "aarch64"
 then
@@ -6158,16 +6168,14 @@ $as_echo "\"$target_os\"" >&6; };
 INCLUDEFLAGS="\
 -I\$(top_srcdir)/include \
 -I\$(top_srcdir)/include/base \
+-I\$(top_srcdir)/include/bypass \
 -I\$(top_srcdir)/include/xpn_client/ \
 -I\$(top_srcdir)/include/xpn_client/xpn \
 -I\$(top_srcdir)/include/xpn_client/xpn/xpn_simple \
 -I\$(top_srcdir)/include/xpn_client/nfi \
--I\$(top_srcdir)/include/xpn_client/nfi/nfi_local \
--I\$(top_srcdir)/include/xpn_client/xpni/ \
--I\$(top_srcdir)/include/xpn_client/xpni/common \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_null \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_ft \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_log"
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_mpi_server \
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_tcp_server \
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_local"
 
 
 ### BEGIN OF NFS BLOCK. Do not remove this line. ###

configure.ac

@@ -33,6 +33,14 @@ AC_PROG_MAKE_SET
 AC_PROG_CC([gcc mpicc icc mpifcc])
 AM_PROG_CC_C_O
 
+#
+# Check __open2 support
+#
+MPICC_INTEL=$(mpicc --version | grep "Intel" | wc -l)
+if test "x$MPICC_INTEL" == "x1"; then
+   AC_DEFINE([HAVE_ICC], [1], [we have icc as mpicc compiler])
+fi
+
 #
 # Checks for libraries.
 #
@@ -82,7 +90,7 @@ LFLAGS+="-L../../../mxml -L../../../xpn/lib "
 
 LIBLINK=$LIBS
 LIBLINK+=" -lmxml"
-LIBS=$LFLAGS" "$LIBS
+LIBS=$LFLAGS" "$LIBS" -lmxml"
 
 if test "$target_cpu" = "AARCH64" -o "$target_cpu" = "aarch64"
 then
@@ -146,16 +154,14 @@ AC_MSG_RESULT("$target_os");
 INCLUDEFLAGS="\
 -I\$(top_srcdir)/include \
 -I\$(top_srcdir)/include/base \
+-I\$(top_srcdir)/include/bypass \
 -I\$(top_srcdir)/include/xpn_client/ \
 -I\$(top_srcdir)/include/xpn_client/xpn \
 -I\$(top_srcdir)/include/xpn_client/xpn/xpn_simple \
 -I\$(top_srcdir)/include/xpn_client/nfi \
--I\$(top_srcdir)/include/xpn_client/nfi/nfi_local \
--I\$(top_srcdir)/include/xpn_client/xpni/ \
--I\$(top_srcdir)/include/xpn_client/xpni/common \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_null \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_ft \
--I\$(top_srcdir)/include/xpn_client/xpni/xpni_log"
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_mpi_server \
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_tcp_server \
+-I\$(top_srcdir)/include/xpn_client/nfi/nfi_local"
 
 
 ### BEGIN OF NFS BLOCK. Do not remove this line. ###

include/base/base_lib.h

@@ -36,6 +36,7 @@
    #include "dtable.h"     
 
    #include "string_misc.h"  
+   #include "path_misc.h"  
    #include "urlstr.h" 
 
 #endif

include/base/ns_tcp.h

@@ -37,7 +37,7 @@
       #endif
 
       #ifndef TCP_SERVER_FILE_DEFAULT
-      #define TCP_SERVER_FILE_DEFAULT "/etc/xpn/tcp_server.dns"
+      #define TCP_SERVER_FILE_DEFAULT "/tmp/tcp_server.dns"
       #endif
 
       #ifndef MAX_TCP_SERVER_NODES
@@ -53,10 +53,10 @@
        *  API
        */
 
-      char * ns_tcp_get_hostname ( void ) ;
-      int    ns_tcp_publish      ( char * dns_file, char * param_srv_name, char * host_name, char * port_name ) ;
-      int    ns_tcp_unpublish    ( char * dns_file, char * serv_name ) ;
-      int    ns_tcp_lookup       ( char * param_srv_name, char * host_name, char * port_name ) ;
+      char *      ns_tcp_get_hostname     ( void ) ;
+      int         ns_tcp_publish          ( char * dns_file, char * param_srv_name, char * host_name, char * port_name ) ;
+      int         ns_tcp_unpublish        ( char * dns_file, char * serv_name ) ;
+      int         ns_tcp_lookup           (                  char * param_srv_name, char * host_name, char * port_name ) ;
 
 
 #endif

include/base/path_misc.h

@@ -0,0 +1,61 @@
+
+  /*
+   *  Copyright 2000-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos, Luis Miguel Sanchez Garcia, Borja Bergua Guerra
+   *
+   *  This file is part of Expand.
+   *
+   *  Expand is free software: you can redistribute it and/or modify
+   *  it under the terms of the GNU Lesser General Public License as published by
+   *  the Free Software Foundation, either version 3 of the License, or
+   *  (at your option) any later version.
+   *
+   *  Expand is distributed in the hope that it will be useful,
+   *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+   *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   *  GNU Lesser General Public License for more details.
+   *
+   *  You should have received a copy of the GNU Lesser General Public License
+   *  along with Expand.  If not, see <http://www.gnu.org/licenses/>.
+   *
+   */
+
+
+ #ifndef _PATH_MISC_H_
+ #define _PATH_MISC_H_
+
+   /* ... Include / Inclusion ........................................... */
+
+      #include "all_system.h"
+      #include "base/string_misc.h"
+
+
+ #ifdef  __cplusplus
+    extern "C" {
+ #endif
+
+
+   /* ... Functions / Funciones ......................................... */
+
+      int hash (char *file,int nServ) ;
+
+      int getFirstDir   ( char *dir, char *path) ;
+      int getSizeFactor ( char *name ) ;
+
+      /* get the last name of a path and erase the file name */
+      int getNameFile(char *file, char *dir);
+
+      /* get the first name of the path and erase the part name */
+      int getNamePart(char *part, char *dir);
+
+      /* erase the initial dir used in the url */
+      int getDirWithURL(char *url, char *dir);
+
+
+  /* .................................................................... */
+
+#ifdef  __cplusplus
+    }
+ #endif
+
+ #endif /* _PATH_MISC_H */
+

include/base/string_misc.h

@@ -109,34 +109,6 @@
       ) ;
 
 
-      // TODO: Unknown for string... filesystem functions???
-      int hash (char *file,int nServ) ;
-
-      // TODO: move into filesystem.h ???
-      int getDir        ( char *arg1 ) ;
-      int multiline     ( char *text ) ;
-      int getFirstDir   ( char *dir, char *path) ;
-      int getSizeFactor ( char *name ) ;
-
-      // TODO: move into urlstr.h ????
-      int ParseURL(   char *url,  
-      		      char *protocol, 
-		      char *login, 
-		      char *passwd, 
-		      char *server,  
-		      char *port,  
-		      char *dir);
-
-      /* get the last name of a path and erase the file name */
-      int getNameFile(char *file, char *dir);
-
-      /* get the first name of the path and erase the part name */
-      int getNamePart(char *part, char *dir);
-
-      /* erase the initial dir used in the url */
-      int getDirWithURL(char *url, char *dir);
-
-
   /* .................................................................... */
 
 #ifdef  __cplusplus