diff --git a/README.md b/README.md index 83b44274..5d9086c1 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ -# XPN 2.2 +# XPN 2.3 *Expand Ad-Hoc Parallel File System* [![License: GPL3](https://img.shields.io/badge/License-GPL3-blue.svg)](https://opensource.org/licenses/GPL-3.0) -![version](https://img.shields.io/badge/version-2.2-blue) +![version](https://img.shields.io/badge/version-2.3-blue) [![Codacy Badge](https://app.codacy.com/project/badge/Grade/ca0c40db97f64698a2db9992cafdd4ab)](https://www.codacy.com/gh/xpn-arcos/xpn/dashboard?utm_source=github.com&utm_medium=referral&utm_content=xpn-arcos/xpn&utm_campaign=Badge_Grade) * *Homepage*: @@ -15,8 +15,22 @@ The Expand Parallel File System (a.k.a. XPN) can be installed on a cluster with local storage per-node (HDD, SSD or RAM Drive) and a shared home directory. - The general steps to deploy XPN with Spack are: + The general steps to deploy XPN are: + ```mermaid + flowchart LR + A[Start] --> B{Spack?} + B -- Yes --> C[1.1 With Spack] + B -- No ---> D[install prerequisites] + subgraph ide1 [1.2 With autotools] + D --> E[download source of XPN and mxml] + E --> F[build XPN] + F --> G[End] + end + ``` + +### 1.1. With Spack + To deploy XPN with Spack: ``` git clone https://github.com/xpn-arcos/xpn.git spack repo add xpn/scripts/spack @@ -25,39 +39,51 @@ spack load xpn ``` - The traditional steps (without Spack) are: - -```mermaid -stateDiagram-v2 - direction LR - state "1. Install prerequisites" as step1 - state "2. Download the source code of XPN and mxml" as step2 - state "3. Build Expand" as step3 - [*] --> step1 - step1 --> step2 - step2 --> step3 - step3 --> [*] -``` +### 1.2. With autotools (configure, make, make install) -### 1.1. Installing prerequisites +### 1.2.1. Installing prerequisites XPN needs the typical C development tools (gcc, make, autotools) and a MPI implementation installed. -Some common use cases: -* If you are administrator of your Linux machine then you need to execute: - ``` - sudo apt-get install -y autoconf automake gcc g++ make libtool libmpich-dev - ``` +* To install the C development tools: + * If you are an user of a cluster with modules then you might try to load the compiler module, for example: + ```bash + module load gcc + ``` + Where "gcc" is the compiler module. + + * If you are administrator of your Linux machine then you need to execute: + ``` + sudo apt-get install -y autoconf automake gcc g++ make libtool build-essential + ``` -* If you are an user of a cluster with already installed software then you might try the following: - ```bash - module load icc - module load "impi/2017.4" - ``` - Where "icc" is the compiler module and "impi/2017.4" is the MPI module. - - -### 1.2. Download the source code of XPN +* To install the MPICH implementation of MPI: + * If you are an user of a cluster with already installed software then you might try to load the MPI module, for example: + ```bash + module load "impi/2017.4" + ``` + Where "impi/2017.4" is the MPI module. + + * From source code and with Infiniband (Omni-Path) support: + ``` + wget https://www.mpich.org/static/downloads/4.1.1/mpich-4.1.1.tar.gz + tar zxf mpich-4.1.1 + cd mpich-4.1.1 + ./configure --prefix= \ + --enable-threads=multiple \ + --enable-romio \ + --with-device=ch4:ofi:psm2 --with-libfabric= + make + make install + ``` + + * If you are administrator of your Linux machine then you need to execute: + ``` + sudo apt-get install -y libmpich-dev + ``` + + +### 1.2.2. Download the source code of XPN You need to download the source code of [XPN](https://xpn-arcos.github.io/arcos-xpn.github.io/) and [minixml](http://www.minixml.org). @@ -72,7 +98,7 @@ git clone https://github.com/xpn-arcos/xpn.git You must do both 'git clone' requests in the same directory (e.g.: $HOME/src). -### 1.3. Building XPN +### 1.2.3. Building XPN To build Expand you need to execute: ``` @@ -94,7 +120,7 @@ Where: * Installation directory will be $HOME/xpn_bin -## 4. Executing XPN +## 2. Executing XPN First, you need to get familiar with 4 special files: * `````` for MPI, it is a text file with the list of host names (one per line) where XPN servers and XPN client is going to be executed. @@ -110,7 +136,7 @@ Then, you need to get familiar with 5 special environment variables for XPN clie * ```XPN_LOCALITY``` with value 0 for without locality and value 1 for with locality (optional, default: 0). -### 4.1 Ad-Hoc Expand (based on MPI) +### 2.1 Ad-Hoc Expand (based on MPI) The typical executions has 3 main steps: - First, launch the Expand MPI server (xpn_mpi_server): diff --git a/config/config.guess b/config/config.guess old mode 100644 new mode 100755 diff --git a/config/config.sub b/config/config.sub old mode 100644 new mode 100755 diff --git a/configure b/configure index 0b4ce751..44c4a60c 100755 --- a/configure +++ b/configure @@ -5057,6 +5057,16 @@ fi +# +# Check __open2 support +# +MPICC_INTEL=$(mpicc --version | grep "Intel" | wc -l) +if test "x$MPICC_INTEL" == "x1"; then + +$as_echo "#define HAVE_ICC 1" >>confdefs.h + +fi + # # Checks for libraries. # @@ -6091,7 +6101,7 @@ LFLAGS+="-L../../../mxml -L../../../xpn/lib " LIBLINK=$LIBS LIBLINK+=" -lmxml" -LIBS=$LFLAGS" "$LIBS +LIBS=$LFLAGS" "$LIBS" -lmxml" if test "$target_cpu" = "AARCH64" -o "$target_cpu" = "aarch64" then @@ -6158,16 +6168,14 @@ $as_echo "\"$target_os\"" >&6; }; INCLUDEFLAGS="\ -I\$(top_srcdir)/include \ -I\$(top_srcdir)/include/base \ +-I\$(top_srcdir)/include/bypass \ -I\$(top_srcdir)/include/xpn_client/ \ -I\$(top_srcdir)/include/xpn_client/xpn \ -I\$(top_srcdir)/include/xpn_client/xpn/xpn_simple \ -I\$(top_srcdir)/include/xpn_client/nfi \ --I\$(top_srcdir)/include/xpn_client/nfi/nfi_local \ --I\$(top_srcdir)/include/xpn_client/xpni/ \ --I\$(top_srcdir)/include/xpn_client/xpni/common \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_null \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_ft \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_log" +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_mpi_server \ +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_tcp_server \ +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_local" ### BEGIN OF NFS BLOCK. Do not remove this line. ### diff --git a/configure.ac b/configure.ac index 4b5fccdb..4c18735f 100644 --- a/configure.ac +++ b/configure.ac @@ -33,6 +33,14 @@ AC_PROG_MAKE_SET AC_PROG_CC([gcc mpicc icc mpifcc]) AM_PROG_CC_C_O +# +# Check __open2 support +# +MPICC_INTEL=$(mpicc --version | grep "Intel" | wc -l) +if test "x$MPICC_INTEL" == "x1"; then + AC_DEFINE([HAVE_ICC], [1], [we have icc as mpicc compiler]) +fi + # # Checks for libraries. # @@ -82,7 +90,7 @@ LFLAGS+="-L../../../mxml -L../../../xpn/lib " LIBLINK=$LIBS LIBLINK+=" -lmxml" -LIBS=$LFLAGS" "$LIBS +LIBS=$LFLAGS" "$LIBS" -lmxml" if test "$target_cpu" = "AARCH64" -o "$target_cpu" = "aarch64" then @@ -146,16 +154,14 @@ AC_MSG_RESULT("$target_os"); INCLUDEFLAGS="\ -I\$(top_srcdir)/include \ -I\$(top_srcdir)/include/base \ +-I\$(top_srcdir)/include/bypass \ -I\$(top_srcdir)/include/xpn_client/ \ -I\$(top_srcdir)/include/xpn_client/xpn \ -I\$(top_srcdir)/include/xpn_client/xpn/xpn_simple \ -I\$(top_srcdir)/include/xpn_client/nfi \ --I\$(top_srcdir)/include/xpn_client/nfi/nfi_local \ --I\$(top_srcdir)/include/xpn_client/xpni/ \ --I\$(top_srcdir)/include/xpn_client/xpni/common \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_null \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_ft \ --I\$(top_srcdir)/include/xpn_client/xpni/xpni_log" +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_mpi_server \ +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_tcp_server \ +-I\$(top_srcdir)/include/xpn_client/nfi/nfi_local" ### BEGIN OF NFS BLOCK. Do not remove this line. ### diff --git a/include/base/base_lib.h b/include/base/base_lib.h index 5cd21aab..03327314 100644 --- a/include/base/base_lib.h +++ b/include/base/base_lib.h @@ -36,6 +36,7 @@ #include "dtable.h" #include "string_misc.h" + #include "path_misc.h" #include "urlstr.h" #endif diff --git a/include/base/ns_tcp.h b/include/base/ns_tcp.h index 0950c733..aa07346a 100644 --- a/include/base/ns_tcp.h +++ b/include/base/ns_tcp.h @@ -37,7 +37,7 @@ #endif #ifndef TCP_SERVER_FILE_DEFAULT - #define TCP_SERVER_FILE_DEFAULT "/etc/xpn/tcp_server.dns" + #define TCP_SERVER_FILE_DEFAULT "/tmp/tcp_server.dns" #endif #ifndef MAX_TCP_SERVER_NODES @@ -53,10 +53,10 @@ * API */ - char * ns_tcp_get_hostname ( void ) ; - int ns_tcp_publish ( char * dns_file, char * param_srv_name, char * host_name, char * port_name ) ; - int ns_tcp_unpublish ( char * dns_file, char * serv_name ) ; - int ns_tcp_lookup ( char * param_srv_name, char * host_name, char * port_name ) ; + char * ns_tcp_get_hostname ( void ) ; + int ns_tcp_publish ( char * dns_file, char * param_srv_name, char * host_name, char * port_name ) ; + int ns_tcp_unpublish ( char * dns_file, char * serv_name ) ; + int ns_tcp_lookup ( char * param_srv_name, char * host_name, char * port_name ) ; #endif diff --git a/include/base/path_misc.h b/include/base/path_misc.h new file mode 100644 index 00000000..f1e0ab39 --- /dev/null +++ b/include/base/path_misc.h @@ -0,0 +1,61 @@ + + /* + * Copyright 2000-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos, Luis Miguel Sanchez Garcia, Borja Bergua Guerra + * + * This file is part of Expand. + * + * Expand is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Expand is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Expand. If not, see . + * + */ + + + #ifndef _PATH_MISC_H_ + #define _PATH_MISC_H_ + + /* ... Include / Inclusion ........................................... */ + + #include "all_system.h" + #include "base/string_misc.h" + + + #ifdef __cplusplus + extern "C" { + #endif + + + /* ... Functions / Funciones ......................................... */ + + int hash (char *file,int nServ) ; + + int getFirstDir ( char *dir, char *path) ; + int getSizeFactor ( char *name ) ; + + /* get the last name of a path and erase the file name */ + int getNameFile(char *file, char *dir); + + /* get the first name of the path and erase the part name */ + int getNamePart(char *part, char *dir); + + /* erase the initial dir used in the url */ + int getDirWithURL(char *url, char *dir); + + + /* .................................................................... */ + +#ifdef __cplusplus + } + #endif + + #endif /* _PATH_MISC_H */ + diff --git a/include/base/string_misc.h b/include/base/string_misc.h index 9581edde..da21fc50 100644 --- a/include/base/string_misc.h +++ b/include/base/string_misc.h @@ -109,34 +109,6 @@ ) ; - // TODO: Unknown for string... filesystem functions??? - int hash (char *file,int nServ) ; - - // TODO: move into filesystem.h ??? - int getDir ( char *arg1 ) ; - int multiline ( char *text ) ; - int getFirstDir ( char *dir, char *path) ; - int getSizeFactor ( char *name ) ; - - // TODO: move into urlstr.h ???? - int ParseURL( char *url, - char *protocol, - char *login, - char *passwd, - char *server, - char *port, - char *dir); - - /* get the last name of a path and erase the file name */ - int getNameFile(char *file, char *dir); - - /* get the first name of the path and erase the part name */ - int getNamePart(char *part, char *dir); - - /* erase the initial dir used in the url */ - int getDirWithURL(char *url, char *dir); - - /* .................................................................... */ #ifdef __cplusplus diff --git a/include/base/urlstr.h b/include/base/urlstr.h index ff5ca902..806dbc65 100644 --- a/include/base/urlstr.h +++ b/include/base/urlstr.h @@ -28,6 +28,7 @@ #include "all_system.h" #include "string_misc.h" + #include "path_misc.h" #ifdef __cplusplus @@ -37,22 +38,16 @@ /* ... Functions / Funciones ......................................... */ - int URLSTR_ParseURL - ( - /*IN */ char *urlstr, - /*OUT*/ char **protocol, - /*OUT*/ char **user, - /*OUT*/ char **machine, - /*OUT*/ int *port, - /*OUT*/ char **file, - /*OUT*/ char **relative, - /*OUT*/ char **params - ) ; - /* - Fill 'url' with information from 'str' string. */ - /* - Rellena 'url' con la informacion de 'str'. */ - - - /* .................................................................... */ + int ParseURL( char *url, + char *protocol, + char *login, + char *passwd, + char *server, + char *port, + char *dir) ; + + + /* ................................................................... */ #ifdef __cplusplus diff --git a/include/bypass/xpn_bypass.h b/include/bypass/xpn_bypass.h index 96f69a81..8913291b 100644 --- a/include/bypass/xpn_bypass.h +++ b/include/bypass/xpn_bypass.h @@ -8,6 +8,8 @@ #define _GNU_SOURCE +#include "config.h" + #include #include #include @@ -81,7 +83,7 @@ struct generic_fd{ int open (const char *path, int flags, ...); int open64 (const char *path, int flags, ...); -///int __open_2 (const char *path, int flags, ...); +int __open_2 (const char *path, int flags, ...); int creat (const char *path, mode_t mode); int close (int fd); @@ -135,6 +137,7 @@ int chown (const char *path, uid_t owner, gid_t group); int fcntl (int fd, int cmd, long arg); int access (const char *path, int mode); char *realpath (const char *restrict path, char *restrict resolved_path); +char * __realpath_chk(const char * path, char * resolved_path, size_t resolved_len); int fsync (int fd); diff --git a/include/config.h b/include/config.h index 5e44da64..a19b5adb 100644 --- a/include/config.h +++ b/include/config.h @@ -16,6 +16,9 @@ /* Define to 1 if you have the `gettimeofday' function. */ #define HAVE_GETTIMEOFDAY 1 +/* we have icc as mpicc compiler */ +/* #undef HAVE_ICC */ + /* Define to 1 if you have the header file. */ #define HAVE_INTTYPES_H 1 diff --git a/include/config.h.in b/include/config.h.in index 74b8cac5..cf795cbe 100644 --- a/include/config.h.in +++ b/include/config.h.in @@ -15,6 +15,9 @@ /* Define to 1 if you have the `gettimeofday' function. */ #undef HAVE_GETTIMEOFDAY +/* we have icc as mpicc compiler */ +#undef HAVE_ICC + /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H diff --git a/include/tcp_server/tcp_server_ops.h b/include/tcp_server/tcp_server_ops.h index 189d0e7c..44843d6f 100644 --- a/include/tcp_server/tcp_server_ops.h +++ b/include/tcp_server/tcp_server_ops.h @@ -39,11 +39,9 @@ * Constants */ -#ifndef TCP_SERVER_ID - #define TCP_SERVER_ID 32 -#endif - - /* Operations */ + #ifndef TCP_SERVER_ID + #define TCP_SERVER_ID 32 + #endif /* Operations */ @@ -135,6 +133,7 @@ struct st_tcp_server_close { int fd; + char path[PATH_MAX]; }; struct st_tcp_server_rename diff --git a/include/tcp_server/tcp_server_params.h b/include/tcp_server/tcp_server_params.h index 25d74c4b..58a59408 100644 --- a/include/tcp_server/tcp_server_params.h +++ b/include/tcp_server/tcp_server_params.h @@ -34,7 +34,10 @@ #include "base/workers.h" - //Constants + /* + * Constants + */ + #define TCP_MAX_PORT_NAME 1024 @@ -58,8 +61,14 @@ // server configuration int thread_mode; + + //mqtt configuration int mosquitto_mode; + #ifdef HAVE_MOSQUITTO_H + struct mosquitto * mqtt; + #endif + //Semaphore for clients char sem_name_server [PATH_MAX]; @@ -87,3 +96,4 @@ void tcp_server_params_show ( tcp_server_param_st *params ) ; #endif + diff --git a/include/xpn_client/nfi/nfi_lib.h b/include/xpn_client/nfi/nfi_lib.h index eac3b8c7..f9bad6b4 100644 --- a/include/xpn_client/nfi/nfi_lib.h +++ b/include/xpn_client/nfi/nfi_lib.h @@ -3,7 +3,7 @@ #include "nfi.h" #include "nfi_worker.h" -#include "base/string_misc.h" +#include "base/path_misc.h" /* protocols */ #include "nfi_local.h" diff --git a/include/xpn_client/nfi/nfi_local/nfi_local.h b/include/xpn_client/nfi/nfi_local/nfi_local.h index aa1284dc..b380bcdb 100644 --- a/include/xpn_client/nfi/nfi_local/nfi_local.h +++ b/include/xpn_client/nfi/nfi_local/nfi_local.h @@ -33,7 +33,8 @@ #define FILESYSTEM_DLSYM 1 #include "all_system.h" #include "base/filesystem.h" - #include "base/string_misc.h" + #include "base/path_misc.h" + #include "base/urlstr.h" #include "nfi/nfi.h" #include "nfi/nfi_local/nfi_local_err.h" #include "nfi_worker.h" diff --git a/include/xpn_client/nfi/nfi_mpi_server/nfi_mpi_server.h b/include/xpn_client/nfi/nfi_mpi_server/nfi_mpi_server.h index d6d491ca..a43cb37a 100644 --- a/include/xpn_client/nfi/nfi_mpi_server/nfi_mpi_server.h +++ b/include/xpn_client/nfi/nfi_mpi_server/nfi_mpi_server.h @@ -31,7 +31,8 @@ #include "all_system.h" #include "base/utils.h" - #include "base/string_misc.h" + #include "base/path_misc.h" + #include "base/urlstr.h" #include "base/workers.h" #include "mpi_server_conf.h" #include "mpi_server_ops.h" diff --git a/include/xpn_client/nfi/nfi_nfs/nfi_nfs.h b/include/xpn_client/nfi/nfi_nfs/nfi_nfs.h index e701f96c..73a2718b 100644 --- a/include/xpn_client/nfi/nfi_nfs/nfi_nfs.h +++ b/include/xpn_client/nfi/nfi_nfs/nfi_nfs.h @@ -2,7 +2,7 @@ #define _NFI_NFS_H #include "all_system.h" -#include "base/string_misc.h" +#include "base/path_misc.h" #include "nfi/nfi.h" #include "nfi/nfi_nfs/nfs.h" #include "nfi/nfi_nfs/nfi_nfs_err.h" diff --git a/include/xpn_client/nfi/nfi_nfs/nfs.h b/include/xpn_client/nfi/nfi_nfs/nfs.h index 5366a8d1..7c3933df 100644 --- a/include/xpn_client/nfi/nfi_nfs/nfs.h +++ b/include/xpn_client/nfi/nfi_nfs/nfs.h @@ -2,7 +2,7 @@ #define _NFS_H_ #include "all_system.h" -#include "base/string_misc.h" +#include "base/path_misc.h" #define NFS_TCP 1 #define NFS_UDP 0 diff --git a/include/xpn_client/nfi/nfi_nfs3/nfi_nfs3.h b/include/xpn_client/nfi/nfi_nfs3/nfi_nfs3.h index 539b78fb..f4494dd1 100644 --- a/include/xpn_client/nfi/nfi_nfs3/nfi_nfs3.h +++ b/include/xpn_client/nfi/nfi_nfs3/nfi_nfs3.h @@ -2,7 +2,7 @@ #define _NFI_NFS3_H #include "all_system.h" -#include "base/string_misc.h" +#include "base/path_misc.h" #include "nfi/nfi.h" #include "nfi/nfi_nfs3/nfs3.h" #include "nfi/nfi_nfs3/nfi_nfs3_err.h" diff --git a/include/xpn_client/nfi/nfi_nfs3/nfs3.h b/include/xpn_client/nfi/nfi_nfs3/nfs3.h index ae88868b..ed94110b 100644 --- a/include/xpn_client/nfi/nfi_nfs3/nfs3.h +++ b/include/xpn_client/nfi/nfi_nfs3/nfs3.h @@ -7,7 +7,7 @@ #define _NFS3_PROT_H_RPCGEN #include "all_system.h" -#include "base/string_misc.h" +#include "base/path_misc.h" #define NFS3_TCP 1 #define NFS3_UDP 0 diff --git a/include/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.h b/include/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.h index 78022169..c58e806e 100644 --- a/include/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.h +++ b/include/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.h @@ -31,7 +31,8 @@ #include "all_system.h" #include "base/utils.h" - #include "base/string_misc.h" + #include "base/path_misc.h" + #include "base/urlstr.h" #include "base/workers.h" #include "tcp_server_conf.h" #include "tcp_server_ops.h" @@ -40,11 +41,19 @@ #include "nfi_tcp_server_err.h" #include "nfi.h" #include "nfi_worker.h" + #ifdef HAVE_MOSQUITTO_H #include #endif + /* ... Const / Const ................................................. */ + + #ifndef MOSQ_OPT_TCP_NODELAY + #define MOSQ_OPT_TCP_NODELAY 0 + #endif + + /* ... Data structures / Estructuras de datos ........................ */ struct nfi_tcp_server_server @@ -53,6 +62,7 @@ char id[PATH_MAX] ; char path[PATH_MAX] ; //struct nfi_tcp_server_connector sd; + struct mosquitto * mqtt; // client stub tcpClient_param_st params ; diff --git a/include/xpn_client/nfi/nfi_tcp_server/tcp_server_ops.h b/include/xpn_client/nfi/nfi_tcp_server/tcp_server_ops.h index 352fe869..f56657b6 100644 --- a/include/xpn_client/nfi/nfi_tcp_server/tcp_server_ops.h +++ b/include/xpn_client/nfi/nfi_tcp_server/tcp_server_ops.h @@ -101,6 +101,7 @@ struct st_tcp_server_close{ int fd; + char path[PATH_MAX]; }; struct st_tcp_server_rm{ diff --git a/include/xpn_client/xpn.h b/include/xpn_client/xpn.h index 1dd3b3f7..ecf20806 100644 --- a/include/xpn_client/xpn.h +++ b/include/xpn_client/xpn.h @@ -12,8 +12,8 @@ extern "C" { #endif // xpn_cwd.c -char* xpn_getcwd (char *path, size_t size); -int xpn_chdir (char *path); +char * xpn_getcwd (char *path, size_t size); +int xpn_chdir (char *path); // xpn_dir.c int xpn_mkdir (const char *path, mode_t perm) ; @@ -43,7 +43,7 @@ int xpn_dup (int fd); int xpn_dup2 (int fd, int fd2); // xpn_opendir.c -DIR* xpn_opendir (const char *path); +DIR * xpn_opendir (const char *path); int xpn_closedir (DIR *dirp); struct dirent* xpn_readdir (DIR *dirp); void xpn_rewinddir (DIR *dirp); @@ -75,7 +75,7 @@ char* xpn_fgets (char *s, int tam, FILE *flujo); /***************/ // xpn_fopen.c -FILE* xpn_fopen (const char *filename, const char *mode); +FILE * xpn_fopen (const char *filename, const char *mode); // xpn_fclose.c int xpn_fclose (FILE *stream); @@ -114,3 +114,4 @@ int xpn_setvbuf (FILE *stream, char *buf, int mode, size_t size); #endif #endif + diff --git a/include/xpn_client/xpn/xpn_simple/xpn_init.h b/include/xpn_client/xpn/xpn_simple/xpn_init.h index cead6541..041c2d61 100644 --- a/include/xpn_client/xpn/xpn_simple/xpn_init.h +++ b/include/xpn_client/xpn/xpn_simple/xpn_init.h @@ -24,6 +24,7 @@ #define _XPN_INIT_H #include "all_system.h" +#include "base/path_misc.h" #include "xpn_policy_init.h" #include "xpn_cwd.h" #include "xpn_file.h" diff --git a/scripts/compile/platform/lab21.sh b/scripts/compile/platform/lab21.sh index e463861f..57a9159e 100755 --- a/scripts/compile/platform/lab21.sh +++ b/scripts/compile/platform/lab21.sh @@ -29,7 +29,7 @@ for P in $PKG_NAMES; do done # 2) working path... -MPICC_PATH=/opt/software/install-mpich/bin/mpicc +MPICC_PATH=/local_test/bin/mpich/bin/mpicc INSTALL_PATH=/local_test/bin/ BASE_PATH=$(dirname $0) diff --git a/scripts/compile/platform/unito.sh b/scripts/compile/platform/unito.sh index 44f34f9a..adc2ae58 100755 --- a/scripts/compile/platform/unito.sh +++ b/scripts/compile/platform/unito.sh @@ -24,11 +24,13 @@ # 1) software (if needed)... #spack load openmpi -spack load mpich +#spack load mpich +spack load pkg-config # 2) working path... #MPICC_PATH=$HOME/opt/spack/linux-ubuntu20.04-zen/gcc-9.4.0/openmpi-4.1.3-4bpvwm3lcbftmjki6en35c4i5od6wjbr/bin/mpicc -MPICC_PATH=$HOME/opt/spack/linux-ubuntu20.04-zen/gcc-9.4.0/mpich-4.0.2-a76rmlxbneoqdvemzjsyewp2akiiuxlj/bin/mpicc +#MPICC_PATH=$HOME/opt/spack/linux-ubuntu20.04-zen/gcc-9.4.0/mpich-4.0.2-a76rmlxbneoqdvemzjsyewp2akiiuxlj/bin/mpicc +MPICC_PATH=$HOME/dcamarma/bin/mpich/bin/mpicc INSTALL_PATH=$HOME/dcamarma/bin/ BASE_PATH=$(dirname $0) diff --git a/src/base/Makefile.am b/src/base/Makefile.am index 6d44ed24..21f7ed2e 100644 --- a/src/base/Makefile.am +++ b/src/base/Makefile.am @@ -10,6 +10,7 @@ BASE_HEADER= @top_srcdir@/include/base/base_lib.h \ @top_srcdir@/include/base/dtable.h \ @top_srcdir@/include/base/math_misc.h \ @top_srcdir@/include/base/string_misc.h \ + @top_srcdir@/include/base/path_misc.h \ @top_srcdir@/include/base/time_misc.h \ @top_srcdir@/include/base/trace_msg.h \ @top_srcdir@/include/base/trace_tags.h \ @@ -28,6 +29,7 @@ BASE_SOURCE= @top_srcdir@/src/base/darray.c \ @top_srcdir@/src/base/trace_msg.c \ @top_srcdir@/src/base/debug_msg.c \ @top_srcdir@/src/base/string_misc.c \ + @top_srcdir@/src/base/path_misc.c \ @top_srcdir@/src/base/time_misc.c \ @top_srcdir@/src/base/urlstr.c \ @top_srcdir@/src/base/ns.c \ diff --git a/src/base/Makefile.in b/src/base/Makefile.in index eb8e2df7..9bfba62e 100644 --- a/src/base/Makefile.in +++ b/src/base/Makefile.in @@ -148,6 +148,7 @@ am__objects_1 = @top_srcdir@/src/base/darray.$(OBJEXT) \ @top_srcdir@/src/base/trace_msg.$(OBJEXT) \ @top_srcdir@/src/base/debug_msg.$(OBJEXT) \ @top_srcdir@/src/base/string_misc.$(OBJEXT) \ + @top_srcdir@/src/base/path_misc.$(OBJEXT) \ @top_srcdir@/src/base/time_misc.$(OBJEXT) \ @top_srcdir@/src/base/urlstr.$(OBJEXT) \ @top_srcdir@/src/base/ns.$(OBJEXT) \ @@ -180,6 +181,7 @@ am__depfiles_remade = @top_srcdir@/src/base/$(DEPDIR)/darray.Po \ @top_srcdir@/src/base/$(DEPDIR)/filesystem.Po \ @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/ns.Po \ + @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po \ @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po \ @@ -358,6 +360,7 @@ BASE_HEADER = @top_srcdir@/include/base/base_lib.h \ @top_srcdir@/include/base/dtable.h \ @top_srcdir@/include/base/math_misc.h \ @top_srcdir@/include/base/string_misc.h \ + @top_srcdir@/include/base/path_misc.h \ @top_srcdir@/include/base/time_misc.h \ @top_srcdir@/include/base/trace_msg.h \ @top_srcdir@/include/base/trace_tags.h \ @@ -376,6 +379,7 @@ BASE_SOURCE = @top_srcdir@/src/base/darray.c \ @top_srcdir@/src/base/trace_msg.c \ @top_srcdir@/src/base/debug_msg.c \ @top_srcdir@/src/base/string_misc.c \ + @top_srcdir@/src/base/path_misc.c \ @top_srcdir@/src/base/time_misc.c \ @top_srcdir@/src/base/urlstr.c \ @top_srcdir@/src/base/ns.c \ @@ -478,6 +482,9 @@ clean-libLIBRARIES: @top_srcdir@/src/base/string_misc.$(OBJEXT): \ @top_srcdir@/src/base/$(am__dirstamp) \ @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) +@top_srcdir@/src/base/path_misc.$(OBJEXT): \ + @top_srcdir@/src/base/$(am__dirstamp) \ + @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) @top_srcdir@/src/base/time_misc.$(OBJEXT): \ @top_srcdir@/src/base/$(am__dirstamp) \ @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) @@ -524,6 +531,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/filesystem.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/math_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/ns.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/path_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/string_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/time_misc.Po@am__quote@ # am--include-marker @@ -715,6 +723,7 @@ distclean: distclean-am -rm -f @top_srcdir@/src/base/$(DEPDIR)/filesystem.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns.Po + -rm -f @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po @@ -776,6 +785,7 @@ maintainer-clean: maintainer-clean-am -rm -f @top_srcdir@/src/base/$(DEPDIR)/filesystem.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns.Po + -rm -f @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po diff --git a/src/base/ns_tcp.c b/src/base/ns_tcp.c index 78833a6e..4db48af4 100644 --- a/src/base/ns_tcp.c +++ b/src/base/ns_tcp.c @@ -25,7 +25,8 @@ /* ... Functions / Funciones ......................................... */ -char * ns_tcp_get_hostname(void) { +char * ns_tcp_get_hostname(void) +{ char * ip; char serv_name[HOST_NAME_MAX]; struct hostent * serv_entry; @@ -37,14 +38,15 @@ char * ns_tcp_get_hostname(void) { return ip; } -int ns_tcp_publish(char * dns_file, char * param_srv_name, char * host_name, char * port_name) { + +int ns_tcp_publish(char * dns_file, char * param_srv_name, char * host_name, char * port_name) +{ int ret; FILE * dns_fd; - dns_fd = fopen(dns_file, "a+"); - printf("[%s][%d]\t%s\n", __FILE__, __LINE__, dns_file); + dns_fd = fopen(dns_file, "a+"); if (NULL == dns_fd) { perror("fopen on DNS File"); return -1; @@ -60,7 +62,9 @@ int ns_tcp_publish(char * dns_file, char * param_srv_name, char * host_name, cha return 0; } -int ns_tcp_unpublish(char * dns_file, char * serv_name) { + +int ns_tcp_unpublish(char * dns_file, char * serv_name) +{ FILE * dns_fd; FILE * new_dns_fd; char new_dns_file[PATH_MAX]; @@ -88,7 +92,8 @@ int ns_tcp_unpublish(char * dns_file, char * serv_name) { } // copy filtering... - while (fscanf(dns_fd, "%s %s %s", aux_name, aux_name_2, port_name) != EOF) { + while (fscanf(dns_fd, "%s %s %s", aux_name, aux_name_2, port_name) != EOF) + { if (strcmp(aux_name, serv_name) == 0) { //Not copy the line found = 1; @@ -112,7 +117,9 @@ int ns_tcp_unpublish(char * dns_file, char * serv_name) { return 0; } -int ns_tcp_lookup(char * param_srv_name, char * host_name, char * port_name) { + +int ns_tcp_lookup(char * param_srv_name, char * host_name, char * port_name) +{ int found = 0; char aux_name[1024]; char * dns_file; @@ -135,10 +142,8 @@ int ns_tcp_lookup(char * param_srv_name, char * host_name, char * port_name) { while (fscanf(dns_fd, "%s %s %s", aux_name, host_name, port_name) != EOF) { //printf("[%s][%d]\t%s %s %s - %s\n", __FILE__, __LINE__, aux_name, host_name, port_name, param_srv_name); - if (strcmp(aux_name, param_srv_name) == 0) - { + if (strcmp(aux_name, param_srv_name) == 0) { found = 1; - //printf("ENCONTRADO\n\n"); break; } } @@ -152,4 +157,6 @@ int ns_tcp_lookup(char * param_srv_name, char * host_name, char * port_name) { return 0; } -/* ................................................................... */ \ No newline at end of file + +/* ................................................................... */ + diff --git a/src/base/path_misc.c b/src/base/path_misc.c new file mode 100644 index 00000000..868eb2d9 --- /dev/null +++ b/src/base/path_misc.c @@ -0,0 +1,174 @@ + +/* + * Copyright 2020-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos, Luis Miguel Sanchez Garcia, Borja Bergua Guerra + * + * This file is part of Expand. + * + * Expand is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Expand is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Expand. If not, see . + * + */ + + + /* ... Include / Inclusion ........................................... */ + + #include "base/path_misc.h" + + + /* ... Functions / Funciones ......................................... */ + + + //calculo del numero de servidor + int hash(char *file,int nServ) + { + int i,max; + int unsigned num; + + num=0; + max=strlen(file)-1; + for(i=0;i=0)&&(dir[i] == '/')) + i--; + if(i == 0){ + file[0]='\0'; + return 0; /* ? */ + } + + /* */ + j = i; + while((j>=0)&&(dir[j] != '/')) + j--; + + if(dir[j] == '/'){ + j++; + } + + strncpy(file, dir+j, i-j); + + dir[j] = '\0'; + file[i-j] = '\0'; + + return 0; + } + + + int getNamePart(char *part, char *dir) + { + int i,j; + + //Printf("original dir = %s\n", dir); + if(dir[0] != '/') + return -1; + i =1; + while((dir[i] != '\0')&&(dir[i] != '/')) + i++; + strncpy(part, dir+1, i-1); + part[i-1] = '\0'; + j = 0; + while(dir[i] != '\0'){ + dir[j] = dir[i]; + j++; + i++; + } + + dir[j] = '\0'; + return 0; + } + + + /* ................................................................... */ + diff --git a/src/base/string_misc.c b/src/base/string_misc.c index 6969965e..83aad08c 100644 --- a/src/base/string_misc.c +++ b/src/base/string_misc.c @@ -192,398 +192,5 @@ } -/* ............................................................... */ - - - //calculo del numero de servidor - int hash(char *file,int nServ){ - int i,max; - int unsigned num; - - num=0; - max=strlen(file)-1; - for(i=0;i=0)&&(dir[i] == '/')) - i--; - if(i == 0){ - file[0]='\0'; - return 0; /* ? */ - } - - /* */ - j = i; - while((j>=0)&&(dir[j] != '/')) - j--; - - if(dir[j] == '/'){ - j++; - } - - strncpy(file, dir+j, i-j); - - dir[j] = '\0'; - file[i-j] = '\0'; - - return 0; - } - - - int getNamePart(char *part, char *dir){ - int i,j; - - //Printf("original dir = %s\n", dir); - if(dir[0] != '/') - return -1; - i =1; - while((dir[i] != '\0')&&(dir[i] != '/')) - i++; - strncpy(part, dir+1, i-1); - part[i-1] = '\0'; - j = 0; - while(dir[i] != '\0'){ - dir[j] = dir[i]; - j++; - i++; - } - - dir[j] = '\0'; - return 0; - } - - - int getDirWithURL(char *url, char *dir) - { - char dir_aux[255]; /* change for a const*/ - int i,j; - - getURLDir(url, dir_aux); - i = 0; - while((dir_aux[i] != '\0')&&(dir_aux[i] == dir[i])) - i++; - - - if(dir_aux[i] != '\0') - return -1; - if(dir_aux[i] == dir[i]){ - //dir[0] = '\0'; /* or '/' */ - dir[0] = '/'; - dir[1] = '\0'; - return 0; - } - j = 0; - while(dir[j+i]!='\0'){ - dir[j] = dir[j+i]; - j++; - } - dir[j] = '\0'; - if((dir[j] == '/') &&(strlen(dir+j) == 1)) - dir[j-1] = '\0'; - return 0; - } - - - -int getFirstDir(char *dir, char *path) -{ - int i, j; - - //printf("++ path = %s ++\n", path); - - i = 0; - while((path[i] != '\0')&&(path[i] != '/')){ - i++; - } - strncpy(dir, path , i); - - dir[i] = '\0'; - - //printf("++ dir = %s ++\n",dir); - - - while((path[i] != '\0')&&(path[i] == '/')){ - i++; - } - - j = 0; - while(path[i+j] != '\0'){ - - path[j] = path[i+j]; - j++; - } - - path[j] = '\0'; - //printf("++ dir = %s path = %s ++\n",dir, path); - return j; -} - -int getSizeFactor(char *name){ - - switch(name[strlen(name)-1] ){ - case 'K': - case 'k': - return atoi(name)*KB; - case 'M': - case 'm': - return atoi(name)*MB; - case 'G': - case 'g': - return atoi(name)*GB; - case 'B': - case 'b': - switch(name[strlen(name)-2]){ - case 'K': - case 'k': - return atoi(name)*KB; - case 'M': - case 'm': - return atoi(name)*MB; - case 'G': - case 'g': - return atoi(name)*GB; - default: - return 1; - } - default: - return 1; - } -} - - /* ...................................................................... */ + diff --git a/src/base/urlstr.c b/src/base/urlstr.c index 58c522d6..25210039 100644 --- a/src/base/urlstr.c +++ b/src/base/urlstr.c @@ -27,351 +27,288 @@ /* ... Functions / Funciones ......................................... */ - static - int URLSTR_Match_protocol ( char **protocol, - char **str ) + + + /* ... Functions / Funciones ......................................... */ + + int getURLProtocol(char *url, char *protocol) { - static char *URLSTR_protocols[] = - { - "http", - "ftp", - "file", - "tcp_server", - "mpi_server", - NULL - } ; - int i, ret ; - - for (i=0; URLSTR_protocols[i] != NULL; i++) - { - ret = strncmp( (*str), - URLSTR_protocols[i], - strlen(URLSTR_protocols[i]) ) ; - if (ret == 0) - { - if (strcmp("news",URLSTR_protocols[i]) == 0) - (*protocol) = STRING_MISC_StrDup("nntp") ; - else (*protocol) = STRING_MISC_StrDup(URLSTR_protocols[i]) ; - - (*str) = (*str) + STRING_MISC_StrLen(URLSTR_protocols[i]) ; - if ( (**str) == ':' ) - (*str) ++ ; - if ( (**str) == '/' ) - (*str) ++ ; - if ( (**str) == '/' ) - (*str) ++ ; - return (1) ; - } - } - (*protocol) = STRING_MISC_StrDup("http") ; - return (1) ; + int i,j; + + if (url == NULL) { + printf("[%s:%d] ERROR: url is NULL\n", __FILE__, __LINE__); + return -1; + } + + // find ':' + i = 0; + while ((url[i] != '\0')&&(url[i] != ':')) { + i++; + } + j = i; + + if (url[i] != ':') { + printf("[%s:%d] ERROR: missing ':' within the url.\n", __FILE__, __LINE__); + printf("Usage: protocol_name://server:port//dir\n"); + return -1; + } + + i++; + if (url[i] != '/') { + printf("[%s:%d] ERROR: missing first '/' within the url.\n", __FILE__, __LINE__); + printf("Usage: protocol_name://server:port//dir\n"); + return -1; + } + + i++; + if (url[i] != '/') { + printf("[%s:%d] ERROR: missing second '/' within the url.\n", __FILE__, __LINE__); + printf("Usage: protocol_name://server:port//dir\n"); + return -1; + } + + i++; + if (protocol != NULL) { + strncpy(protocol, url, j); + protocol[j] = '\0'; + } + + return i; } - static - int URLSTR_Match_user ( char **user, - char **str ) + + int getURLServer(char *url, char *server) { - char *pch ; + int i,j; - pch = strchr((*str),'@') ; - if (pch == NULL) - return (1) ; + i = getURLProtocol(url, NULL) ; + if (i < 0) { + return -1; + } - pch[0] = '\0' ; - (*user) = STRING_MISC_StrDup((*str)) ; - (*str) = pch + 1 ; - return (1) ; + j = i; + while ((url[j]!='\0')&&(url[j]!=':')&&(url[j]!='/')) { + j++; + } + + if (server != NULL) { + strncpy(server, url+i, j-i); + server[j-i] = '\0'; + } + + return j; } - static - int URLSTR_Match_machine ( char **machine, - char *protocol, - char **str ) + + int getURLPort ( char *url, char *port ) { - char *pch1, *pch2 ; - - (*machine) = NULL ; - - /* - * file - */ - if ( STRING_MISC_Equal(protocol,"file") ) - { - int ret ; - - (*machine) = STRING_MISC_StrDup("localhost") ; - ret = strncmp((*str),"localhost",strlen("localhost")) ; - if (ret == 0) - (*str) = (*str) + strlen("localhost") ; - return (1) ; - } - - /* - * http || ftp - */ - pch1 = strchr((*str),'/') ; - pch2 = strchr((*str),':') ; - if (pch2 == NULL) - { - if (pch1 == NULL) - { - (*machine) = STRING_MISC_StrDup((*str)) ; - (*str) = (*str) + STRING_MISC_StrLen((*str)) ; - return (1) ; - } - else // if (pch1 != NULL) - { - pch1[0] = '\0' ; - (*machine) = STRING_MISC_StrDup((*str)) ; - (*str) = pch1 ; - pch1[0] = '/' ; - return (1) ; - } - } - else // if (pch2 != NULL) - { - pch2[0] = '\0' ; - (*machine) = STRING_MISC_StrDup((*str)) ; - pch2[0] = ':' ; - (*str) = pch2 ; - return (1) ; - } - return (1) ; + int i, j; + + i=getURLServer(url, NULL) ; + if (i < 0) { + return -1; + } + + if (url[i] != ':') { + return -1; + } + + i++; + j = i; + while ((url[j]!='\0')&&(url[j]!='/')) { + j++; + } + + if(port != NULL){ + strncpy(port, url+i, j-i); + port[j-i] = '\0'; + } + + return j; } - static - int URLSTR_Match_port ( int *port, - char *protocol, - char **str ) + + int getURLDir(char *url, char *dir) { - char *pch1, *pch2 ; - - /* ... default port ... */ - (*port) = htons(80) ; - if (STRING_MISC_Equal(protocol,"http") ) - (*port) = htons(80) ; - if (STRING_MISC_Equal(protocol,"ftp") ) - (*port) = htons(21) ; - if (STRING_MISC_Equal(protocol,"file") ) - (*port) = htons(0) ; - if (STRING_MISC_Equal(protocol,"nntp") ) - (*port) = htons(119) ; - if (STRING_MISC_Equal(protocol,"news") ) - (*port) = htons(119) ; - if (STRING_MISC_Equal(protocol,"pop3") ) - (*port) = htons(110) ; - if (STRING_MISC_Equal(protocol,"finger") ) - (*port) = htons(79) ; - - /* ... scanning port ... */ - /* - file - */ - if (STRING_MISC_Equal(protocol,"file") ) - return (1) ; - - /* - http || ftp - */ - pch1 = strchr((*str),':') ; - if (pch1 == NULL) - { - return (1) ; - } - pch1 ++ ; /* skip ':' */ - pch2 = strchr(pch1,'/') ; - if (pch2 == NULL) - { - (*port) = htons(atoi(pch1)) ; - (*str) = (*str) + STRING_MISC_StrLen((*str)) ; - return (1) ; - } - else // if (pch2 != NULL) - { - pch2[0] = '\0' ; - (*port) = htons(atoi(pch1)) ; - pch2[0] = '/' ; - (*str) = pch2 ; - return (1) ; - } - return (1) ; + int i; + + if ((i=getURLPort(url, NULL))<0) + { + if ((i=getURLServer(url, NULL))<0) + { + if((i=getURLProtocol(url, NULL))<0) + { + return -1; + } + } + } + + if (dir != NULL) { + strncpy(dir, url+i, strlen(url)-i); + dir[strlen(url)-i] = '\0'; + } + + return strlen(url); } - static - int URLSTR_Match_file ( char **file, - char **str ) + + +/* + int getURLLogin(char *url, char *login) { - char *pch1 ; - - pch1 = strchr((*str),'#') ; - if (pch1 != NULL) - { - pch1[0] = '\0' ; - (*file) = STRING_MISC_StrDup((*str)) ; - (*str) = (*str) + STRING_MISC_StrLen(pch1) ; - pch1[0] = '#' ; - return (1) ; - } - - pch1 = strchr((*str),'?') ; - if (pch1 != NULL) - { - pch1[0] = '\0' ; - (*file) = STRING_MISC_StrDup((*str)) ; - pch1[0] = '?' ; - (*str) = pch1 ; - return (1) ; - } - - /* ... all is file ... */ - if ((*str)[0] != '\0') - { - (*file) = STRING_MISC_StrDup((*str)) ; - (*str) = (*str) + STRING_MISC_StrLen((*str)) ; - return (1) ; - } - - (*file) = STRING_MISC_StrDup("/") ; - return (1) ; + // TODO + return 0; } - static - int URLSTR_Match_relative ( char **relative, - char **str ) + + int getURLPasswd(char *url, char *passwd) { - char *pch1, *pch2 ; - - (*relative) = NULL ; - pch1 = strchr((*str),'#') ; - if (pch1 == NULL) - return (1) ; - - pch1 ++ ; /* skip '#' */ - pch2 = strchr(pch1,'?') ; - if (pch2 == NULL) - { - (*relative) = STRING_MISC_StrDup((*str)) ; - (*str) = (*str) + STRING_MISC_StrLen((*str)) ; - return (1) ; - } - - if (pch2 != NULL) - { - pch2[0] = '\0' ; - (*relative) = STRING_MISC_StrDup(pch1) ; - (*str) = pch2 ; - pch2[0] = '?' ; - return (1) ; - } - - return (1) ; + // TODO + return 0; } +*/ - static - int URLSTR_Match_params ( char **params, - char **str ) + + int clear_slash(char *path) { - char *pch1 ; + size_t i; + int j; + char ant = '\0', s[PATH_MAX]; + + j=0; + for (i=0; i < strlen(path); i++) + { + switch(path[i]) + { + case '/': + if(ant != '/'){ + ant = s[j] = '/'; + j++; + } + break; + + default: + ant = s[j] = path[i]; + j++; + } + + s[j] = '\0'; + } + + strcpy(path, s); + return 0; + } - (*params) = NULL ; - pch1 = strchr((*str),'?') ; - if (pch1 == NULL) - return (1) ; - pch1 ++ ; /* skip '?' */ - (*params) = STRING_MISC_StrDup(pch1) ; - (*str) = (*str) + STRING_MISC_StrLen((*str)) ; - return (1) ; + int ParseURL( char *url, + char *protocol, char *login, char *passwd, char *server, char *port, char *dir ) + { + char *urlaux; + + urlaux = url; + if (protocol != NULL) + { + /* return the next position */ + if(getURLProtocol(urlaux, protocol)<0){ + return -1; + } + } + + urlaux = url; + if (login != NULL) + { + /* return the next position */ + /* + if(getURLLogin(urlaux, login)<0){ + //Not mandatory + //return -1; + } + */ + } + + urlaux = url; + if(passwd != NULL) + { + /* return the next position */ + /* + if(getURLPasswd(urlaux, passwd)<0){ + //Not mandatory + //return -1; + } + */ + } + + urlaux = url; + if(server != NULL) + { + /* return the next position */ + if(getURLServer(urlaux, server)<0){ + return -1; + } + } + + + urlaux = url; + if(port != NULL) + { + /* return the next position */ + if(getURLPort(urlaux, port)<0){ + //Not mandatory + //return -1; + } + } + + urlaux = url; + if(dir != NULL) + { + /* return the next position */ + if(getURLDir(urlaux, dir)<0){ + return -1; + } + clear_slash(dir); + } + + return 0; } - /** - * Given a URL in 'str', this function split in the - * common components: protocol, user, machine, etc. - * @param str the URL as string. - * @param protocol the string where this component will be placed. - * @param user the string where this component will be placed. - * @param machine the string where this component will be placed. - * @param port the string where this component will be placed. - * @param file the string where this component will be placed. - * @param relative the string where this component will be placed. - * @param params the string where this component will be placed. - * @return true (1) if parsing was madden or error (-1) if - any error is found. - */ - int URLSTR_Match_url ( char **protocol, - char **user, - char **machine, - int *port, - char **file, - char **relative, - char **params, - char **str ) - { - int ok ; - - ok = URLSTR_Match_protocol(protocol,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_user(user,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_machine(machine,*protocol,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_port(port,*protocol,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_file(file,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_relative(relative,str) ; - if (0 == ok) return (0) ; - ok = URLSTR_Match_params(params,str) ; - if (0 == ok) return (0) ; - return (1) ; - } - - /** - * Given a URL in 'urlstr', this function split in the - * common components: protocol, user, machine, etc. - * @param urlstr the URL as string. - * @param protocol the string where this component will be placed. - * @param user the string where this component will be placed. - * @param machine the string where this component will be placed. - * @param port the string where this component will be placed. - * @param file the string where this component will be placed. - * @param relative the string where this component will be placed. - * @param params the string where this component will be placed. - * @return true (1) if parsing was madden or error (-1) if - any error is found. - */ - int URLSTR_ParseURL - ( - /*IN */ char *urlstr, - /*OUT*/ char **protocol, - /*OUT*/ char **user, - /*OUT*/ char **machine, - /*OUT*/ int *port, - /*OUT*/ char **file, - /*OUT*/ char **relative, - /*OUT*/ char **params - ) - { - char *pch, *fch ; - int ok ; - - fch = pch = STRING_MISC_StrDup(urlstr) ; - if (NULL == pch) { - return (-1); + + int getDirWithURL(char *url, char *dir) + { + char dir_aux[PATH_MAX]; /* change for a const*/ + int i,j; + + getURLDir(url, dir_aux); + + i = 0; + while((dir_aux[i] != '\0')&&(dir_aux[i] == dir[i])) { + i++; + } + + + if(dir_aux[i] != '\0') { + return -1; + } + if(dir_aux[i] == dir[i]){ + //dir[0] = '\0'; /* or '/' */ + dir[0] = '/'; + dir[1] = '\0'; + return 0; + } + + j = 0; + while(dir[j+i]!='\0'){ + dir[j] = dir[j+i]; + j++; + } + + dir[j] = '\0'; + if((dir[j] == '/') &&(strlen(dir+j) == 1)) { + dir[j-1] = '\0'; } - ok = URLSTR_Match_url(protocol, - user, - machine, - port, - file, - relative, - params, - &pch) ; - free(fch) ; - return ok ; - } + return 0; + } - /* ...................................................................... */ + /* ................................................................... */ diff --git a/src/bypass/xpn_bypass.c b/src/bypass/xpn_bypass.c index 439532ed..6ca18add 100644 --- a/src/bypass/xpn_bypass.c +++ b/src/bypass/xpn_bypass.c @@ -255,7 +255,9 @@ DIR aux_dirp ; aux_dirp = *dir ; - aux_dirp.fd = aux_dirp.fd - PLUSXPN ; + + struct generic_fd virtual_fd = fdstable_get ( aux_dirp.fd ); + aux_dirp.fd = virtual_fd.real_fd; return aux_dirp ; } @@ -478,7 +480,10 @@ return ret; } - /*int __open_2(const char *path, int flags, ...) + +#ifndef HAVE_ICC + + int __open_2(const char *path, int flags, ...) { int fd, ret; va_list ap; @@ -522,7 +527,9 @@ debug_info("[bypass] << After __open_2.... %s\n", path); return ret; - }*/ + } + +#endif int creat(const char *path, mode_t mode) @@ -1563,6 +1570,28 @@ } } + char * __realpath_chk(const char * path, char * resolved_path, size_t resolved_len) + { + debug_info("[bypass] Before __realpath_chk...\n"); + debug_info("[bypass] Path %s\n", path); + + if (is_xpn_prefix(path)) + { + // We must initialize expand if it has not been initialized yet. + xpn_adaptor_keepInit (); + + debug_info("[bypass] Before __realpath_chk...\n"); + strcpy(resolved_path, path); + return resolved_path; + } + // Not an XPN partition. We must link with the standard library + else + { + debug_info("[bypass] Before dlsym_realpath...\n"); + return dlsym_realpath(path, resolved_path); + } + } + int fsync(int fd) //TODO { int ret = -1; diff --git a/src/mpi_server/mpi_server_comm.c b/src/mpi_server/mpi_server_comm.c index d71e1fe5..cd7daea0 100644 --- a/src/mpi_server/mpi_server_comm.c +++ b/src/mpi_server/mpi_server_comm.c @@ -29,7 +29,7 @@ int mpi_server_comm_init ( mpi_server_param_st *params ) { - int ret, provided ; + int ret, provided, claimed ; char serv_name [HOST_NAME_MAX]; //Get timestap @@ -39,10 +39,31 @@ DEBUG_BEGIN() ; // MPI_Init - ret = MPI_Init_thread(&(params->argc), &(params->argv), MPI_THREAD_MULTIPLE, &provided) ; - if (MPI_SUCCESS != ret) { - debug_error("Server[%d]: MPI_Init fails :-(", -1) ; - return -1 ; + + // Threads disable + if (!params->thread_mode) + { + ret = MPI_Init(&(params->argc), &(params->argv)); + if (MPI_SUCCESS != ret) + { + debug_error("Server[%d]: MPI_Init fails :-(", -1) ; + return -1 ; + } + } + // Threads enable + else + { + ret = MPI_Init_thread(&(params->argc), &(params->argv), MPI_THREAD_MULTIPLE, &provided); + if (MPI_SUCCESS != ret) + { + debug_error("Server[%d]: MPI_Init_thread fails :-(", -1) ; + return -1 ; + } + + MPI_Query_thread(&claimed); + if (claimed != MPI_THREAD_MULTIPLE) { + printf("MPI_Init_thread: your MPI implementation seem not supporting thereads\n") ; + } } // params->rank = comm_rank() @@ -73,10 +94,18 @@ if(strncasecmp(version,"Open MPI", strlen("Open MPI")) != 0) { - ret = ns_publish(params->dns_file, params->srv_name, params->port_name); - if (ret < 0) { - debug_error("Server[%d]: NS_PUBLISH fails :-(", params->rank) ; - return -1; + for (int j=0; j < params->size; j++) + { + if (j == params->rank) + { + ret = ns_publish(params->dns_file, params->srv_name, params->port_name); + if (ret < 0) { + debug_error("Server[%d]: NS_PUBLISH fails :-(", params->rank) ; + return -1; + } + } + + MPI_Barrier(MPI_COMM_WORLD); } } else diff --git a/src/mpi_server/mpi_server_ops.c b/src/mpi_server/mpi_server_ops.c index b23dfe1a..75199b97 100644 --- a/src/mpi_server/mpi_server_ops.c +++ b/src/mpi_server/mpi_server_ops.c @@ -104,7 +104,7 @@ void mpi_server_op_write_ws ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; void mpi_server_op_write_wos ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; void mpi_server_op_close_ws ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; - + void mpi_server_op_rm ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; void mpi_server_op_rename ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; void mpi_server_op_setattr ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; @@ -118,7 +118,7 @@ void mpi_server_op_flush ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; void mpi_server_op_preload ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; - + void mpi_server_op_getnodename ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; //NEW void mpi_server_op_fstat ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; //TODO: implement void mpi_server_op_getid ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id ) ; //TODO: call in switch @@ -191,7 +191,7 @@ mpi_server_op_close_ws(th->params, (MPI_Comm) th->sd, &head, th->rank_client_id) ; } break; - + // Metadata API case MPI_SERVER_RM_FILE: ret = mpi_server_comm_read_data(th->params, (MPI_Comm) th->sd, (char *)&(head.u_st_mpi_server_msg.op_rm), sizeof(struct st_mpi_server_rm), th->rank_client_id) ; @@ -719,8 +719,8 @@ void mpi_server_op_setattr ( mpi_server_param_st *params, - __attribute__((__unused__)) MPI_Comm sd, - struct st_mpi_server_msg *head, + __attribute__((__unused__)) MPI_Comm sd, + struct st_mpi_server_msg *head, __attribute__((__unused__)) int rank_client_id ) { @@ -790,7 +790,7 @@ if (ret != NULL){ ret_entry.end = 1; - ret_entry.ret = *ret; + ret_entry.ret = *ret; } else{ ret_entry.end = 0; @@ -839,13 +839,12 @@ { int ret; int fd_dest, fd_orig; - char *protocol; - char *user; - char *machine; - int port; - char *file; - char *relative; - char *params1; + char protocol[1024]; + char user[1024]; + char pass[1024]; + char machine[1024]; + char port[1024]; + char file[1024]; int BLOCKSIZE = head->u_st_mpi_server_msg.op_preload.block_size; char buffer [BLOCKSIZE]; @@ -856,7 +855,13 @@ return; } - ret = URLSTR_ParseURL(head->u_st_mpi_server_msg.op_preload.virtual_path, &protocol, &user, &machine, &port, &file, &relative, ¶ms1) ; + ret = ParseURL(head->u_st_mpi_server_msg.op_preload.virtual_path, + protocol, + user, + pass, + machine, + port, + file) ; // Create new file fd_dest = filesystem_creat(file, 0777) ; @@ -910,23 +915,28 @@ ret) ; return; } - + void mpi_server_op_flush ( mpi_server_param_st *params, MPI_Comm sd, struct st_mpi_server_msg *head, int rank_client_id) { int ret; int fd_dest, fd_orig; - char *protocol; - char *user; - char *machine; - int port; - char *file; - char *relative; - char *params1; + char protocol[1024]; + char user[1024]; + char pass[1024]; + char machine[1024]; + char port[1024]; + char file[1024]; int BLOCKSIZE = head->u_st_mpi_server_msg.op_flush.block_size; char buffer [BLOCKSIZE]; - ret = URLSTR_ParseURL(head->u_st_mpi_server_msg.op_flush.virtual_path, &protocol, &user, &machine, &port, &file, &relative, ¶ms1) ; + ret = ParseURL(head->u_st_mpi_server_msg.op_preload.virtual_path, + protocol, + user, + pass, + machine, + port, + file) ; // Open origin file fd_orig = filesystem_open(file, O_RDONLY) ; @@ -959,7 +969,7 @@ filesystem_lseek(fd_dest, cont, SEEK_SET) ; //TODO: check error write_bytes = filesystem_write(fd_dest, &buffer, read_bytes) ; - if (write_bytes==-1) { + if (write_bytes < 0) { return; } } @@ -980,10 +990,10 @@ //FS Metadata API - void mpi_server_op_getnodename ( + void mpi_server_op_getnodename ( mpi_server_param_st *params, - MPI_Comm sd, - __attribute__((__unused__)) struct st_mpi_server_msg *head, + MPI_Comm sd, + __attribute__((__unused__)) struct st_mpi_server_msg *head, int rank_client_id ) { @@ -995,7 +1005,7 @@ gethostname(serv_name, HOST_NAME_MAX) ; // - // head = head; // Avoid unused parameter + // head = head; // Avoid unused parameter // // show debug info diff --git a/src/mpi_server/mpi_server_params.c b/src/mpi_server/mpi_server_params.c index f4f04b74..39fbe03f 100644 --- a/src/mpi_server/mpi_server_params.c +++ b/src/mpi_server/mpi_server_params.c @@ -72,7 +72,7 @@ params->argv = argv ; params->size = 0 ; params->rank = 0 ; - params->thread_mode = TH_POOL ; + params->thread_mode = TH_NOT ; strcpy(params->port_name, "") ; strcpy(params->srv_name, "") ; strcpy(params->dirbase, MPI_SERVER_DIRBASE_DEFAULT) ; diff --git a/src/tcp_server/tcp_server_comm.c b/src/tcp_server/tcp_server_comm.c index 7fd2976d..c1fd4ed9 100644 --- a/src/tcp_server/tcp_server_comm.c +++ b/src/tcp_server/tcp_server_comm.c @@ -20,33 +20,28 @@ /* ... Include / Inclusion ........................................... */ -//#define DEBUG 1 + //#define DEBUG 1 -#include "tcp_server/tcp_server_comm.h" + #include "tcp_server/tcp_server_comm.h" -#ifdef HAVE_MOSQUITTO_H - struct mosquitto *mosqtcpserver; -#endif /* ... Functions / Funciones ......................................... */ + + // MOSQUITTO FILE #ifdef HAVE_MOSQUITTO_H -void on_message(struct mosquitto *mosqtcpserver, void *obj, const struct mosquitto_message *msg) +void on_message(struct mosquitto *mqtt, void *obj, const struct mosquitto_message *msg) { - printf("%s\n\n%s\n", msg->topic, (char *) msg->payload); + //tcp_server_comm_read_data(params, sd, buffer, to_write, rank_client_id); + //filesystem_lseek(fd, head -> u_st_tcp_server_msg.op_write.offset + cont, SEEK_SET); + + //req.size = filesystem_write(fd, buffer, to_write); + printf("%s\t%d\n\n", msg->topic, msg->payloadlen); } #endif - - - - - - - - int tcp_server_comm_init ( tcp_server_param_st * params ) { int ret, val ; @@ -115,31 +110,43 @@ int tcp_server_comm_init ( tcp_server_param_st * params ) mosquitto_lib_init(); - mosqtcpserver = mosquitto_new(NULL, true, NULL); - if(mosqtcpserver == NULL) + params -> mqtt = mosquitto_new(NULL, true, NULL); + + if(params -> mqtt == NULL) { fprintf(stderr, "Error: Out of memory.\n"); return 1; } - //mosquitto_connect_callback_set(mosqtcpserver, on_connect); - //mosquitto_subscribe_callback_set(mosqtcpserver, on_subscribe); - mosquitto_message_callback_set(mosqtcpserver, on_message); + //mosquitto_connect_callback_set(params -> mqtt, on_connect); + //mosquitto_subscribe_callback_set(params -> mqtt, on_subscribe); + mosquitto_message_callback_set(params -> mqtt, on_message); #ifndef MOSQ_OPT_TCP_NODELAY #define MOSQ_OPT_TCP_NODELAY 1 #endif - mosquitto_int_option(mosqtcpserver, MOSQ_OPT_TCP_NODELAY, 1); + mosquitto_int_option(params -> mqtt, MOSQ_OPT_TCP_NODELAY, 1); - int rc = mosquitto_connect(mosqtcpserver, "localhost", 1886, 0); + int rc = mosquitto_connect(params -> mqtt, "localhost", 1883, 0); if( rc != MOSQ_ERR_SUCCESS ) { - mosquitto_destroy(mosqtcpserver); + mosquitto_destroy(params -> mqtt); fprintf(stderr, "[%d]\tERROR INIT MOSQUITTO TCP_SERVER: %s\n", __LINE__, mosquitto_strerror(rc)); return 1; } - //mosquitto_loop_forever(mosqtcpserver, -1, 1); + + /* Run the network loop in a background thread, this call returns quickly. */ + rc = mosquitto_loop_start(params -> mqtt); + + if(rc != MOSQ_ERR_SUCCESS) + { + mosquitto_destroy(params -> mqtt); + fprintf(stderr, "Error: %s\n", mosquitto_strerror(rc)); + return 1; + } + + //mosquitto_loop_forever(params -> mqtt, -1, 1); printf("[%d]\tEND INIT MOSQUITTO TCP_SERVER\n\n", __LINE__); } @@ -202,6 +209,7 @@ int tcp_server_comm_destroy ( tcp_server_param_st * params ) debug_info("[%d]\tBEGIN DESTROY MOSQUITTO TCP_SERVER\n\n", __LINE__); mosquitto_lib_cleanup(); + mosquitto_loop_stop(params -> mqtt, true); debug_info("[%d]\tEND DESTROY MOSQUITTO TCP_SERVER\n\n", __LINE__); } diff --git a/src/tcp_server/tcp_server_ops.c b/src/tcp_server/tcp_server_ops.c index 35a6600e..0cdd3413 100644 --- a/src/tcp_server/tcp_server_ops.c +++ b/src/tcp_server/tcp_server_ops.c @@ -18,16 +18,15 @@ * */ -/* ... Include / Inclusion ........................................... */ + /* ... Include / Inclusion ........................................... */ -#include "tcp_server_ops.h" + #include "tcp_server_ops.h" -extern struct mosquitto *mosqtcpserver; + /* ... Functions / Funciones ......................................... */ -/* ... Functions / Funciones ......................................... */ - -char * tcp_server_op2string(int op_code) { +char * tcp_server_op2string(int op_code) +{ char * ret = "Unknown"; switch (op_code) { @@ -114,6 +113,7 @@ char * tcp_server_op2string(int op_code) { return ret; } + /* * OPERATIONAL FUNCTIONS */ @@ -310,20 +310,39 @@ int tcp_server_do_operation(struct st_th * th, int * the_end) return 0; } + // // File API // -// MOSQUITTO FILE - +void tcp_server_op_open_ws(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) // WS - With Session +{ + int fd; + char *s; + char *extra = "/#"; + char *sm = malloc(strlen(head -> u_st_tcp_server_msg.op_open.path) + strlen(extra) + 1); + strcpy(sm, head -> u_st_tcp_server_msg.op_open.path); + strcat(sm, extra); + /* + * MOSQUITTO OPEN FILE + */ + if ( params -> mosquitto_mode == 1 ) + { + #ifdef HAVE_MOSQUITTO_H + printf("[%d]\tBEGIN OPEN MOSQUITTO TCP_SERVER WS - %s - %ld\n\n", __LINE__, sm, strlen(sm)); + int rc = mosquitto_subscribe(params -> mqtt, NULL, sm, 0); + if(rc != MOSQ_ERR_SUCCESS) + { + fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); + mosquitto_disconnect(params -> mqtt); + } -void tcp_server_op_open_ws(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) // WS - With Session -{ - int fd; - char * s; + printf("[%d]\tEND OPEN MOSQUITTO TCP_SERVER WS - %s\n\n", __LINE__, sm); + #endif + } s = head -> u_st_tcp_server_msg.op_open.path; @@ -333,68 +352,87 @@ void tcp_server_op_open_ws(tcp_server_param_st * params, int sd, struct st_tcp_s tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) OPEN(%s)=%d\n", params -> srv_name, s, fd); + debug_info("[%d][TCP-SERVER-OPS] (ID=%s) OPEN(%s)=%d\n", __LINE__, params -> srv_name, s, fd); } + void tcp_server_op_open_wos(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) //WOS - Without Session { int fd; - char * s; + char *s; + char *extra = "/#"; + char *sm = malloc(strlen(head -> u_st_tcp_server_msg.op_open.path) + strlen(extra) + 1); + strcpy(sm, head -> u_st_tcp_server_msg.op_open.path); + strcat(sm, extra); s = head -> u_st_tcp_server_msg.op_open.path; - - /* * MOSQUITTO OPEN FILE */ - int rc = 0; + if ( params -> mosquitto_mode == 1 ) { #ifdef HAVE_MOSQUITTO_H - printf("[%d]\tBEGIN OPEN MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); + printf("[%d]\tBEGIN OPEN MOSQUITTO TCP_SERVER WOS - %s - %ld\n\n", __LINE__, sm, strlen(sm)); - rc = mosquitto_subscribe(mosqtcpserver, NULL, s, 0); + int rc = mosquitto_subscribe(params -> mqtt, NULL, sm, 0); if(rc != MOSQ_ERR_SUCCESS) { fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); - mosquitto_disconnect(mosqtcpserver); + mosquitto_disconnect(params -> mqtt); } - printf("[%d]\tEND OPEN MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); + printf("[%d]\tEND OPEN MOSQUITTO TCP_SERVER WOS - %s\n\n", __LINE__, sm); - printf("[%d]\tBEGIN CLOSE MOSQUITTO TCP_SERVER\n\n", __LINE__); + printf("[%d]\tBEGIN CLOSE OPEN MOSQUITTO TCP_SERVER - WOS \n\n", __LINE__); - rc = mosquitto_unsubscribe(mosqtcpserver, NULL, s); - if(rc != MOSQ_ERR_SUCCESS) - { - fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); - mosquitto_disconnect(mosqtcpserver); - } + mosquitto_unsubscribe(params -> mqtt, NULL, sm); + mosquitto_unsubscribe(params -> mqtt, NULL, s); - printf("[%d]\tEND CLOSE MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); - tcp_server_comm_write_data(params, sd, (char * ) & rc, sizeof(int), rank_client_id); + printf("[%d]\tEND CLOSE OPEN MOSQUITTO TCP_SERVER - WOS %s\n\n", __LINE__, s); #endif } - else - { - // do open - fd = filesystem_open(s, O_RDWR); - tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); + // do open + fd = filesystem_open(s, O_RDWR); + + tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); + + filesystem_close(fd); - filesystem_close(fd); - } // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) OPEN(%s)=%d\n", params -> srv_name, s, fd); + printf("[%d][TCP-SERVER-OPS] (ID=%s) OPEN(%s)=%d\n", __LINE__, params -> srv_name, s, fd); } + void tcp_server_op_creat_ws(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { - int fd; - char * s; + int fd, rc; + char *s; + char *extra = "/#"; + char *sm = malloc(strlen(head -> u_st_tcp_server_msg.op_creat.path) + strlen(extra) + 1); + strcpy(sm, head -> u_st_tcp_server_msg.op_creat.path); + strcat(sm, extra); + + if ( params -> mosquitto_mode == 1 ) + { + #ifdef HAVE_MOSQUITTO_H + printf("[%d]\tBEGIN CREAT MOSQUITTO TCP_SERVER WS - %s - %ld\n\n", __LINE__, sm, strlen(sm)); + + rc = mosquitto_subscribe(params -> mqtt, NULL, sm, 0); + + if(rc != MOSQ_ERR_SUCCESS) + { + fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); + mosquitto_disconnect(params -> mqtt); + } + + printf("[%d]\tEND CREAT MOSQUITTO TCP_SERVER WS - %s\n\n", __LINE__, sm); + #endif + } s = head -> u_st_tcp_server_msg.op_creat.path; @@ -408,54 +446,59 @@ void tcp_server_op_creat_ws(tcp_server_param_st * params, int sd, struct st_tcp_ tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) CREAT(%s)=%d\n", params -> srv_name, s, fd); + printf("[%d][TCP-SERVER-OPS] (ID=%s) CREAT(%s)=%d\n", __LINE__, params -> srv_name, s, fd); } + void tcp_server_op_creat_wos(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int fd; - char * s; + char *extra = "/#"; + char *s; + char *sm = malloc(strlen(head -> u_st_tcp_server_msg.op_creat.path) + strlen(extra) + 1); + strcpy(sm, head -> u_st_tcp_server_msg.op_creat.path); + strcat(sm, extra); s = head -> u_st_tcp_server_msg.op_creat.path; - // do creat - fd = filesystem_creat(s, 0770); // TODO: tcp_server_op_creat don't use 'mode' from client ? - if (fd < 0) { - filesystem_mkpath(s); - fd = filesystem_creat(s, 0770); - } + if ( params -> mosquitto_mode == 1 ) + { + #ifdef HAVE_MOSQUITTO_H + printf("[%d]\tBEGIN CREATE MOSQUITTO TCP_SERVER WOS - %s\n\n", __LINE__, s); - tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); + int rc = mosquitto_subscribe(params -> mqtt, NULL, sm, 0); - #ifdef HAVE_MOSQUITTO_H + if(rc != MOSQ_ERR_SUCCESS) + { + fprintf(stderr, "Error subscribing creat: %s\n", mosquitto_strerror(rc)); + mosquitto_disconnect(params -> mqtt); + } - printf("[%d]\tBEGIN CREATE MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); + printf("[%d]\tEND CREATE MOSQUITTO TCP_SERVER WOS - %s\n\n", __LINE__, s); - int rc; + printf("[%d]\tBEGIN CLOSE CREAT MOSQUITTO TCP_SERVER - WOS \n\n", __LINE__); - rc = mosquitto_subscribe(mosqtcpserver, NULL, s, 0); - if(rc != MOSQ_ERR_SUCCESS){ - fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); - mosquitto_disconnect(mosqtcpserver); - } + mosquitto_unsubscribe(params -> mqtt, NULL, sm); + mosquitto_unsubscribe(params -> mqtt, NULL, s); - printf("[%d]\tEND CREATE MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); - printf("[%d]\tBEGIN CLOSE MOSQUITTO TCP_SERVER\n\n", __LINE__); + printf("[%d]\tEND CLOSE CREAT MOSQUITTO TCP_SERVER - WOS %s \n\n", __LINE__, s); + #endif + } - rc = mosquitto_unsubscribe(mosqtcpserver, NULL, s); - if(rc != MOSQ_ERR_SUCCESS){ - fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); - mosquitto_disconnect(mosqtcpserver); + // do creat + fd = filesystem_creat(s, 0770); // TODO: tcp_server_op_creat don't use 'mode' from client ? + if (fd < 0) { + filesystem_mkpath(s); + fd = filesystem_creat(s, 0770); } - printf("[%d]\tEND CLOSE MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, s); + tcp_server_comm_write_data(params, sd, (char * ) & fd, sizeof(int), rank_client_id); - #endif filesystem_close(fd); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) CREAT(%s)=%d\n", params -> srv_name, s, fd); + printf("[%d][TCP-SERVER-OPS] (ID=%s) CREAT(%s)=%d\n", __LINE__, params -> srv_name, s, fd); } @@ -525,7 +568,9 @@ void tcp_server_op_read_ws(tcp_server_param_st * params, int sd, struct st_tcp_s params -> srv_name, head -> u_st_tcp_server_msg.op_read.fd, (int) head -> u_st_tcp_server_msg.op_read.offset, size); } -void tcp_server_op_read_wos(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { + +void tcp_server_op_read_wos(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) +{ struct st_tcp_server_read_req req; char * buffer; long size, diff, to_read, cont; @@ -604,90 +649,67 @@ void tcp_server_op_read_wos(tcp_server_param_st * params, int sd, struct st_tcp_ void tcp_server_op_write_ws(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { - struct st_tcp_server_write_req req; - char * buffer; - int size, diff, cont, to_write; - - debug_info("[TCP-SERVER-OPS] (ID=%s) begin write: fd %d ID=xn", params -> srv_name, head -> u_st_tcp_server_msg.op_write.fd); + if( params -> mosquitto_mode == 0 ) + { + printf("[TCP-SERVER-OPS] (ID=%s) begin write: fd %d ID=xn", params -> srv_name, head -> u_st_tcp_server_msg.op_write.fd); - // initialize counters - cont = 0; - size = (head -> u_st_tcp_server_msg.op_write.size); - if (size > MAX_BUFFER_SIZE) { - size = MAX_BUFFER_SIZE; - } - diff = head -> u_st_tcp_server_msg.op_read.size - cont; + struct st_tcp_server_write_req req; + char * buffer; + int size, diff, cont, to_write; - // malloc a buffer of size... - buffer = (char * ) malloc(size); - if (NULL == buffer) { - req.size = -1; // TODO: check in client that -1 is treated properly... :-) - tcp_server_comm_write_data(params, sd, (char * ) & req, sizeof(struct st_tcp_server_write_req), rank_client_id); - return; - } + debug_info("[TCP-SERVER-OPS] (ID=%s) begin write: fd %d ID=xn", params -> srv_name, head -> u_st_tcp_server_msg.op_write.fd); - // loop... - do { - if (diff > size) to_write = size; - else to_write = diff; - - // read data from TCP and write into the file - tcp_server_comm_read_data(params, sd, buffer, to_write, rank_client_id); - filesystem_lseek(head -> u_st_tcp_server_msg.op_write.fd, head -> u_st_tcp_server_msg.op_write.offset + cont, SEEK_SET); - //sem_wait(&disk_sem); - req.size = filesystem_write(head -> u_st_tcp_server_msg.op_write.fd, buffer, to_write); - //sem_post(&disk_sem); - - // update counters - cont = cont + req.size; // Received bytes + // initialize counters + cont = 0; + size = (head -> u_st_tcp_server_msg.op_write.size); + if (size > MAX_BUFFER_SIZE) { + size = MAX_BUFFER_SIZE; + } diff = head -> u_st_tcp_server_msg.op_read.size - cont; - } while ((diff > 0) && (req.size != 0)); + // malloc a buffer of size... + buffer = (char * ) malloc(size); + if (NULL == buffer) { + req.size = -1; // TODO: check in client that -1 is treated properly... :-) + tcp_server_comm_write_data(params, sd, (char * ) & req, sizeof(struct st_tcp_server_write_req), rank_client_id); + return; + } - // write to the client the status of the write operation - req.size = cont; - tcp_server_comm_write_data(params, sd, (char * ) & req, sizeof(struct st_tcp_server_write_req), rank_client_id); + // loop... + do { + if (diff > size) to_write = size; + else to_write = diff; - // free buffer - FREE_AND_NULL(buffer); + // read data from TCP and write into the file + tcp_server_comm_read_data(params, sd, buffer, to_write, rank_client_id); + filesystem_lseek(head -> u_st_tcp_server_msg.op_write.fd, head -> u_st_tcp_server_msg.op_write.offset + cont, SEEK_SET); + //sem_wait(&disk_sem); + req.size = filesystem_write(head -> u_st_tcp_server_msg.op_write.fd, buffer, to_write); + //sem_post(&disk_sem); + + // update counters + cont = cont + req.size; // Received bytes + diff = head -> u_st_tcp_server_msg.op_read.size - cont; + + } while ((diff > 0) && (req.size != 0)); + + // write to the client the status of the write operation + req.size = cont; + tcp_server_comm_write_data(params, sd, (char * ) & req, sizeof(struct st_tcp_server_write_req), rank_client_id); + + // free buffer + FREE_AND_NULL(buffer); + } // for debugging purpouses debug_info("[TCP-SERVER-OPS] (ID=%s) end write: fd %d ID=xn", params -> srv_name, head -> u_st_tcp_server_msg.op_write.fd); } - void tcp_server_op_write_wos(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { - #ifdef HAVE_MOSQUITTO_H - int rc ; - - if ( params -> mosquitto_mode == 1 ) - { - // MOSQUITTO OPEN (SUBSCRIBE) - - printf("[%d]\tBEGIN OPEN MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, head -> u_st_tcp_server_msg.op_write.path); - - rc = mosquitto_subscribe(mosqtcpserver, NULL, head -> u_st_tcp_server_msg.op_write.path, 0); - if (rc != MOSQ_ERR_SUCCESS) - { - fprintf(stderr, "Error subscribing open: %s\n", mosquitto_strerror(rc)); - mosquitto_disconnect(mosqtcpserver); - } - - printf("[%d]\tEND OPEN MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, head -> u_st_tcp_server_msg.op_write.path); - - //mosquitto_message_callback_set(mosqtcpserver, on_message); - - tcp_server_comm_write_data(params, sd, (char * ) & rc, sizeof(int), rank_client_id); - } - #endif - - printf("%d\n", params -> mosquitto_mode); - if( params -> mosquitto_mode == 0 ) { - struct st_tcp_server_write_req req; char * buffer; int size, diff, cont, to_write; @@ -754,7 +776,6 @@ void tcp_server_op_write_wos(tcp_server_param_st * params, int sd, struct st_tcp } - void tcp_server_op_close_ws(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int ret = -1; @@ -765,13 +786,25 @@ void tcp_server_op_close_ws(tcp_server_param_st * params, int sd, struct st_tcp_ return; } - #ifdef HAVE_MOSQUITTO_H - printf("[%d]\tBEGIN CLOSE MOSQUITTO TCP_SERVER\n\n", __LINE__); + char *extra = "/#"; + char *s; + char *sm = malloc(strlen(head -> u_st_tcp_server_msg.op_close.path) + strlen(extra) + 1); + strcpy(sm, head -> u_st_tcp_server_msg.op_close.path); + strcat(sm, extra); + + s = head -> u_st_tcp_server_msg.op_close.path; - ret = mosquitto_unsubscribe(mosqtcpserver, NULL, head -> u_st_tcp_server_msg.op_write.path); + if( params -> mosquitto_mode == 1 ) + { + #ifdef HAVE_MOSQUITTO_H + printf("[%d]\tBEGIN CLOSE MOSQUITTO TCP_SERVER - WS \n\n", __LINE__); + + mosquitto_unsubscribe(params -> mqtt, NULL, sm); + mosquitto_unsubscribe(params -> mqtt, NULL, s); - printf("[%d]\tEND CLOSE MOSQUITTO TCP_SERVER - %s\n\n", __LINE__, head -> u_st_tcp_server_msg.op_write.path); - #endif + printf("[%d]\tEND CLOSE MOSQUITTO TCP_SERVER - WS %s\n\n", __LINE__, sm); + #endif + } // do close if (head -> u_st_tcp_server_msg.op_close.fd != -1 && params -> mosquitto_mode == 0 ) @@ -782,11 +815,10 @@ void tcp_server_op_close_ws(tcp_server_param_st * params, int sd, struct st_tcp_ tcp_server_comm_write_data(params, sd, (char * ) & ret, sizeof(int), rank_client_id); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) CLOSE(fd=%d)\n", params -> srv_name, head -> u_st_tcp_server_msg.op_close.fd); + debug_info("[TCP-SERVER-OPS] (ID=%s) CLOSE(fd=%d, path=%s)\n", params -> srv_name, head -> u_st_tcp_server_msg.op_close.fd, head -> u_st_tcp_server_msg.op_close.path); } - void tcp_server_op_rm(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { char * s; @@ -808,7 +840,6 @@ void tcp_server_op_rm(tcp_server_param_st * params, int sd, struct st_tcp_server } - void tcp_server_op_rename(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { char * old_url; @@ -833,7 +864,6 @@ void tcp_server_op_rename(tcp_server_param_st * params, int sd, struct st_tcp_se } - void tcp_server_op_getattr(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { struct st_tcp_server_attr_req req; @@ -842,15 +872,13 @@ void tcp_server_op_getattr(tcp_server_param_st * params, int sd, struct st_tcp_s // do getattr s = head -> u_st_tcp_server_msg.op_getattr.path; req.status = filesystem_stat(s, & req.attr); - tcp_server_comm_write_data(params, sd, (char * ) & req, sizeof(struct st_tcp_server_attr_req), rank_client_id); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) GETATTR(%s)\n", params -> srv_name, head -> u_st_tcp_server_msg.op_getattr.path); + printf("[TCP-SERVER-OPS] (ID=%s) GETATTR(%s)\n", params -> srv_name, head -> u_st_tcp_server_msg.op_getattr.path); } - void tcp_server_op_setattr(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, __attribute__((__unused__)) int rank_client_id) { // check params... @@ -879,8 +907,6 @@ void tcp_server_op_setattr(tcp_server_param_st * params, int sd, struct st_tcp_s } - - //Directory API void tcp_server_op_mkdir(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { @@ -900,7 +926,6 @@ void tcp_server_op_mkdir(tcp_server_param_st * params, int sd, struct st_tcp_ser } - void tcp_server_op_opendir(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { DIR * ret; @@ -917,7 +942,6 @@ void tcp_server_op_opendir(tcp_server_param_st * params, int sd, struct st_tcp_s } - void tcp_server_op_readdir(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { struct dirent * ret; @@ -942,7 +966,6 @@ void tcp_server_op_readdir(tcp_server_param_st * params, int sd, struct st_tcp_s } - void tcp_server_op_closedir(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int ret; @@ -959,7 +982,6 @@ void tcp_server_op_closedir(tcp_server_param_st * params, int sd, struct st_tcp_ } - void tcp_server_op_rmdir(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int ret; @@ -976,20 +998,18 @@ void tcp_server_op_rmdir(tcp_server_param_st * params, int sd, struct st_tcp_ser } - //Optimization API void tcp_server_op_preload(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int ret; int fd_dest, fd_orig; - char * protocol; - char * user; - char * machine; - int port; - char * file; - char * relative; - char * params1; + char protocol[1024]; + char user[1024]; + char pass[1024]; + char machine[1024]; + char port[1024]; + char file[1024]; int BLOCKSIZE = head -> u_st_tcp_server_msg.op_preload.block_size; char buffer[BLOCKSIZE]; @@ -1002,7 +1022,13 @@ void tcp_server_op_preload(tcp_server_param_st * params, int sd, struct st_tcp_s return; } - ret = URLSTR_ParseURL(head -> u_st_tcp_server_msg.op_preload.virtual_path, & protocol, & user, & machine, & port, & file, & relative, & params1); + ret = ParseURL(head->u_st_tcp_server_msg.op_preload.virtual_path, + protocol, + user, + pass, + machine, + port, + file) ; // Create new file fd_dest = filesystem_creat(file, 0777); @@ -1058,30 +1084,33 @@ void tcp_server_op_preload(tcp_server_param_st * params, int sd, struct st_tcp_s tcp_server_comm_write_data(params, sd, (char * ) & cont, sizeof(int), rank_client_id); // show debug info - debug_info("[TCP-SERVER-OPS] (ID=%s) PRELOAD(%s,%s) -> %d\n", - params -> srv_name, head -> u_st_tcp_server_msg.op_preload.virtual_path, head -> u_st_tcp_server_msg.op_preload.storage_path, ret); + debug_info("[TCP-SERVER-OPS] (ID=%s) PRELOAD(%s,%s) -> %d\n", params -> srv_name, head -> u_st_tcp_server_msg.op_preload.virtual_path, head -> u_st_tcp_server_msg.op_preload.storage_path, ret); return; } - void tcp_server_op_flush(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { int ret; int fd_dest, fd_orig; - char * protocol; - char * user; - char * machine; - int port; - char * file; - char * relative; - char * params1; + char protocol[1024]; + char user[1024]; + char pass[1024]; + char machine[1024]; + char port[1024]; + char file[1024]; int BLOCKSIZE = head -> u_st_tcp_server_msg.op_flush.block_size; char buffer[BLOCKSIZE]; - ret = URLSTR_ParseURL(head -> u_st_tcp_server_msg.op_flush.virtual_path, & protocol, & user, & machine, & port, & file, & relative, & params1); + ret = ParseURL(head->u_st_tcp_server_msg.op_flush.virtual_path, + protocol, + user, + pass, + machine, + port, + file) ; // Open origin file fd_orig = filesystem_open(file, O_RDONLY); @@ -1141,8 +1170,9 @@ void tcp_server_op_flush(tcp_server_param_st * params, int sd, struct st_tcp_ser // show debug info debug_info("[TCP-SERVER-OPS] (ID=%s) FLUSH(%s)\n", params -> srv_name, head -> u_st_tcp_server_msg.op_flush.virtual_path); -} + return; +} //FS Metadata API @@ -1169,6 +1199,7 @@ void tcp_server_op_getnodename(tcp_server_param_st * params, int sd, __attribute DEBUG_END(); } + void tcp_server_op_getid(tcp_server_param_st * params, int sd, struct st_tcp_server_msg * head, int rank_client_id) { // do getid diff --git a/src/utils/make_bundle.sh b/src/utils/make_bundle.sh deleted file mode 100755 index aa5caa0f..00000000 --- a/src/utils/make_bundle.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/sh -#set -x - -if [ $# -ge 1 ] -then - FILES=$@ -else - FILES=$(/bin/ls -1 ./*.c | sed s/.c$//g) -fi - -echo "Making bundle of $FILES" - -for f in $FILES -do - if [ ! -f "$f" ] - then - echo "Warning: Binary '$f' does not exist!" - fi -done - -LIBS=$(ldd $FILES | grep '=>' | cut -d '>' -f 2- | cut -d '(' -f 1 | sort | uniq) -if echo "$LIBS" | grep globus -then - LIBS="$LIBS /usr/lib/libglobus_thread_pthread.so" -fi - -mkdir xpn_utils_bundle -cp "$FILES" xpn_utils_bundle/ -cp "$LIBS" xpn_utils_bundle/ -tar zcvf xpn_utils_bundle.tar.gz xpn_utils_bundle/ -rm -rf xpn_utils_bundle/ - diff --git a/src/utils/xpn_cp.c b/src/utils/xpn_cp.c index c3f3baf9..d6ee9f56 100644 --- a/src/utils/xpn_cp.c +++ b/src/utils/xpn_cp.c @@ -1,76 +1,184 @@ -#include -#include -#include -#include -#include - -#define BUFFER_SIZE (8192*1024) - -int main(int argc, char *argv[]) -{ - - char buf[BUFFER_SIZE]; - int ret ,fd_src, fd_dest ; - - if(argc < 3) - { - printf("Too few arguments ./xpn_copy \n"); - return -1; - } - - fd_src = open(argv[1], O_RDONLY); - if(fd_src == -1){ - perror("open 1: "); - return -1; - } - - fd_dest = open(argv[2], O_WRONLY | O_CREAT | O_TRUNC, 0755); - if(fd_dest == -1) - { - perror("open 2: "); - return -1; - } - - int cont = 0; - int cont2 = 0; - int buf_len = BUFFER_SIZE; - - do - { - cont = 0; - buf_len = BUFFER_SIZE; - do { - ret = read(fd_src, buf + cont, buf_len); - if (ret == -1) - { - perror("read: "); - return -1; - } - - cont = cont + ret ; - buf_len = buf_len - ret ; - } while ( (cont < buf_len) && (ret > 0) ); - - cont2 = 0; - buf_len = cont; - - do { - ret = write(fd_dest, buf + cont2, buf_len); - if (ret == -1) - { - perror("write: "); - return -1; - } - - cont2 = cont2 + ret ; - buf_len = buf_len - ret ; - } while ( (cont2 < cont) && (ret > 0) ); - } - while(cont > 0); - - close(fd_src); - close(fd_dest); - - return 0; -} - + +/* + * Copyright 2020-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos + * + * This file is part of Expand. + * + * Expand is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Expand is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Expand. If not, see . + * + */ + + +/* ... Include / Inclusion ........................................... */ + + #include + #include + #include + #include + #include + #include + #include + +/* ... Const / Const ................................................. */ + + #define BUFFER_SIZE (8192*1024) + char buf[BUFFER_SIZE]; + +/* ... Functions / Funciones ......................................... */ + + int do_cp ( char *src_path, char *dest_path ) + { + int ret, fd_src, fd_dest ; + int cont = 0; + int cont2 = 0; + int buf_len = BUFFER_SIZE; + + fd_src = open(src_path, O_RDONLY); + if (fd_src < 0) { + perror("open source: "); + return -1; + } + + // TODO: mkdir any directory from dest_path before open it, just in case parent directories doesn't exists... + + fd_dest = open(dest_path, O_WRONLY | O_CREAT | O_TRUNC, 0755); + if (fd_dest < 0) + { + perror("open destination: "); + return -1; + } + + do + { + cont = 0; + buf_len = BUFFER_SIZE; + do { + ret = read(fd_src, buf + cont, buf_len); + if (ret == -1) + { + perror("read: "); + return -1; + } + + cont = cont + ret ; + buf_len = buf_len - ret ; + } while ( (cont < buf_len) && (ret > 0) ); + + cont2 = 0; + buf_len = cont; + + do { + ret = write(fd_dest, buf + cont2, buf_len); + if (ret == -1) + { + perror("write: "); + return -1; + } + + cont2 = cont2 + ret ; + buf_len = buf_len - ret ; + } while ( (cont2 < cont) && (ret > 0) ); + + } while(cont > 0); + + close(fd_src); + close(fd_dest); + + return 0; + } + + int do_cp_recursive ( char *src_path, char *dest_path ) + { + struct stat stat_buf; + int ret = 0; + + ret = stat(src_path, &stat_buf); + if (ret < 0) { + perror("stat source: "); + printf("ERROR at '%s': stat failed\n", src_path); + return -1; + } + + if (S_ISREG(stat_buf.st_mode)) + { + return do_cp(src_path, dest_path) ; + } + else if (S_ISDIR(stat_buf.st_mode)) + { + DIR * dir = NULL; + struct dirent * entry = NULL; + char path [PATH_MAX]; + + dir = opendir(src_path); + if (NULL == dir) { + perror("opendir:"); + return -1; + } + + entry = readdir(dir); + while(entry != NULL) + { + if (! strcmp(entry->d_name, ".")) { + entry = readdir(dir); + continue; + } + if (! strcmp(entry->d_name, "..")) { + entry = readdir(dir); + continue; + } + + sprintf(path, "%s/%s", src_path, entry->d_name); + printf("cp '%s' ...\n", path); + + do_cp_recursive(path, dest_path) ; + + entry = readdir(dir); + } + + closedir(dir); + } + else + { + printf("ERROR at '%s': not a directory or file\n", src_path); + } + + return ret; + } + + + int main ( int argc, char *argv[] ) + { + int ret; + + printf("\n"); + printf(" xpn_cp\n"); + printf(" --------\n"); + printf("\n"); + printf(" Usage:\n"); + printf(" %s ./xpn_copy \n", argv[0]) ; + printf("\n"); + + if (argc < 3) + { + printf(" >> ERROR: too few arguments :-(\n"); + printf("\n"); + return -1; + } + + ret = do_cp_recursive(argv[1], argv[2]) ; + + return ret ; + } + +/* ................................................................... */ diff --git a/src/utils/xpn_ls.c b/src/utils/xpn_ls.c index 37469df7..95f676fe 100644 --- a/src/utils/xpn_ls.c +++ b/src/utils/xpn_ls.c @@ -1,88 +1,118 @@ -#include -#include -#include -#include -#include -#include - -#include "mpi.h" - - -int list (char * dir_name, FILE * fd) -{ - int ret; - DIR* dir = NULL; - struct stat stat_buf; - char path [PATH_MAX]; - - dir = opendir(dir_name); - if(dir == NULL) - { - perror("opendir:"); - return -1; - } - - struct dirent* entry; - entry = readdir(dir); - - while(entry != NULL) - { - if (! strcmp(entry->d_name, ".")){ - entry = readdir(dir); - continue; - } - - if (! strcmp(entry->d_name, "..")){ - entry = readdir(dir); - continue; - } - - sprintf(path, "%s/%s", dir_name, entry->d_name); - fprintf(fd, "%s\n", path); - - ret = stat(path, &stat_buf); - if (ret < 0) { - perror("stat: "); - printf("%s\n", path); - entry = readdir(dir); - continue; - } - - if (S_ISDIR(stat_buf.st_mode)) - { - list(path, fd); - } - - entry = readdir(dir); - } - - closedir(dir); - - return 0; -} - -int main(int argc, char *argv[]) -{ - FILE * fd; - - if(argc < 3){ - printf("Too few arguments: xpn_ls \n"); - return -1; - } - - MPI_Init(&argc, &argv); - - fd = fopen(argv[2], "w"); - if ( fd == NULL ) - { - perror("fopen: "); - return -1; - } - - list(argv[1], fd); - - MPI_Barrier(MPI_COMM_WORLD); - MPI_Finalize(); - - return 0; -} \ No newline at end of file + +/* + * Copyright 2020-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos + * + * This file is part of Expand. + * + * Expand is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Expand is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Expand. If not, see . + * + */ + +/* ... Include / Inclusion ........................................... */ + + #include + #include + #include + #include + #include + #include + #include "mpi.h" + +/* ... Const / Const ................................................. */ + + + +/* ... Functions / Funciones ......................................... */ + + + int list (char * dir_name, FILE * fd) + { + int ret; + DIR* dir = NULL; + struct stat stat_buf; + char path [PATH_MAX]; + + dir = opendir(dir_name); + if(dir == NULL) + { + perror("opendir:"); + return -1; + } + + struct dirent* entry; + entry = readdir(dir); + + while(entry != NULL) + { + if (! strcmp(entry->d_name, ".")){ + entry = readdir(dir); + continue; + } + + if (! strcmp(entry->d_name, "..")){ + entry = readdir(dir); + continue; + } + + sprintf(path, "%s/%s", dir_name, entry->d_name); + fprintf(fd, "%s\n", path); + + ret = stat(path, &stat_buf); + if (ret < 0) { + perror("stat: "); + printf("%s\n", path); + entry = readdir(dir); + continue; + } + + if (S_ISDIR(stat_buf.st_mode)) + { + list(path, fd); + } + + entry = readdir(dir); + } + + closedir(dir); + + return 0; + } + + int main(int argc, char *argv[]) + { + FILE * fd; + + if(argc < 3){ + printf("Too few arguments: xpn_ls \n"); + return -1; + } + + MPI_Init(&argc, &argv); + + fd = fopen(argv[2], "w"); + if ( fd == NULL ) + { + perror("fopen: "); + return -1; + } + + list(argv[1], fd); + + MPI_Barrier(MPI_COMM_WORLD); + MPI_Finalize(); + + return 0; + } + +/* ................................................................... */ diff --git a/src/utils/xpn_rebuild.c b/src/utils/xpn_rebuild.c index 54824c1f..e6607ac4 100644 --- a/src/utils/xpn_rebuild.c +++ b/src/utils/xpn_rebuild.c @@ -1,208 +1,242 @@ -#define _LARGEFILE_SOURCE -#define _FILE_OFFSET_BITS 64 - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "mpi.h" - -#define MIN(a,b) (((a)<(b))?(a):(b)) -#define HEADER_SIZE (0) - -char command[4*1024]; -char src_path [PATH_MAX+5]; -char dest_path [PATH_MAX+5]; - -int copy(char * entry, char * dir_name, char * dest_prefix, int blocksize, int rank, int size) -{ - int ret; - - //FILE *file = NULL; - struct stat stat_buf; - - int fd_src, fd_dest; - char *buf ; - int buf_len; - off64_t offset_src ; - int cont, cont2 ; - - buf = (char *) malloc(blocksize + 1) ; - if (NULL == buf) { - perror("malloc: "); - return -1; - } - - //Generate source path - //sprintf( src_path, "%s/%s", argv[1], entry ); - strcpy(src_path, entry); - ret = stat(src_path, &stat_buf); - if (ret < 0) { - perror("stat: "); - printf("[ERROR] %s\n", src_path); - return -1; - } - - //Generate destination path - char * aux_entry = entry + strlen(dir_name); - sprintf( dest_path, "%s/%s", dest_prefix, aux_entry ); - - if (S_ISDIR(stat_buf.st_mode)) - { - ret = mkdir(dest_path, 0755); - if ( ret < 0 ) - { - perror("mkdir: "); +/* + * Copyright 2020-2023 Felix Garcia Carballeira, Diego Camarmas Alonso, Alejandro Calderon Mateos + * + * This file is part of Expand. + * + * Expand is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Expand is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Expand. If not, see . + * + */ + +/* ... Include / Inclusion ........................................... */ + + #include + #include + #include + #include + #include + #include + #include + #include + #include + #include "mpi.h" + + +/* ... Const / Const ................................................. */ + + #ifndef _LARGEFILE_SOURCE + #define _LARGEFILE_SOURCE + #endif + + #ifndef _FILE_OFFSET_BITS + #define _FILE_OFFSET_BITS 64 + #endif + + #define MIN(a,b) (((a)<(b))?(a):(b)) + #define HEADER_SIZE (0) + + char command[4*1024]; + char src_path [PATH_MAX+5]; + char dest_path [PATH_MAX+5]; + + +/* ... Functions / Funciones ......................................... */ + + int copy(char * entry, char * dir_name, char * dest_prefix, int blocksize, int rank, int size) + { + int ret; + + //FILE *file = NULL; + struct stat stat_buf; + + int fd_src, fd_dest; + char *buf ; + int buf_len; + off64_t offset_src ; + int cont, cont2 ; + + buf = (char *) malloc(blocksize + 1) ; + if (NULL == buf) { + perror("malloc: "); return -1; } - } - else if (S_ISREG(stat_buf.st_mode)) - { - fd_src = open64(src_path, O_RDONLY | O_LARGEFILE); - if ( fd_src < 0 ) - { - perror("open 2: "); + + //Generate source path + //sprintf( src_path, "%s/%s", argv[1], entry ); + strcpy(src_path, entry); + ret = stat(src_path, &stat_buf); + if (ret < 0) { + perror("stat: "); + printf("[ERROR] %s\n", src_path); return -1; } - fd_dest = open64(dest_path, O_CREAT | O_WRONLY | O_TRUNC | O_LARGEFILE, 0755); - if ( fd_dest < 0 ) + //Generate destination path + char * aux_entry = entry + strlen(dir_name); + sprintf( dest_path, "%s/%s", dest_prefix, aux_entry ); + + if (S_ISDIR(stat_buf.st_mode)) { - perror("open 1: "); - return -1; + ret = mkdir(dest_path, 0755); + if ( ret < 0 ) + { + perror("mkdir: "); + return -1; + } } + else if (S_ISREG(stat_buf.st_mode)) + { + fd_src = open64(src_path, O_RDONLY | O_LARGEFILE); + if ( fd_src < 0 ) + { + perror("open 2: "); + return -1; + } + + fd_dest = open64(dest_path, O_CREAT | O_WRONLY | O_TRUNC | O_LARGEFILE, 0755); + if ( fd_dest < 0 ) + { + perror("open 1: "); + return -1; + } - // Write header - ret = write(fd_dest, buf, HEADER_SIZE); // TODO: buf MUST be the header - - offset_src = rank * blocksize ; - do - { - off64_t ret_2; - ret_2 = lseek64(fd_src, offset_src, SEEK_SET) ; - if (ret_2 < 0) { - //perror("lseek: "); - break; + // Write header + ret = write(fd_dest, buf, HEADER_SIZE); // TODO: buf MUST be the header + + offset_src = rank * blocksize ; + do + { + off64_t ret_2; + ret_2 = lseek64(fd_src, offset_src, SEEK_SET) ; + if (ret_2 < 0) { + //perror("lseek: "); + break; + } + + cont = 0; + buf_len = blocksize; + memset(buf, 0, buf_len); + do { + ret = read(fd_src, buf + cont, buf_len); + cont = cont + ret ; + buf_len = buf_len - ret ; + } while ( (cont < buf_len) && (ret != 0) ); + + cont2 = 0; + buf_len = cont; + do { + ret = write(fd_dest, buf + cont2, buf_len); + cont2 = cont2 + ret ; + buf_len = buf_len - ret ; + } while ( (cont2 < cont) && (ret != 0) ); + + //printf("rank %d; ret: %d; offset %ld; nodes %d; blocksize %d\n", rank, ret, offset_src, size, blocksize); + //printf("Buf: %s\n", buf); + + offset_src = offset_src + (size * blocksize) ; } + while(cont > 0); - cont = 0; - buf_len = blocksize; - memset(buf, 0, buf_len); - do { - ret = read(fd_src, buf + cont, buf_len); - cont = cont + ret ; - buf_len = buf_len - ret ; - } while ( (cont < buf_len) && (ret != 0) ); - - cont2 = 0; - buf_len = cont; - do { - ret = write(fd_dest, buf + cont2, buf_len); - cont2 = cont2 + ret ; - buf_len = buf_len - ret ; - } while ( (cont2 < cont) && (ret != 0) ); - - //printf("rank %d; ret: %d; offset %ld; nodes %d; blocksize %d\n", rank, ret, offset_src, size, blocksize); - //printf("Buf: %s\n", buf); - - offset_src = offset_src + (size * blocksize) ; + close(fd_src); + close(fd_dest); } - while(cont > 0); + + free(buf); - close(fd_src); - close(fd_dest); + return 0; } - - free(buf); - return 0; -} - -int list (char * dir_name, char * dest_prefix, int blocksize, int rank, int size) -{ - int ret; - DIR* dir = NULL; - struct stat stat_buf; - char path [PATH_MAX]; - - dir = opendir(dir_name); - if(dir == NULL) + int list (char * dir_name, char * dest_prefix, int blocksize, int rank, int size) { - perror("opendir:"); - return -1; - } - - struct dirent* entry; - entry = readdir(dir); + int ret; + DIR* dir = NULL; + struct stat stat_buf; + char path [PATH_MAX]; - while(entry != NULL) - { - if (! strcmp(entry->d_name, ".")){ - entry = readdir(dir); - continue; + dir = opendir(dir_name); + if(dir == NULL) + { + perror("opendir:"); + return -1; } + + struct dirent* entry; + entry = readdir(dir); - if (! strcmp(entry->d_name, "..")){ - entry = readdir(dir); - continue; - } + while(entry != NULL) + { + if (! strcmp(entry->d_name, ".")){ + entry = readdir(dir); + continue; + } - sprintf(path, "%s/%s", dir_name, entry->d_name); - copy(path, dir_name, dest_prefix, blocksize, rank, size); + if (! strcmp(entry->d_name, "..")){ + entry = readdir(dir); + continue; + } - ret = stat(path, &stat_buf); - if (ret < 0) { + sprintf(path, "%s/%s", dir_name, entry->d_name); + copy(path, dir_name, dest_prefix, blocksize, rank, size); + + ret = stat(path, &stat_buf); + if (ret < 0) { perror("stat: "); printf("%s\n", path); entry = readdir(dir); continue; - } + } - if (S_ISDIR(stat_buf.st_mode)) - { - list(path, dest_prefix, blocksize, rank, size); + if (S_ISDIR(stat_buf.st_mode)) + { + list(path, dest_prefix, blocksize, rank, size); + } + + entry = readdir(dir); } - entry = readdir(dir); - } + closedir(dir); - closedir(dir); + return 0; + } - return 0; -} + int main(int argc, char *argv[]) + { + int rank, size; -int main(int argc, char *argv[]) -{ - int rank, size; + // + // Check arguments... + // + if ( argc < 4 ) + { + printf("Usage:\n"); + printf(" ./%s \n", argv[0]); + printf("\n"); + return -1; + } - // - // Check arguments... - // - if ( argc < 4 ) - { - printf("Usage:\n"); - printf(" ./%s \n", argv[0]); - printf("\n"); - return -1; - } + MPI_Init(&argc, &argv); + MPI_Comm_rank(MPI_COMM_WORLD, &rank); + MPI_Comm_size(MPI_COMM_WORLD, &size); - MPI_Init(&argc, &argv); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); + list (argv[1], argv[2], atoi(argv[3]), rank, size); - list (argv[1], argv[2], atoi(argv[3]), rank, size); + MPI_Barrier(MPI_COMM_WORLD); + MPI_Finalize(); - MPI_Barrier(MPI_COMM_WORLD); - MPI_Finalize(); + return 0; + } - return 0; -} +/* ................................................................... */ diff --git a/src/xpn_client/Makefile.am b/src/xpn_client/Makefile.am index 20ae040b..de6f23af 100644 --- a/src/xpn_client/Makefile.am +++ b/src/xpn_client/Makefile.am @@ -18,6 +18,7 @@ BASE_HEADER= @top_srcdir@/include/base/base_lib.h \ @top_srcdir@/include/base/dtable.h \ @top_srcdir@/include/base/math_misc.h \ @top_srcdir@/include/base/string_misc.h \ + @top_srcdir@/include/base/path_misc.h \ @top_srcdir@/include/base/time_misc.h \ @top_srcdir@/include/base/trace_msg.h \ @top_srcdir@/include/base/trace_tags.h \ @@ -125,6 +126,7 @@ BASE_OBJECTS= @top_srcdir@/src/base/darray.c \ @top_srcdir@/src/base/trace_msg.c \ @top_srcdir@/src/base/debug_msg.c \ @top_srcdir@/src/base/string_misc.c \ + @top_srcdir@/src/base/path_misc.c \ @top_srcdir@/src/base/time_misc.c \ @top_srcdir@/src/base/urlstr.c \ @top_srcdir@/src/base/ns.c \ diff --git a/src/xpn_client/Makefile.in b/src/xpn_client/Makefile.in index 3ea2b82c..6ba5fee7 100644 --- a/src/xpn_client/Makefile.in +++ b/src/xpn_client/Makefile.in @@ -173,6 +173,7 @@ am__libxpn_a_SOURCES_DIST = @top_srcdir@/src/base/darray.c \ @top_srcdir@/src/base/trace_msg.c \ @top_srcdir@/src/base/debug_msg.c \ @top_srcdir@/src/base/string_misc.c \ + @top_srcdir@/src/base/path_misc.c \ @top_srcdir@/src/base/time_misc.c \ @top_srcdir@/src/base/urlstr.c @top_srcdir@/src/base/ns.c \ @top_srcdir@/src/base/ns_tcp.c \ @@ -225,6 +226,7 @@ am__objects_1 = @top_srcdir@/src/base/darray.$(OBJEXT) \ @top_srcdir@/src/base/trace_msg.$(OBJEXT) \ @top_srcdir@/src/base/debug_msg.$(OBJEXT) \ @top_srcdir@/src/base/string_misc.$(OBJEXT) \ + @top_srcdir@/src/base/path_misc.$(OBJEXT) \ @top_srcdir@/src/base/time_misc.$(OBJEXT) \ @top_srcdir@/src/base/urlstr.$(OBJEXT) \ @top_srcdir@/src/base/ns.$(OBJEXT) \ @@ -307,6 +309,7 @@ am__depfiles_remade = @top_srcdir@/src/base/$(DEPDIR)/darray.Po \ @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/ns.Po \ @top_srcdir@/src/base/$(DEPDIR)/ns_tcp.Po \ + @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po \ @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po \ @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po \ @@ -386,6 +389,7 @@ am__nobase_include_HEADERS_DIST = @top_srcdir@/include/config.h \ @top_srcdir@/include/base/dtable.h \ @top_srcdir@/include/base/math_misc.h \ @top_srcdir@/include/base/string_misc.h \ + @top_srcdir@/include/base/path_misc.h \ @top_srcdir@/include/base/time_misc.h \ @top_srcdir@/include/base/trace_msg.h \ @top_srcdir@/include/base/trace_tags.h \ @@ -601,6 +605,7 @@ BASE_HEADER = @top_srcdir@/include/base/base_lib.h \ @top_srcdir@/include/base/dtable.h \ @top_srcdir@/include/base/math_misc.h \ @top_srcdir@/include/base/string_misc.h \ + @top_srcdir@/include/base/path_misc.h \ @top_srcdir@/include/base/time_misc.h \ @top_srcdir@/include/base/trace_msg.h \ @top_srcdir@/include/base/trace_tags.h \ @@ -671,6 +676,7 @@ BASE_OBJECTS = @top_srcdir@/src/base/darray.c \ @top_srcdir@/src/base/trace_msg.c \ @top_srcdir@/src/base/debug_msg.c \ @top_srcdir@/src/base/string_misc.c \ + @top_srcdir@/src/base/path_misc.c \ @top_srcdir@/src/base/time_misc.c \ @top_srcdir@/src/base/urlstr.c \ @top_srcdir@/src/base/ns.c \ @@ -814,6 +820,9 @@ clean-libLIBRARIES: @top_srcdir@/src/base/string_misc.$(OBJEXT): \ @top_srcdir@/src/base/$(am__dirstamp) \ @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) +@top_srcdir@/src/base/path_misc.$(OBJEXT): \ + @top_srcdir@/src/base/$(am__dirstamp) \ + @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) @top_srcdir@/src/base/time_misc.$(OBJEXT): \ @top_srcdir@/src/base/$(am__dirstamp) \ @top_srcdir@/src/base/$(DEPDIR)/$(am__dirstamp) @@ -1015,6 +1024,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/math_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/ns.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/ns_tcp.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/path_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/string_misc.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@@top_srcdir@/src/base/$(DEPDIR)/time_misc.Po@am__quote@ # am--include-marker @@ -1261,6 +1271,7 @@ distclean: distclean-am -rm -f @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns_tcp.Po + -rm -f @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po @@ -1359,6 +1370,7 @@ maintainer-clean: maintainer-clean-am -rm -f @top_srcdir@/src/base/$(DEPDIR)/math_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/ns_tcp.Po + -rm -f @top_srcdir@/src/base/$(DEPDIR)/path_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/string_misc.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/syscall_proxies.Po -rm -f @top_srcdir@/src/base/$(DEPDIR)/time_misc.Po diff --git a/src/xpn_client/nfi/nfi_local/nfi_local.c b/src/xpn_client/nfi/nfi_local/nfi_local.c index bc0f4ff8..cb5fc2b7 100644 --- a/src/xpn_client/nfi/nfi_local/nfi_local.c +++ b/src/xpn_client/nfi/nfi_local/nfi_local.c @@ -168,7 +168,7 @@ serv->ops->nfi_statfs = nfi_local_statfs; // ParseURL... - ret = ParseURL(url, prt, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, prt, NULL, NULL, server, NULL, dir) ; if (ret < 0) { // local_err(LOCALERR_URL) ; @@ -202,15 +202,15 @@ strcpy(server_aux->path, dir) ; serv->private_info = (void *)server_aux; - serv->server = STRING_MISC_StrDup(server) ; // server addrets - if (serv->server == NULL) + serv->url = STRING_MISC_StrDup(url) ; // full URL + if (serv->url == NULL) { debug_error("ERROR: out of memory\n") ; return -1; } - serv->url = STRING_MISC_StrDup(url) ; // server addrets - if (serv->url == NULL) + serv->server = STRING_MISC_StrDup(server) ; // URL.server + if (serv->server == NULL) { debug_error("ERROR: out of memory\n") ; return -1; @@ -335,7 +335,7 @@ { // Don't see the serv retult int ret; - char server[PATH_MAX], dir[PATH_MAX]; + char dir[PATH_MAX]; struct nfi_local_server *server_aux; DEBUG_BEGIN(); @@ -350,7 +350,7 @@ return -1; } - ret = ParseURL(serv->url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(serv->url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { debug_error("nfi_local_reconnect: incorrect '%s' URL.\n", serv->url) ; return -1; @@ -379,7 +379,7 @@ int nfi_local_open ( struct nfi_server *serv, char *url, struct nfi_fhandle *fho ) { int ret; - char dir[PATH_MAX], server[PATH_MAX]; + char dir[PATH_MAX] ; struct nfi_local_fhandle *fh_aux; DEBUG_BEGIN(); @@ -391,7 +391,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // from url -> server + dir - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir); + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir); if (ret < 0) { fprintf(stderr,"nfi_local_open: url %s incorrect.\n",url) ; @@ -436,7 +436,7 @@ int nfi_local_create ( struct nfi_server *serv, char *url, struct nfi_attr *attr, struct nfi_fhandle *fh ) { int ret; - char server[PATH_MAX], dir[PATH_MAX]; + char dir[PATH_MAX]; struct nfi_local_fhandle *fh_aux; struct stat st; @@ -449,7 +449,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // url -> server + dir - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { fprintf(stderr,"ERROR: nfi_local_create: url %s incorrect.\n",url) ; @@ -607,7 +607,7 @@ int nfi_local_remove ( struct nfi_server *serv, char *url ) { int ret; - char server[PATH_MAX], dir[PATH_MAX]; + char dir[PATH_MAX]; DEBUG_BEGIN(); @@ -618,7 +618,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // from url -> server + dir - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { fprintf(stderr,"nfi_local_remove: url %s incorrect.\n",url) ; @@ -643,8 +643,8 @@ int nfi_local_rename (struct nfi_server *serv, char *old_url, char *new_url ) { - int ret; - char server[PATH_MAX], old_path[PATH_MAX], new_path[PATH_MAX]; + int ret; + char old_path[PATH_MAX], new_path[PATH_MAX]; DEBUG_BEGIN(); @@ -656,7 +656,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // Get fields... - ret = ParseURL(old_url, NULL, NULL, NULL, server, NULL, old_path); + ret = ParseURL(old_url, NULL, NULL, NULL, NULL, NULL, old_path); if (ret < 0) { fprintf(stderr,"nfi_local_open: url %s incorrect.\n",old_url) ; @@ -664,7 +664,7 @@ return -1; } - ret = ParseURL(new_url, NULL, NULL, NULL, server, NULL, new_path); + ret = ParseURL(new_url, NULL, NULL, NULL, NULL, NULL, new_path); if (ret < 0) { fprintf(stderr,"nfi_local_open: url %s incorrect.\n",new_path) ; @@ -688,8 +688,8 @@ int nfi_local_getattr ( struct nfi_server *serv, struct nfi_fhandle *fh, struct nfi_attr *attr ) { - int ret; - char server[PATH_MAX], dir[PATH_MAX]; + int ret; + char dir[PATH_MAX]; //struct nfi_local_fhandle *fh_aux; struct stat st; @@ -705,10 +705,10 @@ // copy private information... //fh_aux = (struct nfi_local_fhandle *) fh->priv_fh; - ret = ParseURL(fh->url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(fh->url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { - fprintf(stderr,"nfi_mpi_server_getattr: url %s incorrect.\n",dir) ; + fprintf(stderr,"nfi_mpi_server_getattr: url %s incorrect.\n", dir) ; // mpi_server_err(LOCALERR_URL) ; return -1; } @@ -729,6 +729,7 @@ return 0; } + int nfi_local_setattr ( struct nfi_server *serv, struct nfi_fhandle *fh, struct nfi_attr *attr ) { struct stat st; @@ -756,8 +757,8 @@ int nfi_local_mkdir ( struct nfi_server *serv, char *url, struct nfi_attr *attr, struct nfi_fhandle *fh ) { - int ret; - char server[PATH_MAX], dir[PATH_MAX]; + int ret; + char dir[PATH_MAX]; struct nfi_local_fhandle *fh_aux; struct stat st; @@ -770,7 +771,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // Get fields... - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { fprintf(stderr,"nfi_local_mkdir: url %s incorrect.\n",url) ; @@ -821,8 +822,8 @@ int nfi_local_opendir ( struct nfi_server *serv, char *url, struct nfi_fhandle *fho ) { - int ret; - char dir[PATH_MAX], server[PATH_MAX]; + int ret; + char dir[PATH_MAX]; struct nfi_local_fhandle *fh_aux; DEBUG_BEGIN(); @@ -835,7 +836,7 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // Get fields... - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { fprintf(stderr,"nfi_local_opendir: url %s incorrect.\n",url) ; @@ -946,7 +947,7 @@ int nfi_local_rmdir ( struct nfi_server *serv, char *url ) { int ret; - char server[PATH_MAX], dir[PATH_MAX]; + char dir[PATH_MAX]; DEBUG_BEGIN(); @@ -957,10 +958,10 @@ NULL_RET_ERR(serv->private_info, LOCALERR_PARAM) ; // Get fields... - ret = ParseURL(url, NULL, NULL, NULL, server, NULL, dir) ; + ret = ParseURL(url, NULL, NULL, NULL, NULL, NULL, dir) ; if (ret < 0) { - fprintf(stderr,"nfi_local_rmdir: url %s incorrect.\n",url) ; + fprintf(stderr,"nfi_local_rmdir: url %s incorrect.\n", url) ; local_err(LOCALERR_URL) ; return -1; } @@ -1043,4 +1044,4 @@ /* ................................................................... */ - + diff --git a/src/xpn_client/nfi/nfi_mpi_server/mpi_server_comm.c b/src/xpn_client/nfi/nfi_mpi_server/mpi_server_comm.c index 4b562575..2979403c 100644 --- a/src/xpn_client/nfi/nfi_mpi_server/mpi_server_comm.c +++ b/src/xpn_client/nfi/nfi_mpi_server/mpi_server_comm.c @@ -62,7 +62,7 @@ MPI_Query_thread(&claimed); if (claimed != MPI_THREAD_MULTIPLE) { - printf("MPI_Init: your MPI implementation seem not supporting thereads\n") ; + printf("MPI_Init_thread: your MPI implementation seem not supporting thereads\n") ; } } } diff --git a/src/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.c b/src/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.c index 3e571614..a3f78a26 100644 --- a/src/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.c +++ b/src/xpn_client/nfi/nfi_tcp_server/nfi_tcp_server.c @@ -18,26 +18,71 @@ * */ -/* ... Include / Inclusion ........................................... */ -//#define DEBUG 1 -#include "nfi_tcp_server.h" + /* ... Include / Inclusion ........................................... */ + //#define DEBUG 1 + #include "nfi_tcp_server.h" -/* ... Global Variable / Variable Globales ........................... */ -#define FILESYSTEM_DLSYM 1 + /* ... Global Variable / Variable Globales ........................... */ -#ifdef HAVE_MOSQUITTO_H -struct mosquitto * mosqstr; -#endif + #define FILESYSTEM_DLSYM 1 + +/*#ifdef HAVE_MOSQUITTO_H +#define MAX_MQTT_BROKERS 10 + +struct Broker { + struct mosquitto * mosqstr; + int init; + char ip[256]; +}; + +struct Broker brokers[MAX_BROKERS]; + +//struct mosquitto * mosqstr; +#endif*/ + + /* ... Functions / Funciones ......................................... */ + +/* + +int find_pos_mqtt ( void ) +{ + int pos = -1; + for (int i = 0; i < MAX_BROKERS; i++) + { + if(init = 0) + { + pos = i; + break; + } + } + return pos; +} + +int find_broker ( char *ip ) +{ + int pos = -1; + for (int i = 0; i < MAX_BROKERS; i++) + { + if(strcmp(ip, ip)) + { + pos = i; + break; + } + } + return pos; +} + +*/ -/* ... Functions / Funciones ......................................... */ /* * Communication */ -int tcp_server_write_operation(int sd, struct st_tcp_server_msg * head) { +int tcp_server_write_operation(int sd, struct st_tcp_server_msg * head) +{ int ret; debug_info("[NFI-TCP] (ID=%s) tcpClient_write_data: begin HEAD_TYPE:%d\n", head -> id, sizeof(head -> type)); @@ -49,7 +94,8 @@ int tcp_server_write_operation(int sd, struct st_tcp_server_msg * head) { } debug_info("[NFI-TCP] (ID=%s) write_operation: %d -> \n", head -> id, head -> type); - switch (head -> type) { + switch (head -> type) + { //File API case TCP_SERVER_OPEN_FILE_WS: case TCP_SERVER_OPEN_FILE_WOS: @@ -145,7 +191,9 @@ int nfi_tcp_server_doRequest(struct nfi_tcp_server_server * server_aux, struct s return 0; } -int nfi_tcp_server_keepConnected(struct nfi_server * serv) { + +int nfi_tcp_server_keepConnected(struct nfi_server * serv) +{ if (NULL == serv) { return -1; } @@ -164,11 +212,13 @@ int nfi_tcp_server_keepConnected(struct nfi_server * serv) { return (serv -> private_info != NULL); } + /* * PRIVATE FUNCTIONS TO USE tcp_server SERVERS */ -void NFItoTCP_SERVERattr(struct stat * att, struct nfi_attr * nfi_att) { +void NFItoTCP_SERVERattr(struct stat * att, struct nfi_attr * nfi_att) +{ att -> st_dev = nfi_att -> st_dev; att -> st_ino = nfi_att -> st_ino; @@ -191,7 +241,9 @@ void NFItoTCP_SERVERattr(struct stat * att, struct nfi_attr * nfi_att) { att -> st_ctime = nfi_att -> at_ctime; // time of last change } -void TCP_SERVERtoNFIattr(struct nfi_attr * nfi_att, struct stat * att) { + +void TCP_SERVERtoNFIattr(struct nfi_attr * nfi_att, struct stat * att) +{ nfi_att -> st_dev = att -> st_dev; nfi_att -> st_ino = att -> st_ino; @@ -214,10 +266,12 @@ void TCP_SERVERtoNFIattr(struct nfi_attr * nfi_att, struct stat * att) { nfi_att -> at_ctime = att -> st_ctime; // time of last change } + void TCP_SERVERtoNFIInfo(__attribute__((__unused__)) struct nfi_info * nfi_inf, __attribute__((__unused__)) struct nfi_info * tcp_server_inf) { //TODO } + /* * PUBLIC FUNCTIONS TO USE TCP_SERVER */ @@ -225,7 +279,8 @@ void TCP_SERVERtoNFIInfo(__attribute__((__unused__)) struct nfi_info * nfi_inf, /************************************************************ * Init tcp_server * ************************************************************/ -int nfi_tcp_server_init(char * url, struct nfi_server * serv, __attribute__((__unused__)) struct nfi_attr_server * attr) { +int nfi_tcp_server_init(char * url, struct nfi_server * serv, __attribute__((__unused__)) struct nfi_attr_server * attr) +{ int ret; char server[PATH_MAX], dir[PATH_MAX], prt[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -333,7 +388,11 @@ int nfi_tcp_server_init(char * url, struct nfi_server * serv, __attribute__((__u ret = nfiworker_init(serv); ret = nfi_tcp_server_connect(serv, url, prt, server, dir); - if (ret < 0) { + + printf("\n[%d]\t%s %s\n", __LINE__, url, server_aux -> params.server_name); + + if (ret < 0) + { FREE_AND_NULL(serv -> ops); FREE_AND_NULL(server_aux); return -1; @@ -356,39 +415,43 @@ int nfi_tcp_server_init(char * url, struct nfi_server * serv, __attribute__((__u int rc = 0; server_aux -> params.xpn_mosquitto_mode = 0; char * env_mosquitto = getenv("XPN_MQTT"); - if (env_mosquitto != NULL) + if (env_mosquitto != NULL) { server_aux -> params.xpn_mosquitto_mode = atoi(env_mosquitto); if (server_aux -> params.xpn_mosquitto_mode == 1) //MQTT initialization { + /*int first_av = find_pos_mqtt(); + if (first_av == -1) return -1;*/ + mosquitto_lib_init(); - mosqstr = mosquitto_new(NULL, true, NULL); - if(mosqstr == NULL) + server_aux -> mqtt = mosquitto_new(NULL, true, NULL); + + if(server_aux -> mqtt == NULL) { fprintf(stderr, "Error: Out of memory.\n"); return -1; } - // mosquitto_int_option(mosqstr, MOSQ_OPT_TCP_NODELAY, 1); // TODO: uncomment this line !! - mosquitto_int_option(mosqstr, MOSQ_OPT_SEND_MAXIMUM, 65535); + mosquitto_int_option(server_aux -> mqtt, MOSQ_OPT_TCP_NODELAY, 1); // TODO: uncomment this line !! + mosquitto_int_option(server_aux -> mqtt, MOSQ_OPT_SEND_MAXIMUM, 65535); - rc = mosquitto_connect(mosqstr, "localhost", 1886, 0); + rc = mosquitto_connect(server_aux -> mqtt, server_aux -> params.server_name, 1883, 0); if(rc != MOSQ_ERR_SUCCESS) { - mosquitto_destroy(mosqstr); + mosquitto_destroy(server_aux -> mqtt); fprintf(stderr, "Error: %s\n", mosquitto_strerror(rc)); return 1; } /* Run the network loop in a background thread, this call returns quickly. */ - rc = mosquitto_loop_start(mosqstr); + rc = mosquitto_loop_start(server_aux -> mqtt); if(rc != MOSQ_ERR_SUCCESS) { - mosquitto_destroy(mosqstr); + mosquitto_destroy(server_aux -> mqtt); fprintf(stderr, "Error: %s\n", mosquitto_strerror(rc)); return 1; } @@ -402,10 +465,12 @@ int nfi_tcp_server_init(char * url, struct nfi_server * serv, __attribute__((__u return 0; } + /************************************************************ * Destroy tcp_server * * **********************************************************/ -int nfi_tcp_server_destroy(struct nfi_server * serv) { +int nfi_tcp_server_destroy(struct nfi_server * serv) +{ int ret; struct nfi_tcp_server_server * server_aux; @@ -443,9 +508,11 @@ int nfi_tcp_server_destroy(struct nfi_server * serv) { if (server_aux -> params.xpn_mosquitto_mode == 1) //MQTT finalization { + + mosquitto_disconnect(server_aux -> mqtt); + mosquitto_destroy(server_aux -> mqtt); mosquitto_lib_cleanup(); - mosquitto_loop_stop(mosqstr, true); - } + } #endif @@ -453,10 +520,12 @@ int nfi_tcp_server_destroy(struct nfi_server * serv) { return 0; } + /********************************************************* * Connect to the server * * *******************************************************/ -int nfi_tcp_server_connect(struct nfi_server * serv, __attribute__((__unused__)) char * url, __attribute__((__unused__)) char * prt, char * server, __attribute__((__unused__)) char * dir) { +int nfi_tcp_server_connect(struct nfi_server * serv, __attribute__((__unused__)) char * url, __attribute__((__unused__)) char * prt, char * server, __attribute__((__unused__)) char * dir) +{ int ret; struct nfi_tcp_server_server * server_aux; @@ -485,10 +554,12 @@ int nfi_tcp_server_connect(struct nfi_server * serv, __attribute__((__unused__)) return 0; } + /************************************************************ * Disconnect to the server * * **********************************************************/ -int nfi_tcp_server_disconnect(struct nfi_server * serv) { +int nfi_tcp_server_disconnect(struct nfi_server * serv) +{ int ret; struct nfi_tcp_server_server * server_aux; @@ -517,10 +588,12 @@ int nfi_tcp_server_disconnect(struct nfi_server * serv) { return 0; } + /************************************************************ * Reconnect to the MPI server * ************************************************************/ -int nfi_tcp_server_reconnect(struct nfi_server * serv) { +int nfi_tcp_server_reconnect(struct nfi_server * serv) +{ // Don't see the serv result int ret; char server[PATH_MAX], dir[PATH_MAX]; @@ -561,11 +634,13 @@ int nfi_tcp_server_reconnect(struct nfi_server * serv) { return 0; } + /* * GENERIC FUNCTIONS */ -int nfi_tcp_server_open(struct nfi_server * serv, char * url, struct nfi_fhandle * fho) { +int nfi_tcp_server_open(struct nfi_server * serv, char * url, struct nfi_fhandle * fho) +{ int ret; char dir[PATH_MAX], server[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -649,7 +724,9 @@ int nfi_tcp_server_open(struct nfi_server * serv, char * url, struct nfi_fhandle return 0; } -int nfi_tcp_server_create(struct nfi_server * serv, char * url, struct nfi_attr * attr, struct nfi_fhandle * fh) { + +int nfi_tcp_server_create(struct nfi_server * serv, char * url, struct nfi_attr * attr, struct nfi_fhandle * fh) +{ int ret; char server[PATH_MAX], dir[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -758,7 +835,9 @@ int nfi_tcp_server_create(struct nfi_server * serv, char * url, struct nfi_attr return 0; } -ssize_t nfi_tcp_server_read(struct nfi_server * serv, struct nfi_fhandle * fh, void * buffer, off_t offset, size_t size) { + +ssize_t nfi_tcp_server_read(struct nfi_server * serv, struct nfi_fhandle * fh, void * buffer, off_t offset, size_t size) +{ int ret, cont, diff; struct nfi_tcp_server_server * server_aux; struct nfi_tcp_server_fhandle * fh_aux; @@ -878,8 +957,11 @@ ssize_t nfi_tcp_server_read(struct nfi_server * serv, struct nfi_fhandle * fh, v return ret; } -ssize_t nfi_tcp_server_write(struct nfi_server * serv, struct nfi_fhandle * fh, void * buffer, off_t offset, size_t size) { + +ssize_t nfi_tcp_server_write(struct nfi_server * serv, struct nfi_fhandle * fh, void * buffer, off_t offset, size_t size) +{ int ret, diff, cont; + struct nfi_tcp_server_server * server_aux; struct nfi_tcp_server_fhandle * fh_aux; struct st_tcp_server_msg msg; @@ -902,71 +984,124 @@ ssize_t nfi_tcp_server_write(struct nfi_server * serv, struct nfi_fhandle * fh, debug_info("[NFI-TCP] nfi_tcp_server_write(ID=%s): begin off %d size %d\n", server_aux -> id, (int) offset, (int) size); fh_aux = (struct nfi_tcp_server_fhandle * ) fh -> priv_fh; - /************** LOCAL *****************/ - if (server_aux -> params.locality) + if (server_aux -> params.xpn_mosquitto_mode == 0) { - if (server_aux -> params.xpn_session) + /************** LOCAL *****************/ + if (server_aux -> params.locality) { - real_posix_lseek(fh_aux -> fd, offset, SEEK_SET); - //if(server_aux->params.sem_server != 0) sem_wait(server_aux->params.sem_server); - ret = real_posix_write(fh_aux -> fd, buffer, size); - //if(server_aux->params.sem_server != 0) sem_post(server_aux->params.sem_server); + if (server_aux -> params.xpn_session) + { + real_posix_lseek(fh_aux -> fd, offset, SEEK_SET); + //if(server_aux->params.sem_server != 0) sem_wait(server_aux->params.sem_server); + ret = real_posix_write(fh_aux -> fd, buffer, size); + //if(server_aux->params.sem_server != 0) sem_post(server_aux->params.sem_server); - debug_info("[NFI-TCP] write %s(%d) off %ld size %zu (ret:%zd)", fh -> url, fh_aux -> fd, (long int) offset, size, ret); - } else - { - int fd; + debug_info("[NFI-TCP] write %s(%d) off %ld size %zu (ret:%zd)", fh -> url, fh_aux -> fd, (long int) offset, size, ret); + } else + { + int fd; - fd = real_posix_open(fh_aux -> path, O_WRONLY); // WOS - if (fd < 0) { + fd = real_posix_open(fh_aux -> path, O_WRONLY); // WOS + if (fd < 0) { + debug_error("real_posix_write writes zero bytes from url:%s offset:%ld size:%zu (ret:%zd) errno=%d\n", fh -> url, (long int) offset, size, ret, errno); + return -1; + } + + real_posix_lseek(fd, offset, SEEK_SET); + //if(server_aux->params.sem_server != 0) sem_wait(server_aux->params.sem_server); + ret = real_posix_write(fd, buffer, size); + //if(server_aux->params.sem_server != 0) sem_post(server_aux->params.sem_server); + + debug_info("[NFI-TCP] write %s(%d) off %ld size %zu (ret:%zd)", fh -> url, fd, (long int) offset, size, ret); + + real_posix_close(fd); // WOS + } + + if (ret < 0) { debug_error("real_posix_write writes zero bytes from url:%s offset:%ld size:%zu (ret:%zd) errno=%d\n", fh -> url, (long int) offset, size, ret, errno); return -1; } + } + /************** REMOTE ****************/ + else + { + // send write request + if (server_aux -> params.xpn_session) + { + msg.type = TCP_SERVER_WRITE_FILE_WS; + msg.u_st_tcp_server_msg.op_write.fd = fh_aux -> fd; + printf("[NFI-TCP] write: -> fd %d \n", msg.u_st_tcp_server_msg.op_write.fd); + } else + { + msg.type = TCP_SERVER_WRITE_FILE_WOS; + memccpy(msg.u_st_tcp_server_msg.op_write.path, fh_aux -> path, 0, PATH_MAX - 1); + printf("[NFI-TCP] write: -> path %s \n", msg.u_st_tcp_server_msg.op_write.path); + } - real_posix_lseek(fd, offset, SEEK_SET); - //if(server_aux->params.sem_server != 0) sem_wait(server_aux->params.sem_server); - ret = real_posix_write(fd, buffer, size); - //if(server_aux->params.sem_server != 0) sem_post(server_aux->params.sem_server); + memccpy(msg.id, server_aux -> id, 0, TCP_SERVER_ID - 1); + msg.u_st_tcp_server_msg.op_write.offset = offset; + msg.u_st_tcp_server_msg.op_write.size = size; - debug_info("[NFI-TCP] write %s(%d) off %ld size %zu (ret:%zd)", fh -> url, fd, (long int) offset, size, ret); + debug_info("[NFI-TCP] write: -> offset %d \n", (int) msg.u_st_tcp_server_msg.op_write.offset); + debug_info("[NFI-TCP] write: -> size %d \n", msg.u_st_tcp_server_msg.op_write.size); - real_posix_close(fd); // WOS - } + ret = tcp_server_write_operation(server_aux -> params.server, & msg); + if (ret < 0) + { + fprintf(stderr, "(1)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); + return -1; + } - if (ret < 0) { - debug_error("real_posix_write writes zero bytes from url:%s offset:%ld size:%zu (ret:%zd) errno=%d\n", fh -> url, (long int) offset, size, ret, errno); - return -1; - } - } - /************** REMOTE ****************/ - else - { - // send write request - if (server_aux -> params.xpn_session) - { - msg.type = TCP_SERVER_WRITE_FILE_WS; - msg.u_st_tcp_server_msg.op_write.fd = fh_aux -> fd; - debug_info("[NFI-TCP] write: -> fd %d \n", msg.u_st_tcp_server_msg.op_write.fd); - } else - { - msg.type = TCP_SERVER_WRITE_FILE_WOS; - memccpy(msg.u_st_tcp_server_msg.op_write.path, fh_aux -> path, 0, PATH_MAX - 1); - debug_info("[NFI-TCP] write: -> path %s \n", msg.u_st_tcp_server_msg.op_write.path); - } + // write n times: ... + diff = size; + cont = 0; - memccpy(msg.id, server_aux -> id, 0, TCP_SERVER_ID - 1); - msg.u_st_tcp_server_msg.op_write.offset = offset; - msg.u_st_tcp_server_msg.op_write.size = size; + // Max buffer size + int buffer_size = size; + if (buffer_size > MAX_BUFFER_SIZE) + { + buffer_size = MAX_BUFFER_SIZE; + } - debug_info("[NFI-TCP] write: -> offset %d \n", (int) msg.u_st_tcp_server_msg.op_write.offset); - debug_info("[NFI-TCP] write: -> size %d \n", msg.u_st_tcp_server_msg.op_write.size); + do + { + int bytes_to_write = 0; - ret = tcp_server_write_operation(server_aux -> params.server, & msg); - if (ret < 0) - { - fprintf(stderr, "(1)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); - return -1; + if( diff > buffer_size ) bytes_to_write = buffer_size; + else bytes_to_write = diff; + + ret = tcpClient_write_data(server_aux -> params.server, (char * ) buffer + cont, bytes_to_write, msg.id); + + if (ret < 0) + { + fprintf(stderr, "(2)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); + } + + cont = cont + bytes_to_write; //Send bytes + diff = size - cont; + + } while ((diff > 0) && (ret != 0)); + + ret = tcpClient_read_data(server_aux -> params.server, (char * ) & req, sizeof(struct st_tcp_server_write_req), msg.id); + if (ret < 0) + { + fprintf(stderr, "(3)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); + return -1; + } + + debug_info("[NFI-TCP] nfi_tcp_server_write(ID=%s): write %s off %d size %d (err:%d).\n", server_aux -> id, fh -> url, (int) offset, (int) size, (int) req.size); + if (req.size < 0) + { + fprintf(stderr, "ERROR: nfi_tcp_server_write(ID=%s): Fail write %s off %d size %d (err:%d).\n", server_aux -> id, fh -> url, (int) offset, (int) size, (int) req.size); + tcp_server_err(TCP_SERVERERR_WRITE); + return -1; + } + + ret = cont; } + } + else + { // write n times: ... diff = size; @@ -974,71 +1109,53 @@ ssize_t nfi_tcp_server_write(struct nfi_server * serv, struct nfi_fhandle * fh, // Max buffer size int buffer_size = size; - if (buffer_size > MAX_BUFFER_SIZE) + if (buffer_size > MAX_BUFFER_SIZE) { buffer_size = MAX_BUFFER_SIZE; } - do + do { int bytes_to_write = 0; + char *topic = malloc(strlen(fh_aux -> path) + sizeof(bytes_to_write) + sizeof(cont) + sizeof(fh_aux -> fd) + 4); if( diff > buffer_size ) bytes_to_write = buffer_size; else bytes_to_write = diff; - if (server_aux -> params.xpn_mosquitto_mode == 0) - { - ret = tcpClient_write_data(server_aux -> params.server, (char * ) buffer + cont, bytes_to_write, msg.id); - } #ifdef HAVE_MOSQUITTO_H - else //MQTT sending text - { - printf("CLIENTE ESCRITURA - %s\n", fh_aux -> path); - ret = mosquitto_publish(mosqstr, NULL, fh_aux -> path, bytes_to_write, (char * ) buffer + cont, 0, false); - if(ret != MOSQ_ERR_SUCCESS) - { - fprintf(stderr, "Error publishing write: %s\n", mosquitto_strerror(ret)); - return -1; - } + sprintf(topic, "%s/%d/%d/%d", fh_aux -> path, fh_aux -> fd, bytes_to_write, cont); + printf("\nCLIENTE ESCRITURA - %s - topic=%s\n\n", fh_aux -> path, topic); + + ret = mosquitto_publish(server_aux -> mqtt, NULL, topic, bytes_to_write, (char * ) buffer + cont, 0, false); + + if(ret != MOSQ_ERR_SUCCESS) + { + fprintf(stderr, "Error publishing write: %s\n", mosquitto_strerror(ret)); + return -1; } + #endif - if (ret < 0) + if (ret < 0) { fprintf(stderr, "(2)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); } cont = cont + bytes_to_write; //Send bytes diff = size - cont; - + free(topic); } while ((diff > 0) && (ret != 0)); - - ret = tcpClient_read_data(server_aux -> params.server, (char * ) & req, sizeof(struct st_tcp_server_write_req), msg.id); - if (ret < 0) - { - fprintf(stderr, "(3)ERROR: nfi_tcp_server_write(ID=%s): Error on write operation\n", server_aux -> id); - return -1; - } - - debug_info("[NFI-TCP] nfi_tcp_server_write(ID=%s): write %s off %d size %d (err:%d).\n", server_aux -> id, fh -> url, (int) offset, (int) size, (int) req.size); - if (req.size < 0) - { - fprintf(stderr, "ERROR: nfi_tcp_server_write(ID=%s): Fail write %s off %d size %d (err:%d).\n", server_aux -> id, fh -> url, (int) offset, (int) size, (int) req.size); - tcp_server_err(TCP_SERVERERR_WRITE); - return -1; - } - - ret = cont; } + DEBUG_END(); return ret; } - -int nfi_tcp_server_close(struct nfi_server * serv, struct nfi_fhandle * fh) { +int nfi_tcp_server_close(struct nfi_server * serv, struct nfi_fhandle * fh) +{ int ret = -1; struct nfi_tcp_server_fhandle * fh_aux; struct nfi_tcp_server_server * server_aux; @@ -1085,9 +1202,10 @@ int nfi_tcp_server_close(struct nfi_server * serv, struct nfi_fhandle * fh) { msg.type = TCP_SERVER_CLOSE_FILE_WS; memccpy(msg.id, server_aux -> id, 0, TCP_SERVER_ID - 1); msg.u_st_tcp_server_msg.op_close.fd = fh_aux -> fd; + memccpy(msg.u_st_tcp_server_msg.op_close.path, fh_aux -> path, 0, PATH_MAX - 1); nfi_tcp_server_doRequest(server_aux, & msg, (char * ) & (ret), sizeof(int)); - debug_info("[NFI-TCP] nfi_tcp_server_close(ID=%s): close -> %d \n", server_aux -> id, msg.u_st_tcp_server_msg.op_close.fd); + printf("[NFI-TCP] nfi_tcp_server_close(ID=%s): close -> %d\n", server_aux -> id, msg.u_st_tcp_server_msg.op_close.fd); } // free memory @@ -1102,8 +1220,8 @@ int nfi_tcp_server_close(struct nfi_server * serv, struct nfi_fhandle * fh) { } - -int nfi_tcp_server_remove(struct nfi_server * serv, char * url) { +int nfi_tcp_server_remove(struct nfi_server * serv, char * url) +{ int ret; char server[PATH_MAX], dir[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1154,8 +1272,8 @@ int nfi_tcp_server_remove(struct nfi_server * serv, char * url) { } - -int nfi_tcp_server_rename(struct nfi_server * serv, char * old_url, char * new_url) { +int nfi_tcp_server_rename(struct nfi_server * serv, char * old_url, char * new_url) +{ int ret; char server[PATH_MAX], old_path[PATH_MAX], new_path[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1217,8 +1335,8 @@ int nfi_tcp_server_rename(struct nfi_server * serv, char * old_url, char * new_u } - -int nfi_tcp_server_getattr(struct nfi_server * serv, struct nfi_fhandle * fh, struct nfi_attr * attr) { +int nfi_tcp_server_getattr(struct nfi_server * serv, struct nfi_fhandle * fh, struct nfi_attr * attr) +{ int ret; char server[PATH_MAX], dir[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1272,8 +1390,8 @@ int nfi_tcp_server_getattr(struct nfi_server * serv, struct nfi_fhandle * fh, st } - -int nfi_tcp_server_setattr(struct nfi_server * serv, struct nfi_fhandle * fh, struct nfi_attr * attr) { +int nfi_tcp_server_setattr(struct nfi_server * serv, struct nfi_fhandle * fh, struct nfi_attr * attr) +{ struct nfi_tcp_server_server * server_aux; struct nfi_tcp_server_fhandle * fh_aux; @@ -1300,8 +1418,8 @@ int nfi_tcp_server_setattr(struct nfi_server * serv, struct nfi_fhandle * fh, st } - -int nfi_tcp_server_mkdir(struct nfi_server * serv, char * url, struct nfi_attr * attr, struct nfi_fhandle * fh) { +int nfi_tcp_server_mkdir(struct nfi_server * serv, char * url, struct nfi_attr * attr, struct nfi_fhandle * fh) +{ int ret; char server[PATH_MAX], dir[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1390,8 +1508,8 @@ int nfi_tcp_server_mkdir(struct nfi_server * serv, char * url, struct nfi_attr * } - -int nfi_tcp_server_opendir(struct nfi_server * serv, char * url, struct nfi_fhandle * fho) { +int nfi_tcp_server_opendir(struct nfi_server * serv, char * url, struct nfi_fhandle * fho) +{ int ret; char dir[PATH_MAX], server[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1460,8 +1578,8 @@ int nfi_tcp_server_opendir(struct nfi_server * serv, char * url, struct nfi_fhan } - -int nfi_tcp_server_readdir(struct nfi_server * serv, struct nfi_fhandle * fh, struct dirent * entry) { +int nfi_tcp_server_readdir(struct nfi_server * serv, struct nfi_fhandle * fh, struct dirent * entry) +{ struct nfi_tcp_server_server * server_aux; struct nfi_tcp_server_fhandle * fh_aux; struct st_tcp_server_msg msg; @@ -1521,8 +1639,8 @@ int nfi_tcp_server_readdir(struct nfi_server * serv, struct nfi_fhandle * fh, st } - -int nfi_tcp_server_closedir(struct nfi_server * serv, struct nfi_fhandle * fh) { +int nfi_tcp_server_closedir(struct nfi_server * serv, struct nfi_fhandle * fh) +{ int ret; struct st_tcp_server_msg msg; struct nfi_tcp_server_server * server_aux; @@ -1567,8 +1685,8 @@ int nfi_tcp_server_closedir(struct nfi_server * serv, struct nfi_fhandle * fh) { } - -int nfi_tcp_server_rmdir(struct nfi_server * serv, char * url) { +int nfi_tcp_server_rmdir(struct nfi_server * serv, char * url) +{ int ret; char server[PATH_MAX], dir[PATH_MAX]; struct nfi_tcp_server_server * server_aux; @@ -1620,8 +1738,8 @@ int nfi_tcp_server_rmdir(struct nfi_server * serv, char * url) { } - -int nfi_tcp_server_statfs(__attribute__((__unused__)) struct nfi_server * serv, __attribute__((__unused__)) struct nfi_info * inf) { +int nfi_tcp_server_statfs(__attribute__((__unused__)) struct nfi_server * serv, __attribute__((__unused__)) struct nfi_info * inf) +{ DEBUG_BEGIN(); /* @@ -1655,8 +1773,8 @@ int nfi_tcp_server_statfs(__attribute__((__unused__)) struct nfi_server * serv, } - -int nfi_tcp_server_preload(struct nfi_server * serv, char * url, char * virtual_path, char * storage_path, int opt) { +int nfi_tcp_server_preload(struct nfi_server * serv, char * url, char * virtual_path, char * storage_path, int opt) +{ //char dir[PATH_MAX]; int ret; struct nfi_tcp_server_server * server_aux; @@ -1703,8 +1821,8 @@ int nfi_tcp_server_preload(struct nfi_server * serv, char * url, char * virtual_ } - -int nfi_tcp_server_flush(struct nfi_server * serv, char * url, char * virtual_path, char * storage_path, int opt) { +int nfi_tcp_server_flush(struct nfi_server * serv, char * url, char * virtual_path, char * storage_path, int opt) +{ //char dir[PATH_MAX]; int ret; struct nfi_tcp_server_server * server_aux; @@ -1750,4 +1868,6 @@ int nfi_tcp_server_flush(struct nfi_server * serv, char * url, char * virtual_pa return 0; } + /* ................................................................... */ + diff --git a/test/integrity/nfi/interpreter.c b/test/integrity/nfi/interpreter.c index a8d3eb3f..f9ef8a25 100644 --- a/test/integrity/nfi/interpreter.c +++ b/test/integrity/nfi/interpreter.c @@ -5,7 +5,8 @@ #include #include "all_system.h" -#include "base/string_misc.h" +#include "base/path_misc.h" +#include "base/urlstr.h" #include "nfi/nfi.h" #include "nfi/nfi_lib.h" #include "nfi/nfi_nfs/nfs.h" diff --git a/test/integrity/xpn/open-write-close.c b/test/integrity/xpn/open-write-close.c index 174fe01b..2b160ca9 100644 --- a/test/integrity/xpn/open-write-close.c +++ b/test/integrity/xpn/open-write-close.c @@ -28,7 +28,7 @@ int main ( int argc, char *argv[] ) ret = xpn_write(fd1, buffer, BUFF_SIZE); printf("%d = xpn_write(%d, %p, %lu)\n", ret, fd1, buffer, (unsigned long)BUFF_SIZE); - +// sleep(1); ret = xpn_close(fd1); printf("%d = xpn_close(%d)\n", ret, fd1) ; diff --git a/test/integrity/xpn/kk.sh b/test/integrity/xpn/test-1.sh similarity index 90% rename from test/integrity/xpn/kk.sh rename to test/integrity/xpn/test-1.sh index 00ee469c..9592de5b 100755 --- a/test/integrity/xpn/kk.sh +++ b/test/integrity/xpn/test-1.sh @@ -7,6 +7,7 @@ export XPN_CONF=./xpn.conf export XPN_DNS=/tmp/tcp_server.dns export XPN_LOCALITY=0 +export XPN_SESSION=1 export XPN_MQTT=1 rm -fr /export/data/d4 diff --git a/test/integrity/xpn/xpn.conf b/test/integrity/xpn/xpn.conf index 9a99d062..b42777d7 100644 --- a/test/integrity/xpn/xpn.conf +++ b/test/integrity/xpn/xpn.conf @@ -1,6 +1,7 @@ - + +