Should we have a 'default' location we install tools into?

[MikeFultonDev]

Many of the Open Source packages we are porting have the notion of installing into some 'prefix' location.
This defaults to /usr/local. The Filesystem Hierarchy Standard states:

The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to
be safe from being overwritten when the system software is updated. It may be used for programs and data that
are shareable amongst a group of hosts, but not found in /usr.
Locally installed software must be placed within /usr/local rather than /usr unless it is being installed to
replace or upgrade software in /usr.

Should we be building our software such that the installation prefix we specify is /usr/local or some other well-known location rather than /jenkins/workspace ?
Should we keep the version info when we do this install so that customers can have multiple releases of a software package available?
If we do have multiple versions installed, should we either create, or define how to create, symbolic links so that a 'default' release of a software package can be specified?

One advantage of taking this approach is that if a systems programmer follows this advice and specifies defaults, we could drastically reduce the number of environment variables someone would need to specify since we would 'know' where everything is

[MikeFultonDev]

One approach that would be similar to what I see on Mac would be to have a structure like:

/usr/local
  + zopen
     + bash
        + 5.2.15
           + <contents>
    + git
       + 2.36.1
          + <contents>
  + bin
     + bash -> ../zopen/bash/5.2.15/bin/bash
     + bashbug -> ../zopen/bash/5.2.15/bin/bashbug
     + git -> ../zopen/git/2.36.1/bin/git

Here is the bash brew formulae: https://formulae.brew.sh/formula/bash and: https://github.com/Homebrew/homebrew-core/blob/master/Formula/bash.rb

[AnthonyGiorgio]

I think that /usr/local is perfect for what this project is (currently) doing. When I was running my first Slackware Linux system, any software I downloaded and compiled would get installed there. Stuff that came with the system would end up in /bin or /usr/bin. Since these tools are going to be installed by users or developers, /usr/local is a perfect place for them.

[IgorTodorovskiIBM]

I also like this approach, I think it will make things simpler overall.
Though, I think we would support from a package manager to handle the updating of symbolic links, and downgrading and updating of tool versions. And maybe we can implement something like (--instdir=dir) https://manpages.ubuntu.com/manpages/focal/en/man1/dpkg.1.html to allow for modifying the install dir if the user does not have authority to write to /usr/local, but I think it should be handled by the package manager.

[AnthonyGiorgio]

We're going to need something like etc-alternatives, since that closely mirrors what we're doing. Having different tool versions in different directories is fine, but we really need a single set of directories for handling $PATH, $LIBPATH, $MANPATH, etc. This way users can just update their profiles to point to a single place, and the tooling can handle swapping out the links behind the scenes.

[edelsohn]

There is a difference between the default location for the tool and the default location for the package manager.

Most tools default to install in /usr/local. zosopentools should not change the tool defaults.

Separately, zosopentools is creating a package manager system equivalent to Apt/RPM/DNF . On Linux, the tools are the system environment, installed in /usr . On AIX, the AIX Toolbox for Linux rpm command installs Open Source tools in /opt/freeware. zosopentools should choose a common location for its tools, probably separate from the z/OS USS defaults. But that is the zosopentools packaging, not the tools defaults.

The build system for zosopentools or the default commands to download, configure and build the tools in the user environment should add --prefix= to the configure command to define a common installation directory defined by the package management system for zosopentools.

[jshimoda9323]

I think this is a good idea. One thing we might also want to consider is being able to install the the tools with the same heirarchy in some subdirectory the user's home directory. I can imagine in a lot of cases (especially on z/OS) users will want to try out the tools, but don't want to bother the sys admin to install it into /usr. Having this option allows them to hit the ground running.

[MikeFultonDev]

How about this:

• If you build your own version of an Open Source software package, you will be able to install it into the location of your choosing with our zopen-build tooling (this works today)
• If you download a pre-built pax file from the z/OS Open Tools site, you can install it into the location of your choosing.
• Our pre-built pax files all come with a .env file for sourcing to update your environment for the tool in question. IF you install into the recommended location:/usr/local/zopen/<package>/<version>, and you create symbolic links from /usr/local to /usr/local/zopen/<package>/<version> as appropriate, then the .env file will set fewer environment variables since tools will know where to look. This is consistent with how these software packages function on other platforms as well.
• Sysprogs will want the flexibility to control the sym links - it may be that they need to 'roll out' an update to the version of software after testing - so perhaps the symlink step should be separate?

[perry-ca]

I liked the original idea and this one too. The only request I'd make on top of what you have is that there is a single .env file in /usr/local/zosopentools/ that would set up all the packages. It would be awkward for people to run each of the individual .env files and keep updating their .profile as packages are added/updated//removed.

[MikeFultonDev]

Agreed

[ccw-1]

It shouldn't be limited to /usr/local/, it just need to be a user specified location. with bin etc games include lib man sbin share src directories like linux. The current .env setup keeps on growing PATH, there is a limitation of how long PATH can be, I know that some old shells will crash when it gets too long. Also, a long PATH to search would be a performance issue. And instead of "copying the files" to "INSTALL LOCATION"/bin , etc, man , lib etc. You should optionally allow the use of symlinks.

[lbdyck]

Gone are the days, even in the *nix world, where a system is used by only one user – every system of worth supports 10’s to 1000’s of users and the introduction of new code (new products, updates, changes) must be managed through a change process (some of which are grueling).

A typical change process includes:

1. Installing new/updates/changes to a sandbox, or test system, where the installer can perform a basic validation test.
   a. This is accomplished on z/OS for non-sysres products by using a unique high-level-qualifier (hlq) for the installation (target) datasets.
   b. Or for USS into a unique ZFS that can be mounted by the tester at the ‘correct’ mount point
2. Copying (1) into a development environment. Not a reinstall so as to ensure that nothing changes from (1) to (2)
   a. This may be a separate LPAR, typically is, so the hlq will be the production hlq and the ZFS mount will be at the correct mount point
3. Copying (2) into the production environment with the production level hlq and mount points.
   a. Catalog aliases may be changed to point to the new datasets, including ZFS, at this time if the new datasets have version specific qualifiers (which is fairly common)

In a large environment it is critical that all changes be verified prior to implementation. There may be multiple development and production environments that support mission critical applications and services and even a small change can bring a system down or corrupt data.

What I strongly recommend for zopen is the ability to install one, or more, tools into a user specified filesystem, which may be a new ZFS with version (or other) qualifiers or just a subdirectory. The installer can then perform initial testing from there. Next the installer can copy the filesystem to a new ZFS that can be mounted for additional testing, copied (or used) for development validation, and used as the mount for production.

Then zopen update can be used against (1) and the process followed to move into production.

Hope this makes sense.

[lbdyck]

Something I found on the Rocket forum:

Not wanting to perform an installation to active system, our process is installation and maintenance to a non-active environment which is later copied to a SYSRES based set of files, which are mounted at IPL. The SMPE installation process hardcodes the Installation path /mvsde2/ into the installed files.
e.g.
/mvsde2/usr/lpp1/roa/ro/export_vars.sh

---

#common export of PATH, LIBPATH and MANPATH
export PATH=/mvsde2/usr/lpp1/roa/ro/bin:$PATH
export LIBPATH=/mvsde2/usr/lpp1/roa/ro/lib:$LIBPATH
export MANPATH=/mvsde2/usr/lpp1/roa/ro/share/man:$MANPATH

for file in /mvsde2/usr/lpp1/roa/ro/etc/conda/activate.d/*.sh
do
if [ -f "$file" ]; then
. "$file"
fi
done

---

/mvsde2/usr/lpp1/roa/ro/etc/gitconfig

---

[http]
sslVerify = true
sslCAinfo = /mvsde2/usr/lpp1/roa/ro/bin/ca-bundle.crt

---

/mvsde2/usr/lpp1/roa/ro/etc/conda/activate.d/ant_home.sh

---

export ANT_HOME_CONDA_BACKUP=$ANT_HOME
export ANT_HOME=/mvsde2/usr/lpp1/roa/ro

---

/mvsde2/usr/lpp1/roa/ro/etc/conda/activate.d/git.sh
********************************************************************* Top
export GIT_SHELL=/mvsde2/usr/lpp1/roa/ro/bin/bash
export GIT_EXEC_PATH=/mvsde2/usr/lpp1/roa/ro/libexec/git-core
export GIT_TEMPLATE_DIR=/mvsde2/usr/lpp1/roa/ro/share/git-core/templates
******************************************************************** Botto
/mvsde2/usr/lpp1/roa/ro/etc/conda/activate.d/ncurses.sh

---

export ASCII_TERMINFO=/mvsde2/usr/lpp1/roa/ro/share/terminfo

---

The above from a partial manual review of installed files.

After IPL we have standard SYMLINK for /usr/lpp1 which points to the sysres local version. i.e. /SYSRES/usr/lpp1
Looking for a way to set the future path values during install and maintenance process.
Bruce

---

Bruce Hewson
Infra Tech Lead Analyst
Citigroup Technology Inc

[ccw-1]

I think all the tools need to be symlinked installed into a "TOOLLOCATION" type of envvar that specifies the PATH for bin, etc, man etc. The configuration paths for the tools need to be aware of TOOLLOCATION and find its default own configuration files without the need to source each of the .env files.

[wizardofzos]

Just my 2cc, but I think it would be amazing to ship it as UMODs too :) Store the installers in /SERVICE and then have a 'postinstall' step that installs it to sites desired location.... (I'm currently trying to get that running with the 'gitmaster'-installer)

On the other hand, this stuff doesn't need SysPro installations but can just as easy run from 'user-sphere' ...

[MikeFultonDev]

Good input here. I know people are keen to just 'get it done' but I also want to ensure we 'do it right'.

From a 'user experience' perspective, the following all seem reasonable:

• As a developer on a shared system, I should be able to install a tool (and any dependent tools) into my own file system to use. For example, I decide 'vim' is the coolest thing, but I'm the only one, so I set it up myself under my $HOME directory
• As a small team with a 'sandbox' (i.e. my own second level z/OS system on z/OS, a small native LPAR, or a zPDT-based emulator running Wazi Sandbox, zD&T, zVDT, or Wazi as a Service in the cloud), I should be able to install software into a shared location (e.g. /usr/local/zopen) in the sandbox and use it from different IDs without requiring deep-z/OS systems knowledge
• As a z/OS systems programmer, I should be able to install new Open Source software my development teams need, shared across my SYSPLEX. I may need to install multiple versions of the software simultaneously because different teams want different versions of the software (e.g. different versions of Perl). I need to be able to install the software in my own testing area, validate it, then promote it to be shared. The Open Source software needs to be organized into read-only code that is shared across the sysplex, along with a corresponding writable directory that is per-LPAR so that I can install and maintain the software easily across dozens of LPARs

[lbdyck]

Point 3 - no need for a writeable directory per lpar since a zfs can be re-mounted read/write when needed and 99.99999% of the time remain mounted read-only. Also - updates typically would go into a new zfs and just change the mount point to the new zfs - that way it is easy to back out and the mount-point can be updated on an lpar by lpar basis as needed.

[MikeFultonDev]

Russell Wilson (not sure what your github id is): can you add in your approach here?
@lbdyck Thinking your issue: zopencommunity/gitport#38 should be addressed with our 'larger' solution here? If so, are you ok if we close that issue and continue it here?

[MikeFultonDev]

this is now done.