Thoughts on the FreeBSD ports/package system #161

probonopd · 2021-03-31T13:43:06Z

While we have been working with FreeBSD packages for a while, it seems like the structure of how packages get built and distributed leaves a few aspects to be deserved. This issue describes our gripes in the hope that someone can help us find solutions.

The FreeBSD base system and packages should feel like one consistent system, not like a "split personality". They should be upgraded together, e.g., when one upgrades the FreeBSD base system from 12.1 to 12.2 then all packages that require different binaries to work on 12.1 vs. 12.2 should be upgraded as well (e.g, Intel GPU driver). If the Intel GPU driver that is available in packages does not work with the base system, then this should be a blocker preventing the base system from being released. (In fact I think that Xorg and all GPU drivers should become an optional part of the base system because they are as essential to the operation of a graphical system as the kernel and basic command line userland.)
This also means that for FreeBSD -RELEASE the packages should be equally "slow-changing" (not rolling release!), whereas for -CURRENT rolling release is acceptable
It should be possible to use quarterly packages without the fear that from one day to the next a package disappears (it is ok if the old version of the package stays around until a new one is produced, but packages just going missing is not acceptable). In fact, isn't "quarterly" supposed to not change during the quarter? Users who want daily changing packages can use latest. This could be achieved by freezing a snapshot of latest at the begin of each quarter, and only allow new packages to be added after this point but not to change existing packages except for very urgent security patches
As long as the point before is not solved, we are forced to use release_X packages (which get frozen at the time when the FreeBSD images are built). But there is no way to get new packages into release_X packages which effectively means that we can get new packages only after a quarter has passed (e.g., falkon-qtonly is not available in http://pkg.freebsd.org/FreeBSD:12:amd64/release_2/ but hopefully will be in release_3)
We need to find a way to tell pkg not to touch/overwrite certain files (example: files in /usr/local/share/dbus-1/services/, details)

The text was updated successfully, but these errors were encountered:

crees · 2021-03-31T16:49:30Z

On your third point, this is exactly how quarterly branches work. Breakage is rare and high priority, though I know you were hit by an incident in the past.

For the first point... yes, separation of base and ports is pretty much the number one interesting Thing about FreeBSD. Pkgbase is still very new.

In short, we know about these issues and have been working on them for a while.

probonopd · 2021-04-01T06:27:59Z

Not sure what is going on here.

% cat /etc/pkg/FreeBSD.conf 
(...)
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
(...)

% sudo pkg install firefox
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        firefox: 86.0_2,2 -> 87.0,2

probonopd · 2021-04-02T19:30:14Z

The FreeBSD base system and packages should feel like one consistent system, not like a "split personality".

This also means that there should be a set of packages e.g., for 12.1 that do not suddenly stop working once 12.2 is released.

I fell into the exact issue here:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253452

Even if a FreeBSD minor version is no longer supported, the packages for it should not vanish. Some of us want to set up a system and freeze it in time, sometimes even for decades.

kettle-7 · 2021-04-30T06:02:33Z

So, in other words, what you're saying is that we should have different repositories for different versions, right?

probonopd · 2021-04-30T06:46:41Z

At least different versions of the packages for different FreeBSD versions for those packages that interact with the kernel directly or are otherwhise dependent on the exact FreeBSD version.

probonopd · 2021-05-22T11:28:51Z

py37-xdg has been broken in latest packages for FreeBSD 13 on amd64 for quite a while now.
This has been breaking our FreeBSD 13 based builds for weeks now.

Turns out that depending on the architecture, the package is now called differently: py38-xdg for amd64 but not for e.g, aarch64. Which both are supposedly "tier 1".

What gives?!?

Suggestions:

Do not hardcode the Python minor version in the package name, e.g., call it py3-xdg
Do not have different versions of the same packages depending on architectures

Workaround: Use the pkg install devel/py-xdg nomenclature? (This is a pain, because you have to look up the part before the / for every package.)

helloSystem/hello#161

grahamperrin · 2021-05-22T17:02:12Z

py37-xdg … latest …

Please see https://cgit.freebsd.org/ports/tree/UPDATING?id=210ee04dd3c3f2e82cd3130e412866a182066859#n8

probonopd · 2021-05-23T16:45:29Z

Thanks @grahamperrin. It seems that things are more robust if we use the port names rather than the package names for specifying which packages shall be installed? E.g., pkg install devel/py-xdg will hopefully not break once Python 3.9 is there.

grahamperrin · 2021-05-25T07:57:26Z

#161 (comment)

What gives?

Maybe a combination of issues with FreshPorts; see FreshPorts/freshports#287 (comment)

https://www.freshports.org/devel/py-xdg/#add

To install the port: cd /usr/ports/devel/py-xdg/ && make install clean
To add the package: pkg install py37-xdg

root@mowa219-gjp4-8570p:~ # pkg install -n py37-xdg
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'py37-xdg' have been found in the repositories
root@mowa219-gjp4-8570p:~ # git -C /usr/doc pull --ff-only && git -C /usr/ports pull --ff-only && git -C /usr/src pull --ff-only
Already up to date.
Already up to date.
Already up to date.
root@mowa219-gjp4-8570p:~ # cd /usr/ports/devel/py-xdg/
root@mowa219-gjp4-8570p:/usr/ports/devel/py-xdg # make -V FLAVORS
py38 
root@mowa219-gjp4-8570p:/usr/ports/devel/py-xdg # pkg info -x py-xdg
pkg: No package(s) matching py-xdg
root@mowa219-gjp4-8570p:/usr/ports/devel/py-xdg # pkg install -n devel/py-xdg
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The most recent versions of packages are already installed
root@mowa219-gjp4-8570p:/usr/ports/devel/py-xdg # pkg info -x py38-xdg
py38-xdg-0.27
root@mowa219-gjp4-8570p:/usr/ports/devel/py-xdg #

probonopd · 2021-09-14T22:11:14Z

Point in case today:
#203

TL;DR: Installed some applications, which upgraded some system libraries. As a result, was forced to upgrade Firefox. Cannot access meet.jit.si anymore as a result. No obvious way to downgrade.

grahamperrin · 2021-09-14T23:28:25Z

… upgrade Firefox.

To which version, exactly?

Latest repo, or quarterly?

Which version of helloSystem?

Which version of FreeBSD?

freebsd-version -kru && uname -aKU

Cannot access meet.jit.si anymore as a result.

With recently upgraded firefox-92.0_2,2, the site is accessible:

– and (after clicking Allow) meet.jit.si does recognise the camera and microphone.

https://www.freshports.org/www/firefox/#history there's no more recent version.

No obvious way to downgrade.

Don't forget the boot environment that you created before the upgrade.

If you forgot, then try your cached packages.

xk2600 · 2021-09-23T20:02:41Z

I have often posed the same questions in my head. It feels like there should maybe be an assertion that happens when performing an upgrade but how can the system be aware of where the modules in /boot/modules came from and whether it was a pkg install placed module, ports built module, or compiled from source by the user?

I am personally a huge fan of the pkg and ports system. I believe a lot of the problems with packages/ports getting ousted or breaking stems from lack of developers and maintainers if anything.

My thoughts on sweeping changes... lots of recommendations have been made in the past to overhaul/reorganize the ports system. This would be a pretty complicated endeavor and would probably require a fork of the ports tree. Anything that further segments the BSDs I believe is a disservice to the community. I think there exists enough which can be gleaned from the ports tree to help us make assertions to assist in extending the freebsd-update system. From my perspective, helloSystem is a beautiful distribution of FreeBSD and I would like to see it stay that way. The less we change the more collaboration can exist across both teams on a common code base.

Do not hardcode the Python minor version in the package name, e.g., call it py3-xdg
Do not have different versions of the same packages depending on architectures

On the systems that I must support running linux, I have often seen what you are suggesting break in release systems when addition of a module requires updating the interpreter's package. The virtual-environments (as they use symlinks and aren't a copy of the binary) don't protect against this breakage either. My personal opinion is there is too much codependancy on minor versions of python between modules and the binary, especially when modules are backed by C/C++ based libaries. The same goes for perl and tcl/tk, but my guess would be most interpreters suffer the same plight if they allow modules to be backed by C/C++ binaries. My suggestion would be to try and leverage Meta-ports for this mechanism. Discussed below.

So arguments aside, solutions:

Poudriere-- Build a autoscaling cluster on cloud infrastructure that runs when CI requires it, to host a pkg distribution for helloSystem (if this doesn't already exist). Leverage the FreeBSD ports tree.
Meta-ports-- Heavily leverage Meta-ports to allow multiple versions of prepackaged modules and interpreters to coexist in the pkg repo. The meta-port points to the latest stable in release and is locked at each minor release of helloSystem. Then if another package has a dependency that doesn't care about the minor version, it can depend on the meta-port. If the meta-port gets repointed to a new version during a minor release cycle, CI can have tests written to notify the build team and automatically changes the dependency for that package to point to the previous minor version. If someone reports a bug for a test that hasn't been written, someone should write a test and update the port to point to the static port major-minor. This should trigger CI to have Poudrier rebuild that package, increasing the pkg database version, so if a user runs into the same problem, a pkg update fixes it.
Extend the freebsd-update system or build world and include packages in base.tgz-- Right now the pkg/ports system uses the category 'kld' on any port that contains kernel modules. This might be a good mechanism to lookup port/pkgs which should be upgraded with freebsd-update. I haven't dug too far into helloSystem to see how the system gets built yet, but this distribution could effectively be packaged to includes specific Poudriere built packages for graphics to be supported OOTB in base.tgz: drm-next, nvidia, amd, etc.

Anyways just wanted to dig into my thoughts for a moment and get an idea on what others think. I've got a working install, and plan on contributing. What has been done thus far is beyond impressive.

probonopd · 2021-09-23T20:26:46Z

Thanks for your thoughtful comments @xk2600. As you may have guessed, I am still relatively new to FreeBSD, having spent many years on the Mac and in the Linux universe before. I do see clear advantages in using FreeBSD, and I fully agree that helloSystem should stay a "true FreeBSD", albeit one preconfigured in an opinionated way to work out-of-the-box as a "desktop appliance" (while providing raw FreeBSD power under the hood for those who know what they are doing). For this very reason, I'd like to see those issues to be solved on a FreeBSD level ("for everyone") rather than in helloSystem-specific ways.

Poudriere-- Build a autoscaling cluster on cloud infrastructure that runs when CI requires it, to host a pkg distribution for helloSystem

We are currently not doing this, and I'd like to avoid this for multiple reasons:

Resources. I simply don't have the resources to build and run such a system. Neither in terms of cost nor in terms of time
Compatibility. I'd like to keep helloSystem fully compatible with FreeBSD ports and packages should work, as should books and instructions written for FreeBSD. As you are saying, "anything that further segments the BSDs I believe is a disservice to the community"
Being easy for anyone to fork. By not needing any infrastructure to build helloSystem (other than what GitHub and Cirrus CI provide), anyone can simply fork the repository and create their own modified ISOs. In fact, this is what got me started with helloSystem (by forking the FuryBSD repository), and is what allowed Airyx to have a quick route to an ISO based on helloSystem.

So, right now helloSystem gets built by a shell script from the official FreeBSD binary ingredients plus official FreeBSD ports, at least for the core parts of the system (which for me includes the graphics stack and graphics drivers).

On the subject of graphics drivers, it seems like FreeBSD leadership is aware of the problem and "absolutely need to solve this before FreeBSD 12.3 or 13.1".
Your idea to move all "kld" ports to base.tgz or something like it seems reasonable to me - if FreeBSD does it.

kettle-7 · 2021-09-24T08:10:25Z

Maybe while we're trying to figure out a better long term updating option, run the install shell script after pkg upgrade is called?

kettle-7 · 2021-09-24T08:10:52Z

~~or make a new package manager~~

probonopd · 2021-09-24T19:32:44Z

Making a proof-of-concept Firefox.app

mszoek · 2021-09-24T19:50:40Z

How can we do this?

I would try using a chrooted package build where the chroot environment is a pristine helloSystem install. That way you will only get packages that are not part of the default system.

kettle-7 · 2021-09-25T08:40:13Z

Again, that gnumeric POC comes to mind, I think that's the way to go here.

kettle-7 · 2021-09-25T08:42:56Z

So basically there's a read-only directory containing a pristine helloSystem copy, that is mounted from an image file, then using unionfs mounts we could get the root filesystem used by the system, with /home, /var, /etc etc. mounted from separate ZFS subvolumes, /tmp mounted tmpfs, and use a similar overlay for apps.

probonopd · 2021-09-25T20:25:48Z

(Moving the bundle-related topics to #203)

grahamperrin · 2021-09-26T07:33:19Z

the FreeBSD ports/package system

From Technology Roadmap | FreeBSD Foundation (2021-09-17):

… the pkgbase project is very important, and are ready to help with the effort.

#161 (comment)

graphics drivers

Also from the Roadmap:

– and:

… per-release repositories have been proposed. Release-specific overlays could be used to, e.g., offer DRM packages simultaneously for 13.0 and 13.1 while both point releases are supported. …

probonopd · 2021-10-03T11:03:16Z

Why does sudo pkg install mtools mess around with LibreOffice? That kind of stuff is driving me insane.

probonopd · 2022-10-05T06:24:14Z

Currently this ends up with dependencies like

"py38-beautifulsoup": {origin: "www/py-beautifulsoup", version: "4.10.0"},
"py38-dateutil": {origin: "devel/py-dateutil", version: "2.8.1"},
"py38-pip": {origin: "devel/py-pip", version: "20.3.4"},
"py38-psutil": {origin: "sysutils/py-psutil", version: "5.8.0"},
"py38-pytz": {origin: "devel/py-pytz", version: "2021.1,1"},
"py38-qt5-dbus": {origin: "devel/py-qt5-dbus", version: "5.15.4_2"},
"py38-qt5-multimedia": {origin: "multimedia/py-qt5-multimedia", version: "5.15.4_3"},
"py38-qt5-network": {origin: "net/py-qt5-network", version: "5.15.4_2"},
"py38-qt5-qml": {origin: "lang/py-qt5-qml", version: "5.15.4_3"},
"py38-qt5-webengine": {origin: "www/py-qt5-webengine", version: "5.15.4_3"},
"py38-qt5-widgets": {origin: "x11-toolkits/py-qt5-widgets", version: "5.15.4_3"},
"py38-sqlite3": {origin: "databases/py-sqlite3", version: "3.8.12_7"},
"py38-xdg": {origin: "devel/py-xdg", version: "0.27"},
"py38-xmltodict": {origin: "devel/py-xmltodict", version: "0.12.0"},

We don't want py38-... in the dependencies because FreeBSD packages might switch to py39-..., at which point in time all py38-... packages should be updated to py39-... packages automatically.
How to do this?

Trying to specify py3*-beautifulsoup as a dependency fails with: pkg: Missing dependency 'py3*-beautifulsoup'.

grahamperrin · 2022-10-07T05:14:13Z

… For example, we would want to depend on x11-toolkits/py-qt5-widgets but not on py38-qt5-widgets specifically, so that once py39 is there …

py39 will never be there.

grahamperrin · 2022-10-08T07:55:29Z

Re: #161 (comment)

https://www.freebsd.org/administration/ there's now a FreeBSD Packages Management Team.

probonopd · 2022-10-16T17:54:38Z

Here we go again. I just want to install something yet pkg wants to REMOVE something.

FreeBSD% sudo pkg install linphone
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 37 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
        FreeCAD: 0.20_1
        PrusaSlicer: 2.3.3_4
        frei0r-plugins: 1.8.0
        frei0r-plugins-opencv: 1.8.0
        gdal: 3.5.0_1
        opencv: 4.5.5_7
        openvdb: 9.1.0
        shotcut: 21.03.21_3

New packages to be INSTALLED:
        bcg729: 1.1.1 [FreeBSD]
        bcmatroska2: 0.23 [FreeBSD]
        bctoolbox: 5.1.38_1 [FreeBSD]
        bcunit: 3.0.2p20220210 [FreeBSD]
        belcard: 5.1.32 [FreeBSD]
        belle-sip: 5.1.32_1 [FreeBSD]
        belr: 5.1.32 [FreeBSD]
        bzrtp: 5.1.32 [FreeBSD]
        libantlr3c: 3.4_1 [FreeBSD]
        liblinphone: 5.1.56 [FreeBSD]
        libsrtp2: 2.4.2 [FreeBSD]
        lime: 5.0.8_1 [FreeBSD]
        linphone: 4.4.8_2,1 [FreeBSD]
        mediastreamer: 5.1.55 [FreeBSD]
        openldap26-client: 2.6.3 [FreeBSD]
        ortp: 5.1.55 [FreeBSD]
        soci: 4.0.3_2 [FreeBSD]

Installed packages to be UPGRADED:
        boost-libs: 1.79.0_1 -> 1.80.0 [FreeBSD]
        botan2: 2.19.2 -> 2.19.2_2 [FreeBSD]
        codeblocks: 20.03_2 -> 20.03_3 [FreeBSD]
        libcmis: 0.5.2_5 -> 0.5.2_6 [FreeBSD]
        libixion: 0.17.0_1 -> 0.17.0_2 [FreeBSD]
        liborcus: 0.17.2_1 -> 0.17.2_2 [FreeBSD]
        libreoffice: 7.3.5.2 -> 7.4.2.3 [FreeBSD]
        poppler: 22.06.0 -> 22.09.0 [FreeBSD]
        poppler-glib: 22.06.0 -> 22.09.0_1 [FreeBSD]
        poppler-qt5: 22.06.0 -> 22.09.0_1 [FreeBSD]
        poppler-utils: 22.06.0 -> 22.09.0_1 [FreeBSD]
        sfcgal: 1.4.1_1 -> 1.4.1_2 [FreeBSD]

Number of packages to be removed: 8
Number of packages to be installed: 17
Number of packages to be upgraded: 12

The operation will free 519 MiB.
146 MiB to be downloaded.

Proceed with this action? [y/N]:

This has to be avoided at all cost.

probonopd · 2022-10-16T17:56:04Z

Why the heck does the system want to install Qt6 out of the blue now??? In the middle of the quarter...

FreeBSD% cat /etc/pkg/FreeBSD.conf 
# $FreeBSD$
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

FreeBSD% sudo pkg upgrade
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking for upgrades (458 candidates): 100%
Processing candidates (458 candidates): 100%
The following 475 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        cmake-core: 3.24.0 [FreeBSD]
        cmake-doc: 3.24.0 [FreeBSD]
        cmake-man: 3.24.0 [FreeBSD]
        espeak-ng: 1.51.1_3 [FreeBSD]
        gdk-pixbuf-xlib: 2.40.2 [FreeBSD]
        go119: 1.19.2 [FreeBSD]
        hdf5-110: 1.10.9 [FreeBSD]
        ksanecore: 22.08.1 [FreeBSD]
        libavif: 0.10.1_5 [FreeBSD]
        libdeflate: 1.14 [FreeBSD]
        libxcvt: 0.1.2_1 [FreeBSD]
        mlt7: 7.8.0_2 [FreeBSD]
        openjdk17: 17.0.4+8.1_1 [FreeBSD]
        openldap26-client: 2.6.3 [FreeBSD]
        pcaudiolib: 1.2 [FreeBSD]
        qca-qt5: 2.3.4_1 [FreeBSD]
        qt6-5compat: 6.3.2 [FreeBSD]
        qt6-base: 6.3.2 [FreeBSD]
        qt6-svg: 6.3.2 [FreeBSD]
        qtkeychain-qt5: 0.13.2_1 [FreeBSD]
        sonivox: 3.6.11 [FreeBSD]

grahamperrin · 2022-10-16T18:11:17Z

#161 (comment)

probonopd · 2022-10-16T18:20:50Z

This thread is my dumping ground for all gripes I have with pkg, and documentation why I don't think a package manager is a tool that should be given into the hands of unsuspecting non-technical end users ("mere mortals").

grahamperrin · 2022-10-16T18:53:35Z

… I don't think a package manager is a tool that should be given into the hands of unsuspecting non-technical end users …

In their hands:

System Update utility Utilities#33 (comment)

probonopd · 2022-10-16T18:54:52Z

GhostBSD maintains the type of thing that helloSystem does not want to maintain:

a repository – FreeBSD ports with GhostBSD ports.

grahamperrin · 2022-10-16T18:58:12Z

dumping ground

Fair enough. Increasingly difficult to follow, GitHub issues don't lend themselves to this type of thing.

Methinks, here's no longer the place to answer questions.

grahamperrin · 2022-11-26T09:23:01Z

helloSystem/ISO@8630218 (2022-10-15) referenced #161 (comment), but not in the commit message.

probonopd · 2023-01-20T19:49:18Z

@Cozmo007 which helloSystem are you on? This should (hopefully) be fixed in the recent 0.8.0 experimental builds thanks to
#161 (comment).

crees · 2023-04-03T15:34:09Z

Just a note here that it's occurred to me that Hello could mark all of the leaf packages it expects to keep as vital. This is a flag that makes pkg refuse to uninstall itself without forcing, and I think it might work to stop now missing packages from being deinstalled. I'll let you know how testing goes- going to have a go at the same time as the base package.

probonopd · 2023-06-18T19:05:15Z

Cannot build the ISO today because packages are again missing from one day to the next...

pkg-static: No packages available to install matching 'www/py-qt5-webengine' have been found in the repositories
pkg-static: No packages available to install matching 'falkon-qtonly' have been found in the repositories

Root cause possibly is (https://portsfallout.com/port/26602/, source):

=======================<phase: lib-depends    >============================
===>   py39-qt5-webengine-5.15.6_1 depends on shared library: libGL.soSegmentation fault
 - not found
===>   Installing existing package /packages/All/libglvnd-1.6.0.pkg
[124i386-quarterly-job-07] Installing libglvnd-1.6.0...
the most recent version of libglvnd-1.6.0 is already installed
===>   py39-qt5-webengine-5.15.6_1 depends on shared library: libGL.soSegmentation fault
 - not found
*** Error code 1

Similarly (source),

[124i386-quarterly-job-07] Extracting libX11-1.7.2,1: .......... done
===>   falkon-qtonly-22.12.3 depends on file: /usr/local/libdata/pkgconfig/x11.pc - found
===>   Returning to build of falkon-qtonly-22.12.3
===>   falkon-qtonly-22.12.3 depends on file: /usr/local/libdata/pkgconfig/xcb.pc - found
===========================================================================
=======================<phase: lib-depends    >============================
===>   falkon-qtonly-22.12.3 depends on shared library: libKF5Archive.so - not found
===>   Installing existing package /packages/All/kf5-karchive-5.104.0.pkg
[124i386-quarterly-job-07] Installing kf5-karchive-5.104.0...
[124i386-quarterly-job-07] Extracting kf5-karchive-5.104.0: .......... done
===>   falkon-qtonly-22.12.3 depends on shared library: libKF5Archive.soSegmentation fault
 - not found
*** Error code 1

Stop.
make: stopped in /usr/ports/www/falkon
=>> Cleaning up wrkdir
===>  Cleaning for falkon-qtonly-22.12.3
build of www/falkon@qtonly | falkon-qtonly-22.12.3 ended at Thu May 25 17:44:12 UTC 2023
build time: 00:00:13
!!! build failure encountered !!!

Any chance to get that still fixed in quarterly packages?

For me, that the previous version of each package is not kept until a new build has succeeded (like Linux distributions seem to be doing it) is the number 1 annoyance in using FreeBSD nowadays (along with the issue that graphics drivers are not treated the same as base).

grahamperrin · 2023-06-18T19:23:30Z

… packages … one day to the next … number 1 annoyance …

It seems that you want a repository, or repositories, to always fulfil the requirements of helloSystem.

From helloSystem/ISO#141 (comment) (2022-09-13):

… impossible for a single repository to completely fulfil everyone's requirements 24/7/365. …

I use poudriere to meet my requirements. I don't expect the FreeBSD Project to prioritise my requirements above those of other users (or projects).

Again, please consider using poudriere.

crees · 2023-06-18T19:26:22Z

I strongly disagree that disappearing packages are acceptable on the Project's main and recommended package repositories. You don't see Debian doing this, and portmgr AFAIK agrees that this is an undesirable state of affairs. It's simply a matter of QA, which is incredibly difficult.

grahamperrin · 2023-06-18T19:26:41Z

Any chance to get that still fixed in quarterly packages?

That's a question for the maintainer. https://www.freshports.org/www/falkon/

Postscript

Re: #384 (comment)

a different port
the same maintainer.

grahamperrin · 2023-06-18T20:30:33Z

This might be already amongst the ~~thirty-seven~~ one hundred and six hidden comments:

https://lists.freebsd.org/subscription/freebsd-pkg

#161 (comment)

The suggestion to use poudriere is to be realistic about the status quo.

#161 (comment)

… Root cause possibly is (portsfallout.com/port/26602, source): …

That's for 12.4-RELEASE on a different platform. https://www.freebsd.org/platforms/i386/

… previous version …

Please see:

Previous versions of www/falkon-qtonly #506

probonopd · 2023-06-18T20:52:19Z

That's for 12.4-RELEASE on a different platform. https://www.freebsd.org/platforms/i386/

Sorry, my bad. How can I find out why it is missing for 13.1 arm64?

crees · 2023-06-18T20:59:56Z

Seems to be qt6-webengine failing.

https://pkg-status.freebsd.org/builds/default:default:131amd64:8dc4a71c010e:beefy16

I think the beefy machines that show the logs are IPv6 only, so you may have better luck than me following the links to the build logs.

probonopd · 2023-06-18T21:03:49Z

Thing is, helloSystem isn't even (trying to) use Qt 6.
Hopefully we won't be forced to ship both Qt 5 and 6 in parallel... (man do I dislike major version upgrades).

Why was everything working just 2 days ago when I was able to build hello-0.8.2_0H337-FreeBSD-13.2-amd64.iso just fine? Did some packages switch from Qt 5 to Qt 5 in quarterly packages (which I expect to change only once per quarter)?

Said differently, I wish that all software that depends on Qt 5 should get a -qt6 variant that one can opt into (or not opt into, which helloSystem would like to do at this point until all packages are available with -qt6 variants).

Understandable feature request? Realistic?

grahamperrin · 2023-06-19T01:32:17Z

… quarterly packages (which I expect to change only once per quarter)? …

No, changes (merges) may occur at any time:

https://wiki.freebsd.org/Ports/QuarterlyBranch

grahamperrin · 2023-06-19T07:09:16Z

#161 (comment)

… I think the beefy machines that show the logs are IPv6 only, …

True.

#161 (comment)

… How can I find out why it is missing for 13.1

Reminder:

– NB the most recent answer under 382.

arm64?

AMD64, not 64-bit ARMv8, for helloSystem at this time.

Via https://www.freebsd.org/platforms/:

grahamperrin · 2023-06-21T05:26:54Z

#161 (comment)

… helloSystem isn't even (trying to) use Qt 6. …

6 was irrelevant.

Please see the two most recent answers under:

How to find the reason for missing packages? #384

#384 (comment) (the first of the two) identifies:

the reason for the skip of falkon-qtonly-22.12.3
the subsequent success (2023-06-19).

probonopd · 2023-08-27T09:51:46Z

Looks like there is at least some distant hope for the graphics drivers, as Zirias explained:

The issue is that there's always just one package repository for each major #FreeBSD version, because ABI is stable across minor releases (so, having multiple repositories is just unnecessary).

Unfortunately, this is not true for some in-kernel ABI needed by these drivers. So, the binary package can only work on one or the other minor release.

During the 3-month transition period from one to the next minor, this will be the older one for the official packages. Nevertheless, compiling the port yourself with the newer minor will work perfectly fine.

Indeed, having the drivers in base would solve this particular issue. It might happen again, main issue was licensing.

(...)

It's exactly 3 months where you can't use the packages from official repositories with the newer minor release, and

This can by definition never affect a .0 release, that's a new major, with a new ABI and therefore a new package repository as well.

As I said, (re-)inclusion in base will probably happen when the last piece of GPL-licensed glue-code is rewritten (the drivers themselves are unencumbered).

Until then, is it a huge problem to integrate building of a port in your scripts?

(...)

Here:
https://github.com/freebsd/drm-kmod/tree/master/linuxkpi/gplv2/include

(edit: and yes, there's almost nothing left ...)

There used to be a LOT of gpl-licensed glue-code, and continuous work was done replacing it.

The drm-kmod drivers once were in base. Then, all this glue-code was required to upgrade them, so they could not live in base any more (IIRC this was around between FreeBSD 11 and 12). I now see the chance they might be reintegrated soon. But I don't work in that area myself, sorry 😉

So the way I understand it, https://github.com/freebsd/drm-kmod/tree/master/linuxkpi has two directories: "bsd" and "gplv2". All "gplv2" code needs to be converted to "bsd", so that in the end there is no more "gplv2" directory. Then the graphics drivers might have a chance of being built, tested, and released alongside base.txz (again).

The "bsd" part shouldn't be too large either because what's there is constantly reintegrated in the base kernel (and only kept for as long as the drm-kmod port must support older kernels that don't have it yet).

probonopd · 2024-07-09T19:42:02Z

Point in case why a package manager should never be in the hands of an end user, or else he might end up with a broken system:

https://youtu.be/DG7AHQhxBKY?feature=shared&t=640

He is using a helloSystem released years ago, and is now trying to use pkg (via our GUI wrappers, and via the command line), and it ends in a desaster.

We should probably remove pkg entirely from the system and only offer application bundles. Like the classic Mac, macOS, iOS, and Android do.

(This is not FreeBSD specific; a similar but worse thing happened to Linus Tech Tips: https://www.youtube.com/watch?v=0506yDSgU7M)

darkoverlordofdata · 2024-07-10T23:36:44Z

Indeed, the latest version (hello-0.9.0_0I67-FreeBSD-14.0-amd64.iso) has the same issue. I recently reinstalled it and when I install Firefox, it breaks the system. I don’t see this issue in GhostBSD. It looks like they have their own repo, maybe they use poudriere?

…

On Tue, Jul 9, 2024 at 12:42 PM probonopd ***@***.***> wrote: Point in case why a package manager should never be in the hands of an end user, or else he might end up with a broken system: https://youtu.be/DG7AHQhxBKY?feature=shared&t=640 He is using a helloSystem released years ago, and is now trying to use pkg (via our GUI wrappers, and via the command line), and it ends in a desaster. We should probably remove pkg entirely from the system and only offer application bundles. Like the classic Mac, macOS, iOS, and Android do. — Reply to this email directly, view it on GitHub <#161 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJUE4S3DPLWRVYRLVNX67LZLQ4J7AVCNFSM6AAAAABKTQA4BCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJYGUYTOOJQGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Bruce Davidson

grahamperrin · 2024-07-15T05:26:40Z

GhostBSD

#161 (comment)

… they have their own repo, …

True.

https://github.com/ghostbsd/ghostbsd-ports

This repository combines:

the FreeBSD ports collection

PkgBase-oriented GhostBSD ports.

The combination allows simplified, coherent updates to the (base) operating system and other installed software. …

It's normal to have just one repo enabled: for the base operating system and ports.

grahamperrin@mowa219-gjp4-ghostbsd-14-vm:~ % ls -hln /etc/pkg
total 5
-rw-r--r--  1 0 0  432B Dec 25  2023 GhostBSD.conf
grahamperrin@mowa219-gjp4-ghostbsd-14-vm:~ % pkg -vv | grep -B 1 -e url -e priority
  GhostBSD: { 
    url             : "https://pkg.ghostbsd.org/stable/FreeBSD:14:amd64/latest",
    enabled         : yes,
    priority        : 0,
--
  FreeBSD-ports: { 
    url             : "pkg+https://pkg.FreeBSD.org/FreeBSD:14:amd64/latest",
    enabled         : no,
    priority        : 3,
--
  GhostBSD_FR: { 
    url             : "https://pkg.fr.ghostbsd.org/stable/FreeBSD:14:amd64/latest",
    enabled         : no,
    priority        : 0,
grahamperrin@mowa219-gjp4-ghostbsd-14-vm:~ %

FreeBSD

Official packages for base became available last year.

There's separation between base and ports – the first two of the four repos in this example:

% pkg -vv | grep -B 1 -e url -e priority
  FreeBSD-ports: { 
    url             : "pkg+https://pkg.freebsd.org/FreeBSD:15:amd64/latest",
    enabled         : yes,
    priority        : 2,
--
  FreeBSD-base: { 
    url             : "pkg+https://pkg.freebsd.org/FreeBSD:15:amd64/base_latest",
    enabled         : yes,
    priority        : 0,
--
  aninstaller: { 
    url             : "file:////media/aninstaller/packages/FreeBSD:15:amd64",
    enabled         : no,
    priority        : 0
--
  local-poudriere: { 
    url             : "file:///usr/local/poudriere/data/packages/main-default",
    enabled         : yes,
    priority        : 3
%

https://wiki.freebsd.org/PkgBase

grahamperrin · 2024-07-15T06:01:20Z

#161 (comment) (2021-09-23) it seems that:

helloSystem will avoid offering packages of ports.

From the same comment:

… keep helloSystem fully compatible with FreeBSD ports and packages …

#161 (comment) last week:

… We should probably remove pkg entirely from the system and only offer application bundles. …

FreeBSD ports collection

https://www.freebsd.org/status/report-2024-01-2024-03/#_ports_collection more than thirty thousand ports.

https://docs.freebsd.org/en/books/handbook/ports/#ports-using describes using the collection.

Less verbosely, with Firefox as an example, at https://www.freshports.org/www/firefox/#add the alternative to command line use of pkg is:

cd /usr/ports/www/firefox/ && make install clean

Things such as make install will be impossible without pkg.

In the absence of thousands of ports:

how many application bundles might be maintained by helloSystem?

probonopd changed the title ~~Thoughts on ports/package issues~~ Thoughts on the FreeBSD ports/package system Mar 31, 2021

probonopd added a commit to helloSystem/ISO that referenced this issue May 22, 2021

devel/py-xdg [ci skip]

7488e13

helloSystem/hello#161

probonopd added a commit to helloSystem/ISO that referenced this issue May 22, 2021

devel/py-xdg

45021e0

helloSystem/hello#161

probonopd closed this as completed May 23, 2021

probonopd reopened this May 23, 2021

grahamperrin mentioned this issue Sep 26, 2021

Turn all applications into application bundles #203

Open

This was referenced Sep 26, 2021

Need to think about an update strategy helloSystem/ISO#55

Open

Consider using pkgbase helloSystem/ISO#166

Open

probonopd mentioned this issue Jan 18, 2023

Black Screen Bug after 'pkg upgrade' helloSystem/ISO#442

Closed

Thoughts on the FreeBSD ports/package system #161

Thoughts on the FreeBSD ports/package system #161

Comments

probonopd commented Mar 31, 2021 • edited Loading

crees commented Mar 31, 2021

probonopd commented Apr 1, 2021

probonopd commented Apr 2, 2021 • edited Loading

kettle-7 commented Apr 30, 2021

probonopd commented Apr 30, 2021

probonopd commented May 22, 2021 • edited Loading

grahamperrin commented May 22, 2021

probonopd commented May 23, 2021 • edited Loading

grahamperrin commented May 25, 2021

probonopd commented Sep 14, 2021

grahamperrin commented Sep 14, 2021 • edited Loading

xk2600 commented Sep 23, 2021 • edited Loading

probonopd commented Sep 23, 2021

kettle-7 commented Sep 24, 2021

kettle-7 commented Sep 24, 2021

probonopd commented Sep 24, 2021 • edited Loading

mszoek commented Sep 24, 2021

kettle-7 commented Sep 25, 2021

kettle-7 commented Sep 25, 2021

probonopd commented Sep 25, 2021

grahamperrin commented Sep 26, 2021

probonopd commented Oct 3, 2021 • edited Loading

probonopd commented Oct 5, 2022 • edited Loading

grahamperrin commented Oct 7, 2022

grahamperrin commented Oct 8, 2022

probonopd commented Oct 16, 2022

probonopd commented Oct 16, 2022 • edited Loading

grahamperrin commented Oct 16, 2022

probonopd commented Oct 16, 2022

grahamperrin commented Oct 16, 2022

probonopd commented Oct 16, 2022

grahamperrin commented Oct 16, 2022

grahamperrin commented Nov 26, 2022

probonopd commented Jan 20, 2023 • edited Loading

crees commented Apr 3, 2023

probonopd commented Jun 18, 2023 • edited Loading

grahamperrin commented Jun 18, 2023

crees commented Jun 18, 2023

grahamperrin commented Jun 18, 2023 • edited Loading

Postscript

grahamperrin commented Jun 18, 2023 • edited Loading

probonopd commented Jun 18, 2023

crees commented Jun 18, 2023

probonopd commented Jun 18, 2023 • edited Loading

grahamperrin commented Jun 19, 2023 • edited Loading

grahamperrin commented Jun 19, 2023

grahamperrin commented Jun 21, 2023

probonopd commented Aug 27, 2023 • edited Loading

probonopd commented Jul 9, 2024 • edited Loading

darkoverlordofdata commented Jul 10, 2024 via email

grahamperrin commented Jul 15, 2024

GhostBSD

FreeBSD

grahamperrin commented Jul 15, 2024

FreeBSD ports collection

probonopd commented Mar 31, 2021 •

edited

Loading

probonopd commented Apr 2, 2021 •

edited

Loading

probonopd commented May 22, 2021 •

edited

Loading

probonopd commented May 23, 2021 •

edited

Loading

grahamperrin commented Sep 14, 2021 •

edited

Loading

xk2600 commented Sep 23, 2021 •

edited

Loading

probonopd commented Sep 24, 2021 •

edited

Loading

probonopd commented Oct 3, 2021 •

edited

Loading

probonopd commented Oct 5, 2022 •

edited

Loading

probonopd commented Oct 16, 2022 •

edited

Loading

probonopd commented Jan 20, 2023 •

edited

Loading

probonopd commented Jun 18, 2023 •

edited

Loading

grahamperrin commented Jun 18, 2023 •

edited

Loading

grahamperrin commented Jun 18, 2023 •

edited

Loading

probonopd commented Jun 18, 2023 •

edited

Loading

grahamperrin commented Jun 19, 2023 •

edited

Loading

probonopd commented Aug 27, 2023 •

edited

Loading

probonopd commented Jul 9, 2024 •

edited

Loading