Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stack overflow error during build, during loadup-db-from-full.sh execution #1935

Open
sethm opened this issue Dec 24, 2024 · 27 comments
Open

Stack overflow error during build, during loadup-db-from-full.sh execution #1935

sethm opened this issue Dec 24, 2024 · 27 comments

Comments

@sethm
Copy link

sethm commented Dec 24, 2024

Describe the bug

I am seeing a failure in the loadup-db-from-full.sh when trying to build Medley from a Git checkout. The screenshot and build log below demonstrate the error in full.

To Reproduce

I have three repositories checked out under the directory $HOME/Interlisp/, at the following Git hashes:

  • $HOME/Interlisp/maiko (git hash f23a43f1a0cffebdfa08660dbb5cd1a1dac7f7c3)
  • $HOME/Interlisp/medley (git hash f896885720d31c92f8bc244d586a5b2534107f0f)
  • $HOME/Interlisp/notecards (git hash 8bc5368052bbcedc143e4fd1e3e7e34688b1b732)

Maiko was built with:

$ cd $HOME/Interlisp/maiko/bin
$ ./makeright sdl
$ ./makeright init

Here were the steps I took when I encountered the error after building Maiko:

$ cd $HOME/Interlisp/medley
$ (time ./scripts/loadup-all.sh -apps && time ./scripts/loadup-db.sh) 2>&1 | tee -a build.log

The execution of ./scripts/loadup-all.sh -apps ran successfully to completion. The execution of ./scripts/loadup-db.sh stopped while running ./scripts/loadup-db-from-full.sh when Medley encountered a stack overflow error (attached below).

Expected behavior

The full build should complete without error.

Screenshots

medley_build_failure

The complete build log is here, to the point where it stopped:

build.log

Context (please complete the following information):

  • Are you using online.interlisp.org? no
  • OS: Linux
  • OS Version: Arch Linux kernel 6.12.6-arch1-1
  • Display/window system: Wayland / KDE Plasma 6.2.4
  • Host arch: x86_64
  • Maiko version: f23a43f1a0cffebdfa08660dbb5cd1a1dac7f7c3
  • IL:MAKESYSDATE: N/A

Additional context

I'm sure I'm probably something wrong here or missing a step. Maybe this is a documentation omission rather than a build error?
@pamoroso
Copy link
Contributor

I don't expect any different outcomes but in general it's worth executing ./makeright sdl cleanup prior to ./makeright sdl.

@masinter
Copy link
Member

  • it looks like it would reduce the differences in the environment to also clone 'test' and 'loops' repositories at the same level as medley, maiko notecards.
  • I suspect this might be a SDL problem. Does the problem happen if you do 'makeright x' rather than 'makeright sdl'?

@nbriggs
Copy link
Contributor

nbriggs commented Dec 25, 2024

@masinter, @sethm - that's an XQuartz window on a Mac, right? So this is SDL on the Linux box driving a remote X11 display back on a Mac ? Whether the display subsystem is X11 or SDL (and SDL's choice of back-end display) won't make a difference to things like Lisp's stack.

@pamoroso the Makefiles have the correct dependencies, as far as I know, so ./makeright sdl cleanup should never be necessary unless you're modifying things like the Makefiles themselves, which aren't a dependency of any source file.

@masinter
Copy link
Member

masinter commented Dec 25, 2024 •
edited
Loading

the problem doesn't seem to have anything to do with SDL. At least, I get a stac overflow when I clone the repos anew (mkdir retry; cd retry, then gh repo clone interlisp/maiko and .../medley and /notecards, run cd maiko\bin, .\makeright x and .makeright init.

STACKOVERFLOW.txt

it makes a FULL.SYSOUT that doesn't have TEdit loaded.
but later on the script loads something that loads TEdit.
./medley -y /tmp/loadups-1159/full.sysout -r -

I don't remember where *ENV* comes from, but the stack is full of frames looking like this:
image

@sethm
Copy link
Author

sethm commented Dec 25, 2024 •
edited
Loading

@nbriggs @masinter It's actually KDE Plasma 6.2 (which does look very much like macOS!), so it's running locally under Wayland. I must confess I'm very new to Wayland and KDE myself, I have a lot of X11 history under my belt so this is new territory.

I have tried a few things based on this conversation so far:

  1. I tried the X11 build by cleaning up the maiko directory and then running "makeright x", but it behaves the same and fails in the same place.
  2. I added the "loops" and "test" repositories to my $HOME/Interlisp/ directory. It still fails with the same stack overflow error, but this time the two messages Note: Can't find a clone directory for LOOPS and Note: Can't find a clone directory for TEST that are shown in the first screenshot I attached no longer appear, so that's a nice improvement!
  3. I tried the same process under good old twm running in X11 on the same laptop and operating system, but it had the same stack overflow error.

I think I need to learn basic Interlisp debugging so I can explore the stack trace, hopefully that will give some more info.

EDIT: Looks like @masinter added a comment just as I was completing my own, race condition! :)

@nbriggs
Copy link
Contributor

nbriggs commented Dec 26, 2024

@sethm - I guess KDE Plasma and XQuartz sourced their logo from the same place!
image

@masinter - A full.sysout should always have TEdit loaded, shouldn't it?

@masinter
Copy link
Member

the loadup process doesn't halt when an error occurs during the loadup. This was something Ron brought up but I'm not sure an issue was ever made up. So yes, a full sysout should always have TEDIT loaded.

I'm wondering about the file name of TEDIT -- it's lower case in the arguments in the call, but not in the file name.

The stack overflow I wonder if the difference is a case sensitive file system on linux but not on macos.

@masinter
Copy link
Member

@nbriggs when I change Maiko to an older version from an older release -- e.g., medley-full-wsl-x86_64-240319-2f7972e7_240211-ce20acca

I no longer get the stack overflow. I believe the problem has to do with the changes to the caching of directory enumeration for {DSK} files when Medley is run on a case sensitive file system.

@nbriggs
Copy link
Contributor

nbriggs commented Dec 26, 2024

@masinter that's odd, I just did a loadup-full.sh and loadup-db.sh on FreeBSD which also has a case-sensitive file system -- but I'll try some more tests.

@nbriggs
Copy link
Contributor

nbriggs commented Dec 26, 2024

@masinter - so the first place things go wrong is that it is producing a "full.sysout" where TEDIT is not loaded, right? That then cascades into the loadup-db.sh failing?

@nbriggs
Copy link
Contributor

nbriggs commented Dec 26, 2024

@sethm

$HOME/Interlisp/medley (git hash f896885)

this git hash is before the TEdit round 5 merge to master, can you retry at the current head of the medley branch?

I haven't been able to reproduce the failure on a FreeBSD system which also has a case-sensitive file system. I don't have a Linux system to test on.

@rmkaplan
Copy link
Contributor

rmkaplan commented Dec 26, 2024 via email

@nbriggs
Copy link
Contributor

nbriggs commented Dec 26, 2024

@rmkaplan that works perfectly - both before and after the 5th round merge, with my current maiko, in every configuration I've tried.

@rmkaplan
Copy link
Contributor

rmkaplan commented Dec 26, 2024 via email

@rmkaplan
Copy link
Contributor

rmkaplan commented Dec 26, 2024 via email

@masinter
Copy link
Member

@rmkaplan @nbriggs This is a problem with maiko. I have a clear case where the loadup fails on WSL with current maiko, but doesn't fail with an older maiko.

The loadup also doesn't fail on macos ith current darwin. but renamefile etc. aren't the problem.

Now I see that UPPER-CASE-DILE-NAMES is T and FINDFILE(FILESEts) retrns NILL.

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024 •
edited
Loading

*UPPER-CASE-FILE-NAMES* is T in my full.sysout and *DSK-UPPER-CASE-FILE-NAMES* is NIL

I'm not saying it's NOT a Maiko problem -- but I've tried FreeBSD (because I have one) and Linux Mint 22 (because I had an image I could run on VirtualBox), and I can't develop a smaller test-case because I can't reproduce the problem.

Do you have a test case that doesn't involve doing a loadup?

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024

All variations of case, atom or string, of FILESETS in (FINDFILE "FILESets") returns (on a Linux Mint system) the atom
{DSK}<home>briggs>Projects>medley>sources>FILESETS.;1

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024

@masinter - can you try on WSL with maiko at 86ba485c3eb25e9d734d60c48d7149b69668dd97 (before version array changes) and fae5c4956ee005339c029bcb1b274f51d3560253 (with version array changes)?

@sethm
Copy link
Author

sethm commented Dec 27, 2024

I just wanted to confirm that with the latest changes in medley and maiko, everything is working perfectly now! The full build completes without any errors that I've seen. Thank you all for your work!

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024

@sethm - well that's confusing. I didn't change anything in Maiko and you were already at the latest released version.

@masinter
Copy link
Member

masinter commented Dec 27, 2024 •
edited
Loading

not working for me in WSL:

cd ../maiko
  rm -r *linux*
  git checkout  fae5c4956
  cd bin
  ./makeright x
  ./makeright init
  cd ../../medley/
 time ./scripts/loadup-all.sh -apps
  echo failed

  cd ../maiko
  rm -r *linux*
  git checkout 86ba485
  cd bin
  ./makeright x && ./makeright init
  cd ../../medley/
  time ./scripts/loadup-all.sh -apps
  echo succeeded

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024

@masinter Is your medley branch up to date?

@masinter
Copy link
Member 

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
larry@MSM:~/il/medley$ git log
commit 5b37dd09db4bba3633a5bed30de4bb53a7ced539 (HEAD -> master, origin/master, origin/HEAD)
Author: rmkaplan <[email protected]>
Date:   Wed Dec 25 13:06:35 2024 -0800

@nbriggs
Copy link
Contributor

nbriggs commented Dec 27, 2024

Well, I don't know what to say. Does the dribble output show what actually failed? It's now working for Seth on WSL. It's working for me (I just rolled back to the same commit that failed for you) on Mint with a case-sensitive file system. What version of Linux are you on for WSL - I can see if I can get the same .iso and run it on VirtualBox.

@masinter
Copy link
Member

I'm not sure what's going on. Current master for medley and maiko now work for me with an updated Ubuntu distro in WSL. I get failures of different kinds with the Pengwin distro (WLinux). I had trouble with an older Ubuntu (I think; can't try it out now because I updated). Updating pengwin (using sudo apt update and sudo apt upgrade) gives an error.

I like the pengwin distro but I can use ubuntu for now.

We need to fix the loadup so that it doesn't plow on if there is an earlier error.

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://cli.github.com/packages stable InRelease: The following signatures were invalid: EXPKEYSIG 23F3D4EA75716059 GitHub CLI <[email protected]>
W: Failed to fetch https://cli.github.com/packages/dists/stable/InRelease  The following signatures were invalid: EXPKEYSIG 23F3D4EA75716059 GitHub CLI <[email protected]>'

@nbriggs
Copy link
Contributor

nbriggs commented Dec 29, 2024

That error (signature verification) is because somebody's GPG key has expired.
See hello-robot/stretch_install#80 and fixes it references - perhaps that will help get your pengwin distro back to life.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@sethm @masinter @nbriggs @pamoroso @rmkaplan