repo census

[AstroSnail]

from my analysis of the requires, all the files reachable from runtest are:

• alicorn-expressions
• alicorn-utils
• base-env
• derivers
• environment
• evaluator
• fibonacci-buffer
• format
• format-adapter
• internals-interface
• lazy-prefix-tree
• metalanguage
• pretty-printable-trait
• pretty-printer
• profile
• reducer-utils
• terms
• terms-generators
• terms-pretty
• traits

besides runtest, there are other tests, and few of these give a clear indication of pass/fail:

name	runtime	pass/fail/crash?
alicorn-smoketest	any	crashes, out of date (requires 'alicorn-evaluator', deleted 2023-09-12 e068b1f)
metalanguage-smoketest	any	crashes, out of date (requires 'temp-format', deleted 2023-12-19 5a3788e)
test	luajit	passes
test-derive-eq	luvit	runs to completion, not clear whether pass/fail
test-derive-pretty-print	luvit	runs to completion, not clear whether pass/fail
test-eval	any	crashes, out of date
test-fitsinto	luvit	fails, out of date (uses evaluator.fitsinto, removed 2024-06-22 9ca74db)
test-format-adapter	any	crashes, out of date (requires 'temp-format', deleted 2023-12-19 5a3788e) not a test, an old version of format-adapter used by metalanguage-smoketest
test-map-array	luvit	runs to completion, not clear whether pass/fail
test-pretty-print	luajit	stub
test-syntax-to-terms-to-eval	luvit	runs to completion, not clear whether pass/fail crashes, out of date
test-terms	luvit	runs to completion, likely passes
test-traits	any	runs to completion, likely passes
test-util	luvit	passes

out of all the lua files that are left, terms-gen-meta is used to regenerate the files under types/, and all the others seem to be obsolete code (besides repo-wide refactors, they didn't get meaningful changes since 2023-07-16):

• backend-builder
• core-operatives
• cotuple (mentioned in base-env)
• modules (mentioned in base-env)
• operative-scratch
• syntax-schema
• testlanguage (used in metalanguage-smoketest)
• typesystem (used in alicorn-smoketest, mentioned in alicorn-expressions, base-env, terms)

there's also a samplecode.alc file that seems intended for illustration and not for running

---

• do we want to delete the obsolete files? all of them? or which ones?
• which tests still matter? should we fix those to clearly state pass/fail? should we delete those that don't matter?
• should we move tests to a tests directory?
• should tests be added to checks or automated in some fashion?
• should terms-gen-meta be added to a commit hook?

[ErikMcClure]

• We should delete any obsolete files people do not specifically ask to be preserved

• test-util should be adapted to a more standard testing framework and kept up to date inside whatever new test folder we create. I noticed that the tests seemed to be inconsistent about how they work and don't have a central test script to run off of, but I have no idea what lua tests should look like. Open mentioned wanting to keep the metalanguage-smoketest and the testlanguage, but they might have to be updated.

• yes, move to a tests directory.

• Yes, once we standardize on a test framework of some kind so the tests are actually consistent, we should be running them in CI

• Probably yes? Could also add terms-gen-meta to the start of the test harness so the tests are always running up to date code.

[AstroSnail]

terms-gen-meta doesn't generate code that can be meaningfully tested. it's for lsp only.

it would be helpful to have it in a commit hook like we currently have stylua formatting check in the commit hook, to make sure the lsp types of terms constructors/unwraps/etc are always up to date

[AstroSnail]

currently tests are indeed not centralized afaict. they are run individually and some require a specific runtime (as noted in the table)

there are two test frameworks that i can see currently in use: tap (for luvit, used in test-fitsinto) and luaunit (for luajit, used in test, and formerly test-pretty-print)

[ErikMcClure]

I believe i had trouble getting tap working on windows, so we would need to figure that out if we wanted to use it.

[AstroSnail]

i'll probably need guidance to be sure that i'm maintaining the intent of the tests. particularly seeing how the smoketests and test-syntax-to-terms-to-eval seem similar in shape to runtest

[aiverson]

The smoke tests are intended to run a minimal amount of stuff all the way through their respective target; the metalanguage smoke test defines a minimal interpreted language based on the metalanguage system and evaluates some code in it to exercise a whole metalanguage pipeline. The Alicorn smoke test instantiates the whole Alicorn syntax and type system and makes it run essentially 2+2=4. The various test-$system tests are intended to run their system in isolation and test it on many cases, checking that the outputs match what we expect. Run test then launches a whole Alicorn interpreter and runs actual sizable code that does significant interaction with the compiler and goes through more of the bootstrap path to build and exercise high level capabilities inside the language. It should probably be refactored into a separated prelude which can be used in various tests and files as we build the base library, or have static tests built in with assertions in Alicorn to check them as the prelude gets checked.

[AstroSnail]

the unformatter test in test.lua is noticeably slower than all the other tests, do we want to look into that?

[AstroSnail]

at this time the only failing tests left are alicorn-smoketest, test-eval, and test-fitsinto

[ErikMcClure]

we're ignoring test-fitsinto as it's not relevant right now, but we should probably fix test-eval. Alicorn-smoketest is the exact same as metalanguage-smoketest and has been deleted.

@bootstrap-prime needs to explain why paranthesis parsing is quadratic