Skip to content

Getting Started Guide

Jump to bottom
Alex Ameen edited this page Feb 23, 2023 · 2 revisions

Getting Started

Let’s start out with a simple project with only registry dependencies and no install scripts in the dependency graph.

Our project will be a simple Typescript CLI tool. This means we will have a build stage, a global installation target, and a dist target ( a tarball suitable for publishing ).

I want to highlight the fact that our dependency graph only contains trivial packages that do not require installations - we will cover handling these dependencies in another guide. For now we’re keeping things simple.

Please note that the boilerplate default.nix, foverrides.nix, and a related file floco-cfg.nix are provided in the nix flake init -t github:aakropotkin/floco; template. The template is functionally the same as this examples’ files except that portions of default.nix are moved to a general purpose file floco-cfg.nix ( our trivial example doesn’t have any use for this separation ).

Creating the Project

In the spirit of completeness, this guide uses real sources so that we can automatically detect if it’s out of date and so that you can copy these files to try them on your box.

This executable will be trivial, we’ll just print a version string for lodash, but we’ll gate the output behind a few typed expressions to prove that they work.

// index.ts
import {
  VERSION, flatten, isEqual, last, omit, pick, pickBy, uniq
} from 'lodash';

const f : Array<number> = flatten( [[1, 2], [3, 4]] );
console.log( f );
const l : number = last( [1, 2, 3, 4] )
console.log( l );
const u : Array<number> = uniq( [1, 2, 3, 1, 2, 3, 2] );
console.log( u );
console.log( VERSION );

A JavaScript CLI wrapper that requires the built form of our TypeScript shit. Notably we won’t set executable permissions on this just to prove that the installer handles that for us.

#! /usr/bin/env node
// bin.js
require( './index.js' );

The package.json, nothing special here. The only note I’ll make is that the script names build and prepublish are treated as synonyms, and that prebuild and postbuild are also recognized but preprepublish and postprepublish are not because their names are completely idiotic and they aren’t commonly used so I ain’t supporting them.

{
  "name": "@floco/test",
  "version": "4.2.0",
  "bin": {
    "test": "./bin.js"
  },
  "scripts": {
    "clean": "rm -rf ./node_modules ./index.js",
    "build": "tsc ./index.ts"
  },
  "devDependencies": {
    "@types/lodash": "^4.14.191",
    "typescript": "^4.9.4"
  },
  "dependencies": {
    "lodash": "^4.17.21"
  }
}

Just for kicks we can ensure this project works using npm by running:

$ nix shell nixpkgs#nodejs-14_x nixpkgs#nodejs-14_x.pkgs.npm;
$ npm install --lockfile-version=3;
$ npm run build;
$ node ./bin.js
[ 1, 2, 3, 4 ]
4
[ 1, 2, 3 ]
4.17.21
$ npm run clean;
$ rm ./package-lock.json;  # Optional

Setting up floco and nix

To Nixify our project we’ll generate a file called pdefs.nix from our existing package-lock.json. Once generated the package-lock.json file can be deleted allowing floco and nix to fly solo.

After that we’ll add a default.nix file to expose a few derivations to the nix CLI.

# default.nix
# ============================================================================ #
#
# Package shim exposing installable targets from `floco' modules.
#
# ---------------------------------------------------------------------------- #

{ floco  ? builtins.getFlake "github:aakropotkin/floco"
, lib    ? floco.lib
, system ? builtins.currentSystem
}: let

# ---------------------------------------------------------------------------- #

  fmod = lib.evalModules {
    modules = [
      floco.nixosModules.floco
      { config.floco.settings = { inherit system; basedir = ./.; }; }
      # Loads our generated `pdefs.nix' as a "module config".
      ./pdefs.nix
    ];
  };

# ---------------------------------------------------------------------------- #

  # This attrset holds a few derivations related to our package.
  # We'll expose these below to the CLI.
  pkg = fmod.config.floco.packages."@floco/test"."4.2.0";

# ---------------------------------------------------------------------------- #

in {
  inherit (pkg)
    dist      # A tarball form of our built package suitable for publishing
    prepared  # The "prepared" form of our project for use by other Nix builds
    global    # A globally installed form to run our executable
  ;
  built = pkg.built.packages;  # Our project in it's "built" state
}

# ---------------------------------------------------------------------------- #

Lets generate pdefs.nix and take this bad boy for a spin:

$ nix run github:aakropotkin/floco#fromPlock;
$ rm *~||:;  # Delete any backup files that might've been created

# Run our executable from the `global' target.
# We add the flag `-L' to show build logs.
# If this is your first time building with `floco' this may take a minute to
# initialize your box's cache, but successive builds will fly.
$ nix run -f ./. -L global;
...
test-built> unpacking sources
test-built> unpacking source archive /nix/store/4xna8iwywa57wrv8j64p4cimhy819sq3-basic
test-built> source root is basic
test-built> patching sources
test-built> configuring
test-built> building
test-built> installing
test-built> post-installation fixup
test-built> shrinking RPATHs of ELF executables and libraries in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0
test-built> patching script interpreter paths in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0
test-built> /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0/bin.js: interpreter directive changed from "#! /usr/bin/env node" to "/nix/store/mwd1dxh5rcy0wi9vgv2brlxpr5gmngr7-nodejs-14.20.1/bin/node"
test-built> checking for references to /build/ in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0...

# If we run again you'll see we skip the build:
$ nix run -f ./. -L global;

# Lets build our tarball:
$ nix build -f ./. dist;

$ tar tzf ./result;
package/bin.js
package/index.js
package/package.json
package/default.nix
package/pdefs.nix
package/index.ts

Pretty slick. Right off the bat you might be asking: how is this any different from npm, aside from the fact that I had to write extra files and read a guide? It’s a fair question, and in the next few sections we’ll try to win you over.

Opportunities to Optimize

Upfront let’s just say that there isn’t a practical reason to optimize this trivial package; but as an exercise let’s just treat it as a playground to show techniques that can be used out in the field where it really matters.

Globalization

The largest opportunity to speed up most builds is by treating CLI tools as “global” dependencies. The reason this speeds up builds is that rather than copying the contents of these dependencies into the build areas we can instead add them to PATH. Doing so allows us to avoid copying the entire dependency closure - not just the target package. As an added bonus this tends to simplify ideal tree formation.

In a tool like npm this is like doing npm i -g foo except that in the case of floco we actually have the ability to declare these in a standardized way. With our example project typescript can be handled this way.

To mark typescript as a globally installed dependency we will delete it from a fragment of our config metadata named treeInfo, and then move it to the buildInputs field of our built target. We could accomplish this same goal with other types of config settings, which we might prefer for projects that we regenerate frequently; but this is the simplest approach.

Pop open the pdefs.nix file and we’ll drop typescript.

# `pdefs.nix'
{
  floco = {
    pdefs = {
      "@floco/test" = {
        "4.2.0" = {
          ident = "@floco/test";
          version = "4.2.0";
          # ...
          treeInfo = {
            "@types/lodash" = {
              key = "@types/lodash/4.14.191";
              dev = true;
            };
            "lodash" = {
              key = "lodash/4.17.21";
            };
            # We're removing `typescript':
            ## typescript = {
            ##   key = "typescript/4.9.4";
            ##   dev = true;
            ## };
          };
        };
      };
      # ...
    };
  };
}

Next we’ll make a new file called foverrides.nix to get the global form of the package added to the sandbox. The seperation between these files is somewhat arbitrary but we’ll revist that later in a discussion about project organization.

# `foverrides.nix'
{ config, ... }: {
  config.floco.packages."@floco/test"."4.2.0".built.extraBuildInputs = [
    config.floco.packages."typescript"."4.9.4".global
  ];
}

This file allows you to explicitly fill config values by hand. Separation from the default.nix file allows it to be used “anywhere” especially external projects.

To use an override file in our project we just need to add it to our list of modules in default.nix ( order doesn’t matter among list members ).

# default.nix
# ---------------------------------------------------------------------------- #
{
# ...
  fmod = lib.evalModules {
    modules = [
      floco.nixosModules.floco
      { config.floco.settings = { inherit system; basedir = ./.; }; }
      ./pdefs.nix
      # Add an override file
      ./foverrides.nix
    ];
  };
}
# ---------------------------------------------------------------------------- #

Now if you rebuild you won’t have to copy typescript, instead you will just add its executables to PATH.

This same approach can be used for any type of config setting.

Next Time

In later guides we’ll cover a few more common customizations like cleaning local source trees, activating symlinked node_modules/ dirs, adding tests, custom build hooks, and more.

Clone this wiki locally