Nix

This directory contains Nix expressions defining the packages, and containers used to run/build Poseidon.

Directory Structure

nix/
├── packages/                    # Individual Poseidon service packages
│   ├── default.nix              # Entry point - builds all packages with dependencies
│   ├── serverpack.nix
│   ├── dataagent.nix
│   ├── atlantis.nix
│   ├── atlantis-client.nix
│   ├── atlantis-client.json     # Client dependencies metadata
│   ├── atlantis-deps.json       # Atlantis backend dependencies metadata
│   ├── sorcerer.nix
│   └── archivist.nix
├── secrets/                     # Age encrypted files
│   ├── secrets.nix
│   └── *.age
├── containers.nix               # Docker & Singularity container definitions
│   ├── atlantis
│   ├── sorcerer
│   └── archivist
└── pre-commit.nix               # Pre-commit hooks for code quality

Usage

Note: nix-build can be switched for nom-build for a pretty-printed output (using Nix Output Monitor)

Secrets

# To run nix-build
$ set -x NETRC $(agenix -d netrc.age)

Secrets used for development and production are stored as *.age files and decrypted/encrypted using agenix via ssh-keys.

# To encrypt
$ agenix -e name.age # And add to secrets.nix

# To decrypt
$ agenix -d name.age

In case a new key is added to the secrets.nix file we need to re-key the age files with:

$ agenix -r

Building Individual Packages

# Build a specific service
$ nix-build -A packages.atlantis
$ nix-build -A packages.sorcerer

# Build all packages
$ nix-build -A packages

Building Container Images

# Build Docker containers
$ nix-build -A containers.atlantis
$ nix-build -A containers.sorcerer

# Build Singularity container for HPC
$ nix-build -A containers.archivist

Development Shell

# Enter development environment
$ nix-shell

# Or use a specific package's shell
$ nix-shell -A packages.atlantis

Running Services

# Run Atlantis server
$ ./result/bin/atlantis

# Run Sorcerer API
$ ./result/bin/sorcerer

Update dependencies

When updating the npm dependencies, the outputHash in atlantis-client.nix needs to be updated. Simply run

nix-build -A packages.atlantis-client # in the root

It will then fail on the wrong hash and provide the correct one.

Deterministic builds, vendor hashes, and lock files

Nix aims for deterministic (reproducible) builds. A key part of this is fixed-output derivations (FODs) such as fetchurl, buildDotnetModule’s vendor step, fetchgit, etc. For any FOD, Nix requires a content hash up front. After the build/fetch runs, Nix verifies that the resulting output’s hash exactly matches what was declared; if it doesn’t, the build fails with a “hash mismatch in fixed-output derivation” error. This protects you from drifting dependencies and ensures that CI and local builds use the exact same inputs.

Why vendor hashes need to be pinned

Language ecosystems resolve and download a lot of upstream content (Go modules, npm/yarn, cargo crates, vendored tarballs…). To make those fetches deterministic, Nix needs the expected content hash. For example, with makeDerivation, you must set outputHash so Nix knows what the fully-resolved module tree should hash to. If you bump a version or change dependencies, the content changes and the old hash becomes invalid—Nix will (correctly) refuse the build until you update the pinned hash.

3.6 KiB Raw Blame History Unescape Escape

Nix