110 lines
3.6 KiB
Markdown
110 lines
3.6 KiB
Markdown
# Nix
|
||
|
||
This directory contains Nix expressions defining the packages, and containers used to run/build Poseidon.
|
||
|
||
## Directory Structure
|
||
|
||
```
|
||
nix/
|
||
├── packages/ # Individual Poseidon service packages
|
||
│ ├── default.nix # Entry point - builds all packages with dependencies
|
||
│ ├── serverpack.nix
|
||
│ ├── dataagent.nix
|
||
│ ├── atlantis.nix
|
||
│ ├── atlantis-client.nix
|
||
│ ├── atlantis-client.json # Client dependencies metadata
|
||
│ ├── atlantis-deps.json # Atlantis backend dependencies metadata
|
||
│ ├── sorcerer.nix
|
||
│ └── archivist.nix
|
||
├── secrets/ # Age encrypted files
|
||
│ ├── secrets.nix
|
||
│ └── *.age
|
||
├── containers.nix # Docker & Singularity container definitions
|
||
│ ├── atlantis
|
||
│ ├── sorcerer
|
||
│ └── archivist
|
||
└── pre-commit.nix # Pre-commit hooks for code quality
|
||
```
|
||
|
||
## Usage
|
||
|
||
Note: `nix-build` can be switched for `nom-build` for a pretty-printed output (using [Nix Output Monitor](https://github.com/maralorn/nix-output-monitor))
|
||
|
||
### Secrets
|
||
|
||
```bash
|
||
# To run nix-build
|
||
$ set -x NETRC $(agenix -d netrc.age)
|
||
```
|
||
|
||
Secrets used for development and production are stored as `*.age` files and decrypted/encrypted using agenix via ssh-keys.
|
||
|
||
```bash
|
||
# To encrypt
|
||
$ agenix -e name.age # And add to secrets.nix
|
||
|
||
# To decrypt
|
||
$ agenix -d name.age
|
||
```
|
||
|
||
In case a new key is added to the `secrets.nix` file we need to re-key the age files with:
|
||
```bash
|
||
$ agenix -r
|
||
```
|
||
|
||
### Building Individual Packages
|
||
```bash
|
||
# Build a specific service
|
||
$ nix-build -A packages.atlantis
|
||
$ nix-build -A packages.sorcerer
|
||
|
||
# Build all packages
|
||
$ nix-build -A packages
|
||
```
|
||
|
||
### Building Container Images
|
||
```bash
|
||
# Build Docker containers
|
||
$ nix-build -A containers.atlantis
|
||
$ nix-build -A containers.sorcerer
|
||
|
||
# Build Singularity container for HPC
|
||
$ nix-build -A containers.archivist
|
||
```
|
||
|
||
### Development Shell
|
||
```bash
|
||
# Enter development environment
|
||
$ nix-shell
|
||
|
||
# Or use a specific package's shell
|
||
$ nix-shell -A packages.atlantis
|
||
```
|
||
|
||
### Running Services
|
||
```bash
|
||
# Run Atlantis server
|
||
$ ./result/bin/atlantis
|
||
|
||
# Run Sorcerer API
|
||
$ ./result/bin/sorcerer
|
||
```
|
||
|
||
### Update dependencies
|
||
|
||
When updating the `npm` dependencies, the `outputHash` in `atlantis-client.nix` needs to be updated. Simply run
|
||
|
||
```bash
|
||
nix-build -A packages.atlantis-client # in the root
|
||
```
|
||
|
||
It will then fail on the wrong hash and provide the correct one.
|
||
|
||
#### Deterministic builds, vendor hashes, and lock files
|
||
|
||
Nix aims for deterministic (reproducible) builds. A key part of this is **fixed-output derivations (FODs)** such as `fetchurl`, `buildDotnetModule`’s vendor step, `fetchgit`, etc. For any FOD, Nix requires a **content hash** up front. After the build/fetch runs, Nix verifies that the resulting output’s hash exactly matches what was declared; if it doesn’t, the build fails with a “hash mismatch in fixed-output derivation” error. This protects you from drifting dependencies and ensures that CI and local builds use the exact same inputs.
|
||
|
||
#### Why vendor hashes need to be pinned
|
||
|
||
Language ecosystems resolve and download a lot of upstream content (Go modules, npm/yarn, cargo crates, vendored tarballs…). To make those fetches deterministic, Nix needs the **expected content hash**. For example, with `makeDerivation`, you must set `outputHash` so Nix knows what the fully-resolved module tree should hash to. If you bump a version or change dependencies, the **content changes** and the old hash becomes invalid—Nix will (correctly) refuse the build until you update the pinned hash.
|