How do you add new programming languages to Cratera?

Toolchains are defined declaratively in languages.toml and baked into the guest rootfs image. Adding a new language requires zero Rust engine recompilation.

CRATERA

Q: What is Cratera and how does it execute untrusted code?

Cratera is a self-hosted code execution and judge engine built in Rust using Firecracker and Linux KVM. It runs untrusted programs in ephemeral Linux microVMs with separate guest kernels, no guest network interface, and configurable CPU, memory, and process limits.

Q: How does Cratera achieve ~5ms snapshot restore latency?

Cratera takes a memory snapshot of a pre-booted guest Linux kernel and supervisor agent in RAM. When an execution request arrives, Cratera restores the microVM memory state in approximately 5ms, bypassing guest Linux boot overhead.

Q: How does Cratera differ from Docker containers and WebAssembly sandboxes?

Docker containers share the host Linux kernel, where a kernel vulnerability can compromise the host. WebAssembly provides sandboxing but requires compilation to Wasm bytecode and cannot run arbitrary native binaries without specialized porting. Cratera uses KVM hardware virtualization with dedicated guest kernels and runs standard native Linux toolchains.

Q: How are network access and resource exhaustion handled?

MicroVMs are launched without a virtual network interface (no guest NIC). Fork bombs and resource exhaustion are mitigated via host cgroups v2 (memory.max, cpu.max, pids.max), guest pidfd SIGKILL watchdogs, and a one-microVM-at-a-time coordinator.

Q: How is Cratera deployed in production?

Cratera includes a production systemd unit with Firecracker Jailer (UID 20001), IPAddressDeny=any, and cgroups v2. Production serve refuses Jailer-off, non-loopback bind, placeholder API keys, and missing or mismatched kernel/rootfs SHA-256 checksums.

MicroVM Execution Engine

Hardware-isolated code execution with Firecracker and Linux KVM.

Run untrusted programs in ephemeral Linux microVMs with separate guest kernels, no guest network interface, and configurable CPU, memory, and process limits.

View on GitHub ↗ Quick start

Rust 2024 • Firecracker • Linux KVM • Jailer • Apache-2.0

Specifications

System Metrics

Isolation

KVM /dev/kvm

Separate guest kernel via KVM

Snapshot restore

~5 ms

Bypasses guest Linux boot via memory snapshot

Network

No NIC

MicroVMs start with no virtual network devices

Workspace

256 MiB tmpfs

In-memory /tmp, zero persistent disk writes

Execution metrics

μs Monotonic

In-guest monotonic timing & live RssAnon sampling

Languages

30 Toolchains

Defined in languages.toml manifest

Host Requirements

[*] Linux Kernel 5.10+ with /dev/kvm virtualization
[*] CPU Architecture: x86_64 or aarch64 (ARM64)
[*] cgroups v2: Unified hierarchy (memory, cpu, pids)
[*] Firecracker Jailer: UID/GID 20001, seccomp-level 2 (required in production)

Guest Environment

[*] Linux Kernel: Minimal microVM kernel (vmlinux.bin)
[*] Rootfs Image: Read-only SquashFS / ext4 filesystem
[*] cratera-agent: In-guest PID 1 supervisor process
[*] Host IPC: AF_VSOCK (Port 52) point-to-point stream

Architecture

Isolation Model

Host
  │
  ├── Cratera coordinator (Axum, 127.0.0.1:3100)
  │
  ├── Firecracker Jailer
  │     ├── privilege drop (UID/GID 20001)
  │     ├── chroot (CRATERA_WORK_DIR/firecracker/<id>)
  │     └── cgroups v2 (memory.max, cpu.max, pids.max)
  │
  └── Linux KVM (/dev/kvm)
        │
        ▼
     Guest kernel (vmlinux.bin)
        │
        ├── cratera-agent (PID 1 supervisor)
        ├── tmpfs workspace (/tmp, 256 MiB RAM)
        ├── compiler & runtime sandbox
        └── AF_VSOCK:52 (point-to-point IPC)

Host Coordinator

Listens on 127.0.0.1:3100, validates Bearer token authentication, manages worker slot pools, and orchestrates fast snapshot restores.

Firecracker Jailer

Drops privileges to UID/GID 20001, chroots under CRATERA_WORK_DIR, applies seccomp-level 2, and sets cgroups v2 memory.max, cpu.max, and pids.max. Required when NODE_ENV=production.

KVM Boundary

Provides hardware virtualization. Workloads execute inside an independent guest kernel with isolated address spaces and CPU state.

Guest Agent (PID 1)

Serves as the guest init process, mounts an ephemeral 256 MiB tmpfs in RAM, and communicates strictly over point-to-point virtual sockets.

Toolchain Sandbox

Compiles and runs code from languages.toml with a pidfd SIGKILL watchdog. Guest processes start from a cleared environment (PATH, HOME, TMPDIR, XDG only).

Telemetry Stream

Measures in-guest time in microseconds and live RssAnon over AF_VSOCK:52. Each job also emits one host job_record log (verdict, timings, cgroup stats).

Security

Defense-in-Depth

Defense-in-depth isolation boundary

Cratera removes the shared-host-kernel isolation model used by traditional containers and adds the KVM/Firecracker microVM boundary as the primary workload isolation layer. Layered host containment (Jailer, seccomp, cgroups v2, and network isolation) provides additional protection against hostile payloads.

Separate guest kernel

Each execution runs inside a dedicated microVM rather than a container sharing the host kernel. Workloads interact only with their private guest kernel.

Jailer containment

Firecracker runs under the unprivileged Jailer with UID/GID 20001, chroot under the work directory, and seccomp-level 2. Production serve refuses to start if Jailer is off.

No guest network interface

MicroVMs are started without a virtual NIC (no tap/tun, no eth0). The systemd deployment also applies IPAddressDeny=any at the host service boundary.

Resource bounds

Host cgroups cap memory.max, cpu.max (one quota period per vCPU), and pids.max. The coordinator runs one microVM at a time. Guest timeouts use pidfd SIGKILL.

Production fail-closed

With NODE_ENV=production, Cratera refuses Jailer-off, non-loopback bind, placeholder API keys (dev-key…), and missing or mismatched kernel/rootfs SHA-256 checksums.

No host env in the guest

Jailer, Firecracker, and guest jobs start from a cleared environment. The API stays on loopback; a leaked Bearer key cannot be used from the public internet if Zero Trust ingress is kept.

Quick start

Setup & Run

01.

Clone repository

Clone the open-source repository from GitHub:

git clone https://github.com/cratera-project/cratera.git && cd cratera

02.

Install dependencies

Run the setup script or install the crate via Cargo:

./scripts/install.sh # or: cargo install cratera

03.

Verify host environment

Verify /dev/kvm, SMT/CPU mitigations, Jailer, bind address, checksums, and cgroups v2:

cratera doctor

04.

Execute test workload

Submit a test evaluation in Rust, Python, C++, or any of the 30 toolchains:

cratera test rust # or: ./examples/submit.sh python

API

POST /harness

cURL Request Example

HTTP/1.1 :3100

curl -s -X POST http://127.0.0.1:3100/harness \
  -H "Authorization: Bearer <api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "language": "rust",
    "code": "fn main() { println!(\"Hello from isolated microVM!\"); }",
    "mode": "submit"
  }'

JSON Response Payload

application/json

{
  "compilationSuccess": true,
  "passed": true,
  "status": "Passed",
  "verdict": "AC",
  "stdout": "Hello from isolated microVM!\n",
  "executionTime": 124,
  "memoryKb": 192,
  "compileMs": 384,
  "bootMs": 45,
  "wallMs": 510,
  "restored": true
}

Accepted

Test passed, exit code 0.

Wrong Answer

Assertion / test failure.

Compilation Error

Compiler error stream.

TLE

Time Limit Exceeded

Execution timeout tripped.

MLE

Memory Limit Exceeded

OOM or cgroup cap hit.

Runtime Error

SIGSEGV / panic / crash.

Internal Error

Hypervisor fault.

Languages

30 Toolchains

languages.toml

Cratera 30 Supported Programming Languages and Runtime Configurations
#	Key	Language	Compile Command	Run Command	Install Strategy
01	rust	Rust 2024	`rustc --edition 2024 -C opt-level=2 ...`	`/tmp/job`	curl_tar
02	python	Python 3	-	`python3 {file}`	apt_core
03	node	JavaScript (Node 24)	-	`node {file}`	docker_image
04	typescript	TypeScript	`esbuild {file} --bundle ...`	`node /tmp/job.js`	curl_tar
05	cpp	C++ (C++20)	`g++ -O3 -std=c++20 -o /tmp/job {file}`	`/tmp/job`	apt_core
06	c	C (C17)	`gcc -O3 -std=c17 -o /tmp/job {file}`	`/tmp/job`	apt_core
07	go	Go 1.24	`go build -o /tmp/job {file}`	`/tmp/job`	curl_tar
08	zig	Zig 0.14	`zig build-exe -O ReleaseFast {file} ...`	`/tmp/job`	curl_tar
09	java	Java (OpenJDK 21)	`javac -d /tmp {file}`	`java -cp /tmp Solution`	apt_core
10	swift	Swift 6.0	`swiftc -O -o /tmp/job {file}`	`/tmp/job`	docker_image
11	csharp	C# (Mono)	`mcs -out:/tmp/job.exe {file}`	`mono /tmp/job.exe`	apt_core
12	kotlin	Kotlin 2.1	`kotlinc {file} -include-runtime -d ...`	`java -jar /tmp/job.jar`	curl_tar
13	scala	Scala 3.6	`scalac -d /tmp/classes {file}`	`scala -cp /tmp/classes Solution`	curl_tar
14	ruby	Ruby 3.3	-	`ruby {file}`	apt_core
15	php	PHP 8.3	-	`php {file}`	apt_core
16	julia	Julia 1.11	-	`julia {file}`	curl_tar
17	haskell	Haskell (GHC 9.6)	`ghc -O2 -o /tmp/job {file}`	`/tmp/job`	apt_core
18	lua	Lua 5.4	-	`lua5.4 {file}`	apt_core
19	perl	Perl 5.38	-	`perl {file}`	apt_core
20	r	R 4.4	-	`Rscript {file}`	apt_core
21	dart	Dart 3.6	`dart compile exe {file} -o /tmp/job`	`/tmp/job`	docker_image
22	elixir	Elixir 1.17	-	`elixir {file}`	apt_core
23	erlang	Erlang (OTP 27)	`erlc -o /tmp {file}`	`erl -noshell -pa /tmp -s solution start`	apt_core
24	ocaml	OCaml 5.1	`ocamlopt -O3 -o /tmp/job {file}`	`/tmp/job`	apt_core
25	clojure	Clojure	-	`clojure -M {file}`	curl_tar
26	fsharp	F# (.NET / Mono)	`fsharpc --out:/tmp/job.exe {file}`	`mono /tmp/job.exe`	apt_core
27	d	D (DMD)	`dmd -O -release -of=/tmp/job {file}`	`/tmp/job`	curl_tar
28	nim	Nim 2.0	`nim c -d:release --out:/tmp/job {file}`	`/tmp/job`	curl_tar
29	fortran	Fortran (GFortran)	`gfortran -O3 -o /tmp/job {file}`	`/tmp/job`	apt_core
30	bash	GNU Bash 5.2	-	`bash {file}`	apt_core

Adding a language (languages.toml)

Toolchains are defined declaratively in languages.toml and baked into the guest root filesystem. Adding or updating toolchains requires zero Rust engine recompilation.

1. install = "curl_tar" Downloads and unpacks release tarballs (e.g. Zig, Julia, Gleam).

2. install = "docker_image" Extracts compiler binaries from container images (e.g. Node, Swift, Dart).

3. install = "apt_core" Installs packages via apt during rootfs build (e.g. GCC, Python, Java).

4. install = "docker_image_base" Uses a container image directly as the rootfs base layer.

Example: Adding Gleam to languages.toml

languages.toml

[gleam]
enabled = true
name = "Gleam"
source = "main.gleam"
compile = "gleam build --target=erlang"
run = "gleam run"
install = "curl_tar"
source_url = "https://github.com/gleam-lang/gleam/releases/download/v1.8.1/gleam-v1.8.1-x86_64-unknown-linux-musl.tar.gz"

Bake toolchains into the guest rootfs:

./scripts/build-rootfs.sh

Deployment

Systemd & Ingress

Development

Run locally with cargo or shell scripts for testing and language integration:

cratera doctor && cratera serve

Production

Deploy via systemd with Firecracker Jailer, cgroups v2, and host network filters:

sudo systemctl enable --now cratera.service

deploy/cratera.service

[Unit]
Description=Cratera Firecracker harness judge service
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/cratera
Environment=NODE_ENV=production
Environment=CRATERA_BIND=127.0.0.1:3100
Environment=CRATERA_FIRECRACKER=/usr/local/bin/firecracker
Environment=CRATERA_JAILER=/usr/local/bin/jailer
Environment=CRATERA_KERNEL=/opt/cratera/images/vmlinux.bin
Environment=CRATERA_ROOTFS=/opt/cratera/images/rootfs.ext4
Environment=CRATERA_WORK_DIR=/var/lib/cratera
Environment=CRATERA_USE_JAILER=1
Environment=CRATERA_JAIL_UID=20001
Environment=CRATERA_JAIL_GID=20001
Environment=CRATERA_USE_SNAPSHOT=1
Environment=CRATERA_SNAPSHOT_DIR=/opt/cratera/images/snapshot
EnvironmentFile=-/opt/cratera/.env
ExecStart=/opt/cratera/cratera
Restart=on-failure
RestartSec=3
LimitNOFILE=65536
Delegate=yes
KillMode=mixed

IPAddressDeny=any
IPAddressAllow=localhost

[Install]
WantedBy=multi-user.target

Directive / Variable	Value	Function
User	`root`	Required for `/dev/kvm` ioctls; Jailer launcher drops privileges to UID/GID `20001`.
CRATERA_USE_JAILER	`1`	Required in production. Jailer chroot, UID/GID 20001, seccomp-level 2, and cgroups v2. `serve` exits if this is off when `NODE_ENV=production`.
Delegate	`yes`	Grants Cratera control over its cgroup v2 subtree (`memory.max`, `cpu.max`, `pids.max`).
IPAddressDeny	`any`	Blocks inbound and outbound IPv4/IPv6 traffic at the host service level.
IPAddressAllow	`localhost`	Allows local loopback (`127.0.0.1`) for reverse proxies and coordinator clients.
CRATERA_USE_SNAPSHOT	`1`	Restores microVM state from memory snapshot in ~5ms, bypassing guest Linux boot.

Recommended Ingress

Cratera binds to 127.0.0.1:3100 by default. In production environments, place the service behind a private ingress such as a Cloudflare Zero Trust Tunnel (cloudflared) or a Tailscale / WireGuard mesh without exposing inbound ports to the public internet.

Built with Cratera

Projects

Used in production

Projects and platforms using Cratera for isolated code execution.

Cratery

Interactive Rust learning platform with quizzes, coding trials, and weekly contests evaluated in real time on Cratera microVMs. Powered by Cratera in production since February 2026.

Domain: cratery.cratera.org

Open Cratery ↗

Building a platform or service on Cratera? Let us know on Zulip or submit your project.

Zulip Chat ↗ Submit Project ↗

FAQ

Questions

What is Cratera and how does it execute untrusted code?

▶

Cratera is a self-hosted code execution engine built in Rust using AWS Firecracker and Linux KVM. Rather than running workloads inside containers that share the host Linux kernel, Cratera runs each execution inside an independent microVM with its own dedicated guest kernel.

This separates untrusted workloads from the host kernel and avoids the shared-kernel escape model used by containers.

How does Cratera achieve ~5ms snapshot restore latency?

▶

Standard virtual machine boots take hundreds of milliseconds to several seconds. Cratera pre-boots a minimal Linux guest kernel with the in-guest cratera-agent and creates an in-memory snapshot.

When an execution request arrives, Cratera restores the microVM snapshot in approximately 5 milliseconds, bypassing the guest kernel boot sequence.

How does Cratera differ from Docker containers and WebAssembly?

▶

Docker / LXC: Workloads share the host kernel. A vulnerability in the host kernel can compromise the host machine or other containers.

WebAssembly: Offers sandboxing but requires compilation to Wasm bytecode and cannot run arbitrary native Linux binaries or standard C-extensions without specialized porting.

Cratera: Uses KVM hardware virtualization with dedicated guest kernels and runs standard native Linux toolchains.

How are network access and resource exhaustion handled?

▶

No guest network interface: MicroVMs are launched without any virtual network devices (no tap/tun, no eth0). The systemd unit applies IPAddressDeny=any. Production also refuses a non-loopback CRATERA_BIND.

Resource bounds: Host cgroups cap memory, CPU, and PIDs. Guest execution is killed via pidfd SIGKILL. The coordinator runs one microVM at a time.

How do I add or customize programming languages?

▶

Adding toolchains requires zero Rust engine recompilation. Compiler commands, source filenames, runtime paths, and installation strategies (apt packages, official Docker image extraction, or tarballs) are defined in languages.toml and baked into the guest rootfs image.

How is Cratera deployed in production?

▶

Cratera includes a systemd unit (deploy/cratera.service) with Jailer (UID/GID 20001), cgroups v2 (Delegate=yes), and IPAddressDeny=any. Production serve refuses Jailer-off, public bind, placeholder API keys, and missing image checksums. Keep the API on localhost behind Zero Trust ingress so a leaked key cannot be used from the internet.