Methodology

Every number on this site comes from a stated source. This page explains how we get the numbers and how we label the pages where we did not measure anything ourselves.

How a test runs

1. The app runs in Docker on our lab host. We use the pinned image version from the Compose file and record that version and the test date on the page.
2. We bring the stack up from a clean state and wait for it to report healthy.
3. We run the app's setup steps (for example creating the first user), then measureidle RAM and CPU once the stack's memory has settled. What counts as settled is defined below.
4. We apply a stated load, then record peak RAM under that load. The page says what the load was.
5. We record image size for every container and startup time from start to first healthy response.
6. We keep the verified Compose file and the raw results so you can check them.

How we measure idle RAM

Some apps use more memory for a few minutes after they start than they do afterwards. A fixed wait would measure that start-up phase and call it idle, so we wait for the memory to settle and then measure. The rule, exactly:

1. From the moment the stack is healthy, we sample memory continuously: one docker stats pass for every container, about every 2 to 3 seconds. Memory is the cgroup working set (usage minus inactive file cache), in MiB.
2. The settle clock starts when the setup steps finish (or at healthy, if the app has none). The stack is settled when, over the last 60 seconds, the total memory of all containers at the same instant varied (highest minus lowest sample) by less than the larger of 2% of the window's mean and 10 MiB.
3. We never call it settled before 120 seconds, and we stop waiting after 600 seconds. Both count from the settle clock. All five numbers (window, percentage, floor, minimum, maximum) are defaults that a test can override. The Test box prints the ones used.
4. Then we take the idle window: at least 30 seconds and at least 10 samples. If the idle window is not itself flat by the same threshold, or its mean differs from the settle window's by more than the threshold, the stack was not really settled and we keep waiting (still within the 600-second limit).
5. If 600 seconds pass without settling, we still measure an idle window, but the result is marked not settled and the Test box says so in plain text.

We also record the startup peak (the highest memory, per container and for the stack total, between healthy and the end of the settle phase) and a memory timeline from healthy to the end of the idle window. The Test box shows the startup peak, the time it took to settle and a small chart of the timeline with the same data as a table.

What this does not catch: a stack that is flat for a whole 60-second window and only then changes. The idle window check catches a change within the next 30 seconds, nothing later. Memory growth over days is not measured at all.

About the lab host

All tests run on one lab host. We print its CPU model and RAM next to every result. RAM and image size carry over to other machines well. CPU, throughput and tokens-per-second figures are only valid for our host, so we label them that way and never compare them with numbers from other people's hardware.

Evidence labels

Each page shows one label near the top.

A page only changes label when new data is actually published on it, and the change is noted in its update log.

What we do not claim

We never claim hands-on testing of hardware we have not tested. If we have not run a mini PC, NAS, drive or VPS ourselves, the page says "Researched" and tells you where the specs came from.

Who does the work

Research, testing and drafting are carried out by AI agents (Anthropic's Claude) running the lab. Their work is reviewed under the editorial oversight of Simon Carter, who is accountable for what is published. We say this openly because you should know how the site is made.

Methodology changes

2026-10-02: idle is measured after memory settles, not after 60 seconds

Before. The lab waited a fixed 60 seconds after the app was healthy and set up, then sampled for about 30 seconds and published that as idle RAM.

Why we changed it. On Oct 1, 2026 we watched Immich's server container for ten minutes after a start. It held 1.4 to 1.5 GiB for the first minutes, then dropped to about 0.7 GiB roughly two minutes after the stack became healthy. A 60-second wait plus a 30-second window lands before that drop, so our idle figures for Immich described a start-up phase and overstated the steady-state RAM. We publish these numbers as measurements, so they had to describe steady state.

What changed. The settle rule above, plus the startup peak and the memory timeline in every result. We re-ran every lab result under the new rule on 2026-10-02 and updated every page that quotes them; each page's update log says so. Result files now carry schema_version 2.

What did not change. Peak RAM under load is measured as before: the highest simultaneous total during the load step. Image sizes, startup time and the load itself are unchanged.

Prices and affiliate links

Prices come from vendor pages and carry the date we read them. Affiliate relationships never change a result or a ranking; see the affiliate disclosure.