Forecast: a private lab server (22 cores, 128 GB). Written 2026-10-16 21:21 CEST, before any code
The situation
Up to 16.5.1 the app was shaped by a 2-core, 3 GB server:
- one Python process, with one thread per request;
- at most two protein sites at a time, each fragment at most 1500 atoms with hydrogens;
- SMILES of at most 120 atoms, and structure files of at most 20 MB;
- small caches.
On a machine with 22 cores, one Python process still uses about one core. Python's global interpreter lock lets only one thread run Python code at a time, and nearly all of the orbital layer is Python. Two visitors computing at once each get half a core.
The plan
One code base, two profiles, chosen at start. vps keeps today's behaviour; lab is for the new machine; auto picks lab with at least 8 cores and 32 GB.
Under lab, the expensive routes run in a pool of worker processes:
- the routes:
/api/site,/api/model,/api/pi-modeland the SVG render; - each structure (by its SHA-256) and each SMILES goes to the same worker each time, so the parsed file and the computed results stay cached there;
- workers send back finished JSON, so the front process only moves bytes.
Bigger limits, larger caches and batch analyses come on top. Every number below is checked in tools/lab/RESULTS.md. This sandbox has 2 cores and 7 GB. What needs more cores is measured on the lab machine with tools/lab/bench.py, and the forecast says which is which.
Forecasts
| forecast | where it is tested | |
|---|---|---|
| P1 · overhead | A site request to a worker that already holds the structure costs < 30 ms more than in-process. The first request with a 20 MB structure costs < 200 ms more. | here |
| P2 · throughput | k different ligand requests sent at once, to k workers on k free cores, all finish within 1.25× the time of one alone. That holds up to k = cores − 2: k = 2 here, k = 20 on the lab machine, so ≥ 16 requests in the time of one. | here (k = 2), lab |
| P3 · no regression | A single request computed in a worker takes within 10 % of the same request in-process. | here |
| P4 · same answer | Pool and in-process return byte-identical JSON for the five examples (two motifs, three ligands). | here |
| P5 · memory | An idle worker costs < 250 MB (NumPy and qo_chem; with RDKit on the server). 20 workers cost < 5 GB. With every cache budget full, the server stays < 70 GB. | here (worker), lab (total) |
| P6 · largest site | The site pipeline (fragment, orbital layer, JSON) stays close to linear. A 12 000-atom fragment with hydrogens takes < 20 s on one core, with JSON < 40 MB. | here (one sandbox core) |
| P7 · no re-uploads | After the first request, a structure is not uploaded again in a session: later requests are < 1 kB. | here (browser test) |
| P8 · batch | Perceiving every ligand of the 23 biotite test structures (156 ligands, all four routes where available): < 10 s on one core, and ≥ 1.6× faster on 2 workers. On a folder of files, the speed-up on 20 workers is ≥ 12×, limited by reading and parsing. | here (1 and 2), lab (20) |
| P9 · local dictionary | Indexing the PDB's components.cif (~400 MB) once takes < 60 s; then a component is looked up in < 5 ms. | here (synthetic excerpt, timing per MB), lab (real file) |
What would change the plan
- If P2 fails (k requests take more than 2× one), the pool is not worth its complexity. Ship the lab limits and run several plain server processes behind nginx instead.
- If P6 fails (worse than linear past 5000 atoms), the site limit stays where it holds 20 s, not at 12 000.
- If P1 fails for large files, the browser must stop resending the structure (P7) before anything else.