The data our models live and die against.

What we mean by "data" at an observatory

An observatory does not own its sky. The light has already been arriving for several billion years, and any organisation that pretends to monopolise its analysis has misread the room. Solstice Pro AI works with five principal archives — four of them public, one of them curated by us — and a small library of internal calibration sequences taken on test-bench sensors. We do not gate any of them behind a login wall; where we cannot redistribute (NASA, GOES, ZTF) we point at the canonical mirror and ship the loaders.

What we do claim is the validation harness. Each model we release is paired with a complete instrument-night replay: a chronologically ordered sequence of frames, masks and human-adjudicated labels that the model has never been allowed to touch during training. The replay is the artefact a buyer should care about. The trained weights are the by-product.

Below is the working register, kept in the same observatory-log style we use for nightly operations. Sizes are uncompressed working-set sizes; on-disk footprint after lz4 is roughly 0.55× for image cubes and 0.72× for FITS tables.

The five archives

Two of the five are ours to give away, and we do. The cosmic-ray corpus in particular took a junior astronomer and a senior one the better part of four months to label, double-adjudicate and triple-spot-check. We did not consider keeping it private; the community gains more from a published benchmark than we lose from a moat.

Deterministic builds and the validation notebook

An astronomical paper that cannot be reproduced is gossip. An astronomical model that cannot be reproduced is worse, because it gives the appearance of operational reliability without the substance. Solstice Pro AI commits, at the level of company policy, to the following reproducibility constraints on every model release.

Deterministic CUDA builds

Every Solstice container is pinned to a specific CUDA toolkit (currently 12.4), a specific cuDNN minor version, a specific PyTorch wheel and a specific set of torch.use_deterministic_algorithms(True) guard rails. The non-deterministic kernels we cannot avoid — scatter-add, some atomic reductions — are flagged in the release notes with the expected pixel-level variance across runs. A buyer who reruns inference on the same input on the same GPU class will see byte-identical output for everything else.

Validation notebook packaged with every release

The notebook is not a marketing artefact. It loads the held-out replay set, runs inference, computes the headline metric, and produces every plot in the release announcement. If we change a number on a dispatch post, we have changed the notebook, and the diff is in the git history. The notebook is the contract.

Frozen replay sets, not random splits

A random 80/20 split is the default sin of machine-learning practice and the one that an instrument night punishes hardest. Adjacent frames in a sky survey are correlated: same airmass, same seeing, same satellites passing overhead. A model evaluated on a random split will overstate its real-night performance by a half-step to a full step. We therefore evaluate on whole instrument nights — a complete observing session, sunrise to sunrise — that the model has not been allowed to see in any form during training.

Where each archive sits in the pipeline

Not all of these archives are training data. Some are validation only, some are calibration only, and the ZTF replay is both, in a strict temporal partition. The table below makes the role explicit; readers from observatory data-management offices should find it the most useful page on this site.

How to cite us, and how we cite you

If you use the Solstice cosmic-ray segmentation corpus, cite the Zenodo DOI in the table above and, if convenient, the dispatch post that announced it. If you use the EMCCD twilight calibration set, cite the Photon-001 release notes and acknowledge the bench (Andor iXon 888, sensor serial pool S-44/S-45/S-46). If you use a trained Solstice model under a research-collaborator agreement, cite the model SHA and the dispatch post that published the headline metric; the SHA is printed on the splash screen of every container.

In the other direction, we cite every archive we depend on in the standard astronomical-journal pattern: lead author, year, journal, DOI, on the first reference and an abbreviation thereafter. The bibliography for any given model release is included in the validation notebook and re-rendered every time the notebook is rerun, so it cannot drift from what the model actually consumes.

For commercial reuse — operations integrators who want to redistribute Solstice corpora inside a paid product — the cosmic-ray segmentation corpus is CC-BY-SA-4.0, which means a derivative product is share-alike. The EMCCD set is CC-BY-4.0, which is permissive. If either license is incompatible with a buyer's pipeline, talk to us before the integration begins, not after.