VIEB
One place to segment animal behaviour from pose — with the controls that tell a real state from an artefact built into every result.
In development · runs locally · not yet publicWhat it is
Unsupervised behaviour segmentation is easy to fool: a method will find states in almost anything, and a clean ethogram says nothing about whether the states are behaviour. VIEB is a research platform where a researcher brings their own pose tracking, chooses and tunes a method, and gets every result back beside the same five controls, so the reading never depends on a picture alone.
A web app for the whole loop, a command line for scripts and clusters, and one engine underneath, so a run started anywhere shows up everywhere. Methods are plugins: a new one is a single class and appears in the app with its parameters, ranges and help.
Methods
| Method | What it does |
|---|---|
| VIEB v1 | 91 engineered features (kinematics, posture, wavelets) → UMAP → HDBSCAN → HMM smoothing. |
| Keypoint-MoSeq | An autoregressive HMM over low-dimensional pose with sticky syllables; fitted and applied from the platform in its own environment. |
| HDBSCAN | Density clustering of a (delay-embedded) pose representation. |
| Koopman | Attractor decomposition of local linear operators. |
| Ulam | Transfer-operator coarse-graining into metastable states. |
The five controls
| Control | Question it answers |
|---|---|
| Surrogate null | The same method, same settings, on pose with the behaviour removed (phase-randomised; white noise). Does it find as many states there? |
| Session identity | Do the states tell apart two sessions of the same animal — camera, lighting, tracking — more than behaviour should? |
| Seed stability | Does another random seed give the same partition? Identical labels count as "not tested", not as stable. |
| Known answer | Where a human labelled behaviour: do the states recover it as bouts, beyond shifted-label chance? |
| Coverage | How much of the data did the method decline to label? |
Each control writes its numbers, a verdict and the rule behind the verdict. A lab can set its own thresholds.
What the controls catch
A synthetic project with a known answer: two behaviours, still and move, planted in bouts of about two seconds. VIEB v1, swept over its main parameter:
| min_cluster_size | States (real) | States (white noise) | Label F1 | Surrogate |
|---|---|---|---|---|
| 100 | 10 | 15 | 1.00 | red |
| 200 | 8 | 4 | 1.00 | amber |
| 500 | 3 | 3 | 1.00 | red |
| 1000 | 4 | 2 | 1.00 | amber |
The labels alone would look perfect: both behaviours recovered at every setting. The surrogate shows the other half — the method also finds 2 to 15 states in pure noise, and splits two behaviours into three to ten. The state count is partly the method's own. That is the reading the platform exists to make visible.
Watching the states
For any run, the platform cuts example bouts of each state from the source videos — typical and longest, from different animals — with the skeleton drawn and the in-state frames outlined, as clips that play in the browser.
min_cluster_size = 500. State 0 is
the moving posture; states 1 and 2 are both the still one — the split the surrogate warned about, visible at a
glance.Also
Parameter sweeps with every control per value; import of DeepLabCut projects and SLEAP files; a keypoint-roles and skeleton editor; metadata from file names; per-run provenance (code version, library versions, the exact request) so any run can be repeated.
Status
VIEB is in active development and runs locally; it is not public yet. The research record behind it is in VIEB docs. Interested in trying it on your data? Email me.