QEVD Guide ← annotation Examples

How to annotate

Ten minutes of reading, then look at two or three examples and start. Ask whenever something is unclear — a question now is much cheaper than 100 segments annotated the wrong way.

1 · What you are producing

Each video shows somebody exercising while a coach talks. We are building the link between what the coach says and which individual execution they are talking about. So for every clip you tell us two things:

  1. Where each execution starts and ends — the boundaries.
  2. How each execution went, and what a coach would say about it.
The single most important habit: judge one execution at a time, not the clip as a whole. "The squats were sloppy" is not what we need; "this squat, the fourth one, was shallower than the three before it" is.

2 · Four words

WordMeaning
segmentone clip in the left-hand list — one exercise, from start to finish
setone complete execution of the exercise: one squat, one push-up, one jumping jack. This is the thing you are counting.
cycleone up-and-down of the joint-angle curve. That is what the automatic detector produces (blue triangles). A cycle is not always a set — the detector miscounts, which is exactly why you are here
rest / transitionframes where nothing is being executed. rest = a pause within the same exercise; transition = moving to a different exercise (walking over, picking up a dumbbell, standing up afterwards)

3 · The flow: two passes

Open a segment from the list. It already has the detector's cycles in place as your sets — you are correcting them, not starting from nothing. Then:

  1. Pass 1 — boundaries. Play at 0.5× and check the sets. Fix only the ones that are wrong. Press A if the detector was right, C if you corrected it.
  2. Pass 2 — judging. Press V. Now go set by set: is anything off about this one? Would the coach say something about it? Then ] for the next.
  3. S to save and move to the next segment.

They are separate on purpose. Boundaries need the whole clip. Judging must only use what happened up to that point — a coach watching live cannot see the future either, so in judging mode the timeline after the current set is greyed out.

4 · Pass 1 — set boundaries

A green bar on the timeline = one set. Its right edge is the moment that set is completed. Watch for the four things the detector gets wrong:

What you seeWhat to do
one green bar covers two executionsput the playhead between them, press X to split
two bars cover one executionG merges into the previous
a bar where nothing happenedD deletes it
an execution with no bardouble-click the curve there, or click the red hatching
the boundary is a bit early/latedrag the line between two bars, or step to the frame with and press .

Not every frame is exercise. The bar at the top of the timeline says what each frame is: set (exercising), rest (a pause in the same exercise) or transition (changing exercise). Only sets inside a set block count. If a stretch of the clip is marked set but has no set in it, you get red hatching — either add a set there or mark that stretch rest (⇧3).

How exact? Within about a quarter of a second is fine. What matters is that each execution is one set, and the boundary is inside the pause between them, not that you nail the exact frame.

5 · Pass 2 — judging (V)

Three separate questions per set. They are independent — answer each on its own.

① This set on its own — pick one of four

KeyMeaningExample
Qnormal — the default. You do not have to press it a decent rep
Wnot completed — gave up part-way through the movement goes half-way down into the squat and stands back up
Enot the exercise — the movement finished, but it is not really this exercisea "squat" where only the arms move
Rdone, but poor — it is this exercise, it was completed, it is just not done wella squat far too shallow, knees caving in
Only flag what is off. Normal is the default, so you do not confirm every set. Press ] when you are done looking at a set — that is how we know how far you actually reviewed. Skipping ] is not the same as marking normal.

W matters more than it looks: a half-rep does not count and takes no number. Automatic counters almost always count it, which is exactly the failure we want in the data.

③ Compared with the person's own previous sets

8 smaller range · 9 larger · 5 slower · 6 faster — compared with their own previous 2 sets (highlighted in blue on the timeline; not-completed ones are skipped). Press the same key again to clear.

Why this is separate from ①: ① asks "is this rep good?", ③ asks "has it changed?". A perfectly normal rep can still be slower than the two before it — that is fatigue setting in, and it is exactly the kind of thing a coach reacts to. Mark both, or either, or neither.

6 · ② Coaching comments

This is the heart of the task. The panel lists what the coach actually said in this clip. For each set, attach the lines that are about this particular execution.

Most of what a coach says should NOT be attached. Roughly two thirds of it is encouragement and chatter — "Nice!", "Keep going!", "Five more!". Those are not about how a particular rep was executed. Attach only the lines that describe this execution: "go deeper", "keep your back straight", "slow it down". A low acceptance rate is the expected result, not a sign you are doing it wrong.

7 · Saving — and what is safe

WhatWhenWhere it goes
your unsaved editsas you workkept in your browser, so closing the tab by accident does not lose them
S savewheneverthe server. Save as often as you like and come back to change it later
⇧S finalise"this segment is done"a separate archive; the segment locks. ⇧U unlocks it if you need to change something — nothing is ever destroyed
Your work is yours alone. Everyone annotates the same segments and nobody can see anyone else's answers. That is deliberate: we compare independent annotations afterwards. Please do not discuss specific clips with other annotators.

8 · Questions people ask

The video and the curve disagree — which do I trust?

The video. Always. The curve and the blue triangles are one automatic joint angle; they are a hint, not the truth.

The person does two exercises in one clip.

That happens. Select the sets that belong to the other exercise and rename them in the "exercise for this set" box. The exercise reference shows what each name looks like — please use those names so everyone writes the same thing.

I cannot tell whether it counts as an execution.

Ask yourself what a coach counting out loud would do. If they would say "three!", it is a set. If they would say "no, that one doesn't count" — that is exactly W.

The person is out of frame / the video is unusable.

Press U (unusable) and move on. Do not guess.

Do I have to judge every set?

You have to look at every set (] through them). You only press a key on the ones where something is off.

I made a mess of a segment.

Z undoes any single step. clears the whole segment and you can start over. Nothing you do is unrecoverable.

Roughly how long does a segment take?

Once you are used to it, a couple of minutes: most of the time the detector's boundaries are nearly right and the judging is a fast pass. Do not rush the comments though — that layer is the whole point.