How AI Is Changing Fitness — From Wearables to Real-Time Form Judging

For most of the last decade, 'AI in fitness' meant a wearable estimating your heart rate and calories. Useful, but passive — it measured what already happened. The more interesting shift over the last few years is active: computer vision models that watch your body move, in real time, from an ordinary camera, and judge the movement itself.

What pose estimation actually does

Pose estimation models map the human body to a skeleton of key points — shoulders, elbows, hips, knees, ankles — dozens of times per second, from a normal 2D video feed. From that skeleton, software can calculate joint angles, depth, tempo, and symmetry: exactly the inputs a human coach would use to judge a rep, just measured with far more precision and consistency than an eye can manage.

Why on-device matters

A meaningful design choice in modern pose-estimation is running the model directly in the browser or on-device, rather than uploading video to a server. It's faster (no round-trip latency), cheaper to run at scale, and — importantly for user trust — means your actual video never has to leave your device for the scoring to happen. REPPED's rep-counting and form-scoring runs entirely client-side for exactly this reason; only the final numeric result is ever sent anywhere.

What a joint angle can and cannot tell you

It is worth being precise about what this technology measures, because the marketing around it rarely is. A pose model returns coordinates. Everything else — depth, tempo, symmetry, whether a repetition counted — is arithmetic performed on those coordinates by code somebody wrote. The model is not judging your form. It is reporting where your knee is, and a separate set of rules decides what that means.

That distinction matters because the rules are where the opinions live. Deciding that a squat counts below 110 degrees is a judgement call, not a measurement. A different system could reasonably pick 100 or 120 and be equally defensible. What the camera gives you is consistency: whatever threshold is chosen gets applied identically to every rep, by every person, on every device, which is more than can be said for a human counting in a gym.

The honest summary is that computer vision does not remove subjectivity from judging exercise. It moves the subjectivity out of the moment and into the rules, where it can be written down, argued with and applied the same way to everybody.

Two-dimensional video has real blind spots

A single ordinary camera sees a flat projection of a three-dimensional body, and some information simply is not recoverable from it. Rotation toward or away from the lens compresses limbs in ways that look identical to those limbs being shorter. A joint hidden behind another part of your body has to be estimated rather than seen.

This is the actual reason systems ask you to stand side-on with your whole body in frame. It is not a preference. It is the arrangement in which the angles that matter — elbow, hip, knee — lie roughly in the plane the camera can measure, so they can be read rather than inferred.

It also explains a category of complaint that sounds like a bug and is not. If your reps stop counting when you drift toward the edge of the frame, nothing has broken; the joints the scoring depends on have left the picture. A system that kept confidently counting in that situation would be worse, not better, because it would be making the numbers up.

The privacy argument is a technical one

There is a version of on-device processing that is a marketing claim and a version that is an architectural fact, and they are easy to confuse. The distinguishing question is simple: does the video have to leave the device for the product to work at all? If the scoring happens in a data centre, the answer is yes, and every promise about deletion is a promise about what someone chooses to do with a file they already have.

If the model runs in the browser, the answer is no, and the promise is structural instead. There is no upload to delete because there was never an upload. What travels is a number — a rep count, a score — which is not reconstructable into a picture of anyone's living room.

For a product where the whole premise is pointing a camera at yourself, often at home, often as a teenager, this is not a small detail. It is most of the reason the thing is defensible at all.

Where this is heading

• Real-time form correction during a workout, not after it
• Objective scoring that removes subjective 'that counts, right?' arguments
• Fair matchmaking based on measured performance, not self-reported fitness level
• Competitive formats that weren't possible when judging required a human referee for every rep

The honest limitations

Camera-based pose estimation isn't a medical device and isn't perfect in poor lighting, unusual camera angles, or with the body partially out of frame — which is why fair systems build in visibility checks and graceful fallbacks rather than pretending to be flawless. REPPED, for instance, flags a rep when a player isn't sufficiently visible rather than guessing. Read our full explanation of how the AI judges every rep for the specifics.