← Blog

An intro to pose detection and biomechanics

July 22, 2026 · 7 min read

Ten years ago, measuring joint angles meant a motion-capture lab: reflective markers, calibrated cameras, expensive software. Today a neural network can find the joints of a moving body in ordinary video — on your own device. Here's how that works, and what the numbers do and don't mean.

What a pose model actually does

A pose-estimation model is trained on hundreds of thousands of images where humans have hand-labeled body landmarks: "this pixel is the left knee." From those examples it learns to predict the same landmarks on images it has never seen. Vi.S.In detects 26 keypoints — the classic 17 (nose, eyes, ears, shoulders, elbows, wrists, hips, knees, ankles) plus head, neck, hip center, and six foot points: both heels, big toes, and little toes.

Modern systems work in two stages. First, a detector finds the person and draws a box around them. Then the pose model looks only inside that box, at high resolution. This two-stage approach is why keypoints stay accurate even when the person is small in the frame — the model never wastes attention on background.

Each keypoint comes with a confidence score. A wrist hidden behind the body scores low, and Vi.S.In hides low-confidence points rather than guessing — an honest gap beats a wrong number.

From points to angles

Once you have keypoints, biomechanics is geometry. Two points define a segment — hip to knee is the thigh, knee to ankle is the shank. The angle between two segments is a joint angle: thigh vs. shank is knee flexion, trunk vs. thigh is the hip angle a bike fitter measures. Segments can also be compared to gravity: trunk vs. vertical is forward lean, and the heel-to-toe line vs. horizontal is your foot-strike angle. Vi.S.In computes 22 such pairs on every frame, which is what turns a video into a chart.

Why angles matter

Quantified movement answers questions that eyeballing can't:

• Range of motion — is your squat actually reaching depth? Has knee flexion improved since last month's physio session?
• Symmetry — does your left knee track like your right? Asymmetries often show up in numbers before they show up as pain.
• Technique — cadence, trunk lean, and shin angle at footstrike for runners; knee extension and hip closure for cyclists. Sport science has published target ranges for many of these, and presets load them for you.
• Progress — the same movement, filmed the same way, weeks apart, compared frame by frame.

The honest caveats

A single camera sees the world in 2D, so an angle is measured as projected onto the camera's view. Film a knee from the side and you get true flexion; film it from 45° and the number is distorted. That's why filming angle matters more than camera quality: side-on for knees, hips, and trunk lean; straight-on from behind for pelvic drop.

Occlusion, motion blur, and baggy clothing all reduce accuracy — and no pose model is flawless even in good conditions. Treat single-frame values with mild suspicion, trends and comparisons with confidence, and remember that none of this is medical advice: numbers inform judgment, they don't replace it. When a keypoint does land wrong, you can drag it into place and the angles recompute.

Where it runs

Pose detection executes in your browser using your device's own compute. The model weights download once (about 40 MB) and detection typically runs at tens of milliseconds per frame on a modern laptop.

Two details worth being precise about, because "runs in your browser" is often used loosely. Photos never leave your device at all. A video is uploaded to our server for one narrow job — decoding it into frames, which browsers cannot do reliably across formats — and it is deleted as soon as those frames are returned. It is never stored, never written to a database, and never sent to a cloud GPU. The analysis itself, every keypoint and every angle, is computed on your device and stays there.