AI Workout Assistant App
We built a fitness app that uses AI to guide users through workouts.
Client
Businesses are increasingly investing in artificial intelligence, and retailers are no exception. In 2021, a large U.S. fitness brand with a network of brick-and-mortar stores and an established online presence reached out to us.
The company aimed to build an AI-powered mobile workout app to strengthen its core retail business.
Challenges
Through the app, the company aimed to attract new customers and convert one-time purchases into ongoing digital engagement with the brand.
The product was aimed at users who train on their own. It was not intended as an alternative to a professional trainer. Its role was to monitor exercise technique and flag incorrect movement.
The assistant had to remain simple and safe for the user, while being designed to support ongoing engagement with the brand.
Interpreting Movement
Training the model required properly labeled data reflecting correct exercise technique. The starting point was to determine how to divide the human body into distinct points and define how those points relate to one another in space during movement.
Custom Datasets
Pre-trained models are available. They handle standard tasks well: object detection, camera angle estimation, keypoint extraction. However, they are not sufficient for this type of application, as off-the-shelf data alone would not be sufficient.
The idea was straightforward: use a pre-trained pose model (for example, ML Kit) to detect keypoints, then train a custom model on top of them to count repetitions and assess technique.
Building an AI Workout App with Real-Time Pose Detection
Base Pose Detection
As with SignLab, our internal project, we manually collected, cleaned, and annotated the datasets used to train the workout assistant. We analyzed a substantial dataset of correctly performed exercises in three-dimensional space.
In the first version of the product, we trained the application to track body part positions and determine correct execution for five basic exercises: plank, push-ups, squats, pull-ups, and lunges.
To clarify, this does not involve full 3D geometry. It relies on keypoints: two-dimensional coordinates (X and Y) plus a Z value representing relative depth.
Custom Technique Analysis Model
To process the video stream and determine the user’s position in the frame, we used Android’s standard tool, CameraX. It handles camera operations such as preview display, focus adjustment, and exposure control.
However, raw camera input alone is not sufficient. The ML Kit library and the Pose Detection API are used to convert the image in real time into a set of 33 body keypoints, from the head and shoulders to the knees and feet. This creates a skeletal model the algorithm uses to interpret user movement.
Our custom model operates on these keypoints. It counts repetitions, evaluates technique, and provides hints and feedback.
How the Core Function Works
– Step 1. The user opens the camera within the app and positions it to ensure their full body is visible during the exercise. They then select an exercise from the list, and the workout begins.
– Step 2. The model first evaluates the camera angle and lighting conditions, then identifies the user’s pose and the current phase of the exercise. Based on the camera input, the application compares joint angles and movement trajectories with reference patterns defined for each exercise. It builds skeletal keypoints, counts repetitions, and evaluates technique in real time. Feedback is delivered with a delay of 100 to 200 milliseconds on Android devices common in 2021. This is not medical-grade precision, but it is sufficient for its intended use. Camera access remains active only during the exercise.
– Step 3. If an error is detected, the app displays a visual cue. The incorrectly positioned body part is highlighted in red, along with a brief suggestion on how to improve. The highlight appears over the relevant joints, and the recommendation is usually based on one or two common rules, such as keeping the knee aligned during a squat. The goal is to guide the user, not to enforce strict instructions.
Users can also log their physical and psychological state before or after a session. This allows the application to adjust workload or recovery time accordingly.
Additional Features
In the second version, we expanded the functionality:
— Automatic saving of workout results in the user’s profile, including repetition count, workout duration, rest intervals, and intensity level.
— Detailed reference information for each type of exercise, including instructions on how to perform it, the muscle groups involved, and the recommended number of sets depending on the user’s fitness level.
— Generation of a personalized training plan based on the user’s goals, age, and activity level, with the plan added to the built-in calendar. Goals and preferences can be described in a conversational format, with AI assisting in formulating objectives such as weight loss, endurance, or posture improvement and collecting the necessary inputs. We describe the chatbot platform in a separate case study. The virtual trainer’s chatbot follows the same architectural principles.
— Automatic adjustment of the training program based on user progress, without providing medical conclusions.
— Progress tracking that takes into account nutrition and calories burned through manual input provided by the user.
Privacy
— All processing takes place on the user’s device rather than in the cloud, and video data remains on the device.
— Personal data is not transmitted outside the device. Only anonymized, aggregated data is stored on the server, without biometric information, and only with the user’s consent.
— Data is retained for 30 days and complies with GDPR and CCPA requirements.
Technologies
Backend
Firebase
Algorithm
Python
Mobile
Android (Java)
Library
TensorFlow Lite
Library
ML Kit Pose Detection
Database
Firestore
Result
We delivered an application that uses AI to monitor exercise technique and track both the quality and number of repetitions.
The product met its core business objectives. It contributed to user growth and strengthened the link between training engagement and product purchases. Personalized training logic improved user outcomes while maintaining safety standards. Importantly, the application was positioned strictly as a fitness tool, without diagnostic claims or medical functionality.
As with any camera-based system, performance depends on environmental conditions, including lighting, camera placement, and full-body visibility in the frame. Detection may degrade in low light, when joints are partially obscured, during very fast or abrupt movements, or in confined spaces. Improvements in these areas are planned for future updates.
The client also planned to expand the range of supported exercises and integrate the application with IoT devices. Connected devices would allow the system to monitor fatigue indicators automatically and adjust training intensity in real time, as users do not always recognize these signals themselves.
in click-throughs from the app to the product catalog, indicating strong cross-channel engagement.
in promo code usage from the app.
What happens next:
Having received and processed your request, we will reach you shortly to detail your project needs.
After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time and cost estimates.
We arrange a meeting with you to discuss the offer and come to an agreement.
We sign a contract and start working on your project as quickly as possible.