Could the calories shown on fitness trackers be incorrect?

We are all guilty of getting in a great workout at the gym, and in knowing we have burned a certain number of calories, treating ourselves with an ice cream (or two) afterwards, and after doing so, not feeling quite so bad as our trusty calorie counter fitness watch told us it was alright. But, what if these junk food rewards are wrong? What if we are rewarding ourselves, or rather, replenishing calories based on incorrect results?

In light of this thinking, a recent experimental study was conducted by Stanford University in the United States, as well as the Swedish School of Sport and Health Sciences based in Stockholm, to determine just how accurate certain fitness trackers are.

Before this study was conducted, there was not a great deal of research done into the accuracy of fitness devices. The issue arises when results obtained from certain devices are shared by the user with dietitians, who in turn base their decisions on these results when assessing patients. Or when users use their own results to inform their decisions regarding their diet.

Researchers assessed the accuracy of seven popular fitness trackers in their ability to measure one’s heart rate and the calories burned when being compared to the FDA-approved gold standard assessment of one’s heart rate and as a result, the energy that is expended. Therefore, the study set out to determine the accuracy and trustworthiness of popular fitness devices when compared to their medical and clinically approved counterparts.

The study was done in a controlled environment, in an exercise laboratory and theoretically, informs users on which devices to choose based on the accuracy of the results.

The study involved the following seven fitness trackers:

1. Apple Watch
2. Samsung Gear S2
3. Mio Alpha 2
4. Fitbit Surge
5. Basis Peak
6. Microsoft Band
7. PulseOn

The fitness trackers were chosen according to their abilities, or rather, specifications to:

• Be worn on one’s wrist
• Have a battery life lasting longer than 24 hours
• Measure one’s heart rate consistently
• Be available to the consumers at the time that the study was conducted

The study consisted of 60 people, 29 of them being men, and the other 31 women. They were older than 18 years old and were selected in order to be accurate representatives of diversity in the general population according to their weight, height, skin tone, fitness level and range of their ages.

The individuals had to wear the fitness trackers whilst they were cycling, sitting, running and walking. While wearing their fitness trackers, they were monitored simultaneously with clinical instruments, these were an ECG (electrocardiogram), as well as clinical grade indirect calorimetry that was continuously being used. This was able to give accurate measurements of energy expenditure of each participant.

To simply explain what indirect calorimetry is, it is a technique used by laboratories to measure one’s fitness levels through monitoring the maximum oxygen intake when breathing while cycling on a bike (ergometer) or running on a treadmill and measuring this intake with a standard formula in order to formulate an estimate of the energy that has been expended. This is where we refer to the ‘gold standard’ that is used for measuring the number of calories burned.

Following this, the data received from the fitness trackers was then analysed against the data from the clinically approved devices. The heart rate obtained from the fitness trackers was then compared to the data that the ECGs captured, as well as this, the number of calories that were burned was also compared to the data obtained from the indirect calorimetry. From this, the error percentage in relation to the clinical instruments could be measured.

The results were interesting, with researchers noting that the majority of the fitness trackers had the ability to measure one’s heart rate on a fairly accurate scale, however, their calories burned estimate was purely calculated.

According to the study, the Apple Watch was found to be the top performer when it comes to accuracy, and the worst performer was the Samsung Gear 2, having the largest amount of errors seen when measuring the heart rate and calories that were burned by the participants.

The summary of the results are as follows:

• Not all fitness devices have been created equal. The Apple Watch is the clear winner, achieving the lowest error percentage according to heart rate, being 2%. The Samsung device received an error percentage of 6.8%. When the calories burned were tracked, the Fitbit Surge had the best outcome with 27.4%, and the PulseOn had the worst with 92.6%.
• The devices were all under-performers when it came to tracking the energy expenditure, as seen above, the most accurate device was still 27% off. Researchers noted that the rate should be 10% or less.
• Over the years, the ability of heart rate monitors has improved.
• The devices were shown to be more accurate in the collection of data when cycling as opposed to walking.
• The errors obtained seemed to be more present in men as opposed to their female counterparts, as well as those who had a larger body mass index and a darker skin tone.

Therefore, the findings of the heart rate accuracy were acceptable, however, the accuracy of calories burned was unacceptable for any setting. Researchers noted that doctors and individuals should be aware of the limitations, as well as the strengths of fitness devices. 

The research team explained that they hope another study can be conducted on individuals when they are exercising and functioning in their everyday settings and not in a lab. The scale of the above study can be regarded as small and more individuals, as well as more devices would need to be tested in order for a larger range and therefore, more accurate results to be obtained.

In conclusion,fitness devices can still have their useful benefits and can be used by individuals as long as they are aware of their downfalls. Therefore, to go back to our initial thoughts and questions on the matter, it is not advised that anyone replace their calories burned with a treat to reward themselves. The best idea is to focus on weekly fitness goals as opposed to the number of calories burned during each workout.