21 May 2024
This is a technical rabbit hole post, which ultimately ends up blaming not one, but all three of my young daughters for my Garmin watch woes. Hope you enjoy!
Next week will mark a year since Garmin introduced Endurance Score on their Epix Pro & Fenix 7 Pro products. Since then they’ve expanded it to other top-end watches, including the Garmin Forerunner 955 and 965. Endurance Score was designed as a bit of a counter-point to watching your VO2Max number never actually move. There’s a simple reality that your VO2Max score can’t infinitely increase, and especially as you age, it’s actually going to slowly decrease.
Quick Endurance Score Background:
Thus Garmin’s Endurance Score gave a new metric that essentially rewarded longer aerobic activities. Or more specifically, time in mostly higher heart rate zones. The longer and harder a workout was, the higher impact it had on your score, which trended over longer periods of time, and had values upwards of 11,000 or more. A year ago when I first released my review of those two watches and Endurance Score (and Hill Score that was released concurrently), I said “there’s shortcomings that seem hard to connect”, but went onto note that “I think Endurance Score is reasonably in the ballpark of things for me”.
A year on, with plenty of data, my understanding of these two metrics has improved slightly. With an emphasis on ‘slightly’. I’ve learned/observed the following:
– Endurance Score won’t often increase immediately after a workout like Acute Load, but instead the next day.
– Except for really big workouts (e.g. many hours), in which case it will often pop more quickly
– Endurance Score really seem to care more about the last 90 days average trending in some capacity
– If you get sick and stop training, it’ll quickly drop your Endurance Score (to rub salt in your wound)
– When you resume training again, it’s unlikely those first few workouts will change the score at all (like, zero), instead, it’ll take at least a week or so to see any score change upwards
– It actually rewards consistently above outright duration. DesFit is the poster child for this, doing roughly 90-120 minutes of riding daily, but rarely (purposefully) doing anything more than 3-3.5 hours. His Endurance Score has reached 11,000, though currently sits just below 10,000
That said, a year later, and I think it’s still generally in the ballpark of were I’m at. Not in a fast-race way, but rather just a ‘do I have long endurance’ way. So in other words, matching the name of the feature. It’s got quirks, but again, I think it’s roughly aligning to what my body is capable of (of course, ignoring any feel-good titles it gives me like ‘Elite’).
How it all went wrong:
So last Friday, my endurance score was floating around 8,000pts. The last little while it’s been in the 8,000-8,200 range. But given I had a bunch of travel in the preceding 7 days, my score was down a bit due to jet lag driving lower training volumes.
For this Friday workout, I picked something at random from Zwift’s workout library, and ERG workout of sorts to test out some gear I’m poking at. It wasn’t super hard, nor super long. I wanted to get out and watch my 6 & 7yo’s sail their tiny little boats solo around the lake. The workout was slated at 39 minutes, though I’d end up riding 52 minutes:
I had both Zwift recording, but also the Garmin Epix Pro recording the workout, like usual. Some 45 minutes later I completed the workout without any difficulty, ready to go about my day. It was then though, that I noticed my Endurance Score had skyrocketed, gaining about 1,000pts due to this single mundane workout. Here’s a picture I took from the dock watching the kids:
I texted Des of DesFit, and we went through a quick troubleshooting list, notably wondering whether one of the power meters produced a crazy-high erroneous power spike, or inaccurate data (e.g. 800w for 45 mins)? Nope, checked both files. Did Acute Load randomly skyrocket? Nope, also good. Any weird heart rate skyrocketing issues? Nope, totally clean too.
Also, he later had a funny comment: “FirstBeat engineers be like, Oh FML”.
At this point, I couldn’t figure out any obvious reason why it’d have sky-rocketed. Everything seemed to be fine, except this number. Then, while sitting there later that night watching a TV show, I bumped the watch and noticed an odd value for my FTP:
(Apologies for somewhat blurry couch photos)
For context, my FTP is normally around 3.4-3.5w/kg. Some 16w/kg puts me well over 3x the highest world class Tour de France level athletes.
But that gave me a clue. Given I knew my power values from the workout were normal and find, FTP in the context is literally written as “watts per kilogram”. So, had my weight changed?
I dug into the watch settings for my weight, and sure enough, it was now showing:
Well, there are many things I am, but weighing 19kg (42lbs) is not one of them. But how’d it get to that? Had I accidentally somehow pressed a boatload of buttons and changed a setting on the watch?
Instead, I went to Garmin Connect app, to see what it said as my weight there, and sure enough, I found three distinct weigh-ins, for values between 18kg (40lbs) and 52kg:
And then it hit me: My kids.
Specifically, a few hours before my trainer ride, the doctor’s office had called and needed an updated weight for one of my daughters to fill a prescription. The kids were home with a babysitter, and we texted to have the kids jump on the scale and get their weights.
Which scale? The Garmin Index S2 scale.
Now normally, that wouldn’t be a problem. The scale automatically assigns weights to users based on weight. Thus it vaguely knows the difference between me, my wife, and if our kids jump on, and suggests which account user it should be assigned to (via WiFi).
But such technical prowess vastly underestimates a trio of 4/6/7 year old’s interacting with any display-laden object. You see, you can change the assigned user by simply tapping left or right on the scale, as it scrolls through a list of names. Undoubtedly, for whatever reason, the kids tapped again and it assigned the first one to me. Once that baseline was established, it then assigned the other two weigh-ins (because even know we only needed one child’s weight, all three wanted to partake).
Within a few minutes of them weighing in, the scale would have automatically updated my Garmin Connect account, which in turn would have updated my watch. All without ever telling me. Then, as I went off to do the workout, it was effectively a ticking time bomb for my Endurance Score. As soon as I completed that next cycling workout, boom, my Endurance Score was broken.
I was able to reset my weights of course, and upon my next ride, my endurance score went back to normal. But there’s no why to remove that spike, it’s there apparently forever.
Going forward:
Now, like I said above, I think the Endurance Score (as a number) roughly aligns with my endurance capability. It’s a ‘good enough’ metric, even if some might argue it’s just another useless metric. But realistically, virtually every ‘score’ or metric in sports is a made-up metric. There’s almost no scientific agreement on any metric in sports/fitness, as to how it should be achieved. The much regarded FTP is a great example of that. While it’s generally thought of as the max wattage “held for 1 hour”, even its creator has since said that’s more of a “rough” estimate, and it could be some other value.
And even those metrics that do have scientific agreement (VO2Max), can often be screwed up during actual lab testing. I did one two years ago by an incredibly reputable location/lab that handles a boatload of pro athletes, where it spit out an absurdly low (for me) VO2Max of 46. A look at my data by some other reputable people I know, shows they very much screwed up the test (I’ve previously been tested in the 56-61 range, depending on fitness levels).
Point is, every metric has its quirks. And in this failure case, the quirks aren’t directly with Endurance Score. Instead, it’s with the data that Endurance Score builds atop of. Frankly, I’m surprised this hasn’t happened to others before, as certainly plenty of parents have connected Garmin scales in their homes that kids might poke at.
Thus, a few things should be tweaked to avoid this:
1) To much weight change: First and foremost: Garmin Connect simply shouldn’t allow a 150lbs change in weight from one connected scale measurement to the next, on a given user profile. That should always be rejected, unless manually updated on the Garmin Connect profile.
2) Unlikely weight values: Given my account shows my age and height, Garmin should be either rejecting my 44lbs scale weigh-in for my account, or requiring manual approval. There’s no adult that’s 6’2”/188cm that weighs 44lbs/20kg.
3) Non-Human FTP Values: Again, Garmin simply shouldn’t allow a 16w/kg FTP value. Collectively the world agrees this is 3x that of any known human effort. In the same way Garmin throws away power values above a given threshold (e.g. 5,000w), they should also be discarding these values.
4) Massive Unlikely Endurance Score jumps: The Endurance Score algorithm should be looking at whether a jump is justified. I’ve had massive jumps before, mostly around massive efforts. So I’m not saying discard big jumps. But rather, there should be something in the logic that says a 53min trainer ride shouldn’t cause a 1,000pt jump in score.
Of the list above, I think #1 and #3 are really the easiest ones to implement. And would likely solve for #2 and #4 ever being needed. And of course, in hindsight, it’s easy to ask why these things weren’t done before. But ultimately, this hasn’t been a problem for me in the 14 months I’ve had Endurance Score. Though, looking back I can see my kids did weigh-in to my name back in October, but it had been timed only between some runs, so it never had an effect.
Garmin has a saying internally (like many companies I deal with), when something really funky happens. It’s asked whether this is “specifically a Ray-only problem”. Meaning, is this something that only really happens on my account because I have so many devices and files uploading, far more than the normal user, that can often trigger weird edge cases?
Sometimes the answer is ‘Yes, this is a Ray-only problem”, but perhaps is applicable to a few other people, and things ultimately get changed. Sometimes the answer is ‘No, Ray just stumbled into it first”, but will undoubtedly be found by others. And sometimes the answer is as Des roughly described it above “Out of all the people that would stumble into this, why the fudge does it have to be Ray?”
I’m pretty sure this one falls into the last camp, but is something that I’m betting happens to more people than anyone realizes, and people can’t quite figure out why it happens (especially if they then weigh back in themselves, resetting the values).
In any case, welcome to the end of my rabbit hole – and thanks for reading!