12 March 2023
I’ve been testing the Shimano latest generation power meter since last summer, comparing it to virtually every other viable power meter and smart trainer on the market today during that time frame. And as one who tests power meters for a living, it’s glaringly clear the Shimano R9200P follows in the footsteps of the previous generation Shimano power meters in one key area: It’s horribly inaccurate.
Mind you, this isn’t just one unit, or just my test unit loaner provided by Shimano. Instead, it’s looking at data from multiple units, sourced from multiple bikes, on multiple continents. Data which is supported by numerous WorldTour Pro cycling teams and other pro riders reaching out after seeing their own data issues. It’s a constant yet frustratingly inconsistent pattern of inaccuracy that I’ll outline in the data section of this review.
However, before we get to that – a quick and unusual preface.
Normally, my reviews are pretty long and detailed – be it for a power meter, smart trainer, or anything else I cover. These reviews usually cover all aspects of the product: From unboxing, to installation, to general use features (like the app, connectivity, etc…)…all sorts of stuff. The point of all that extra stuff being to help people understand all aspects of the product, so even if one portion of the product isn’t to my liking – it might be to their liking, and thus, the review is still useful to them.
However, in the case of a power meter, the *ONLY* thing that matters is accurate data. Or, at the very minimum, consistent data. In this case, it is neither accurate, nor consistently offset (not ideally either, but sometimes workable). The parameters for when and why it’s inaccurate vary within a ride. Thus, all of the typical review components (unboxing/install/app/etc..) simply don’t matter because I can’t see any viable electronic use for this product. And pro teams agree too – they’ve been actively trying to avoid using it for training and racing, but most are bound by Shimano sponsorship agreements. Despite that, we do actually see some top Shimano-sponsored pro teams using alternate power meters on their R9200P-equipped bikes.
Thus for this review, I’m just going to focus on the single thing that matters here: Power accuracy.
With that, let’s dive into it.
Power Accuracy Deep-Dive:
In this case, the unit I was loaned from Shimano came direct from them, brand new in a box, via their media office. I assume they probably did additional QA/checking on it, as many companies do before sending a product for evaluation. Though I’ve found that, generally speaking, that won’t help much in power meters. The issues I usually find are more algorithm-based than simply one bad unit (in fact, it’s been probably a decade since I’ve had a supplied unit that was physically bad/broken – virtually everything is poor design/algorithms these days if it fails). I’m going to mostly focus on my data, but also splice in some data from other individuals with dual power meters.
From a power meter testing standpoint, you can read a detailed deep-dive on how I do it here. In short though, I do both indoor and outdoor testing, against multiple trusted power meters, in numerous environmental conditions (dry/rain/hot/cold). In this case, I’ve tested it against the following other units over the past 8 months. You can actually find data from the Shimano unit in countless other reviews I’ve written since then.
– Favero Assioma Duo
– Garmin Rally RS-200 Dual
– Garmin Rally RK-200 Dual
– Garmin Vector 3 Dual
– Tacx NEO 2T
– Wahoo POWRLINK ZERO Dual
– Wahoo KICKR V5/2020
– Wahoo KICKR V6/2022
– Zwift Hub
Obviously, with 8 months of testing, I have an absurd amount of data. So instead, in this post, I’m going to try to cover a wide variety of scenarios/data. With that, let’s start indoors, and then transition outdoors. If there’s any takeaway here, it should be that sometimes it’s horrifically inaccurate, sometimes it’s OK. The problem is? You just don’t ever know which ride is going to be good versus bad.
Here’s an indoor trainer ride on Zwift. In this case, it’s compared against the Garmin Rally power meter pedals and the Zwift Hub smart trainer. It’s an ERG workout, with either 60s or 30s intervals. However, there’s one big difference between the first half and the second half – I shifted from the big ring to the little ring:
And as you can see, when I did that, the accuracy of the Shimano R9200P power meter immediately went out the window, offsetting by 20w (the other two units stayed identical/consistent).
Here’s an even more crispy-clear view that I did on one workout, alternating big ring/small ring each interval. It’s astounding how it instantly goes askew. In this case, I kept the rear cassette in the middle and exactly the same position. You’ll notice the wahoo KICKR V6 & Garmin Rally stay in agreement, despite the Shimano R9200P going off-kilter.
Obviously, this means anytime you’d be climbing in the smaller chainring (such as up a long mountain pass), your wattage is incorrectly overstated by 20w+ (which, is a crapton to be wrong). Here, for fun, I did an entire workout this way, just to demonstrate this. In this ERG workout you can see the 20w offset the entire time!
Or again, this time on TrainerRoad and a Wahoo KICKR V6. Mind you, for virtually all structured workouts on virtually all smart trainers, you actually *WANT* to do these in the small chainring to increase the responsiveness of smart trainers (not to go down a rabbit hole sidebar, but basically when in ERG/structured workout mode, lower speeds help smart trainers more quickly stabilize on a set-point. This has nothing to do with accuracy directly, but rather, think of it like riding a bucking horse. A higher flywheel speed makes it harder for the trainer to stabilize. Thus why every training app recommends the small chainring for ERG/structured workouts). Anyways, here’s that TrainerRoad workout – again, an offset:
You may be noticing the first few minutes there the KICKR & Rally aren’t quite as close as later, then at the 20-minute marker I do a calibrate there. You should do that anytime you’ve moved pedals around, or if you bring the bike in from a colder spot to a warmer spot (as I did here).
But why limit this data to just mine? Some DCR readers, as well as fellow reviewers, have sent me their data. For example, here’s a DCR reader doing a test on Garmin Vector 3 pedals but also the highly respected power-accuracy-wise Tacx NEO 2T. Here you can see when he uses the small ring, he gets the exact same offset. In this case, he’s riding Zwift in regular mode (non-ERG, called SIM mode) – meaning this is *PRECISELY* what you’d see outdoors if climbing.
And then here’s highly respected power meter reviewer Shane Miller, with one of his (many) indoor workouts, showing variance between the Shimano R9200P and the Favero Assioma Duo + Wahoo KICKR V5, repeatedly. This is interesting because it shows yet another quirk of inaccuracy depending on the exact unit – right side dropouts (meaning, the right side of the power meter, so-called drive side). These are visible specifically in sprints, once you zoom in.
Check out one of these repeating sprints (compared to Favero Assioma Duo’s and Wahoo KICKR V6):
Notice how the Shimano R9200P drops below the rest? While getting exactly the same max-power on a sprint from multiple units is always tricky (due to transmission/recording timing issues), this gap is huge – about 60w. But the real ‘tell’ here is actually that the Shimano right-side is failing during this sprint. Check-out the breakout here showing left/right power. Here you can see that the Shimano unit undershoots the right side (while the Favero correctly reports it, matching the KICKR’s total power as well);
This repeats itself four times incorrectly on these sets. Except, then for the final and fifth sprint, it magically matches. Why? Who knows. That’s the problem – it’s just not *always wrong* or *always right*, instead, it’s *always inconsistent*.
About now you may be wondering why different people see different inaccuracies. Shouldn’t they all be the same? Well, not really. In talking with a variety of people in the power meter industry about this, there are a few theories. But talking to one company that actually cut apart one of the Shimano R9200P units and inspected it, they believe the culprit isn’t actually Shimano’s strain gauge placement, or even crank arm casting/design, but rather, the specific other electronics Shimano is using as part of the larger power meter design. In their own testing they’ve also seen many of these fluctuating inaccuracy problems.
For example, here’s a ‘regular’ (SIM mode) Zwift ride; in this ride, there’s very little to complain about. It largely matches the Garmin Rally power meter pedals and Wahoo KICKR V5 smart trainer just fine. The only somewhat minor thing to complain about is it’s oversmoothed, actually matching the KICKR’s slight over-smoothing (due to the flywheel). You see it on some of the surges, where the Rally responds faster (both ups and downs). Not a huge deal, but again – how do you know on which ride it’s happy, and on which ride it’s upset?
So…let’s head outside. I’ve got endless piles of outside data. And like indoors, sometimes it’s fine, and other times it’s wonk. Take this little snippet from a ride in July. What’s going on here? You can see the power drops-out, at the same time the cadence does some weird dropages too:
Remember: In most power meters, cadence is a critical component of determining power. If the cadence briefly stumbles, so will the power. You can’t have power without cadence. However, as I’ll show in a second, this isn’t about connectivity dropping. Instead, it’s about how the unit handles resumption from when you stop pedaling (like you see above/below to the left after a stoplight), as well as how it handles easy-pedaling.
The above ride was an easier ride with my wife, but still, it demonstrates how it struggles to maintain a consistent story. A month later, in August, there’s this ride, and things here seem just fine for the most part – with very few items to meaningfully complain about:
The temperature and climate weren’t much different that day, rather my pedaling was. On the July ride, it was an easier intensity with my wife, while the August ride was a bit higher intensity. That, in turn, impacts how the crank arm bends, and from there, how the electronics handle that.
And then this ride in late September seems largely fine too. The pattern? Well, in this case it’s relatively constant intensity (medium), but also all in the big ring. No climbing (only flats where I live), and no major sprints. Oh, actually wait, there was one surge – and during that, the right side spiked out by 100w over, breaking the consistent power that held true the rest of the ride. This is the exact same pattern that Shane Miller saw with the right-said basically failing.
Here’s a 3hr ride from January on a cold windy day. The windy part likely means nothing. For most of the ride it’s relatively consistent. Then after a few hours of riding, I do some lazy sprints. Maybe not proper sprints, but half-assed efforts. As you can see, the actual sprint data isn’t that different, but, if you look closely, you can see the right side again starting to falter. However, check out what happens in between the sprints when I stop pedaling a few times briefly. The Shimano doesn’t stop producing power despite *CLEARLY SHOWING* I’ve stopped pedaling (RPM goes to zero). This is identical to a typical peloton/group ride scenario where you are frequently coasting for a few seconds to maintain positioning. No wonder the pro teams struggle with this thing.
And just to be clear on the above, we know this isn’t a connectivity issue to the Edge 1040, since it’s only tied to when I stop pedaling, and atop that, the other ANT+ accessory at the time (the chest strap) continues recording flawlessly the entire time.
Or there’s this moment on a 3.5hr ride last week, coming from coasting to pseudo-sprint (700w), the Shimano entirely missed it. Like, fart in the wind missed it – off (lower) by 300w at its peak, because of the latency:
Or, if you prefer, on that same ride where, due to the right-side failures, it undercuts this surge by 150w:
Again, you can pick your poison – which failure do you prefer? Is it the right-side failures, or perhaps the cadence freeze-ups, or maybe it’s the small ring inaccuracies?
And, of course, again, it’s not just me. Here’s some data showing the exact same thing from another unit literally half a world away in Australia with Shane Miller. This set of surges shows how it just entirely misses, over and over again – undercutting the power due to the cadence tracking.
Normally, at this stage in the accuracy section, I’d try and summarize it all into a tidy paragraph or two. But I’m guessing by now it’s crystal clear. Nonetheless, here’s the distilled version:
– The right side is inaccurate in sprint, undercutting your power significantly
– The smaller chainring data is consistently inaccurate in most situations
– It incorrectly shows power output even after you’ve stopped pedaling
– It undercuts many short surges by 300w+
– It doesn’t handle easy pedaling well (such as in a group ride), giving incorrect data
– The cadence sensor data is often laggy by about 2-3 seconds (despite having a magnet)
I think that’s about it. Frankly, the IQ2 dumpster fire of a power meter project produced a more accurate first gen pre-production prototype three years ago, than Shimano has made on their next-gen version of their power meter, even after Shimano acquired Pioneer (who actually made quite good power meters). It’s absolutely mind-boggling that most of these weren’t caught in testing. Or, they were caught, and Shimano just didn’t care. Cross-chaining issues like small-ring data being incorrect was a problem 10-15 years ago in power meters. Every power meter company in the world knows to test for it, and it’s a trivial test to do. There’s no excuse for not getting that right in a modern power meter.
(Note: All of the charts in these accuracy sections were created using the DCR Analyzer tool. It allows you to compare power meters/trainers, heart rate, cadence, speed/pace, GPS tracks, and plenty more. You can use it as well for your own gadget comparisons, more details here.)
Wrap-Up:
As if this post somehow needed a wrap-up, I’ll give it a whirl.
For the second edition in a row, the Shimano power meter simply isn’t accurate. Not even close. And it’s been ‘not accurate’ for a long time. The previous model wasn’t accurate, and then Shimano made a big deal of implying this one would be accurate. Shimano stalled for many months to get review units out to any reputable reviewer, and I finally got one last summer. I’ve put a lot of rides in a lot of conditions on it, and ultimately, the answer is unequivocally clear: It’s simply bad.
And mind you – that’s the same overall result that another industry-trusted power meter reviewer found, as well as numerous other media outlets. And now you can see why I skipped going through the motions of spending hours consolidating photos of installation, general usage, screenshots of the app, or anything else. With a power meter – if it’s not accurate, then nothing else matters. Period.
Unfortunately, for many bike consumers and pros alike, you’re stuck with this unit. I’ve been looking at new bikes recently, specifically triathlon bikes, and many of them are equipped with the Shimano R9200P as a non-removable configuration. Thus, you’re paying an extra $1,000 (typically) for the privilege of something that’s not accurate or consistent. And lest you think you’re alone here, WorldTour Pro teams and their riders are suffering this same problem, except in their case Shimano has paid their team vast sums of money to ride the R9200P. And while these teams are riding the R9200P in racing, a number of them are deferring to other power meters in training. And even last year we saw Lotto-Soudal as a sponsored Shimano team also ride with 4iiii power meters as their power meter sponsor.
Complicating matters further is that many media outlets also have extremely heavy advertising spend from Shimano, which is one reason why you don’t quite see as many detailed accuracy-focused reviews of the Shimano power meters. But again, if a power meter is inaccurate, it serves no purpose as a product.
And while some might ask about the Shimano Ultegra R8100P, I haven’t tested that unit. Though it’s based on the exact same electronics design as this one, and there’s no reason to think (and no data to support) it being any more accurate. I don’t, at this point, have plans to test that unit. Instead, I’m already working on testing other Shimano-compatible options. For example, the Stages LR (dual-sided) version for the Shimano R9200 DuraAce crankset is on the bike and the first rides completed. That unit tested extremely well by GPLAMA/Shane Miller, and my initial data is looking good too. 4iiii also has a variant as well for the R9200.
Stay tuned for more testing to come. With that, thanks for reading!