The day after releasing our groundbreaking 2016 Enduro Mountain Bike Review, we set out to apply our singularly rigorous testing process to trail bikes. Backed by nearly two years of R&D involving 17 bikes, a team of 20, and thousands of riding miles across iconic western trails — our bike testing and evaluation process is the most objective and comprehensive in the world. The enduro review may have set the bar, but we just jumped over it. We upped the ante to stress test the intense climbing/descending balancing act that a trail bike is expected to perform. We brought in more testers, added time trials, and included even more diverse terrain to evaluate how each bike performs across the broad range of territory that trail bikes seek out, or accidentally find. We wanted to know which of the top six 2017 trail mountain bikes is the very best. Find out how below.
We approached Trail Bike testing with a solid protocol developed during our 2016 Enduro Bike review. Several key findings from that review influenced the trail bike specific process. For example, our Enduro review proved that a rider can feel fast on a bike without actually being fast on that bike. Timing rides, using our benchmarking time trial process, helps us discover these disparities. As such the benchmark courses are one of the key components of our testing process.
Combining this empirical data with the objective opinion of six professional testers is the key to our comprehensive review protocol. Hard data provides the backbone and initial basis for our comparison, while the opinions of our expert testers fill in the experiential details such ease of handling, comfort of suspension and the ever-mysterious fun factor. We pay particularly close attention to any tester conclusions that reach consensus. These obviously hold more weight than individual opinions.
We also know that consistent benchmark time trials and accurate rider opinion are dependent on setting the bikes up perfectly for each test rider. As such, we spend a great deal of time with expert consultants to ensure proper suspension setup and rider positioning. Following set up, each tester spends some quality time with each bike to get a feel for their idiosyncrasies. This basic level of experience with each bike is critical to nailing down proper results during benchmark testing.
So what exactly are benchmarking time trials? Well, we made our six expert bike testers ride repeatable-effort intervals (i.e. just under race pace) up and down marked courses. We had one downhill course that combines challenging and more mellow feature and two uphill courses to compare and contrast climbing characteristics on smooth and rocky technical terrain. We were often surprised by the results. The times challenge us to separate performance perception from reality. But the merit of a bike is more than these numbers alone can express. That's why all us bike-loving-freaks are out there riding them all the time.
We tested the Yeti SB5.5, Ibis Ripley LS, Santa Cruz Bronson, Pivot Mach 429 Trail, Specialized Stumpjumper FSR Pro Carbon 6Fattie, and the Intense Recluse — two 29ers, and two 27.5"+/29" wheelsets (the Mach 429 set up as a 29er and the Stumpjumper with 3" plus size tires) and two 27.5" bikes.
That's quite a range, but what's in a name anyway? Not centered around a particular racing style, such as cross country or enduro, trail bikes vary as widely as trails do. They can hedge into the upright, uber-light, barebones feel of cross country, or into the heavier, burly suspension systems and more aggressive geometry of enduro, covering the wide-ranging space in between. Traditionalists consider trail bike travel to range between 120mm and 140mm. But with a slew of other design decisions affecting a bike's ability to take a hit, climb a hill, or handle a corner at speed, travel is no longer a singularly defining feature. We wanted to know how the test bikes stacked up across the board in actual performance on the trail. So we threw them all in together, pitting pairs of similar bikes against each other, and against the group, to find out which performed the very best.
Our six testers are beasts; they're kind and polite beasts, but they have a deep need to devour entire mountainsides on two wheels. It's an intimidating quality. Varying in height, weight, riding style, and post-beer preference, these five men and one woman are all ridiculously opinionated and know how to stress test a bike. They've been riding and working on bikes for a collective 145 years, after all. Recording, comparing and contrasting each of their opinions on every aspect of these bikes makes us very confident in our resulting assessments. Group agreement is our highest indicator of a bike's actual performance. Though the personal perspectives of these very different riders are invaluable to the quality of our review, in the end, we seek consensus. We always seem to find it.
We took a cue from Fight Club: we don't talk bike testing. Alright, we do — obviously — but not until the bitter end. And it's a total pain. At least 5% of the fun is shop talk, and we hate missing out on fun. Until we're ready to come together and make rating decisions, we opt to hold on to the integrity of each tester's personal experience on every bike to eliminate groupthink. It also increases our confidence in similar opinions that testers came to separately.
The only release valve is the interview process. After each ride, testers step aside one at a time to answer performance questions covering handling characteristics, pedal efficiency, suspension performance, cornering capabilities, and, well, fun.
The Test Courses
While we set the testers loose on the local mountain ranges to rage far and wide, we bring them together to complete benchmarking time trial laps on a combination of relentless test courses. We use two uphill and one downhill course for trail bike testing. The trail bike test has a greater emphasis on climbing performance than the enduro review since people expect trail bikes to rally on the uphills and descents equally while enduro bikes have a downhill bent.
The Race Courses
Our test trails are a gauntlet of technical and rolling singletrack. There is a flow trail to compare and contrast pedaling performance and feel on a 10-mile loop, a technical climb, a smoother one and a descent that balances the hardcore with the more nuanced aspects of downhill riding.
Flow Trail Rides, AKA The Lunch Hour Loop
The Lunch Hour Loop is a 10-mile ride that combines all of the features found in our benchmark time trial courses. It starts with a non-technical single track climb to get the blood flowing, before tackling a fast, flowy and sometimes rough downhill with layers of tight switchbacks. Some rolling terrain follows, with a short technical rock garden followed by a paved climb. The loop finishes with a steep and loose descent. This course mimics the terrain that most trail riders will encounter. It also gives our testers time to play with compression settings, and formulate bike impressions. Most everyone has a short go to ride like this for midweek sessions in between longer adventures on the weekend. This is ours and it is the quintessential trail ride. (And yes the name is a shameless play on the famed backyard playground in Grand Junction, CO).
Technical Uphill Course, AKA The Soul Grinder
Short, extremely technical, and punishing, The Soul Grinder puts the trail bikes through their paces. Long sections of trail with chunky tire stopping rocks test rollover characteristics of different tire sizes and volumes. Tight corners featuring rocky stair steps are an ideal test for low-speed cornering characteristics and rear wheel traction. An interspersed series of rugged, short and steep drops allow us to assess acceleration under load and suspension feedback. While the course is short, it requires concentration and maximum effort. It's the perfect testing ground for a technical trail bike.
Smooth Uphill Course, AKA The Crank
Unique to the trail bike review is The Crank, a three-minute long section of smooth, winding single track with only switchbacks, a few steep sandy sections, and a five-foot rock slab between you and the top. A trail bike needs to perform well on a wide variety of terrain. The Crank requires a steady application of power versus the short, intense bursts of power needed on the Soul Grinder. This course tests suspension efficiency, ability to maintain momentum, and rolling resistance in the absence of technical trail features.
Downhill Course, AKA The Scorpion
We kept the same downhill test track used in the enduro test to push the upper limits of the trail bikes' technical capabilities. While the testers take the same line every time, they can choose to ride over rollers on shorter travel bikes and launch off them on the more aggressive models. The big features are linked by flat, sandy straight-aways that give the more pedal-focused bikes a chance to catch up. The chunky rock gardens and narrow rock pinches are tricky features that every trail bike is sure to come across.
Bike Setup and First Impressions
To set up the bikes for each rider we spent a day dialing in suspension settings such as sag, compression and rebound on every bike. A team of mechanics and suspension consultants recorded each rider's static SAG by getting them on the bike and setting it to the manufacturer's recommendations. Then everyone rode around and jumped off a bunch of rock drops. We dialed it in from there. We also address fit issues such as saddle placement and stack height to make sure every rider is in their optimal position on each bike.
Then we sent the testers out on bikes, switching them out with the fervor of a magician dealing cards. This serves the dual purpose of breaking in suspension and giving us time to shake out and tune up the parts so mechanical issues don't compromise benchmark-testing times. We pay careful attention to tire pressure during this time, making sure each rider dials in their preferred PSI for each bike. We keep the same numbers throughout benchmark testing to eliminate as many variables as possible.
Benchmark Time Trials
Going fast is fun, and improving speed either climbing or descending is something we can all get behind. For the racers it is everything, and for the rest of us, it means keeping up with friends or pushing our own limits. Our benchmark tests are designed to provide qualitative data to tell of which of our bikes is the fastest.
Most of us use Strava to track our performance, but it doesn't meet our exacting standards as a timing tool for testing. We need accuracy down to the 100th of a second. The Freelap timing system provides it. A handlebar-mounted sensor automatically records start and stop times when a rider passes the transmitter gates. We make sure that the transmitter gates are in the exact same spot on ever trial to ensure that accurate times are recorded.
Chaos and How to Minimize It
We make every possible to limit time trial corrupting factors. Rider fatigue, changing trail conditions, mechanical issues, tires, trail familiarity, and trail user conflicts are all possibilities that we work hard to minimize.
Number of Laps Needed
We work to get four clean laps per tester per bike per course, giving us a solid base of data for comparison.
We knew from our past experience, that consistent times are critical for our comparison process. As our testers fatigue, the quality of data decreases. Thus we limit the number of time trial runs per day and enforce mandatory rest days to ensure our testers are fresh and ready to go. Even with these protocols in place, we constantly monitor rider fatigue between every lap, and will call a testing day if times start getting screwy or any of our testers start struggling. This means that bike testing is spread out over multiple weeks to ensure the consistency and quality of data.
We ride each bike only twice each test day before switching out to another one. This keeps us switching out rides, ensuring that each bike gets an equal share of any shifting trail or rider fitness conditions (i.e. drying out dirt or overly energized testers). Bike order is also carefully planned so that every tester rides the each bike at a different time during the testing day. Meaning they might ride a bike on the first and second laps one testing day when they are fresh and on the fifth and sixth laps the next when they are fully warmed up. Allowing testers to self-select bike order on test day is unreliable, and can lead to bias, or fights.
Every tester maintains the same line through all features on the benchmark courses throughout testing. The testers are intimately familiar with the test courses due to protesting practice runs. Maintaining consistent line choice is critical to revealing differences in bike performance as opposed to showing increased proficiency on the test courses.
The Human Factor
In an effort to maintain consistency over the testing period, testers must adhere to the following:
- Be in top physical condition at the onset of testing. If significant physical improvements are made during the testing process, times will be reflections of the riders, not the bikes.
- Have a high level of confidence and comfort on all testing courses to ensure that increasing proficiency does not skew test results.
- Agree not to disclose testing times to remove competitive drive from the test results.
- Each benchmark test run should be approached as a timed training interval. Test riders push themselves, but the effort is repeatable. Having a strong background in training and racing allows riders to generate submaximal, repeatable efforts with knowledge of pacing strategy; meaning do not blow yourself up on the first run! Proper nutrition, recovery, and body care are essential to this process.
- We also require riders to wear similar kit each day to limit aerodynamic drag changes during testing; basically, don't show up in baggies one day and a skin suit on the next.
The Tire Challenge
Tire choice has a huge impact on bike performance. The manufacturer spec'd tires on our test bikes run the gamut from wide to narrow with a variety of tread patterns. These differences in tires do effect the bikes' trial times. However, we test in a wide range of conditions, and any given tire will perform well in some conditions and underperform in others. In addition, we are committed to the principal of testing bikes as they come from the manufacturer. Buying a bike is a package deal and manufacturers spec tires as they deem appropriate.
Several factors impact their tire choice including personal preference, contracts with tire manufacturers, and price. Not all tires are created equal, and a bike manufacturer may choose a more expensive, better-performing tire at the cost of lost profit or increased base price. These marketing decisions impact the end user and thus we feel testing bikes with the original tires provides the potential buyer with the most accurate review. We also realize that tires are an easily replaceable item; we make every effort to note in the individual reviews when we feel that a poor tire choice is having substantial impact on times or tester perception of the bike.
Lap times slowed by mechanical problems such as a broken chain or stuck dropper post are thrown out, and repeated following repair. However, we keep times that are impacted by frame design, such as pedal strikes due to a low bottom bracket height.
We make every effort possible to limit the impacts of our testing on other trail users. This includes the use of spotters equipped with radios at the top and bottom of benchmark courses to ensure that the trail is free of traffic. When necessary, we cease testing to provide an open trail for public users. Safety of the public and our testers is paramount and always supersedes the need to accomplish testing. Whenever a lap time is impacted by trail traffic, the test run is repeated.
Flow Trail Testing
Our testers all complete at least one lap on the Lunch Hour Loop, described above, on each bike. This is an untimed, low stress, get out, have fun and get comfortable on the bike experience. It allows testers to feel out differences between bikes and experiment with cornering, pedaling, and suspension settings.
At times, our experiences on the bikes just don't make sense. The Intense Recluse was a bit of a conundrum during testing. With geometry similar to the Santa Cruz Bronson, and the same VPP suspension design, we expected the bikes to ride similarly. Extensive time was spent tuning suspension in an effort to give the bike as fair of an assessment as possible. Similarly, we added a volume spacer to the easily bottomed rear shock on the Ibis Ripley LS and switched out its unpopular handlebar. Then we took it out on some extra rides after testing wrapped up to give the consumer more information about this award-winning ride.
This is where all of the hard work comes together. Pages of notes and spreadsheets of data are compiled to rate each bike on a scale of 1-10 across five rating metrics. Each metric is weighted based on relative importance, giving us a score for each bike.
More About the Bikes
In-depth research led us to the selection of the top five 2016 trail that we bought for the test. Manufacturers specify all the components on the bikes as part of their factory builds. In cases where the manufacturer gives build options, components are selected that most closely match the builds on the other bikes in the test. An example of this is the dropper post length option available on the Ibis Ripley LS. We choose a 150mm dropper post rather than the 130mm post, to bring the build in line with the other bikes we tested.
While it's tempting to custom build each bike with the same components to provide an apples to apples comparison, we believe in honoring their creators' vision by buying complete builds. Manufacturers spec components on their frames that they feel accentuate the qualities of their frame design. It also addresses bikes as the average consumer will be purchasing them. All of our test bikes are the mid-priced, high-end build options that are readily available to consumers.
Bike angle finding and geometry measurement methods vary amongst manufacturers. In order to make direct comparisons across bikes, we take all of our measurements and weights in-house using a six-foot box beam level, six and three feet straightedges, a tape measure, and an extended digital protractor goniometer angle finder. The measurements and methods are described below. We provide the results and the manufacture's claimed measurements in a table at the bottom.
Effective Top Tube Length — We used a grease pen to mark the center of the head tube and then marked the center of the seat post at the same height with the help of a beam level. Finally, we measured the distance with a six-foot straightedge.
Reach — We used those same grease marks to get the reach. We placed the six-foot straightedge along the marks and leveled a vertical laser beam from the floor to the ceiling, running through the center of the bottom bracket. It also ran through the straight edge, giving use the measurement from the center of the head tube to the point in space directly above the bottom bracket.
Head Tube Angle — We used a digital protractor goniometer angle finder to get the head tube angle, leveling it along the back of the stanchions for each bike.
Seat Tube Angle — The same digital protractor goniometer angle finder found the seat tube angle by running from the bottom bracket to the grease mark on the seat tube where the effective top tube length was taken. Using this location helped us to standardize the measurement.
Bottom Bracket Height — Measuring from the ground to the center of the bottom bracket with a three-foot straightedge gave us our bottom bracket height.
Standover Height — After having our test riders measure the average location where the frontline of their body met the top tube, we decided to measure our standover height 7" in front of the bottom bracket. We set up the vertical laser beam there using the straight edge and measured the height between the ground and the top tube.
Chainstay Length — We used a straightedge to find the length between the center of the bottom bracket to the center of the wheel's axle, accommodating for the arch in the chainstays as best we could.
Wheelbase — Using a six-foot straightedge we measured from the center of the front axle to the center of the back axle.
Weight — We hung a Park Tool Digital Scale on a tree and weighed each bike with its pedals off.