Grading Climbs with Data Science - Ft. Climb On
HTML-код
- Опубликовано: 28 сен 2024
- Nathaniel from Climb On shows us how to grade climbs with data science!
Climb On Blog: bit.ly/climbon...
Kaya: bit.ly/KayaClimb (20% discount on KAYA PRO)
My Climbing/Workout Gear: amzn.to/3J6fNdD
Climbing Shoes: amzn.to/2tCoPet
Climbing Tape: amzn.to/339wGMv
Hangboard: bit.ly/frictit... (Use coupon code GEEKCLIMBER for free shipping)
Doorway Screw Free Hangboard Mount: bit.ly/frictit... (Use coupon code GEEKCLIMBER for free shipping)
Portable Hangboard: bit.ly/frictiti... (Use coupon code GEEKCLIMBER for free shipping)
Pinch Block: bit.ly/frictiti... (Use coupon code GEEKCLIMBER for free shipping)
Wrist Roller: bit.ly/redgeroller
Foam Roller: amzn.to/2yOeSMU
Massage gun: bit.ly/relxbit (Use coupon code GEEKCLIMBER for 15% off)
Lacrosse Balls amzn.to/2D87wmn
Doorway pull up bar: amzn.to/2HoVtoy
Freestanding pull up bar: amzn.to/2V5PoUc
Resistance band: bit.ly/hybridc... (Use coupon code GEEK10 for 10% off)
Gymnastic rings: bit.ly/hybridc... (Use coupon code GEEK10 for 10% off)
My Apparel: amzn.to/3SEA50U
Flip Flops: amzn.to/2AT7YE2
Shorts: amzn.to/3Zt3A89
My Filming Gear: amzn.to/3ZlFR9J
Main Camera: amzn.to/3KJz0Tp
Budget Main Camera: amzn.to/35bXWfj
Action Camera: amzn.to/2oXlBPZ
Tripod: amzn.to/2LUnyW6
Wireless Microphone: amzn.to/3XYZBit
Wired Microphone: amzn.to/35eZaGz
Video Editing Software: amzn.to/2nvt8VK
Memory Card for Main Camera: amzn.to/2nrHWol
Memory Card for Action Camera: amzn.to/2IwED6p
Memory Card to USB Adapter: amzn.to/2OpkBPc
DISCLAIMER: This video and description contain affiliate links, which means that if you click on one of the product links, I’ll receive a small commission at no additional cost to you. This helps support the channel and allows me to continue to make videos like this. Thank you for your support!
Connect with me:
Official website: geekclimber.com/
Instagram: / geekclimb
![Yailin La Mas Viral, @kreizyk.official - Dale 2 (Remix) [Video Oficial]](http://i.ytimg.com/vi/JaYHBzgqz94/mqdefault.jpg)
Hi - Nathaniel here! Thank you all for watching the video and bringing such good energy. Let me know if there's any direction you're excited to see explored in the future and I'll do my best to write about it on the blog!
Is it feasible to use data sources other than kaya, like mountain project? Or is that data not accessible/accurate enough? It doesn't have indoors obviously but it would give more data points for outdoors and you could also use it to compare different outdoor climbing areas.
EDIT: Difficulty by year the route/boulder was set would also be really interesting to see since older routes tend to be sandbagged.
Thanks again for coming on to present your incredible work, Nathaniel! There are already so many great discussions and suggestions in the comments section, and I can't wait to see what's next from you!
@@GeekClimber Truly, my pleasure! Thanks for the opportunity!
@@jensenrice3236 Totally, that's a great idea
Adding data from different system boards may also be helpful! If we have label the routes then we can collect more attributes and do more cool analysis. Mapping climbers across app might be tricky - but it may also be irrelevant.
Data science is aid, all climbs are a v2 in my gym
@@kevwjin dont think thats a casually explained reference
Dude 😂😂😂
You just watched a casually explained video and all of the sudden you think you are funny
@@kevwjin yes it was.
@@НиколайТобиас ah ok. ty
As a data scientist who sprained my A2 pulley on Serengeti, this makes me feel a little better
I totally understand that A2 pulley strain is no fun. Hope you recover soon!
I have an O(1) algorithm for this: any climb I cannot do is V12
Cool stuff. As a data scientist who climbs at Dogpatch and uses Kaya, I have a couple thoughts. In addition to accounting for climb type in your multilevel model, you could incorporate knowledge of the route setter. You could regress out the individual’s improvement so you don’t have to limit yourself to three months of their data, which won’t be that effective at the beginning of a climber’s career anyway. Inference about an individual’s improvement over time could be cool to see and useful for reflecting on training. Finally, I’d love to see a comparison between gyms. Should be easier than comparing with outdoor climbs since there’s probably more overlap with e.g. Movement than with Bishop to bridge the two gyms, and you have all the internal data there.
Great video! As Nathaniel's friend and housemate, it's been really exciting to see this idea develop over time. One thought I had while watching the video: In addition to using the relative # of climbing attempts to indicate "difficulty preference", we could also use route setter ratings as another indication of preference. If a route setter calls route A a V6 and route B a V5, we should count that as a vote that A is harder than B. You should be able to apply different weight factors to the votes that we get from Kaya data vs. the votes that we get from the route setters. This might stabilize the results when there isn't enough climb data, especially the outdoor cases.
@everyone - we have this man to thank for our beautiful violin plots! I had no idea what those were before this.
This is a very interesting analysis. I've been a Route Setter for 7 years and have thought about these kinds of things extensively over the years.
Grading by analyzing # of attempts data seems like a good place to start but obviously it's not a perfect system. Similar to what he said in the video about slabs being difficult to analyze, I think factors like low percentage moves, balancey moves, morpho moves, dynos, poor beta, weather conditions, fatigue level of the climber, etc make it almost impossible to use # of attempts as a metric. I can't even say how many times I've put 50 attempts on a boulder only to "figure it out" and suddenly it feels easy. # of attempts is more of a measurement of how difficult it is to learn the positions and moves, which doesn't always indicate the physical strength required.
There truly are so many factors in climbing and I believe it will forever be impossible to find the "real" grade. I think the truth is that there is no such thing as a "real" grade. Grades are completely made up. There is no consistency. There is no precision. I find this to be freeing, as I can instead rely on a system of personal grades and only focus on climbs that inspire me.
That being said, I love this type of research and your analysis of the data. It's super interesting and I can't wait to see more.
Thanks so much Austin! Lovely to hear from a veteran route setter. I think you say it really well - there is probably no such thing as a "real" grade...but it's really fun to think about it anyway :D
@@natoparkway I totally agree! It is really fun to think about. I've been thinking about the "real" grade as a concept for a long time. The one thing I know is that if there is such a thing as a real grade, it will certainly be based on the data. Just like how you've been researching. Keep it up man!
Interesting idea, but lacks a lot of considerations. So I'd rather say that this model works only for the mentioned gym vs mention climbing area and won't be very effective when directly translated to other areas or types of climbing.
- How do you know all logged data is accurate? What if not all attempts are logged in?
- How to account for climber progression?
- How to account for personal strengths and weaknesses?
-- Low mobility, or climber is better with pockets rather than slopers?
-- Good technique vs Bad technique?
- How to account for conditions?
- Different rock types: sandstone vs gridstone vs granite, etc.
- Highball V4 vs Lowballs V4
- Classic climbing problems vs Modern Comp-Style problems
- Should you count working single, separated move as an attempt or only when attempt was performed from the start?
- Different gyms may have sand- or -airbagged boulder problems
-- E.G.: In my hometown same subjective difficulty climb would be in one gym V4 and V2 in another
There are a lot more to take into account... but this is surely fun little experiment. Would love to see where it will lead you eventually. Good luck!
AWESOME work, you two!!! The whole KAYA team was super excited to see our data being used in this way. We'd love to contribute to future projects in any way we can! The more data the better. If you're reading this and haven't joined the KAYA community yet, give us a try! Find boulders, create your logbook and track your progress, contribute beta videos to the community, and connect with your crew. 🪨💥👊💚
This is not intended to be a tirade but rather a means of expressing my frustration with the app in the hopes that changes might be implemented. With that said, I personally find the KAYA app is very frustrating due to one main drawback that is preventing me from wanting to use it: When I'm at the gym I don't want to be using my phone more than I have to, and when I open the app I seem to have to select my gym, then figure out what area of the gym I'm in or search for a route and spend time figuring out whether I have the correct one. By this point I'm frustrated and reminded why I don't use the app more often. I've often wondered what is stopping the gym from posting QR codes next to each route than I can quickly scan and get info on. Some gyms have QR codes by section, but not all sections have them, and I have to go in search of a QR code wondering whether it's even the right one. The second thing I would love to see is route setters providing info into KAYA when they set the route. Moreover, a video of someone else (another route setter perhaps) climbing the route. One of the gyms I used to go to in Boston would post videos of specific boulder problems/sets to their facebook page. It would be great if KAYA integrated something similar directly on the app.
Thanks! Hope that more San Diego climbers will start to use Kaya!
This is awesome! I would love to see a comparison with moonboard grades as well
That would be useful - and there'd be a lot more common data there than anywhere else. I imagine also two grades sandbagged? The only V3s moon board climbs I've tried felt much harder than normal gym V3s!
Moonboard is definitely more similar to outdoor grading. But it also depends on the Moonboard layout. For example 2016 and 2019 feel much harder than 2017.
My anecdotal evidence suggests that moonboard "official grade" also fluctuate from the "perceived grade" by a pretty consistent 1 grade in either direction. Same as the data about the gym in the video
Really interesting analysis! I actually attempted a similar project where I used measurable features of climbs--how incut a hold is, hold angle/slopeness, number of footholds, etc. to try and create a model. In the end, the most clear result I got from it is what you said towards the end of the video: Climbing is a complex sport, and grades are really for fun. It is so difficult to encapsulate the complexity of each climb with a single model, but I think approaches like the one in this video have a lot of potential.
When comparing outdoor grading to indoor grading it's also important to note that as a setter, mother nature doesn't care about your height or reach. So a lot of problems can be significantly harder or easier if you don't fit the right box.
When a human setter creates a problem in a gym, they usually try to add holds or force moves so that the grade is fairly consistent for most climbers.
Using data from a commercial chain like TS is going to make everything else seem "sandbagged". Climbed at a TS gym for two years and it was always inflated by two grades to attract more business.
Does kaya have pictures of the climbs? It would be cool to train a deep learning model to predict grades based on pictures of indoor climbs, or at the very least identify features of a climb (crimpy, overhung, etc).
I believe users can submit photos/videos for climbs but they don’t have pictures by default of them being in Kaya
This sounds like such a cool idea, but in my experience with teaching deep learning models the pictures (data) you feed it need to be quite standardized in order to create accurate predictions. So idk how feasible this would be with user submitted photos.
Boulder problems on a standardized board (something like a moonboard or kilterboard) might be able to solve this issue the easiest which would be exciting to see!
Some gyms/setters do include photos or beta videos for every problem, and our outdoor guidebooks include photos and topos for every problem. However, what may be more beneficial is our climb tags!! Setters and users can tag climbs with a huge range of descriptors for style and movement such as "Crimpy, dyno, press," for example. If you and your crew hasn't joined KAYA yet, we'd love to welcome you into the community! The more data the better :)
@@kayaclimbing yeah those tags would be great for training the model!
I just realized my dad went to the same university as this guy with the same major, but just at different decades.
This is in the running for best climbing video ever
I would love if they hooked this up to a computer vision system with multiple angles of every climb, and use it to predict the difficulty of unclimbed boulders and walls
love this!! I climb at Dogpatch and I use Kaya, so this was really fascinating!!
👊💥
Dude, I was projecting fake Eppulator as well! Need to get back to San Diego someday
How does this process take into account a climber's improvement over time? If I took 12 attempts on a V3 last year, but I've been training really hard, and I just sent a V5 on my third try, this analysis would call the V3 harder than the V5.
And this would really affect the data from a very significant number of climbers using the app; I don't think I know a single climber who isn't making at least some attempt to improve their climbing, and climb harder grades (and if such a climber is out there, I doubt they're logging their sends on an app).
as soon as I saw the graph I just knew that buttermilk stem was a slab climb since people fall off them so much.
interesting approach. I would suggest look into IRT, and Rashka's ordinal NN regression (or latent growth model or CDM for including time in the model). Is there a way to read more details about this experiment?
So rad! Thanks for sharing!
Nathaniel: smiles
Nathaniel’s teeth: also smiles
super interesting applying this statistical technique to something as multifaceted as climbing grades.
I am not sure what your data looks like but it seems it could be subject to selection bias. How many "did not complete" data points were reported. I would have thought having fewer climbers able to complete the climb is more representative of its grade than number of attempts to complete the climb. This feels like it could be an example of simpsons paradox where you are actually finding an inderect corrolation which mainfecsts itself with this behaviour such as your average climbers preference for more attemps on harder climbs.
As you pointed out, some styles just lend themself to more attempts, but I wouldn't say that
necessarily makes them harder. e.g. I would probally only have a hadful of attempts on a powerful climb as it is usually clear pretty quickly if it is beyon me, but I rarely flash even a moderatly easy dyno.
nice work, it would be interesting to see if you still got the same results if you approched the analysis from a different angle
Cool data and video. It’s all about fun and it’s complex…nothing beats that. It’d be interesting to see the data for height, weight, ape, years climbing, ape, etc. Another weird tough problem in Bishop is The Birthing Experience/The Womb V1. Having gone there my first trip in 2015 with my then teen daughter, her, bff and climb coach-no one got it. We were at 4 months into climbing, and my ripe age of 46(now 54) I was the only one to send it. Had some other youth boy climbers crushing V7 in bishop and could not get this V1 (that I got in 10 tries; nectarines year took me 60 tries).
Really cool use of the data. What I'd be most interested in (and would be impossible to account for) would be conditions. the biggest data point, Solarium, is notorious for baking in the sun, and 2 climbers of the same style and ability can have a very different experience on that boulder only a couple hours apart.
Also, sign of the cross is a historical sandbag that will stay V3 (I think it gets V4 if you can't reach the original starting holds and pull on one move lower) due to the history. I climbed it in January and it is for certain harder than anything V6 and lower in that area (C Sharp V6 and B flat V4 for example)
There is an analog to this in Kraft which is generally graded pretty soft, but I think its poker chip v1 that if you start lower is definitely atleast a v4, and the v1 is quite a bit harder if your finger are not tiny tiny fingers that can latch the starting holds comfortably. But then thats also a source of error in the data, because it assume everyone knows where the true start of every route is. If you tried potato chip and thought it was a sit start, you would log quite a few attempts for a v1. I did this because for some reason REI didn't sell guide books for Kraft when I was there and I had to rely on mountain project. When we finished the session, and looked up beta, we instantly realized it was not a sit start but a stand start with high hands.
People not understanding where the route actually starts and logging incorrect routes are not filtered out of the data.
I found what was said about slab quite interesting. I think if there were data points logged as well for climbs that took many attempts but where not sent would help make more " techy" climbs not overhraded. Like if a v5 slab took 10 attempts for a V8 climber it might appear as harder than a physical v7 they may have flashed in the overhang. But when accounting for all the v5 climbers who were able to send the slab after 20 attempts but no matter how many they put into the overhang they can't send it.
From experience situations like this usually happen because the v8 climbers has snapped an ankle before, and is scared of the insecure nature of slab where their strength advantage won't save them from a mistake, and they are either unwilling to commit, or take many more attempts to get comfortable enough to commit. A lot of slab is just about willingness to trust feet and commit. I wouldn't say that is a skill, as much as a personality trait. While slab definitely takes skill, there is a bigger aspect of knowing if you fall you are going to cheese grate down the climb in play.
interesting video! thanks!
What is going on at the top off the rock @7:15 - the view from the camera makes me thing snake, but the shadow makes me think maybe a cat?
Anyways - super cool vid.
What he was saying with certain boulders was very similar to how a lot of trad routes are graded where you consider everything from difficulty of the boulder to how it feels emotionally to the environment that the boulder is in. That being said I don't think these factors should be used to rework v grades but rather the idea of a new grading system seems like a better idea. Most previous systems were made by someone who casually used it to differentiate hard vs easy or was adopted by a small community and because there was no real previous grading system it became widely used so having a newer system that takes into account past grades but backs them up with statistical analysis or other factors seems like a harder but overall better solution.
It's more pertinent with trad routes where you cant see the sketchy parts from the ground to grade that way. With a boulder, you can tell if it's sketchy or not. The difficulty of the moves doesn't change because it's scary.
Some ones that is less scared isn't capable of harder moves just because they are less scared on that climb. Willingness to commit is defacto included in this metric, and that really can skew data.
Really cool vid. Bishop grades I think are sandbagged by a full number vs. grades in Washington state.
Is there a particular outdoor bouldering area in Washington state that you would like to see us analyze?
Flashing some v7s in my gym but can’t send one of the v5s feel like it all depends on the setters limit, If they climb max v7 it’ll probably be easy if they climb v14 the same grade will be brutal.
I used to think I could take any fall indoors, then rip acl, meniscus, and climbing
Love this!
Fascinating
I think you somewhat touched on this, but one limitation I notice with your methodology is that US climbing grades are typically intended to account for difficulty with perfect beta. However, some climbs are harder to find beta for than others of the same grade, meaning your metric would consider those climbs with unintuitive beta as harder for the same difficulty. Do you have any ideas on how to mitigate this limitation?
That system doesn' take the 'readability' of the climb into account. Some climbs are easy once you know the beta and the feeling of the moves, however it takes many tries to get there. This is more true for outdoor climbs. Indoor climbds are easier to read, because tyically every hold is there for a reason. That's not the case for the structures on rock. Indoors there is the additional benefit of knowing the holds een before you've tried the climb. They have probably been used in another boulder already. So comparing indoor and outdoor in terms of tries is questionable.
My right leg is graded 8c sadly no other parts are graded above 5a
Cool analysis! It'll be better to establish causal relations between the reported grade and aspects of the movement involved in the climb (eg, dynamicism, max strength, balance, etc). These variables more directly reflect the grade than "attempts taken" (which nonetheless is still a good and cheap one dimensional proxy).
Attempts taken assumes a lot of things. #1 the person was starting the route from the correct starting position, starting from the incorrect hold can make a route several grades harder, or not even the correct route. #2 people get scared outside and bail because of fear an not falling off. Bailing out of fear does not make the route harder, just scarier.
"outdoor grades are sandbagged," the grades were invented outdoors, Indoor grades are typically soft.
Are RStudio based calisthenics trainings the future?
Now you can simply take a picture of the route and have the AI determine the grade and possible betas
Was interested in seeing grading climbs with Data Science until I saw the data that is using is opinions. And that it puts outdoor and indoor climbs on the same place. This only favors indoor climbers, outdoor have other factors like the difficulty to see all the holds and more scary situations that doesn't affect the grade.
Or saying that slabs are easier because when you know the technique it doesn't feel as physically hard. Probably is another climb feels physically hard is because someone needs to improve technique.
If you are going to use data science or AI for this you should use it for the moves, like catalog "all" the moves and set a grade of each move regarding the distance and holds. Cheking the possible rests to lower difficulty and sum up everything adding the fatigue after a sequence of moves.
Alright so, outdoor grades aren't sandbagged, you're just bad at climbing outside. Yeah some spots are sandbagged, but no more than one random gym being sandbagged. Lots of people climb mostly inside nower days, so they're not practiced at going outside.
You should look at the V to font grade conversion too. There are 2 full font grades between V3-4 and between V4-5. Most of the other V grade increments cover 1½ font grades and from V11 onwards each V grade changes at the same rate as the font grade.
This is based on the grade conversion table in my 'Peak District Bouldering' guidebook
Didn’t know the Guy who did Family Guy also does data science
This analysis has a day 0 flaw... the number of attempts does not relate to the difficulty of the climb, you can see how un accurate the analysis at @6:33 when a v1 gets rated the same as a v7 because it took several attempts for a v9 climber to do it, yet he admit that once you know the beta on this climb is very straight forward. A climber that knows the beta before hand is likely to climb a boulder in way fewer attempts than one who is going for the onsight... thats the reason there is such an enfasis in the climbing world between onsight and flash.
Another point to make is that most of the outdoor guidebooks place the grade after the community consensus has been made (and many ppl has try all possible betas) and in most indoor gyms the routesetter has to give a grade on the spot without any consensus sometimes... of course there is always going to be a greater error margin and indoor climbers are going to think outdoor grades are harder, because normally for the first ascensionist the climb most of the time going to feel harder
I have never climbed
Considering the high variance in grades, would it be time to add half grades to the V scale? I mean would it help more people to say a V4.5 rather than a hard V4, I guess the question becomes what fraction do you stop at. Anyway great video!
They kinda have that already. Sometimes a climb is given a half grade like it's a V4/5 or V4+, V4-, etc.
I can already see the gigatoxic reddit threads on this
One big thing usually affecting the seeming grade difference between indoors and outdoors is people's experience outside and their comfort on real rock - I have noticed as I climb outside much more often the grades seem more accurate. Where I assume Dogpatch climbers only go outside a few times a year.
This is another factor that's difficult to account for. Last year I went outdoor bouldering for the first time and was falling off v2 boulders as a gym v7 climber. After a few weekends out I was starting to get some v5s outdoors. Just recently I went out for the first time this year and my outdoor bouldering had sort of regressed back to v4. Outdoor is so different it might not necessarily be much physically harder but an indoor climber will still struggle to climb at the same grade level outside
@@ThatLaggyNoob It could be done with a mixed model in which you control statistically for the number of sends indoors and outdoors
EXACTLY! For me, outdoors is easier. I think outdoor/indoor time are important to consider. And most probably where you started (I started outdoors for many years). And climbing are (sandbagging areas are real).
This happens both ways. When I have been doing a lot of indoor climbing I get worse at outdoor climbing. However, when I spend most of my time climbing outdoors my indoor grades suffer. The styles are so different and getting to know the rock at your local crag makes a big difference climbing outdoors.
This was my first thought just on personal experience. I am comfortable climbing slab, and I have friends that climb much harder than me that are not.
They might be able to flash v10 outside, but if they are sketched out on a v5 slab, they are going to bail many times, and take many attempts to send if they ever commit at all, where as I will probably tick it off in fewer tries.
This is a very evident flaw in their dataset where Buttermilk Stem, which is a consensus v1 with the crux being actually getting on the boulder, getting a v7 grade by the methodology, but in reality, its down to people without outdoor experience being uncomfortable stemming a granite slab, and bailing out of fear.
Having climbed Buttermilk Stem, its definitely an outdoor v1, its a bit slabby and maybe a bit stiff for a v1 if you have no stemming experience, but its in no way even close to an indoor v7 unless the gyms grading system is really messed up where they are calling indoor v4 v7.
"outdoor grades are typically sandbagged by 2 grades" nah, indoor grades are graded 2 grades too easy to encourage a feeling of accomplishment.
Fooking gym climbers, right
Amazing to have some data-driven approach! I think this is a great start but the study has some limitations. Firstly, the turnament approach implies transitivity. Let's pretend "greater" means "harder". If for one person X>Y>Z and for another B>Y, we imply for transitivity that B>Z and this is often not the case in climbing because of the style of the climb and of the climber. For me an "easy" slab can be much easier than for a super stronger bro. And the biggest issue is that people tend to avoid styles that they are not good at.
Second, experience indoors and outdoors impact a lot the perception of the climb difficulty.
I think these issues could be fixed with an experimental approach in which you have different climbers (hopefully not only indoor climbers) do and rate different boulders indoors and outdoors and then control statistically (or do a mixed model) for the amount of experience they have indoor and outdoor. Hopefully we will have more research in the future!
That would resolve itself over time with more tournaments and more "participant graphs" (not sure what to call it... basically more climbers participating in this tournament) where newer participants have graphs containing both B and Z. But until you actually get such data, you're right that it will assume transitivity. To me, that's a totally valid assumption though. If all we know is that someone thought Y > Z and that someone else thought that B > Y, then I'd say we should lean towards the assumption that B > Z, although the confidence shouldn't be too high.
Basically, I think the approach is solid, but they need much more data.
@@Ermude10 This assumption is used in Neoclassical microeconomics models, but when applied to social choices it doesn't work and you end up with cycles. Maybe with enough turnaments it would work but only with the assumption that you have roughly equal amount of people that prefer each style, and we know this is not the case (many more people that prefer crimpy, slightly overhanging boulder than those who prefer slabs or crack climbing, for example). Still I would try to have more statistical control of the data, like making a catalogue of different styles and at least control for that. And for indoors/outdoors experience. And maybe for height and ape index.
@@AllegraClimbingPsychologist Aha, thanks! I only thought about it briefly and thought the concept was sound, but I think you have some good points here.
It would be amazing to use data from the moonboard for this. I think what the moonboard has that is beneficial is that the holds are consistent so you could add descriptors to routes like crimpy, for a route that uses mostly yellow holds for example. It also has a consistent angle and a large user base and were daddy Ben Moon to give use the user data for all the user ratings, attempts and initial suggested grade it would be very interesting to play around with.
I think that is best application for this methodology, as it also eliminates people making multiple attempts on something they could have flashed if they didn't bail because they were scared.
I'm happy to see that data from Kaya is being used in this way! I can't comment on the outdoor stuff, but I climb at both of the indoor gyms you used in your data set, and the data reflects my observations pretty well. It's usually pretty easy to find the softest grades in the gym using Kaya because those tend to be the ones that have the most video uploads (unless it's a particularly flashy problem maybe). You can also cross check by looking at the profiles of people who sent the problem. But anyway, I was thinking that it could be interesting if Kaya could algorithmically suggest climbs to users based on how their sends/attempts match those from other climbers in the area. There's probably enough data on there (in our area at least) to get that working reasonably well!
Another random observation I've made, which you might actually be able to detect, is that grades at gyms seem to fluctuate between being soft and hard over time. About a year and a half ago, Sunnyvale used to have very hard grades, at least in the V0-V3 range. It was noticeable enough that, for a while, I could send about 50-60% of the V3s in nearby gyms, but only maybe 1 or 2 at Sunnyvale. Eventually that changed and Sunnyvale grades became easier than average for a brief period, and now it feels pretty on par with the rest, with occasional outliers in both directions.
Anyway, cool video!
Great ideas - Kaya totally has enough data to make it happen.
I've noticed difficulty fluctuations as well when climbing at different gyms! I always chalked it up to me getting more used a setting style. It would be a little difficult to identify that using data because there's so many confounding factors (climbers getting stronger / weaker for one), but it's a neat idea.
@@natoparkway Yeah, in isolation, you can't really use an individual's relative performance between time periods as an indication of grades tending softer/harder -- but I think what you can do is compare relative difficulty between gyms (for any pair of gyms where there are enough people who frequent both of them) at different time periods, kind of like you did in the analysis here, and see if that relative difficulty shifts. With an individual comparison between two gyms, you wouldn't be able to differentiate one gym getting harder from the other getting softer, but with enough comparisons like these between different gyms, I think you could possibly work that out! This is assuming there's enough good quality data from people who are active at multiple gyms anyway
@@zetorux I think you can detect fluctuations over time within a gym as well if you've got enough data points. If you exclude beginners a lot of climbers have a fairly predictable improvement-curve. E.g. one grade per year. I.e. the gradient of the line measuring their grade that they climb over time.
If you take the average gradient of the improvement-curves of all climbers in a gym in a given moment you'll see little fluctuations.
If the gym sets a batch of harder boulders it will look like lots of people synchronised and got slightly worse
But the difference between a soft batch of routes and a hard batch might only be ~0.25 grades so it might be hard to see in the data
Oh damn, surprised to see you here Zetorux. A pad player AND a climber, thats awesome
Indoor routes tell you where the holds are, which makes a huge different for people how don't climb outdoors so often. For many people who climbed mainly indoors, the hardest thing outdoors is to figure out where the holds are and finding the best ones, as there are usually betas which work but are much harder. This is less the cast indoors where holds are given. This might explain the shit in grades. People just climb a V3 with a beta that makes it a V4.
I have a similar hypothesis. Even after 10 years of climbing indoors and outdoors. When I stand on the ground at the bottom of a problem indoors I already see every hold, can guess how good or bad it is, and identify rests. Outdoors i can only see maybe half the holds from the ground, that already means i'm gonna maybe need a couple of attempts just to find out which holds are good, and then remembering where the holds are is going to take more concentration
I think one of the things that wasn't taken into account is age of problems. I've progressed relatively quickly, and distinctly remember inputting the dozen tries a V3 took me into Kaya, whereas I've put in half a dozen V7s this year. If you just looked at my data points, my personal progression would account for huge variation in points.
True, ideally the tournament would only compare logged ascents that occurred within a few months of each other. Easy to account for, but probably makes the dataset a lot smaller.
@@benja_mint if you had a second algorithm performing a linear regression on people's flash grade and then input each of their data points with the date into that algorithm you'd probably get decent data.
Very cool! Just a quick note, comparing climbs from a climber by attempts would not account for improvement over time. For example, when someone starts climbing, maybe they take 10 attempts for a v1. Then a few months later, they might take 3 attempts on a three, because they have gotten stronger. But according to the metric used here, the v3 would be ranked easier. Just some food for thought, and a cool vid!
This is a great point, particularly for lower grades where someone might improve quite quickly.
Hi! We tried to limit this by looking at a small time period (3 months) - but you're totally correct that this would affect the analysis. In general, we can of this analysis as largely accurate but with definite room for improvement.
I think most people will agree with “sandbagged” grades if given enough time climbing outside. Rock rewards technique and grace, which takes years of practice to develop.
I mean that just sounds to me like most climbs outdoors are hard and require more skill, which corresponds to either outdoors being sandbagged or indoors being soft graded. It's seems silly to insist that the grade is accurate while still assuming it takes more practice and skill.
@@johnmorrell3187 Its just different when you have 20 potential foot/hand placements outdoors to choose from instead of 1. It doesn't make the ultimate grade harder, it just introduces a new skill set of finding the best of those 20 foot placements to use. With experience you start to see which ones are good, which ones are bad, and on different types of rock too, because what is a good smear on sand stone is not a good smear on granite.
But routes are graded on the easiest beta, it's just harder to find that beta with more options to chose from, that doesn't make the easiest beta harder.
Eveb though it's true, the data strikes me as gym grading not necessarily being intentionally "kinder". It seems that gym grading is a different, more consistent and linear scale since setters have very clear benchmarks, and can set to any difficulty they want.
Outdoor V0-3s are much harder because there are fewer boulders, and generally get the grade because they *seem* relatively easier than others, so they get a lower grade. Rather than a V0 or V1 getting the grade for truly being easy, it's graded because it's less difficult.
Whereas routesetters can actually set truly easy V0s, and then set slightly more difficult V1s, V2s, V3s, and so on. Maintaining a fairly steady difficulty curve that just isn't possible outdoors where boulders are graded purely on who has been able to climb them.
This ends up with indoor grades being easier, but as the data shows, generally more consistent with what a climber should expect the difficulty to actually be relative to other indoor climbs.
My climbing partner and I (2 nerdy PhD) are watching this analysis seriously at 2 am 😅😅😅
About the last third of the video, all of this is why boulders aren't only judged by their grades, since they cover both physical, mental and technical aspects. Similar to alpine grades, typical guide books in Bleau show the a bit the split between the 3, by indicating the type of climb (usually slabs will require more tries, despite not being physically demanding, but requiring a technical level that most people do not have, especially in commercial gyms) and the commitment required (risk of injury when falling, do not fall zone, etc.)
love your videos
what I want to know is, does being stronger make it harder to grade easier problems? For example say you're a V9 climber, and you climb a V1. Would you know it is a V1 or would it be harder to know since you're so strong all the easy grades seem the same? So it's better to grade a V1 climb with a v3 climber than a v9?
I would say yes based on my own experience. I once spent about 2 years climbing strictly outdoors once and had progressed roughly 2 or 3 grades because I was focusing on sending a bunch of projects that were above my ability. When I accomplished my goal I eventually returned to the gym.
My previous max for about 12 years had always been a V5 pretty much anywhere I had climbed indoor or outdoor. While at the gym, I hopped on what I had thought was a V3. I flashed it pretty easily and it felt about V3 or maybe a soft V4. When I looked at the tag I realized I had misread it because it was a V5. It was a pretty cool feeling.
Since then I noticed my perception has changed.
I can't tell the difference between anything between a V1 and V3. Outside V1s are much more technical and have holds like crimps and slopers, so sometimes I get in a slabby V3 and think it feels like an outdoor V1. V2s and 3s might as well be the same thing.
I can't tell what a V4 is. It used to be ose to my max but now they feel so easy I have no idea.
The same thing for V2s and V3s can go for 5s, and 6s. Most the time I just don't know.
Unless the problem is at or near my max it feels difficult to know and as my max has increased that means there is a greater variety of climbs at different difficulty ranges that I am able to identify.
Shouldnt the points system you described be inverted, so that easier climbs get more "points"? then higher points corresponds to easier grades. This then takes into account that not that many people can climb higher grades, and it would have the lowest point score, corresponding to the hardest grade. New climbs would be the "hardest climb in the world" until someone sends it.
So one problem with using Kaya as a dataset is that there's a good chance that there are many climbers that don't properly list the # of attempts or the perceived difficulty of climbs they send. They may just want to check it off as sent and select the default options. You can see why this might be problematic especially if they have the tendency to report data more accurately for outdoor climbs because they subjectively feel more important to report on than indoor climbs.
There is also the issue in climbing with how difficulty is reported. Should a reported grade reflect the pure physicality of a boulder? The problem-solving elements? Should it reflect how accustomed the climber is to the type of hold texture (i.e. classic plastic holds, wooden hold, limestone, etc)? Should it reflect the average weather patterns in that area and the frequency with which a climber can be expected to climb in perfect conditions (i.e. indoors, a fairly ideal outdoor area, vs the windiest and wettest boulder year-round)? What about safety (i.e. how many mats you bring, highball boulders, etc)? Style?
I understand that addressing limitations is not very conducive to youtube videos but I feel that an equally interesting video can be made just in tackling these issues to determine the degree to which all of these other factors contribute to the perceived difficulty of outdoor boulders to indoor ones. Otherwise, I feel that this may be misleading to the average person who does not think to consider these limitations in research when viewing the results of your data.
I think it might be interesting to log the reasons for failing each attempt. (i.e. "excuses".) These could be things like pump, sequence, technique, reach, psych, pain, or conditions. Then it might be possible to categorize that "V1 psych" or "V3 pain" problem for what puts it outside the normal grading progression.
This is very cool the one thing I have to say is experience is such a big factor. Outdoor foot placement grabbing uncomfortable holds body positioning are much more crucial in outdoor climbing. You could take 2 people with equal strength and climbing experience in a gym but if you brought them outside if one has tons of outdoor experience you could see him climbing close to his indoor grade whereas the person with less experience might be 4v grades lower than the other climber.
Climbs can be graded higher for a variety of reasons and being able to identify the difficult aspect of each climb would help understand why certain climbs take more tries in general. If there was a question during the logging of the climb that asked people to describe the climb (Technical, Slab, Endurance, Strength, All-Round, etc.) and to rate difficulty of the climb from 1-10 in each of the following categories: Technique, Arm Strength, Core Strength, Grip Strength and a Personal Difficulty rating. It could help give a more accurate grade. For instance, if a V1 slab climb takes people 10 tries on average, but has a 1/10 in all strength factors, you could reduce the weight of each attempt in the system. Also, if a V8 endurance climb is rated highly in all strength factors, but was completed in 2 tries, you could increase the weight of the attempts. This would help normalize the system.
Hey! Thanks for the insightful analysis and great presentation. I’d like to offer some suggestions:
1. Feature Inclusion: Relying solely on the count of tries is a good starting point, but incorporating additional features like the number of previous climbs that day and climber-specific metrics could enrich your analysis and make your statistical model more robust.
2. Indoor vs. Outdoor Comparison: It’s tricky to compare these statistically, and I appreciate your caution. It’s crucial to consider distribution changes when comparing different climbing styles. For instance, climbers who specialize in dynamic indoor climbing often struggle outdoors, and vice versa. This isn’t just about technique differences but also about how we categorize and analyze these groups (outdoor, indoor, both, ect.). There’s lots of research on this topic that could help with your future analyses. Just search for domain adaption :).
Thanks again for your work, and I look forward to seeing how it evolves!
2:38 Sorry but this is *really* poorly explained. What does it mean that "Climb 3 earns more points from Climb 2, because it is 'defeated' in this tournament, and so it's going to earn two points form Climb 2 and one point from Climb 1"? After writing this quote down verbatim, it still makes zero sense to me.
I personally think that outdoor slabs are significantly harder then indoor slabs, but overhanging problems to mee feel harder indoors then outdoors.
We need a sliding scale for grades. I think in order not to be sexist, grades need to take height and ape index into account. Compressed climbs could be harder for tall climbers and reachy problems could be harder for the short. The grade you are awarded should take this in to consideration.
Dogpatch got a lot of Kaya activity because they have signs with Kaya QR codes on every wall to look up beta
Absolutely loved this video - thanks for the insights!
I think slabs should just be put into a separate grading system
Questions:
1. Do climbers have a bias towards not logging climbs they don't send? If so, how does that affect the result?
2. Do you use data of climbers who do not eventually get the send? If so, how do you incorporate it? If no, how does that bias the data?
3. By using a log scale for scoring, are you assuming that each grade is exponentially harder than the last?
4. What happens if you define difficulty this way: p = C_0 * (x%)^V, where p is the probability that a randomly chosen person could send a problem (an objective measure of difficulty), C_0 is the fraction of the population that can climb V0, x% is a model parameter that represents the fraction of the population that couldn't climb a grade higher (assumed constant), and V is the V grade. Could you infer V using p? Is this a poor model of real-world V grades?
1 + 2. We don't have much and don't use the data of climbers who attempted but failed to send a climb. I'm hopeful that we have enough data (10k sends) anyway such that it doesn't affect our findings significantly. If you take a look at the blog post you can read a lot of these caveats that we didn't focus on in the video.
3. That's not really what the log scale is for. The log scale says that a climb that took 10 attempts is not twice as hard as a climb that took 5 attempts - it's more like 30% harder.
4. I bet this would work pretty well for the core indoor grades (V3 - V7)! It's probably less of a good model with outdoor grades which have higher difficulty variance.
So indoor grades are very soft. Confirmed
My guy did a whole ass study to confirm that his gym is soft
One thing that I don’t think was mentioned here are the effects of conditions on outdoor climbs. For many polished boulders, the difference between trying in very good vs. very bad conditions can be multiple grades of difficulty, OR, many more attempts.
That is covered in the grade spread and the median.
Pls Learn the Dip bar Muscle up
very kewl and big brain
Cool video
Loved this... really interesting. Thank-you!
very interesting. my favorite part was the chart that showed an outdoor V0 COULD be up to a V6 in the gym. that feels about accurate... also, the idea that i could log climbs, look at the data and use it to determine weakness in my climbing is fascinating. more please. :)
There probably should be at least 3 grades for each boulder problem: Power, Endurance, Technique.This could solve the problem of grading overhang vs slab boulders.
Hi Nathaniel, is the data you use publicly available somewhere? That'd be a nice dataset to do some exploratory analysis in!
This is definitely a suggestion aimed at Kaya, but it'd be cool to add a feature like "tags" that you can associate to a problem, that'd make it way easier to do what you said at the end, with generating profiles for individual climbers.
Perhaps style needs to be taken into account as well as number of tries in a session.
So slab may take a lot of attempts to find the sequence but easy to repeat.
Step jug haul are physical so after a couple of tries your chances go down.
etc.
You can also take climb height and where the crux is at the bottom or top
The grade of a climb should be based on the easiest Beta! So when a problem is tricky or hard to flash it should not change the difficulty of the climb.
I think you should learn the v-sit next.
I love your videos and channel, thanks.
How much data science do I need to study, so I can become V5er from V4?
first
This is super interesting! I'd love to see a similar data analysis for my local gym/crag, but I doubt the data exists anywhere. I wonder if someday climbing apps will be so universal that we can start taking data for climbs all over the world!
Let us know the name of your local gym/crag! Nathaniel and I might be able to figure something out!
@@GeekClimber I climb at Boulderz Etobicoke (Near Toronto), and the closest crag is called Niagara glen. This is in Ontario, Canada! I actaully think some people in the area use an App called vertical life climbing to record information, but I doubt there's a ton of data on there. No idea about other apps!
Would a climber who climbs mostly outdoor find indoor problems to be harder? That would be another way to look at all this. I.e. you find it easier to do what you are used to.
I think so. You’re best at the thing you practice the most. So some outdoor specialists will have a harder time on comp style indoor boulders than gym climbers who climb not quite as hard outdoors.
The fear factor should not have any effect on the grade. That's just how it is outdoors. A v1 should never be graded higher because it is terrifying. And the simple reason why is that when there's a guy who's not that afraid of the heights as someone else, the route is easier. So in that case the grade is not about physicality or technique but rather about personal feelings.
For me indoor grades feel generally harder. It's a matter of global technique and proprioception.
And regarding grading system you can mostly compare similar styles. You will never be able to grade with a computer based on size of the grips, how distant they are and how steep. That's why climbing is so unique.
I'm a data analyst. It is a small dataset, yet still very cool! Great work
Grades are subjective. There are to many things to factor in. Gyms are absolutely not a +/- a grade. Maybe a handful of gyms are but the truth is most gyms grade way harder than the truth to incentivize climbers progress. Outdoor grades are all over the place and we’re graded 20 years ago. Plus the top climbers are creating new ways to grade a boulder just so they can say it’s a higher grade. Burden of dreams is a prime example with two hard moves v15 and v14 but yet they decide it’s a V17. With that theory I can climb a hard v8 with multiple hard moves and yet call it a v10…
That isnt a new way to grade boulders. Also, if a certain climber sends a v16 in a certain amount of attempts/ days and it takes them much longer on Burden of Dreams for example, it is clearly more difficult. Lots of v16 climbers have failed on Burden, clearly the grade however it was determined, is accurate.
@@GosseBoard not exactly how it works. Many climbers can climb certain boulders because they fit into a certain box for that climb. There are many people who can climb a v10 Crimpy that requires a small box and some that can climb it. The handful of people who can even attempt burden is just to small to get proper numbers on it. Just cause they can’t do that boulder but can climb another v15/16 doesn’t make it harder.
@@bradracine3426 However we do in fact have a large sample size for burden, from climbers who fit into different "boxes". Jimmy Webb, D-Woods, Hukkataival, Roberts, Raboutou, Bosi, Nakajima and others. All very different sizes, heights, "boxes" strengths and weaknesses. Its disingenuous to to say Burden has not had enough people try and express their opinion on it. I do agree with you that its difficult to generalize grades from different areas/ gyms. However, i travel full time for work so Im at different gyms all over Canada and the USA and I have to say that the majority of the gyms grade easier than outdoor grades. I agree, its probably to encourage people to stay and feel like they are progressing very quick.