Spent a while on this video! I love being able to tell stories of developers doing incredible things! Stay tuned, we are also building incredible things as well :)
@@Kitulous 293 likes after 2 weeks, wouldn't call this blowing up. Edit: since you're comment is 11 days old, and you commented that the original comment was written 8 days ago. That means the comment is now 19 days old, which is over 2 weeks and slows down the growth rate even more 🤔 Edit2: that means in the first 8 days this comment got an average of 8 likes per day. From day 8 to 19 this comment gained on average 20.818181..... likes per day. That means an increase of 2.6022727... times the likes the comment got in the first 8 days. This means it is definitely picking up in speed, at least the few points of data we have so far suggests this trend.
Crazy (in a good way) that Rockstar still payed 10k although it wasn't a Bug Bounty programm. I mean good for Toast but kinda unexpected from such a huge company tbh
@@Ghakimx Also, it's a PR move. Once someone shows you up for such a stupid problem, you need to get the goodwill of your users back. $10,000 is cheap for that. Not doing anything might have cost them millions.
@@rida_brahimyeah that's the point He put in the leg work for them, they take the results and (in the case of good, or at least smart companies) pay the programmer for their time Sure they didn't HAVE to pay him, but that'll look bad on them and could open the case for theft if you really wanted to push it
So problem number 2 occurred because the programmers forgot that a hashset can check for a string in constant time rather than using an array which will take linear time in worst case ! Somebody skipped DSA classes
Maybe the problem is that nobody bothered to bug fix the absurd industry laggard load times for the possible causes. GTA was always the worst and not even close
In an industry like gaming where cpu intensive coding is the norm. I highly doubt if this wasn't intentional. Introducing bugs and wait times for profit. Volkswagen has done it before for pollution control. And apple is doing it still.
@@harkitnebamake THANK YOU!!!! Finally someone who realizes companies may be in talks with Intel/Nvidia/AMD to make games more "heavy" on purpose so they can sell their new hardware. Finally someone that has a brain in his head! I've been saying this for YEARS!
I wonder if AI software testing companies will come out that automatically scan each commit to a code base (such as GTA's source code) to find inefficiencies lime this
specifically this problem is pretty simply to find if you look for it. It's honestly really surprising it even existed at all imo. at the time of writing, github co-pilot x already has this feature, and if you pasted in the source code in chatGPT it would probably point out this problem (as it's pretty good to find un optimized code in my experience)
There already are such things, albeit for CVE issues I believe. It'd be awesome if that would expand and not just mark issues like Sonar often does, but also fix them.
For the fix he did, he deserved 10 times more. 6-15 minutes times the number of concurrent players will be a huge number of people not being able to play a game for a significant amount of time. R* probably got a lot of business(shark cards) through that. He was really paid a pittance.
Turns out, this bug was the reason I never played GTA V Online. *Honestly tho, 10k is a pretty good bug bounty. Patches for serious security flaws in widely used and critical services often pay way less.
Some people say that a CS degree is useless in the workplace, but optimizing your program like this is exactly what it teaches you. This is a perfect example of turning algorithms with O(n²) and O(n) time complexity into O(n) and O(1).
Exceptionally well researched and well explained video Lewis🌟 Great editing. And thanks for inserting a progress bar for the sponsorship section 😂 that's new!
@@justyahz796 I don’t think you know how open source works. Rockstar would have the last say of what was merged into their source code. Does linux get vulnerabilities and back doors added everyday? Blender?
@@justyahz796 oh the AAA corporate game studio said so it must be the truth. There’s 3 reasons why companies keep their source closed. 1. They don’t want to spend the money fixing vulnerabilities in their code. 2. They don’t want you to know what data they’re mining. 3. They have back doors for governmental agencies. There’s plenty of massive open source projects that prove this answer wrong. And any vulnerabilities in open source code either intentional or by mistake are found way faster than any closed source projects.
@@emptydata-xf7ps yes im sure random youtuber with the letter E as a profile picture knows more than the triple A game studio that's had hundreds of projects spanning years
The unique check was probably useful in development to remove duplicate values. Since the json file was likely small initially the performance hit may not have been seen as a problem.
@@snowwsquire I read in another comment that the piece of code in question was a check for the uniqueness of JSON keys... If that's the case, whoever wrote that is dumb as a rock because JSON specifies that the represented object *cannot* have duplicate keys. This is because JSON objects are hash maps, so assigning a key twice replaces the value of the key with the latest value. Thus the check is not necessary at all. I have to check the blog post for verifying this statement. Edit: read the blog post. Yeah, the problem is very similar to that. You don't need to check for uniqueness if JSON already did that for you. And the context of the problem allowed for using a hash map if it was really such an issue. But overall the check was unnecessary because you're reading from a JSON anyway. The keys are always unique.
@@HappyGick If the code ran after the json was parsed, thats dumb. If it ran before somehow , it still makes sense; to throw up an error in CI if there was a duplicate value.
10,000??? From a multi billion dollar company?? Who knows how much extra revenue he must have provided rockstar with the much lower loading screens. Such a scummy move from them
From what I know of Rockstar, very few if anyone working there are on baller salaries. At Rockstar North's offices in Edinburgh you could walk 10 minutes up the road to a bank and get an immediate 20% payrise to write code there, with more growth potential and much less riding on it. Last I heard, Glassdoor had Rockstar programmers on an average of something like £52k, which really isn't a lot considering the literal billions GTA is making. Meanwhile in London finance you can literally add on an extra zero to that figure.
I just did the math on how much time Toast essentially saved millions of gamers, and it is absolutely insane. If you take the relatively faster load time of 6 minutes(before the patch). Then do some relatively simple math. That would mean that the average monthly player base of 2.5 Million players around the world spent a combined 28.5 YEARS JUST LOADING THE GAME. After the patch Toast cut that in HALF and then some, bringing the collective load times of the entire community to an EVEN 10 YEARS. That is absolutely INSANE. That is even more insane if you use a higher timed loading average such as the ten or fifteen minutes some players experienced. It is amazing how much one person can accomplish. Big thanks to toast, and a very well put together video by Coding with Lewis. If I messed up my math, please feel free to correct me. Although it is some pretty simple math, mistakes can still happen. Much Love, Peace Out.
I remember hearing about this fix back when I hadn't played GTAO for about three years. I logged back on again on the PS4 last November forgetting about the fix and was surprised how fast it loaded.
my pc back in 2017 used an HDD, and it took a little over half an hour to load GTA V, plus a little over 20 minutes to start up, and another 5 minutes to log in epic games
Honestly surprised that Rockstar didn't see 15 minute load times as a problem and they must've had one hell of a product for people to be willing to wait that long
This reminds me of a PowerShell script I was given to fix/optimise by a client. I'll obfuscate the details for confidentiality. In short, it needed to check if an array of objects existed in the cloud before either creating them or skipping them. The original code queried the cloud for a list to compare against for every object, which made the script crash as the query took a minute to run every time, and the array consisted of hundreds of objects. I instead queried the cloud once, storing the result in an array, and then iterated against that array which made the process a lot faster and more stable.
I remember finding tost's blog post ages ago when I was looking up "why does gta online take so long to load" and I found it through reddit, it slashed my 12 min waiting time to less than 2 mins on a SSD and about 4 mins on a HDD, mans a legend. Just goes to show how "lazy" rockstar can be and the fact that programmers and modders always have to fix games we love and not even be working at the company... a company with 4000 devs... how? how was 1 person able to fix the game but not like maybe 10 people at RG, it's mind boggling.
As a game developer i can tell you that most of the time we have objectives to achieve at the end of the day so many devs most probably chose an optimistic approach which isn't very complex in logic. Most of the time the easiest approaches are supported becuase of Crunch issues and also so next developer can understand it too. Once the first approach start working no one want to disturb it 😂....... This is also the reason why so many games are not so so optimized sadly ........
Good on rockstar for at least giving the guy some cash and being grateful that he made everyone’s experience of their game better. Had it been Microsoft they would have patched it and then sued the independent programmer who found the bug.
In 2021 I forwarded this fix to Rockstar via all the media I could find them present on. I think many off us did just like me. Hoping for them to implement the fix. I also hated the load time when my PC is 12 cores 24 thread's, 128Gb DDR4 Quad channel, NVME for games dedicated. I couldn't understand why it took so long too boot and I have a optical wired internet connection of 1Gb... Thanks for the video Im glad he got paid $10,000 for his work. I actually quited playing GTA 5 Online due too the load times.
So everything had a unique hash which could have been checked probably in O(1) time, yet they were checking individual plaintext characters over, and over, and over. It seems really rather disappointing that nobody at Rockstar questioned this during testing, nobody questioned the load times, and that such rotten code passed QA or peer review. Doubly so when it remained in the game and was prominently on show for so long. Loading screen for GTA Online? Who cares, nobody plays that game anyway. I once did a university project that had a similar issue and in the feedback the lecturer a) noted it and b) told me why it was actually happening in the first place.
I don't hold the QA testers responsible for the issues found in GTA5. Their role is to identify problems and report them, but ultimately, it's the responsibility of the developers and executives to determine whether or not to address them. It's highly likely that the QA testers flagged the extended loading times as an issue, but the decision-makers may have opted not to address it.
I remember whenever I wanted to play GTA:O I always made myself do something while it loaded. Like take a shower, do the dishes, vacuum the room etc. It's kind of nostalgic now that I think of it, it always took 10-15 minutes.
Great video, but I take a little bit of an issue with saying "each item in the array was unique, there was no purpose to them scanning the array to ensure that it was unique". As someone who has worked on semi-popular products (nothing anywhere near the same league as GTA Online), it's always been wise to take a _safe rather than sorry_ approach. If it's even conceivable that something _could_ happen - whether through a bug, an oversight or even a malicious actor - it's best to write something in which can handle that edge case appropriately, with the best approach being to seamlessly work around it. Using hashmaps would have been a great substitute, but removing the sequence entirely would not have.
Now also remember that Grand Theft Auto 5 is the best-selling entertainment product in history, eclipsing any medium. That is massive and it's hard to comprehend how much this guy contributed to the success of the game.
extra info bit. that json file is exclusive to the GTAO PC version. so it doesn´t exist on consoles (RDR2 has it too and also on consoles) it was implemented in GTAO as a way to make life harder for modders. bc the json file is the catalog that contains every single item that can be bought in the game. from 1$ candy bar, to a 10 million $ yacht. every single one, and they do that so modders can´t for example modify the tunables (the file R* uses for event weeks to adjust payouts, discounts and bonuses) and set the price for vehicles to 0 for example and buy them for free. the catalog has the default values and gets increased/decreased in there too for bonues/discounts. and if the tunable value is not identical to the catalog value, then the game will throw at you a "invalid price data" error and not let you complete the purchase for example
there's a lot of redunt operations in pretty much all modern software. The reason any given program is so much slower than it was 20 years ago isn't just "extra features" as many defensive programmers and marketing departments would claim.
2023 loading online (solo session on PC) takes about 25 seconds! Normal public lobby takes about 45 seconds to a minute! Loading story still takes FOREVER though...
another thing someone commented on a python meme video thing, was to never use loops if theres a better alternative, like, in this case, just doing the loop once as nothing more was needed or as you suggested, if you gotta check if some value is present, just hold it in a type of list that has a method, specifically for checking if a value already exists
The company awarding him money to find the bug is the correct thing to do, a lot of companies would deny and implement software to ban those who implement such a hot fix.
So, super interesting video. I was playing when this all came out, and though I very much appreciated the performance improvement I didn't know much about how. A small note on the video, I would avoid using the term "final solution" in any context. Just sort of elicits harsh feelings.
Kudos to Rockstar for compensating him for what he fixed in THEIR game. I've heard too many stories of game studios just taking a community patch and not crediting the original creator.
6:56 DRM is bad, however in this case that is related to code obfuscation, which is fairly standard in production applications to prevent reverse engineering
This example clearly shows that it is a bad practice and it's a shame this is standard. Video game industry definitely would gain a lot from developers open sourcing their code
@@sylvie_v2939 Well, this clearly isn't helping. Cheaters will always find the way, even when your anticheat runs at kernel level. And Genshin Impacts's case shows, why we shouldn't allow this. But, in this case the game is openly spyware so...
@@ayanami-rei-san cool story. Just offering engineering insight. Rockstar is here to make a profit, I'm not letting them off the hook here; merely a minor correction and a little Occam's razor
I did some code fixing/patching for NFS HPR a while ago using cheat engine lol. When you had more than 100 nfs hpr friends, you would be unable to save/quit a solo world mission. This happened because the game only allocated enough space for 100 Friendds. By having more than 100 the game would cause an overflow over the code running right below it. Before, and during solo mission this code was not being access but trying to hit ESC or finishing a mission the game would crash because of the friend list load when you start a mission. I looked at how the game was writting the strings and mimic-ed how it was ending after writing the 99 friend. Not a pretty patch but enough for the game to never crash after applying the patch. I also made other QoL patches like toggle-able ingame microphone.
It's nearly a shame that this happened as we might have gotten gta 6 by now, rockstar were making too much money from gta online to bother investing in a new game.
Really impressive!! But there’s no way they didn’t know that I think they just didn’t care since GTA is a money making machine regardless Cause those are pretty basic concepts Again props to the guy cause reverse engineering to that point is what takes real skills, incredible!! He deserved way more than 10k!
My guess (as a very novie programmer) is that they were using an old library that combined the strlen and scanf function that worked well for other games with less data. So it would actually help to be in assembly and see that strlen was being called too often, whereas programmers looking at the source code, especially not familiar with the library, would just assume that the library isn't the problem.
@@giphe that makes sense, cause if U imagine how much money they spent in the servers to millions of players, if they know about this issue, doesn't make any sense let it running...
he tries to load gta v, in a wii u, with a gamecube controller 💀💀 amazing video btw, i dont watch too much yt cuz i get bored instantly but this video hooked me up, new sub
Any person with the most basic experience in programming should've been able to spot this. It's obvious that whoever was responsible for that specific functionality either had no experience or didn't use their brain
Rockstar never really gave a sh** about fixing GTA V. The recent issues with GTA Online showed this. It's been awefull to play GTA Online on PC for years. Modders, hackers... and nothing was done against that, ever!
The fact that this guy immediately found the problem by doing a little bit of debugging while the original devs couldn't do it even though they have the source code and are the ones who made it, just shows how little they care.
This underscores a much more subtle problem in the CS community - schools have defocused fundamentals (like algorithms) and turned it into a "grab bag programming" mindset. Don't worry about the details of the function - just call it. Object oriented! I can't count the number of times I've seen things like qsort() being used on a sorted list to insert items, and when confronted, they have no idea why that's a bad thing.
the thing i always did to access gta online was to go into story mode then into a invite only session. it was way faster then pressing go or the online button at start up. it only took like 2 min 3 tops. i still do this since i can just invite my friends
Rockstar was too incompetent to fix a bug in their own game, caused by one simple function. This makes me think that they didn't even take time to go back and profile an obvious code bottleneck. It's sad that someone had to reverse engineer the game just to find and fix the issue for them.
"Most players were waiting between 3-15 minutes while some players had to wait up to 15 minutes" That was some enlightening information, I'm glad I found this channel that is source to such unknown information who knew the difference between 15 minutes and 15 minutes would be so much, hats off for the work you're doing man, not all will tell you such deep truths of life.
Honestly i surprised to see that Rockstar never noticed this. I started playing GTA 5 on PC when it came out, did a shitton of online over the years. Back then i used to game on a I7-860 CPU which handled GTA 5 fine and solid, but my god, the loading was abysmal. That CPU was getting a bit dated and single-core really tanked it hard. And still my experience was relatively fast compared to most shit i read on the forums. Still Kudos for actually rewarding the guy, Nintendo could take some notes.
How does this even happen? How could Rockstar possibly not find this after years of horrible load times? I've made maybe 10 programs in C ever, and even I know not to do this stuff.
Spent a while on this video! I love being able to tell stories of developers doing incredible things!
Stay tuned, we are also building incredible things as well :)
Talk about cyberpunk bugs
You did a great job with the video, it was incredibly insightful, thanks!
Ever played PS5 or just loading into invite only
my man just talk faster and stop trying to be soo gay
I don't know why, but your way of presentation pissed me off so much. Why are you talking like a valley girl?
lets have some claps for this guy. He is helping the small indie company called Rockstar Games fix the game
I know that $10,000 must have really hurt them
rockstar and indie company? Dude you must have been crazy.
@@overlordprincekhan do people have to write sarcasm when it's apparent
@@overlordprincekhan bro 💀
@@overlordprincekhan Someone has no idea what sarcasm is
Im surprised, they paid him $10k rather than suing him for $1m for reverse engineering their code.
How is it suable to reverse engineer a code running on my pc ?
@@tuncaybasak4953 terms of service, corporate greed, etc/
if i remember properly, it says its forbidden to reverse engineer the game
@@bbergeron8471yeah if u read the loading screen while entering the game u can find it is a crime
would be if he made money by doing it , he just fixed a problem , he didn´t exposed any code
The embodiment of "FINE, ILL DO IT MYSELF"
🤣🤣
Here before this comment blows up.
What if it were their dog who did it for them?
--Kermie
@@plashplash-fg6hd 8d ago, 64 likes, did it blow up yet?
@@Kitulous 293 likes after 2 weeks, wouldn't call this blowing up.
Edit: since you're comment is 11 days old, and you commented that the original comment was written 8 days ago. That means the comment is now 19 days old, which is over 2 weeks and slows down the growth rate even more 🤔
Edit2: that means in the first 8 days this comment got an average of 8 likes per day. From day 8 to 19 this comment gained on average 20.818181..... likes per day. That means an increase of 2.6022727... times the likes the comment got in the first 8 days. This means it is definitely picking up in speed, at least the few points of data we have so far suggests this trend.
Crazy (in a good way) that Rockstar still payed 10k although it wasn't a Bug Bounty programm. I mean good for Toast but kinda unexpected from such a huge company tbh
10k is chump change for Rockstar but quite the amount for an annoyed programmer. 💰
@@Ghakimx Also, it's a PR move. Once someone shows you up for such a stupid problem, you need to get the goodwill of your users back. $10,000 is cheap for that. Not doing anything might have cost them millions.
@@JamesRelokI didn't imply any obligation, instead, I mentioned it is a good business move.
@@JamesRelokdidn't they also use the same technique to make their game faster? i think i have seen this in the video
@@rida_brahimyeah that's the point
He put in the leg work for them, they take the results and (in the case of good, or at least smart companies) pay the programmer for their time
Sure they didn't HAVE to pay him, but that'll look bad on them and could open the case for theft if you really wanted to push it
So problem number 2 occurred because the programmers forgot that a hashset can check for a string in constant time rather than using an array which will take linear time in worst case ! Somebody skipped DSA classes
Maybe the problem is that nobody bothered to bug fix the absurd industry laggard load times for the possible causes. GTA was always the worst and not even close
Or maybe they don't care as the needy fanboys will still buy their game?
In an industry like gaming where cpu intensive coding is the norm. I highly doubt if this wasn't intentional. Introducing bugs and wait times for profit. Volkswagen has done it before for pollution control. And apple is doing it still.
@@harkitnebamake THANK YOU!!!! Finally someone who realizes companies may be in talks with Intel/Nvidia/AMD to make games more "heavy" on purpose so they can sell their new hardware. Finally someone that has a brain in his head! I've been saying this for YEARS!
Nah, it really was because they forgot that JSON objects cannot have duplicate keys. ... or maybe they thought their data was buggy and not true JSON?
I wonder if AI software testing companies will come out that automatically scan each commit to a code base (such as GTA's source code) to find inefficiencies lime this
There's already programs for this that don't use AI. Likely that Rockstar devs are just underpaid and don't care.
specifically this problem is pretty simply to find if you look for it. It's honestly really surprising it even existed at all imo.
at the time of writing, github co-pilot x already has this feature, and if you pasted in the source code in chatGPT it would probably point out this problem (as it's pretty good to find un optimized code in my experience)
🍋
There already are such things, albeit for CVE issues I believe. It'd be awesome if that would expand and not just mark issues like Sonar often does, but also fix them.
@@nordgaren2358 modern gaming in a nut shell.
For the fix he did, he deserved 10 times more. 6-15 minutes times the number of concurrent players will be a huge number of people not being able to play a game for a significant amount of time. R* probably got a lot of business(shark cards) through that. He was really paid a pittance.
Turns out, this bug was the reason I never played GTA V Online.
*Honestly tho, 10k is a pretty good bug bounty. Patches for serious security flaws in widely used and critical services often pay way less.
same. shit took forever to load, so I only loaded it up like 2 times. thats it.@@sporqist
not really
Some people say that a CS degree is useless in the workplace, but optimizing your program like this is exactly what it teaches you. This is a perfect example of turning algorithms with O(n²) and O(n) time complexity into O(n) and O(1).
Exceptionally well researched and well explained video Lewis🌟
Great editing.
And thanks for inserting a progress bar for the sponsorship section 😂 that's new!
“What can we learn”: we learn that if the source code was open source this bug would’ve been found within a few months of release.
and also the game wouldn’t work since there would be open ended vulnerabilities and back doors being created daily
@@justyahz796 I don’t think you know how open source works. Rockstar would have the last say of what was merged into their source code. Does linux get vulnerabilities and back doors added everyday? Blender?
@@emptydata-xf7ps this is literally the reason stated as in why gta 4s online hadn’t had open sourced net code
@@justyahz796 oh the AAA corporate game studio said so it must be the truth. There’s 3 reasons why companies keep their source closed. 1. They don’t want to spend the money fixing vulnerabilities in their code. 2. They don’t want you to know what data they’re mining. 3. They have back doors for governmental agencies. There’s plenty of massive open source projects that prove this answer wrong. And any vulnerabilities in open source code either intentional or by mistake are found way faster than any closed source projects.
@@emptydata-xf7ps yes im sure random youtuber with the letter E as a profile picture knows more than the triple A game studio that's had hundreds of projects spanning years
He deserved more than 10k for sorting that mess out tbh
The unique check was probably useful in development to remove duplicate values. Since the json file was likely small initially the performance hit may not have been seen as a problem.
But the values were apparently unique by definition...
@@Mernomprobably not in development, when adding items into the game it makes sense that there could be duplicate items by accident
@@snowwsquire I read in another comment that the piece of code in question was a check for the uniqueness of JSON keys... If that's the case, whoever wrote that is dumb as a rock because JSON specifies that the represented object *cannot* have duplicate keys. This is because JSON objects are hash maps, so assigning a key twice replaces the value of the key with the latest value. Thus the check is not necessary at all.
I have to check the blog post for verifying this statement.
Edit: read the blog post. Yeah, the problem is very similar to that. You don't need to check for uniqueness if JSON already did that for you. And the context of the problem allowed for using a hash map if it was really such an issue. But overall the check was unnecessary because you're reading from a JSON anyway. The keys are always unique.
@@HappyGick If the code ran after the json was parsed, thats dumb. If it ran before somehow , it still makes sense; to throw up an error in CI if there was a duplicate value.
10,000??? From a multi billion dollar company?? Who knows how much extra revenue he must have provided rockstar with the much lower loading screens. Such a scummy move from them
From what I know of Rockstar, very few if anyone working there are on baller salaries. At Rockstar North's offices in Edinburgh you could walk 10 minutes up the road to a bank and get an immediate 20% payrise to write code there, with more growth potential and much less riding on it.
Last I heard, Glassdoor had Rockstar programmers on an average of something like £52k, which really isn't a lot considering the literal billions GTA is making. Meanwhile in London finance you can literally add on an extra zero to that figure.
Rockstar could even sue toast to reverse engineer the source code is illegal.
@@GalileoTaps7858 what, sue them for fixing their terrible code? That sounds like a public death sentence for their reputation
@@MirrorHall_Clay ban someone for adding files to the game files
@@MirrorHall_Clay exactly, that why he didnt got sued...
I wish there was a rough estimation of how much time he collectively saved for everyone.
1.0445 million hours, approximately.
or 119 years. @@maleck25
70% of the old load times, for all pc players (a fraction of which you can see with steam concurrent players), since rockstar added it
I just did the math on how much time Toast essentially saved millions of gamers, and it is absolutely insane. If you take the relatively faster load time of 6 minutes(before the patch). Then do some relatively simple math. That would mean that the average monthly player base of 2.5 Million players around the world spent a combined 28.5 YEARS JUST LOADING THE GAME. After the patch Toast cut that in HALF and then some, bringing the collective load times of the entire community to an EVEN 10 YEARS. That is absolutely INSANE. That is even more insane if you use a higher timed loading average such as the ten or fifteen minutes some players experienced. It is amazing how much one person can accomplish. Big thanks to toast, and a very well put together video by Coding with Lewis. If I messed up my math, please feel free to correct me. Although it is some pretty simple math, mistakes can still happen. Much Love, Peace Out.
you cant know how many people loaded the game what number of times prior to the patch if youre not rockstar themselves
I remember hearing about this fix back when I hadn't played GTAO for about three years. I logged back on again on the PS4 last November forgetting about the fix and was surprised how fast it loaded.
And seeing how a kind programmer, is helping to make us understand the work of toast teaches me of community.
my pc back in 2017 used an HDD, and it took a little over half an hour to load GTA V, plus a little over 20 minutes to start up, and another 5 minutes to log in epic games
Honestly surprised that Rockstar didn't see 15 minute load times as a problem and they must've had one hell of a product for people to be willing to wait that long
This reminds me of a PowerShell script I was given to fix/optimise by a client. I'll obfuscate the details for confidentiality.
In short, it needed to check if an array of objects existed in the cloud before either creating them or skipping them. The original code queried the cloud for a list to compare against for every object, which made the script crash as the query took a minute to run every time, and the array consisted of hundreds of objects.
I instead queried the cloud once, storing the result in an array, and then iterated against that array which made the process a lot faster and more stable.
Awesome video, good explanation! Toast is an legend for this
Add T0ST to the MCU, he's a superhero
On my xbox 360, I would go make an entire meal knowing that GTA Online still wouldn't be loaded by the time I got back
I remember finding tost's blog post ages ago when I was looking up "why does gta online take so long to load" and I found it through reddit, it slashed my 12 min waiting time to less than 2 mins on a SSD and about 4 mins on a HDD, mans a legend. Just goes to show how "lazy" rockstar can be and the fact that programmers and modders always have to fix games we love and not even be working at the company... a company with 4000 devs... how? how was 1 person able to fix the game but not like maybe 10 people at RG, it's mind boggling.
Rockstar actually implementing a fix? And a fix made by the community at that?
Wow..... I actually have trouble believing that.
As a game developer i can tell you that most of the time we have objectives to achieve at the end of the day so many devs most probably chose an optimistic approach which isn't very complex in logic. Most of the time the easiest approaches are supported becuase of Crunch issues and also so next developer can understand it too. Once the first approach start working no one want to disturb it 😂....... This is also the reason why so many games are not so so optimized sadly ........
Reverse Engineering needs to become more popular, we're gonna need it.
Good on rockstar for at least giving the guy some cash and being grateful that he made everyone’s experience of their game better.
Had it been Microsoft they would have patched it and then sued the independent programmer who found the bug.
In 2021 I forwarded this fix to Rockstar via all the media I could find them present on. I think many off us did just like me. Hoping for them to implement the fix. I also hated the load time when my PC is 12 cores 24 thread's, 128Gb DDR4 Quad channel, NVME for games dedicated. I couldn't understand why it took so long too boot and I have a optical wired internet connection of 1Gb...
Thanks for the video Im glad he got paid $10,000 for his work. I actually quited playing GTA 5 Online due too the load times.
Random Programmer? A fucking legend is the most appropiate way to speak about him.
If he so smart why has no social links
So everything had a unique hash which could have been checked probably in O(1) time, yet they were checking individual plaintext characters over, and over, and over.
It seems really rather disappointing that nobody at Rockstar questioned this during testing, nobody questioned the load times, and that such rotten code passed QA or peer review. Doubly so when it remained in the game and was prominently on show for so long. Loading screen for GTA Online? Who cares, nobody plays that game anyway.
I once did a university project that had a similar issue and in the feedback the lecturer a) noted it and b) told me why it was actually happening in the first place.
I don't hold the QA testers responsible for the issues found in GTA5. Their role is to identify problems and report them, but ultimately, it's the responsibility of the developers and executives to determine whether or not to address them. It's highly likely that the QA testers flagged the extended loading times as an issue, but the decision-makers may have opted not to address it.
0:39
"Gamers always finding ways to break things"
*shows gamer breaking keyboard* lmao
Just goes to show Rockstar didn’t even care until someone did the work for them.
I remember whenever I wanted to play GTA:O I always made myself do something while it loaded. Like take a shower, do the dishes, vacuum the room etc. It's kind of nostalgic now that I think of it, it always took 10-15 minutes.
"three to fifteen minutes, while some people had to wait fifteen minutes" ???
This sentence does not make any sense.
Great video, but I take a little bit of an issue with saying "each item in the array was unique, there was no purpose to them scanning the array to ensure that it was unique". As someone who has worked on semi-popular products (nothing anywhere near the same league as GTA Online), it's always been wise to take a _safe rather than sorry_ approach. If it's even conceivable that something _could_ happen - whether through a bug, an oversight or even a malicious actor - it's best to write something in which can handle that edge case appropriately, with the best approach being to seamlessly work around it. Using hashmaps would have been a great substitute, but removing the sequence entirely would not have.
Now also remember that Grand Theft Auto 5 is the best-selling entertainment product in history, eclipsing any medium.
That is massive and it's hard to comprehend how much this guy contributed to the success of the game.
Seven years later he came to save the franchise from bankrupt 😂
extra info bit.
that json file is exclusive to the GTAO PC version.
so it doesn´t exist on consoles (RDR2 has it too and also on consoles)
it was implemented in GTAO as a way to make life harder for modders.
bc the json file is the catalog that contains every single item that can be bought in the game.
from 1$ candy bar, to a 10 million $ yacht.
every single one, and they do that so modders can´t for example modify the tunables (the file R* uses for event weeks to adjust payouts, discounts and bonuses)
and set the price for vehicles to 0 for example and buy them for free.
the catalog has the default values and gets increased/decreased in there too for bonues/discounts.
and if the tunable value is not identical to the catalog value, then the game will throw at you a "invalid price data" error and not let you complete the purchase for example
there's a lot of redunt operations in pretty much all modern software. The reason any given program is so much slower than it was 20 years ago isn't just "extra features" as many defensive programmers and marketing departments would claim.
2023 loading online (solo session on PC) takes about 25 seconds! Normal public lobby takes about 45 seconds to a minute! Loading story still takes FOREVER though...
all my homies got their degrees in 6 minutes or less
I love this example of random player making miracle, because I encountered some people that assume that big companies always do the best job.
another thing someone commented on a python meme video thing, was to never use loops if theres a better alternative, like, in this case, just doing the loop once as nothing more was needed
or as you suggested, if you gotta check if some value is present, just hold it in a type of list that has a method, specifically for checking if a value already exists
At first Rockstar denied the issue. Till it became famous and people started to complain to them.
8:25 where O(N²) really took a toll😭😭
The company awarding him money to find the bug is the correct thing to do, a lot of companies would deny and implement software to ban those who implement such a hot fix.
It’s a shame that I can’t eat lunch during my loading screens anymore though..
Knowing Rockstar I thought they will give the money to Tost in GTA online
bro I have an 11 year old cpu that can’t do anything yet I still get into gta in like 2 minutes… definitely not hardware.
Timmy had to improve his city, he went out himself.
So, super interesting video. I was playing when this all came out, and though I very much appreciated the performance improvement I didn't know much about how.
A small note on the video, I would avoid using the term "final solution" in any context. Just sort of elicits harsh feelings.
agreed until that last sentence. don't whine about something that makes perfect sense in the context.
bro decided to fix a close code 100GB game
Kudos to Rockstar for compensating him for what he fixed in THEIR game. I've heard too many stories of game studios just taking a community patch and not crediting the original creator.
6:56 DRM is bad, however in this case that is related to code obfuscation, which is fairly standard in production applications to prevent reverse engineering
This example clearly shows that it is a bad practice and it's a shame this is standard. Video game industry definitely would gain a lot from developers open sourcing their code
@@sylvie_v2939 Well, this clearly isn't helping. Cheaters will always find the way, even when your anticheat runs at kernel level. And Genshin Impacts's case shows, why we shouldn't allow this. But, in this case the game is openly spyware so...
@@ayanami-rei-san cool story. Just offering engineering insight. Rockstar is here to make a profit, I'm not letting them off the hook here; merely a minor correction and a little Occam's razor
On my low end laptop it took me about 30 to 1 h for online to load, good that they fixed it
I did some code fixing/patching for NFS HPR a while ago using cheat engine lol.
When you had more than 100 nfs hpr friends, you would be unable to save/quit a solo world mission. This happened because the game only allocated enough space for 100 Friendds. By having more than 100 the game would cause an overflow over the code running right below it. Before, and during solo mission this code was not being access but trying to hit ESC or finishing a mission the game would crash because of the friend list load when you start a mission.
I looked at how the game was writting the strings and mimic-ed how it was ending after writing the 99 friend. Not a pretty patch but enough for the game to never crash after applying the patch. I also made other QoL patches like toggle-able ingame microphone.
It's nearly a shame that this happened as we might have gotten gta 6 by now, rockstar were making too much money from gta online to bother investing in a new game.
I kinda wonder how much power has been consumed worldwide purely because of this one bug
Loving these reverse engineering videos! Please dont stop
Seriously, GTA 5 didn't use HashMap's 🤣
Really impressive!!
But there’s no way they didn’t know that
I think they just didn’t care since GTA is a money making machine regardless
Cause those are pretty basic concepts
Again props to the guy cause reverse engineering to that point is what takes real skills, incredible!!
He deserved way more than 10k!
I agree. This was simply genius. He deserved at least a $100k for this feat
My guess (as a very novie programmer) is that they were using an old library that combined the strlen and scanf function that worked well for other games with less data.
So it would actually help to be in assembly and see that strlen was being called too often, whereas programmers looking at the source code, especially not familiar with the library, would just assume that the library isn't the problem.
@@giphe that makes sense, cause if U imagine how much money they spent in the servers to millions of players, if they know about this issue, doesn't make any sense let it running...
he tries to load gta v, in a wii u, with a gamecube controller 💀💀
amazing video btw, i dont watch too much yt cuz i get bored instantly but this video hooked me up, new sub
I remember i thought the problem was my computer spec that made the loading so slow but now i know that everyone also has this problem.
Average gamer is broke. So yes it was your pc.
NGL, i thought you were gonna say "toast was sued for finding this bug"
With all the recent great content i keep getting recommended from this channel, you just earned a new sub 😊
waiting a full decade for gta 6 was nothing to stop grand theft auto's growth
Any person with the most basic experience in programming should've been able to spot this. It's obvious that whoever was responsible for that specific functionality either had no experience or didn't use their brain
Respect to these guys who know all the tech stuff.
its a diffrent world for me
And respect to you, who probably has better eyesight than us
when the website is HTML only you know the programming tool is top-notch
1:15 "i would start cleaning my house" yup i used to do this LMAO
Technically he took his own hands into the matter.
Rather than taking matter into their own hands.
Hey man great video don’t listen to any of those negative comments.As a nonprogrammer and just a gta fan I found this video very interesting.
Rockstar never really gave a sh** about fixing GTA V. The recent issues with GTA Online showed this. It's been awefull to play GTA Online on PC for years. Modders, hackers... and nothing was done against that, ever!
Wow I did not expect Rockstar would compensate Toast for finding the bug, huge W to everyone.
Toast should have been award 10% of the company shares.
The fact that this guy immediately found the problem by doing a little bit of debugging while the original devs couldn't do it even though they have the source code and are the ones who made it, just shows how little they care.
Not that they couldn’t, it’s that they WOULDNT
"Now he needs to work on implementing the final solution" 💀💀💀
Standing ovation for that guy, absolute chad!
You would think a triple A studio can hire someone with basic Data structures and algorithms experience
me in 2023 with a nasa pc: loads online in 2 seconds
thats how peoples get cancaled 10:07
$10.000? what a rats
better than nothing.
I think Tost has been hired at rockstar now.
This underscores a much more subtle problem in the CS community - schools have defocused fundamentals (like algorithms) and turned it into a "grab bag programming" mindset. Don't worry about the details of the function - just call it. Object oriented! I can't count the number of times I've seen things like qsort() being used on a sorted list to insert items, and when confronted, they have no idea why that's a bad thing.
Good developers are often great investigators.
Remember that in FiveM it takes basically 20 seconds from the menu to spawning in game
I don't understand any of this but it sounds cool. Good on the guy for finding and fixing this issue then getting paid.
the thing i always did to access gta online was to go into story mode then into a invite only session. it was way faster then pressing go or the online button at start up. it only took like 2 min 3 tops. i still do this since i can just invite my friends
fun fact: the first computer bug was a moth trapped in the computer
A moth doesnt fit in an ipad 🤡
@@mike.1 there weren't ipads in the 1980's
That file was found out to be the store page. That show how much more they care about you spending money than have a good performing game.
people dont mind spending their whole day on tiktok and waste HOURS of their day and they mind a 15 minutes loading and say it's wasting my time
Mostly because the expectation is that loading shouldn’t take more than 30-45 seconds at most.
Imagine how much time, both CPU and people, this bug has wasted.
Now imagine, some of your software also has this and wasting your time.
Your average screen time is 8 hours a day watching tiktoks. What u talking about
Rockstar was too incompetent to fix a bug in their own game, caused by one simple function. This makes me think that they didn't even take time to go back and profile an obvious code bottleneck. It's sad that someone had to reverse engineer the game just to find and fix the issue for them.
When I hear about such mistake I start to feel more confident.
And people say leetcode questions are not useful... A novice leetcoder would have implemented better loading then the engineer at Rockstar :D
"Most players were waiting between 3-15 minutes while some players had to wait up to 15 minutes" That was some enlightening information, I'm glad I found this channel that is source to such unknown information who knew the difference between 15 minutes and 15 minutes would be so much, hats off for the work you're doing man, not all will tell you such deep truths of life.
chill bro lmao
Honestly i surprised to see that Rockstar never noticed this.
I started playing GTA 5 on PC when it came out, did a shitton of online over the years.
Back then i used to game on a I7-860 CPU which handled GTA 5 fine and solid, but my god, the loading was abysmal.
That CPU was getting a bit dated and single-core really tanked it hard.
And still my experience was relatively fast compared to most shit i read on the forums.
Still Kudos for actually rewarding the guy, Nintendo could take some notes.
$10,000 is nothing for a company like rockstar, and a solution like that probably saved rockstar hundreds of thousands of dollars worth of revenue.
10k is peanuts compared to the size of the game. they should give him another 90k :)
How does this even happen? How could Rockstar possibly not find this after years of horrible load times? I've made maybe 10 programs in C ever, and even I know not to do this stuff.