small update: how the 100 player server ""never"" ""crashes"" - a plugin
HTML-код
- Опубликовано: 11 июл 2024
- SERVER INFO:
search for the "shounic" tag in the server browser, server name is "shounic trenches"
or: add | 45.62.160.71:27015 | to your server browser favorites
but you really should join the discord specifically for the server, besides having a 1 click link to join the server, / discord -- important updates, info, voting & report a player :)
channel discord --- / discord
channel patreon --- / shounic
channel tracker --- trello.com/b/L3B65jUX
[ todo list / current progress on upcoming videos / channel roadmap ]
feedback --- forms.gle/bkuGF6attQrRPc6o9
custom files --- pastebin.com/raw/e32aG4nP
[ hud / crosshairs / hitsound / killsound / gfx cfg ]
Music: BananaSlug ( / user-426347780 )
big thanks to:
nosoop
FAQ:
Q: How did you make that? What editing software do you use?
A: I use Adobe Photoshop, Premiere Pro & After Effects. Photoshop for image editing and thumbnails. Premiere Pro for compositing and After Effects for motion graphics (the fancy animations). 
Игры
![Why [TF2] has (Almost) Perfect Melee Weapons](/img/1.gif)
SERVER INFO in video description, gamers
more importantly, allow me to cope: i really like it when i make a video saying the server won't crash, then valve releases a buggy halloween update hours after that, and now i'm a stinking fraud and liar. (sorry for people who experienced a crash in the first few days after the video on the 9th! but it's not even my fault & we're good now though :))))
also hope this video doesn't feel awkward? i'm not sure how interesting this is but people i've spoken to are like "oo wow that's cool". also making another server update video so soon after the last, but at the same time it seems better to do it now than later when everyone forgets about it 🤔
i always like that more technical video format :), maybe live fixing issues?, with a bunch of people commenting probably you can get feedback faster
"oo wow that's cool" was my exact reaction when watching this video
Don't worry about talking too much about the server, it's genuinely a really cool and impressive project
The only issue would be if a person finds a method to deliberately cause crashes, assuming their Internet allows them to reconnect, they could hold the entire server hostage if the method was reliable.
This is super interesting!
It's a neat solution to the problem.
"Server crashes are bad. But what if you can make the server never crash?" The question every IT expert is asking to themselves.
and water is wet
@I'm wet.
Just make it not crash! Genius! I don't know how I didn't think of this before.
thats genius. just-not-crash it!
Easy, restart the server when it looks like it's going to crash. It's not a crash
Reminds me of one of the developers for Sonic 3D Blast. They needed the QA testing to go well for the holidays, so he wrote a crash handler that warps directly to a secret level select, just to make it look like an intentional secret. The side effect is that you can whack the cartridge while playing and get this screen.
that's hilarious
Workarounds in programming will never be not funny or interesting
3d blast was an interesting game. it had a banger soundtrack too
Reminds me of the wing commander error message. Direct quote from dev:
"Back on Wing Commander 1 we were getting an exception from our EMM386 memory manager when we exited the game. We'd clear the screen and a single line would print out, something like 'EMM386 Memory manager error. Blah blah blah.' We had to ship ASAP. So I hex edited the error in the memory manager itself to read 'Thank you for playing Wing Commander.'"
Classic example
Reminds me of how Morrowind on the original Xbox(2001) would often run out of memory during loading times and the devs just stealthily made it save, reboot the whole console to purge ram & automatically load the latest save once the xbox's finished booting
Wild that the Xbox let you do that, why deal with memory leaks if we can just remove all the memory. Also Deus Ex Infinite War does the same
u not old enough for morrowind silly
@@BottomOfTheDumpsterFire tbf any executable that u manage to be able to load on the og xbox will always have full permissions and access to the entire machine
Dang man. Seems like I missed out. The OG Xbox sounds better than the later variations.
"Modern problems require modern solutions"
This is like if every so often, everyone leaves the bus theyre on and then a new bus comes by and picks everyone up and then the old bus drives off because its about to explode
You just described memory management in a very hilarious way
This is just that one scene in deltarune where the car explodes for no reason
What an unusual example 😂
Hey don't worry about that memory leak thing. You'd never get a 24 hour period of no crashes because people vote for ZI once every hour anyways.
I'm so tired of 100 player ZI, I have seen the humans win only ONCE in weeks of playing on this server. Might be bad luck but...
@@joedingo7022you just need more gun
But what if they'rr already in ZI when hour hits?
What's ZI?
@@Mandarin9900 VScript zombie infection
"seamless crashing" is such a good description of a feature
oh hey i recognize you from fediverse
wild seeing you here
@@EeveeEuphoriasame to both of you
@@bb010g same here too
oh ur one of the rd devs right? nice
It's called programming to fail gracefully. A crash is an exception which is basically the program not being sure what to do with an instruction. Exceptions exist to protect your computer more than being a difficult barrier to program around. Generally hiding crashes aka error throwing with generic handlers is a bad practice. Handling it gracefully so the user doesn't panic is the ideal, and you show a friendly error to the user instead of a stack trace dump.
"theres always future updates" bold words
I am hereby predicting tf2 to get its last update in 2025, when tf3 drops and suddenly world peace is restored and all hunger is solved
Shovelware level updates, sure, but updates
@@WangleLine What the hell would Team Fortress 3 even look like???
@@WangleLineRemember valve never makes anything with the certain number between 2 and 4
@@stephenwoolverton2139 Team Fortress Forever, then ;p
the retry command doesnt actually require you to be in a server, it just connects you to the last server you were in, so if you leave a server to the main menu, if you use retry you will connect back
additional note: it only "remembers" the last server you were in since the game has been opened, if you close the game that history is wiped and retry will not do anything until you connect to a server
no wonder people bypass getting kicked from mvm so easily holy shit
@@jetsflyingoffatrain4338 While I'm unsure of how it works with Valve's mvm servers, I know this definitely doesn't work for Valve's casual servers.
@@jetsflyingoffatrain4338 Pretty sure retry as a bypass for kicking was fixed no?
"ackshully...." & posts something irrelevant
@@zexc not for mvm lmao
I have to thank you shounic for making this server in the first place, it's been the most fun I've had in TF2 in a long time.
Is the server located in NA?
"Hello, IT. Have you tried turning it off and on again?"
It's interesting how common this kind of deal seems to be. I know that a good few Minecraft servers will have a proxy to boot you to a waiting room server if the main one crashes for example.
Minecraft servers also tend to have multiple servers running at once and the player just gets silently switched to another server when one crashes or they choose a specific server. it's essentially just giving data from server a to the player instead of data from server b along with reloading stuff so I wonder if that would also be possible for tf2
It’ll be functional until the server agrees to go all Heavy and fire their Miniguns all at the same time cause your PC to explode
then add a class limit or just force others heavies to switch classes when theres too many heavies (the point where crash can happen)
No one this server can agree to do anything
Nah, that’s soft.
All demos with Scottish resistance however is a guaranteed crash
It only takes 1 player playing soviet march for whole team to go heavy
That'd cause client side issues, anyway.
I like how shounic has evolved from noting all of TF2's issues.. To just up and fixing them himself and creating a small but growing utopia built on actual competence and consistent care.
The Gaben quotes at the end usually don't make me chuckle... until today.
I love your editing and very visual and easy way to explain boring and complex topics.
Keep up the good work man 👍
Years ago I wrote a plugin to restart the server when player count was low enough, while also giving a clear warning that it was going to.
Had a bunch of settings for how many players to stop a restart, when it would happen anyway and such things.
thanks for letting us know
I love how your performance report only has 12 minutes of scout gameplay and nothing else.
couldn't you use this to hide a "secret restart" inside, for example, one of every ten map changes? allows you to preempt memory leaks in exchange for a slightly longer loading time for one of the maps
Sonic did this
@@Voryn13 are you refering to the hedgehog or the guy who made the video
From power up t: @@MadContendery
@PowerUpT
Reminds me of one of the developers for Sonic 3D Blast. They needed the QA testing to go well for the holidays, so he wrote a crash handler that warps directly to a secret level select, just to make it look like an intentional secret. The side effect is that you can whack the cartridge while playing and get this screen
@@Voryn13nah, that was a crash handler like how this video worked, not anything preemptive.
Yeah… I know. I was saying that that game also had a secret restart@@MudakTheMultiplier
I truly love all of your videos. There is something insanely cool in all of them. Thank you for your work.
every time you talk about cool plugins like this it disappoints me that they're not released in some way :(
unless i just don't know where to look for them, this just seems cool to have for community servers in general
this anticrash plugin alone could be really useful on every source game that supports the ability of sending retry on a client from the server.
The 100 player server is some of the most fun I've had in a long time. Thank you for hosting it!
so your telling me you just told it "if crash: don't" and it actually worked
I love your editing style. It's so unique and fresh.
I really appreciate your 100 player server! Really fun!
i thank god every upload people are still making content for this game 😊 bless you
Love your videos, man.
Wow... That is actually very good plugin! Hope 100 players servers support others things like Class Wars, Randomizer or maybe even RTD and more.
Edit:Ok maybe RTD is not a good idea, first the cpw mangler shots RTD, that may reach the limit objects and crash the server
that sounds completely garbage. you have bad taste
Not sure about rtd. If like 10 rolled that one where a ton of cow mangler shots are shot out, I'm not sure the server would be able to handle it
@@ruileite4579 that and the fact that RTD's a garbage mechanic that only exists to satiate dopamine for losers
you can customise the plugin do it wouldn't be bad
@@k90v85 it'll always be bad. "Random buffs with zero way to predict them that can be done instantly at a moment's notice" is just inherently a bad game mechanic.
I feel like I've seen something like this in gmod servers in the past. It would make sense too, given it's the same exact engine
EDIT: also the fact a lot of gmod servers are a 24/7 of one map w/ no reloads, and have much higher player limits (I remember as high as 128 for GMT back in the day)
Is it possible to just sneak in a restart during map changes anyways to deal with the memory issues?
Map changes are restarts. This is why even 24/7 map servers will occasionally restart.
I've been on servers trying to switch maps take longer and give up after 4 retries (even official valve servers) than the time it takes for your 100 player server to restart. Wonder if other servers are just oddly slow or have a bunch more pre-recs than what you got. (could also be a me issue, but I've manual retried servers at times and they're usually dead as a majority of players assumed the server crashed on map change.)
I know nothing about video editing but in my layman's view yours is exceptional
"hey guys today we're going to solve the halting problem in tf2"
Great video! Did you manage to do the same thing for the Linux client? Or does proton take care of it automatically?
A tangent to this video, but the power of the console and what you can do to a client with it has always amused me. I remember running an admin plugin on a gmod server that would let you run console commands on any client connected to the server. So you could do really naughty stuff like: bind all the movement controls to quit the game. To the user, it might just look like their game CTD'd and they'd have to think to go in and reset their controls before rejoining a server. I imagine the console lets you do even worse stuff than this.
This is pretty incredible.
Nothing like creeping up behind enemy lines, ready for a quick backstab, and then everyone stops moving
I'm still curious if it's possible to keep track of the object limit, and when it gets too close to an arbitrary buffer, just start deleting things based on a priority list. Not sure how happy either the server/client would be if a map entity like ropes just vanished.
EDIT: I have the memory of a goldfish. Already implemented in the running smoothly video.
This is cool!
hey shounic! i really enjoy your content! I was wondering if you could make a video explaining why sometimes when you and a teammate damage an enemy one person (the one who shot first and didnt get the killing blow) doesnt get an assist and instead the feed just says the second person (the killer) gets the kill alone. Maybe you have already discussed this and i haven't seen it.
shounic did make a video on how the assist system works at one point.
This guy is an actual advanced devolper how did valve not hire him yet
Just imagine a spy just pushing through like all of blue team trying to get behind them like a crowd
I remember old Counter Strike 1.6 zombie servers using a similar mechanism, where server crash would just cause us to reconnect. We always thought it was a scheduled daily server reset happening at random times, but it may just be crashing.
Shounic upload, time to stop everything
are you going to look into MvM anytime soon? i really want to see more than 22 bots on the field at once, i think that would be amazing.
shounic solves the halting problem
On one hand, this is like the epitome of treating the symptoms instead of the cause. On the other hand, it's almost poetically in line with how TF2 spaghetti works behind the scenes anyway, and I mean it works?
I'll say its fair since, like scream fortress proved, the cause cannot be fixed by shounic's team.
Even if tf2 had the cleanest, most well optimized code, server crashes/freezes would STILL be an inevitability eventually. That's why making it more seamless when a crash DOES occur is the right call imo
You should do a video on face-stabs. Basically how much lag is enough to make a perfectly targeting bot get backstabed.
Maybe an improvement would be to "save" the current game state like the map and the score, to then put back on server restart.
I'm not a source dev, so can't say
That was my first thought too, you could keep like a running log of score and everyone's chosen class / loadout(?) so that from the players perspective they just go back to spawn. Not sure if health / ammo / position restoring would be useful since some people will probably load in faster and get an advantage if you restore people back into battle. But either way that would be a super smooth crash recovery mechanism.
That runs the risk of getting in an infinite loop of crashes because something in the saved state is causing the crash
having game state persist while also feeling fair for different connection speeds is basically impossible, on top of that you have to deal with vanilla clients not knowing how to handle it
meanwhile, making it like a map change puts everyone on the same page about the new situation
@@inakilbss That's why I think only restoring the score and possibly auto selecting their previous class would be best - it would still look like a map change to a vanilla client and everyone would still go back to spawn but it wouldn't be too much of a hassle to resume the progress of the match
@@StephenSmith304 i don't think auto selecting a class would even be possible, going through class selection is part of the flow of joining a server
as for scores, i suppose there wouldn't be much harm in saving them, but also not enough benefit to be worth making such a mechanism for
I'm alive!
I'm alive!
I'm alive!
I'm alive!
I'm alive!
I'm alive!
I'm alive!
I'm alive!
*WARNING: SERVER STOPPED SAYING "I'm Alive"*
CHECKING ANY ERRORS...
1 ERROR FOUND
RESTARTING SERVER.....
RESTART COMPLETED
REFLESHING SERVER...
REFLESH COMPLETED
CHECKING ANY ERRORS....
0 ERRORS FOUND
SERVER READY
RESTARTING "I'm Alive" MENSSAGES...
RESTART COMPLETED
*ALL SAFE* *👍*
@@yellowball1948I am DEAD
wait why is meester here what
@@endarkenedgd I've been a TF2 fan for a decade now
Maybe I'm stupid, but is it possible to get around the 24 hour time period by say. At the last 1 or 2 hours before the memory leak crash would happen, scheduling a server restart when the round would usually change maps? It'd be incredibly seamless and it'd just seem like the round had ended.
I was wondering what causes this on your server :D
Hi! I was wondering is there hidden speed cap on how fast sniper's rifle can charge up? I tried on mvm to upgrade machima's chargning speed fully and than Bazaar bargain with 4 heads, even though Bargain should have 300% charging speed rather than only 200% like machima, they both charge up in 1.5 second
interesting vid!
You're carrying the TF2 Community on your back, keep up the good work.
Real Hero
now if you could get a plugin that intercepts server traffic & host redundant servers all simulating the same game, you could seamlessly transfer player traffic from one server to another without interrupting the player experience (much)
like bungeecord basically
completely unrelated to this video, but would you know how to go about making a tool/server plugin that would put a TFbot in control of the player? i'm just interested in letting a crappy default training bot loose in casual to see how it fares in a world of humans
Shounic, do you think you could help with the bot problem?
I can't believe no one except you has come with this idea after so many years lol
1:15 OHHHHH, THAT'S WHAT THAT WAS!
How much extra stress does this "I'm alive" message put on the network connection?
Not much, I guess, but maybe doing so every 10 or so ticks would be enough? Sending this message every tick sounds like an overkill.
There's another comment about this but bear in mind TF2 is throwing back and forth tons of data all the time - the heartbeat message is basically nothing compared to everything else
How would this affect contract progress?
Two games collide: crash bandicoot and neverhood to create... Neverhood
What if every hitscan weapon (and syringes) could headshot?
Maybe these weapon could do mini-crits instead of full crits (to avoid making ambassador a complete downgrade and """balance""" it a little bit)
How would this affect whole game?
So it just pulls a Richard Nixon every time it’s about to crash which is kinda funny
I think you could have made a proxy that starts and manages the server process and passes through all the ip packets from players to the running server and vice versa, at the same time looking at the responses from server and if it stops sending responses or if its process ends then this proxy can send the packet asking all the clients to reconnect and start proxying ip packets to a reserved instance of a server that's already running. It's just my thought though, never done that.
This is amazing. No more "aw, guess I'll go to 2fort."
3:58 valve did WHAT??
At this point, Valve should just hire Shounic
In essence, surround the start function with a try catch that on any error, forces the server to restart and all players to reconnect.
i'm afraid that summation isn't correct even "in essence"...
this plugin is undoubtedly written in pawn, which has no try/catch control flow to speak of.
the crash handler functionality is provided by the windows api -- and that's only a third of the solution...
@@antcosentino27
It's a joke. I watched the video.
"There's always future updates," very optimistic
With enough stickybomb Demos, the map will always be changing.
I think it'd be cool to see a video about if there was no player limit, how many players could you hypothetically have on one server before it crashes? How crazy would you have to make the optimization to fit as many players as possible? How much room would playing an empty map with no props make?
isn't 100 the new technical limit?
i use "retry" myself to the point i have a dedicated button for it on my main menu
could this custom plugin be uploaded somewhere?
I had a good idea, arena with 100 players. There wouldn’t be a point to cap and the time limit would be 10 or 5 minutes.
There’s a program called ram map by sysinternals that lets you clear un uses ram like memory leaks and I think it might work for the dedicated server
nope
me realizing just how overpowered engineer's dispenser is in a 100v100 gamemode. you can quite literally have one dispenser healing half your team at any given time all at once while the other half is applying constant pressure.
Amputator Medics are popular in 100 player servers for this very reason. It's also a surprisingly good source of ubercharge.
So now the server restarts every time it lag spikes for 7 seconds?
Insane idea: checkpoint the TF2 server process and restore its memory upon the crash event (clients get horribly, horribly confused)
I wonder if you can make deeper changes to the server to make seamless crashes more seamless
Source games have the ability to save, but my understanding is multiplayer games it’s usually non functional.
If you can use the saving code to serialize the current state of the game, and then delete a bunch of objects in the case of object limit crashes, then you can test reloading it, and if it works, reconnecting players get automatically put back to where they were with another plug-in.
More often then not when you crash it's for a reason. So serialising and deserialising the state would likely end up with the same crash again and then you'd end up in a loop
I remember there being a tweet about blue screens of death being ignored through a driver. Imagine if there was a way to make C/C++ programs ignore crashes and continue execution, regardless of any bad state.
but why?
@@grubo8884for fun, just out of curiosity lol
One gotcha to be aware of is when an application crashes typically anything outside looking in can't be sure what exactly went wrong and if anything is salvagable.
So you can try to force the server to send out retry commands but if the crash comes from corrupted memory the server may no longer be able to send out those commands. Best case scenario you detect this and skip sending the commands, worst case you don't detect it and crash harder. Though in this case you're no worse off than if you didn't have the plugin.
Unless it's the crash handler code that is actually restarting the server, and waits until after sending out the commands to do it. If you're using a more traditional method of restarting the server on crash though, you're good.
"If memory usage passes 4gb it will crash no matter what"
You sure about this? The server process (at least the one you've shown) is 32-bit so the limit should be 2gb in Windows.
It is possible to patch 32-bit apps to give them a 3gb limit, but this isn't done automatically since some old apps assume a 2gb limit and will malfunction if provided with memory above the 2gb barrier. Seems like something someone would have tried already since it's just patching a single bit in the executable IIRC.
Full 4gb is not possible in 32-bit processes since some memory space has to be reserved for kernel use. The exact amount needed to be reserved depends on drivers in use and such. Technically 64-bit processes also need this reserved space but 64-bit address space is so huge it is not a concern.
That said I do not run Source servers so I'm not familiar with the actual practical limits.
heres a riddle for you: how does he send commands when the server is frozen?
16 secs ago and i didn't even realize
What are the specs that you are using for the server? I 6ghz or something like that? Also, is the code that mircosoft provided the stuff sending the retry command? If not, how is the server able to send the retry command if it is frozen?
the server is a PC, clients are sent data from TF2 but can also be sent data from other applications. so i think the server is running the TF2 server software alongside a basic crash-handler script (and maybe a few other things).
(edit:) so when it crashes, it's just the TF2 server software that crashes; the crash-handler script is still running.
and yeah, the crash-handler script was built for TF2. microsoft's stuff doesn't automatically know how to send data to TF2 clients
in the last video he said something like ryzen 7 58003d or something like that
7800X3D ruclips.net/video/OIvbxikwEbU/видео.htmlm
"retry" connects you to the last server you were on as documented by VALVe: "Retry connection to last server." It's always the current one too.
"ackshully...."
I had some suspicion that an AI voiceover tool was used at the beginning to read the script...
Are you running server on windows? Isn't linux easier and more efficient to run server?
Also retry mechanism reminds me kexec crash handler from linux, where when kernel crashes it does One Last Thing and boots emergency kernel to dump memory.
EDIT: I think you can make emergency server this way and when main server crashed, you will just spam retry to everyone and then route all new traffic to backup process.
The issues at hand are effectively the same on Windows as on Linux. The srcds server process is also 32-bit and exhibits the same memory leaks, just with modern compiler settings (SSE2 instruction set). The server anti-crash (SIGSEGV handler) and watchdog functionality could be handled similarly in a plugin, with the advantage that you could outright replace the failing process using execve() rather than starting a new server, provided that all the file descriptors are properly marked close-on-exec.
Cool
I do wonder how stable the the Linux version would be vs the Windows one.
Maybe Windows has the advantage?
While there's no fancy API for it, it would not be hard to write a plugin for the Linux version of the Source server (using the same Source / SourceMod interface as the Windows version) that registers for OS signals associated with crashes (in particular SIGSEGV, but also uncommon ones like illegal instruction) and acts upon them in a similar way: Issue a reconnect command to all connected clients, close their connections and re-execute the server in place.
Ive used retry to keep from leaving servers that cheaters are trying to crash.
Was this video sponsored by UEAKCrash?
nice
isn't the linux server more stable, or it don't have any difference?, if the valve didn't detected the fault halloween, was because of the OS?, or they just didn't tested lol
You hold the server on windows?
wait, the server is hosted on WINDOWS?! no wonder there were performance issues before x.x;
"What if you could make the server never crash? First of all, this is TF2, so that's not possible."
But how to tell the clients to reconnect after the server has frozen? Who detects the freeze?
that would be your mother
thanks microsoft 👍