I discovered the importance of playtesting while working on my very own game. You can follow my development journey by watching the series "Developing" - ruclips.net/p/PLc38fcMFcV_uH3OK4sTa4bf-UXGk2NW2n
"Whiplash" is such a strange word for me, since there's no real equivalent for it in my first language. So thanks for applying it here, giving me another chance of finally wrapping my head around it. (The movie "Whiplash" did very little in that regard for me, but that might have been on me.)
One would think that a job at valve is heaven, but wheeew. The unwritten part is that expertise and performance are largely irrelevant. Your productivity is controlled by what is allowed by the survivor show-modeled organizational structure. And that one’s career being the same as a contestant on survivor is extremely unhealthy for anyone but managers and their friends. But there are no managers? There absolutely are. Humans will always organize in a hierarchy, and the same is true at valve. Success is being in the right social circle, nothing else.
I just imagine him putting up his hand saying "That's enough." with the whole team just looking at each other confused before he continues with "You all start on Monday."
Thanks. As a hobbiest game dev that spends nowhere near the amount of time I want on making the hundreds of incompatible and half formed ideas in my head into a real game, I spend a fair amount of time thinking about this stuff.
Kinda reminds me of the (probably fake but who cares) Henry Ford quote “If I would have asked people what they wanted, they would have said faster horses.”
My brother actually found out “players never look up” the hard way - as a player. There’s a temple in Twilight Princess where one of the last puzzles is simply looking up - super easy in hindsight, but it took him over an hour to solve it.
People never look up either There's a prankster that hung $100 at a college campus free for grabs and it took the students a few hours to see and claim it
But @@Klipschrf35 is right - people never look up, even in real life. Because why would they? Above us is nothing than a blank sky. There were never real predators coming from above. Humans don't need to look up.
This type of development requires a very special style of leadership that doesn't punish their employees when they get negative feedback. If tests are done every week, and every week you hear what doesn't work with your current approach, your boss needs to be fully behind you to consider it a chance for improvement and not a "trick" to point out all of your mistakes.
Fully agree, a lot of leadership actually don’t like this type of stuff since they can’t hide their emotions when they know this is just part of the process since negative feedback create stress and usually make someone vulnerable and people will feel bad often.
Not just insecurity, stupidity and trauma projection. Most bosses will just scream at the devs, "I thought you said it was ready" and even when the devs are like "dude you gave us 3 days this is a 4 year dev cycle, we told you this would happen" the manager will hold that against them the rest of their careers, and say the devs have bad attitudes, and if they hired some new people the new people would "do it right". (The new people are Indian and get paid a fraction of what the old crew got paid.)
Kind of funny how Portal, a game all about testing and test chambers, would never have been as succesful as it was without its extreme amounts of player testing.
I'm convinced they only made Portal the way it is (focus around testing) because they enjoyed watchig players playtesting their game and making it perfect much like what GLaDOS does in the game.
The Talos Principle is a game where the (somewhat secondary) story (with an antagonist) started to interest me a lot and become the main reason to keep playing for me
It would probably explain why both GlaDos and Wheatley can't help you solve tests. That seems like a sly nod to Valve's game designers painstakingly watching playtesters and can't help them 😂
I guarantee that several of Glados' mean-spirited quips originated from playtester notes saying something similar. And then they developed her personality in that exact direction because the players LOVED it even more, each week, as the writing iterated through the same process. Being one of those playtesting supervisors who remembered writing similar notes, and then having them show up as dialogue in the actual builds of the game, must have been a surreal but cathartic experience.
One of the best playtesting stories out there is when Shigeru Miyamoto's son first picked up and played Super Mario 64. Miyamoto was a little concerned with the fact that he wasn't showing interest in going into the paintings of Peach's Castle to go after the power stars, but spent over half an hour just running around and jumping and trying to run up and slide down slopes, walls, and attempt wall kick jumps outside Peach's Castle. It ironically was a good sign, because as his son and so many other children pointed out in the post-playtest interviews for Super Mario 64, they *LOVED* how well Mario controlled. Miyamoto and his team spending a year focusing on Mario's controls, statistics, and animations really paid off if the simple act of running and jumping around in a 3D game could be fun.
I still remember my dad bringing the game home for an early Christmas present. He's not a gamer in any way, but when he saw someone playing the game on one of those Wal-Mart demo setups he thought it looked incredible. At 8 years old I was right around that perfect age where I was starting to develop the dexterity to play the game. For many years after that it was like a family tradition to bring the game out on Christmas and show off my skills and how they had improved over the year.
Isn't one of the quirks of their design process that they basically won't work on content until they've stuck their platformer protagonist in a featureless room and polished the movement until it's fun even with nothing to interact with?
@@si2foo tbf, didn't Portal's story already end? I can understand Half Life 3 because of the cliffhanger, but Portal actually resolved its story. I don't really see how they could do much else. And a Half Life-Portal crossover just feels too fan service fan fiction-y for me.
For most of the first year of Portal's development, the "actual game" *was* the tutorial. The final game's sequence of test chambers were all initially concepted as being the tutorial section for a much larger game that they scaled back on significantly.
Ohh that explains why it always felt exactly as that lol. Except for the last proper chamber 18 which has every possible mechanic crammed together in a convoluted way, as if they were compensating for the rest of the game that never was :D
@@Tomlacko Chamber 18 is really the only real "graduation chamber" in the game (well, maybe chamber 19 as well.) Even the escape sequence is a tutorial, with escape_00 being a moving panel tutorial, escape_01 being a rocket turret tutorial, and escape_02 being... well the boss fight.
I replayed Portal a few months back and had forgotten you don't even get to place both portals until like two thirds of the way into the game. Which was surreal but it makes perfect sense if the whole game is effectively just a dolled-up antepiece for the final boss.
@@annakrawczuk5221Except they designed Portal 2 to be playable by people who haven't played the first game. And introduced far more mechanics, which require time to teach. So Portal 2 is also very tutorial-y
14:03 This is especially shown in the climax of Portal 2. Realistically, "shoot the moon" is the easiest puzzle solution of all time. It's basically the only interactive option on the screen. But seeing everything fall apart around you, tension at the highest point, and calling back to the source of portal surfaces; that made the climax incredibly satisfying without being hard.
@@LazyBuddyBan hence why my comment was worth making. because something that should've been common knowledge after beating the game twice only just recently clicked.
@@Mate_Antal_Zoltan half life alyx is probably amazing but it’s also kind of lame that their biggest release recently isn’t able to be played by 99% of people
Pretty fitting that the developers of Portal also loves testing ;) Seems like Valve realized this too since they love using Portal to showcase their new tech like VR and the Steam Deck. It's basically their mascot nowadays
@@SimuLord in their defense, it is a good business strategy from an economic standpoint, making videogames is expensive and also very very risky economically these days, especially for AAA studios like Valve. So why risk the cost of making a new game when all they have to do is sit back relax and just take a cut from games people published on Steam and get a crap ton of money.
@@robbieaulia6462 Who becomes a game developer to not make games? I just disagree with this take being the default "wow valve were so clever" moment. I think it's sad they lost their spirit, and it's a cautionary tale not one of some money focused triumph. They lost to the suits.
@@SimuLord you know thats literally not true? just cus they have not released many games since then does not mean there was none in development, there was literally a lfd3 in development and got really far before it got canceled
Valve’s developer commentary is fascinating. Hearing about how they changed the puzzles to make it so players didn’t get stuck but also not easy enough for it to be a cake-walk (pun not intended) is really cool and gives a lot of insight into what it’s like to develop a game. If only they still developed games.
They're releasing Counter-Strike 2 this summer! It's not really a new game, but they rebuilt CS:GO from the ground up to modernize it and enable its developers to make better updates over the next 10 years.
When Valve was busy developing CS2 which is coming out soon, they invited bunch of community figures and ex-pros to playtest it at Valve HQ. These people later said that there were dozens of Valve employees watching them play live in the same room. And all the time they were asked feedback and sometimes changes based on that were implemented the same day. This video gave me some background to that story.
SPUNJ also mentioned that the Valve employees would listen and watch, but if you asked them a question, they wouldn't really answer. That alligns perfectly with what was said at 11:10. Really interesting stuff, and knowing the masterpieces Valve has released in the past, this makes me very excited for CS2
Yeah that’s pretty standard procedure for user experience testing in general, try not to influence the result too much with your input. Customers playing at home won’t have access to an expert developer, so you need to pretend your playtesters don’t either.
Testing early is also really important because people are much more willing to criticise something that doesn't look like it's received a ton of love and if it looks like it would be easy to make changes to it at its current stage. This doesn't only work with games mind you, it also works with websites, apps, and pretty much anything you can think of. In university I learned about wireframe designs in web design, where everything looks so barebones you sometimes are even just giving people things written on paper, it really helps the iteration process.
The best (and occasionally the most soul crushing) thing you can do after submitting your game to a jam is checking out streamers who are playing games from it. You can usually submit your game to their playlist and discover all the things that could go wrong, in realtime!
@@cantflyforshit Once it is submitted to itch anyone can playtest it. But they might resubmit a final version before the time is up, in which case it seems the old commentary is replaced. If you want to playtest for a jam that's already over, many of them stay available on itch. I recently played games from GMTC's 2022 competition.
Mark, I cannot understate how much better your already phenomenally-good videos have gotten since you started making your magnet game. I am a game designer and graduate of coincidentally relevant DigiPen. The moment you started working on your own project with the goal of not just creating for fun but "finishing" something, I started to recognize the genuine insights of development experience infect your videos. In the past, your videos have always been extremely well researched, well produced, and informative. Something to share with my friends who aren't in the industry but are interested in the topics at hand. Now, it feels like your scripts are more... substantive is the wrong word. Accurate? Relatable, maybe. They feel like more than just talking points extracted from GDC talks, interviews, and postmortem articles. I wince and groan with empathy at the genuine struggles you face in your Developing series because I and everyone else I know who works in games has hit those same roadblocks. And the non-Developing videos feel like their topics and scripts reach so much deeper into the fundamentals of game design than perhaps they did in the past. A video about the single player level design can be really engaging, but it's not as critical to the job as Playtesting or Prototyping is. I have felt sometimes that topics may have come across more simplified or shallow than I think they could've been, even for a youtube audience full of non-developers. I felt less satisfied about some videos. I no longer feel that way. You've been killing it recently. Keep it up.
I've been watching your videos for years and this might be your best work imo. You don't just "still have it", you have improved like crazy. Genuenly thank you for making videos like this.
This game has the most adorable sentry guns ever made. It's actually kinda mad when that sentence makes people go: yes, yes indeed. You're absolutely right. Who on earth decided to make sentry guns of all things, cute and lovable? And make you feel sorry when taking them out? I fear Gabe adds something fishy to the office coffee maker every morning 😆
@@tecinplace1316 It takes a special kind of mind to design those for civilian use instead of military lol. Whoever came up with the concept of Cave Johnson is a genius.
Cute? I found their flat tone and their kind invitations so they can kill you to be warnings of absolute psychopathy. Nothing is more terrifying than playing the game on your laptop at night, and hearing the dying sentrybot say, "I don't blame you..." because they want to devote their dying energy into making you feel guilty whilst pretending to do the opposite. Maybe it's because I associate them with the 102 Dalmatians game's Water Gun Teddy Bear. "We're best friends! Let's play a game! You need a hug!" they say, but once you are in range, they rip out machine gun fire while yelling, "Yeeeeeeehaaaaaaaaw!"
A note for the very end, Viewfinder is insane. The amount of head spinning you initially got from Portal is there immediately in the demo. Everyone should try that if they even have a passing interest in the Portal series.
14:00 I thought this was incredible when it happened. The entire time you're trained on clear delineated puzzles, with black and white panels on walls. I died in the fire a few times despite not having any hiccups playing the main game; the swap from slow cerebral puzzles with clear black and white panels, to a 5 second decision in a flaming garbage pit, felt like one of the best thematic shifts in a game I've ever experienced. Everything after that _felt_ harder, not because the puzzles were more difficult, but because from exactly that point, on to the end of the game, you were trying to find a solution to a problem that you weren't meant to solve (Yes Valve had a solution in mind, but the theme was that GLaDOS no longer personally placed white tiles to lead you to a solution). All of the garbage cluttered the solutions significantly, and one time I stacked boxes to escape into an air duct rather than the correct solution which was to have the rocket-firing robot destroy the glass. But it matches the thematic shift. Portal 2 continued right off the bat with that theme.
It's cool how a games style can build from playtesting and feedback, like Portal being clean and sterile to make puzzles more clear. Or for example Sonic 1's Green Hill Zone having checkered and striped patterns, to make running feel faster as the patterns fly by.
That was an amazing watch! Comin from the QA world myself I can fully understand their philosophy and it shows in their games. Funny how you called the work in progress maps with the orange walls ugly, I personally love seeing old builds of finished games and the Hammer engine always feels clean with those height maps written all over the place, it has a strange almost liminal vibe. Keep up the creativity!
Hi GMTK from Vietnam! I have been a long time subscriber to you. I really love your channel and deep analysis of video game mechanics. You have inspired me to think more deeply about game designs and how much effort goes into making sure games convey the right experiences. I have recently started a small channel to try to analyze game mechanics for Vietnamese. I am no where near as good or knowledgeable as you, but I am trying! Thank you so much for your amazing videos, and for inspiring me. I wish you the very best!
Being in the process of re-designing a game ourselves, for the DBZ Budokai 4 Project, I can't thank you enough for all these informations. Also, I can testify how much playtesting is needed in order to advance further in a Project: we got like a dozen of playtesters and they found out a lot of issues my friend and I would've never realized to begin with. Playtest your games, dudes. It's all worth it.
Yup, playtesting is super important. I was surprised myself how quickly people broke the game I was making and didn't pay attention to things that I thought were obvious. I remember putting a sign that explained how to play, but everyone kept missing it because it just blended with the background.
I love GLaDOS as a character, to the point that of all the characters I myself have come up with as a writer, the one inspired by her is one of my favorites. And heck, funny enough, I did actually have an experience "playtesting" her, where I found that what I was trying to go for with the personality didn't work. I ended up switching gears a bit, and I'm reasonably proud of where it's since ended up going.
@@ConcavePgons Chuck them in a scenario and let your fingers dissociate feom your body so you watch them reacting according to their personality, in text form, live. A good character has escaped your control of them and will do what they do, and all you can do is try to organize them by controlling what the environment throes at them.
I realise this comment will get buried, but GMTK is such a great channel/game jam organiser/studio. It’s the only channel that’s felt relevant at every step of learning game dev, from inspiring me to get into it in the first place, to giving me something to aspire to when I started the absolute torture that is trying to learn coding, to giving me experience in one of the GMTK game jams (can’t remember which one), and now as I get ready to release my first game this channel still feels relevant. Thank you guys, and see at GMTKGJ 2023.
@@coffin7904 I've got a couple that I'm doing dual development on, and while both of them have their core game loops I want to get them slightly polished before I take to social media. I'll give you a shout when I'm at that stage, if that's the kinda thing you're interested in.
i found this super interesting because as a web developer if you asked me what the most important principles to usability testing are in my field i would tell you (almost word for word) the same things turns out when you want to make sure your design or vision are actually working and not frustrating your users, there's only a few good ways to get that feedback
"You could almost think of the player as another designer." That is the best description of what my game design philosophy since studying it in college. A textbook I had emphasized and stressed the importance of regular playtesting, and I've stayed true to that philosophy whenever working on a game. Knowing that Valve does the same explains why I love their games so damn much.
I'm developing my own TTRPG, and it's kind of astounding and reassuring that I've come up with a playtest-focused philosophy very similar to Valve's. Being able to play the game twice a week gives me so much information on what systems need to be changed or rules need to be added. This is even more true when I'm playing under someone else as the GM.
If i was playtesting your comment i would be annoyed at not being able to figure out what ttrpg stands for without googling. Gm, isnt that a game moderator? 😏 Just a small rant. Read a 600 page book at work last week, filled with random acronyms i had to repeatedly google as i kept mixing up and forgetting them.
@@mtfoxtrot5296 Also fair to talk to your audience. This is a GMTK video; the chances that a person reading your comment here knows what TTRPG stands for are pretty damn high.
Great video as per usual! If I can humbly bring a little something I learned being a user researcher who runs playtest at a big studio, I think there's two important elements that might add to what you are saying. 1) One of the biggest issue reducing the usefulness of playtests is the culture/vision the devs have on it. Sadly, devs sometime develop an antagonistic relationship with GUR (Game User Research) due to the fact that most of what will come out of it is gonna be critical. We often refer to our job as having to tell someone its baby is ugly. Ego is at the core of creation and we often see defensivness as a primal response to the results of a playtest. What I see from your video is that Valve seems to have created a great culture around iterating where people don't see "bad reviews" from playtesters as an attack but rather as a problem to solve. In my experience its pretty rare to see this and it is quite commandable. We also live with pressures for playtests to be marketing tools and it also hinders our process. 2) On the tip 4 "Designer should run playtests", I might push back here, not because I think they shoudln't be involved and that user researchers should be the only people working on this, but rather because I think designers should participate (as observers) but not be responsible of 1) the methodology employed, 2) running the actual playtest and 3) analyzing the data. These three steps demand a specific scientific background to not include biases and draw wrong conclusion who might harm the project. It's a really fine line to navigate and we often see devs who quickly observe without context jump to conclusion due to an anecdotal evidence. All of this to say that the scientific method applied to make the best use of playtests is often conterintuitive and having specialist with research background is definitely a must if you have the means to pay such an expense. Again thanks for your work, we often pass around your videos at our job (just so you know :) )!
This was really interesting to view as someone who predominantly plays CS:GO. As valve are currently play testing CS2, this added a lot of insight to their “limited access” approach to a beta. Great video!
Can't believe that a video about the two of my favorite games came out on my birthday. I'm not a game developer but your content is such an entertainment. Massive thank you
I appreciate the addition of having content right until the end of the video. I always feel bad clicking off vids when Patreons are being shouted out but I cant sit through random usernames for a minute or more. Nice work.
I like how some games have used Early Access as a way to get a lot of playtest information for design changes, like Hades did. Though not as early in development as Valve would like, Supergiant's approach to Early Access yielded great results in Hades, and this will likely be the case for Hades 2's Early Access later this year. However, unlike Valve, they seem to base their early access impressions on freeform feedback in their discord server. This is wildly different from the sciency approach of silent observation, and definitely brings eskewed results, but Supergiant's team handled it very well. They're my favourite game studio.
Yea I think different methods of gathering feedback isn't as important as Mark's final tip which is the devs themselves need to have a clear vision of what they want in the game and then use playtesting to validate it. I've always wondered how Valve managed to avoid their games from feeling watered-down despite other games using similar testing methods can come out feeling very *"committee-designed"*... Turns out blindly incorporating player feedback isn't enough.
the tips from the video: Tip1: - test as early as possible (9:23) Tip2: - test often (10:17) Tip3: - shut up and watch (11:07) Tip4: - designers should run play teats (12:08) Tip5: - get the right people (12:53) Tip6: - challenge your assumptions (13:43)
As you were talking about Glados boss fight, i was mixing it in my head with Wheatley boss fight at the end of portal 2. And I was thinking, if the ending isnt just the test for if people were actually giving attention to the story, that they are moon stones that you shoot portals at
Not just that. I love the subtle details in that fight - I went in with no spoilers, and at that climatic scene, I was just shooting randomly. I hit the moon, and everything instantly clicked as the hints let me understand what happened. But in the actual fight itself, I noticed (after re-playing) that to solve the previous puzzle, the player needs to have a portal at the center of the room. Afterwards, no matter which mouse button used, you never erase that portal. The moon is also the only bright object in the middle of the screen, making it an instinctive target to aim at. Wheatley is even screaming about the moon! All in all, I got the right answer because the game basically put me on rails, but I FELT like it was all me/my luck.
Originally, "shoot a portal to the moon" was a secret ending at the middle of the game but playtesting showed that pretty much no one could find it out so it was scrapped. It was only later in the development when faced with a dillema of how to end the game someone remembered this scrapped moon ending and suggested to re-implement it as an actual end.
So many streamers seem to completely miss the significance of the moon-rock-dust in the white paint (and its effect on Cave Johnson) because they are so busy being personalities, instead of playing the game.
Interesting to hear that Portal’s final boss design changed from an ultimate puzzle finale to a more story-driven finale. That’s exactly the change I’m thinking about for my game
@@Dailyfiversorry, I didn’t see this at the time! I’m making a third person action adventure (kind of like breath of the wild, only with sci-fi rules). You play as a character who discovers a device able to unlock a buried smart kingdom, one with the power to lead the land’s indoctrinated people to improve their rationality skills
@@Dailyfiver amazing! I don’t have either of those yet (I’m still deep in prototyping so not at any kind of marketing stage yet), but I can post back on here when I finally do have something. Out of interest, what specifically has made you want to follow this project?
@@EmperorsNewWardrobe I’m actually playing through tears of the kingdom right now and I love that type of gameplay loop. As an indie I don’t expect a massive scale like Nintendo has, but even a small game like that sounds super fun to me. Also Sci-fi is my absolute favorite. I just published my first little game on steam recently just to learn the process. It’s a little game called “NugQuest” lol. Started off as a joke but once I finished college I decided to finish it up.
I was one of the smooth-brains who didn't realize the rocket turret breaking the window in Portal was a hint that it could also break glass tubes that had cubes in them. I still got into the vent... by stacking hard drives and other junk items. Oof.
the devs over at id software and the creators of age of empires 2 were also obssessed with playtesting. Sandy, one of the level designers of Doom 2 and Quake, talks a lot about this on his youtube. It's a resource I highly recommend watching, not only as a historical testemony but also because he explains it so well.
2:23 As hinted at here, playtesting is also the reason we have the Companion Cube. Players didn't understand they were supposed to take it with them, and kept leaving it behind, so they gave it a different design and made GLaDOS talk about it a lot.
Portal 1 & 2 is an absolute masterpiece. The mechanics are easy to pick up, as the levels get harder they don't feel frustrating yet are challenging and feel rewarding to solve. I think this a the best game you can show to a beginner that's new to video games. I also love how big the portal universe feels because of the hidden lore sprinkled throughout the game with the secret rooms, easter eggs, and pre-recorded Cave Johnson messages, GLaDos backstory etc. It makes the game a little more eerie and really pulls you in because you if you pay close attention you notice there use to be actual people within the aperature labs but it is now abandoned even though on the surface it just looks like a mere puzzle game with a bunch of rooms, some robots and one human test subject.
14:12 This is absolutely true. The first time I played Portal, it was terrifying slowly approaching the pit of fire with seemingly no way to escape, then getting the chance to do so in the last few seconds. Practically, you just have to place two very easy portals. But the whole scene was visually so unexpected (the first and I think only time when fire appears in the game) and the pacing was so point on (the premise of slow, unavoidable death) that it already established the feel of difficulty. One minute, you are solving puzzles, one minute later, suddenly you are facing hell. When you replay Portal, you already know that you are going to escape and you just need to place two little Portals but the scene never ceases to hype you up and satisfy you in the end. Great design.
The entire thing really reminds me of what I've read about Agile development in the realm of software development. That being: Part of it is to regularly sit down with the client/end users every two weeks or so, and show them the progress you've made, presumably via live demonstration of new features. The feedback you get from that then informs you about how you're going to spend the next two weeks: new features, tweaks 'n' whatnot. Rinse and repeat. That said, I've never seen it in practice. My experience so far is: You sit in a room for months on end, working on half-remembered information told to you by your boss, it's shipped out the door one dreary day, and you never meet the client. Oh, and Valve's "cabals" also ring true for Agile: Multidisciplinary teams, basically.
The 'Agile' model is a little different to that... it's not about constantly engaging the client/end user in the feedback loop, it's about having a feedback/testing loop _between the teams working on the project_ which enables regular iterative improvements. It's effective in software development because you can regularly test your code to check that it performs as expected, doesn't have downstream impacts, etc and then make more frequent, smaller changes based on everyone's results. Your client/users are involved in regular testing & feedback, but far less frequently than the daily/half-weekly/weekly "buzz" meetings between colleagues (which are intended to get everyone on the same page, sharing ideas, tracking progress & determining next steps). Speaking from experience, the Agile method of project management doesn't translate well into other industries for this exact reason... few have the ability to constantly test & iterate at the rate that software engineers can. When you're dealing with physical products or service industries you need a different (but still critically important) feedback & change management approach... usually with a longer change cycle.
@@medea27 I... am confused by your reply, on how to constructively reply, and I am just generally confused. My mental curveball is the notion of "having a feedback/testing loop between the teams working on the project". I am unaware of Agile being used as a discipline for inter-team coöperation on a project (except for the likes of Scrum@Scale, but then only more fleetingly than Scrum itself or Agile). In fact, what ballpark number do you have in mind when talking about the total number of people working on a project? Mine was 8 +/- 3. It just occurred to me, you could be thinking of projects that require ~100 people, which would then indeed be split up into many small teams. I wholeheartedly agree with your third paragraph, and that was my main point: I'm surprised by the amount of end-user testing Valve does, and I draw parallels between this behaviour and that single component of Agile development; of very regular communication with the customer.
I love this kind of history and information! Great stuff! This also helps me out as a software developer to think about the subtleties of human experiences that you might not get from a simple "feedback form"
This was my literal experience with portal: "What a cool tutorial, and a great segue from tutorial to game by making me reach the surface. I can't wait to apply this portal stuff combined with weapons here in the actual - ..." credits roll.
15:25 This reminds me of something I've praised Axiom Verge for - although as a metroidvania the core gameplay loop involves a lot of finding new places, meeting exotic lifeforms, and killing them, Trace spends a good deal of his dialogue trying to ask questions first and not shoot at all until he understands the situation from all sides. He starts every boss fight trying to talk his way out of it, and a good deal of his conversations with friendly (?) NPCs go the same way. So it feels *incredibly* well-earned when he finally concludes he's heard enough.
Mark, I am so impressed with how helpful and informative your videos are. Thank you for producing these! It's obvious that you put A LOT of preparation and work into them. I used to work at a well-known video game company *many* years ago and I've always been fascinated with the craft required to produce a well-made game. Your videos de-mystify so much of the theory and process that goes into that craft. Well done! You've just gained one more subscriber.
I loved this. I really hope more game development companies can implement this creation method so we stop seeing games that are utter disasters like Redfall coming out.
What made Portal 1 and 2 such great games is that, despite being puzzle games, I was never once stuck on a level, neither did I breeze through, not even once did I have to turn to google for an answer, because I got sick of a puzzle and wanted to move on quick. It has a really fantastic gameplay loop, and great boss fights that use everything you've learned during the game.
This sort of explains why it takes Valve a long time to make a game. This is a good thing, taking the time to make sure everything is polished to a mirror shine thanks to feedback and talent is what makes game studios like Valve great
What crazy idea to release this video before the GMTK Game Jam! It would surely have its fun influence on the Jam! I will be so excited to see many quirky games!
Great summary of something I didn't really see much: Valve! However, I'd like to stress one important point: While I believe designers might get more from running the playtests themselves, there are multiple reasons why this might be a bad idea (learned at my university and from following how studies are done today). Firstly, it may impact the moral negatively. That's why you should have at least one person between you and potentially hurtful feedback. Especially if you run playtests online, with possibly anonymous players. Secondly, it may impact the actual performance and behavior of the players. If they speak with the designers who are directly responsible for the level, they might say something different or think something is "not that bad of a deal" and leave it out. That's why in most playtests (and software tests in general) you have the standard phrase at the beginning of "I'm not a part of the design team." You actually have to stress you have no direct "investment" in this playtest. On the other hand, designers might not get to see the actual recording of the playtests and I'm not sure that's a good thing (especially if it's muted). But that is a slightly different topic, I think.
that bit about game designers making wonky assumptions reminds me of the Subnautica devs. in the early stages of development, the Aurora was just a static png plopped down to fill out space in the skybox. If you've played the game, you know it's an actual location you can visist. playtesters would immediately go towards it (because it's a cool-ass space ship wreck. Why wouldn't you do that?), so the devs expanded it into a full area for the final game (though i think it was actually first added during an early access update).
Subnautica is a great example of how effective playtesting can be. They initially intended the game to be a relaxing adventure through an alien world where you could scan and read about different species, but during the playtesting they noticed everyone was genuinely terrified for 90% of the game so they leaned into it and added aggressive leviathans to contend with, which are now synonymous with Subnautica's identity.
Would love to see you tackle Dota 2 from a game dev perspective. It’s such a weird story in that it was basically just a purely player tested, unprofitable mod for 8-10 years before spawning one of the largest genres of video games. While League can work as an analogue for the moba genre, I would argue that Dota is far more ambitious from a game design perspective (even if to a fault, given how annoyingly high the skill floor is). Really appreciate these videos you’re putting out and think Dota would be a good topic. On a different note, recently playing through Signalis and this game is a gem! Really demonstrates how to do a lot with a little.
Please don't reference the "Offical handbook for employees" as it was manufactured by an employee to give the company a good image. Its content is mostly a sweetened lie for good PR. There is a reason it was "'accidently leaked", and it's because it's not real.
One caveat to keep in mind with that "Test Often" advice is that Valve has more money than god and can afford to slow down its production to a crawl to playtest everything continually. Testing your game every week makes it incredibly difficult for a team to work on big features that take way longer than a week to prototype and implement. It puts enormous amounts of pressure on devs to bugfix at the same time as prototyping. It eats up a good amount of time each week as well to recruit playtesters, schedule them, watch them play, have meetings about the session, etc. So that frequency is not viable if you're a small team working on tight deadlines.
However it's necessary, otherwise you may end up wasting time on mechanics and features, that will be removed later in development. As main cause of exceeding deadlines is lack of precise directions and goals.
@@ceu160193 No other company than Valve tests that frequently. I wouldn't call it necessary. My point is it's only viable under a strict set of circumstances.
I love the actual quote saying the process is repeated until its no longer “excruciatingly painful” to watch the playtests, because i can definitely relate to the pain of watching someone try to play a game that im fluent in, and being like “why dont they get that they just have to do this thing and then this will happen??” And i assume that goes double for the people who actually made the game
This reliance on playtester feedback can bite them in the ass though sometimes. Namely how they neutered the Director AI in the L4D games. Particularly the dynamic level pathing in L4D2. When initially designing the sequel they had a very robust AI pathing system where most levels the director could pick and choose from 3-4 paths through the level based on how well the team was doing(short for struggling teams, longer for pro teams). But apparently, lots of play testers found this very confusing on repeated play throughs so they scrapped almost the whole thing. Only remnants of it are in three levels that can change, but are not dynamic(it is randomly chosen when the map loads). As somebody who played a fair bit on L4D2 I find the fact they removed something entirely based on feedback frustrating. They should have left it as an optional feature or even only usable by the AI on higher difficulties. shame really how that played out.
Similarly, the only reason they retconned the original ending to _Portal_ so that Chell doesn't really get to escape, is because when they were working on the sequel, their dumbass testers didn't remember that ending and were confused as to why they weren't playing as Chell anymore.
I needed this video. I got my brother to playtest a game I'm making a few days ago and I felt like crying lol. I thought the whole game was intuitive and it was impossible to get stuck. But no he did everything the most incorrect way possible. I'm glad it's not just me and it's just a part of game dev.
@@EmptyMag a pointless tech demo for 1% of 1% who have VR gear. Call me when it's made into a proper game that the full fanbase of HL can enjoy (note if I sound irritated, it's not at you, it's at Valve).
I remember finding out about viewfinder towards the start of college when i got into game dev. It was a really cool concept and to find out it was inspired by the game that was meant to be portal 2 makes it that much better
Amazingly well prepared episode. Thank you. It just made me think of the `Lean Startup` paradigm developed by Eric Ries and presented in the book of that title. It talks about `build-measure-learn` feedback loop as the primary tool for development of any product in any business. It was conceived to protect entrepreneurs from building a product that nobody wants by testing assumptions early and adjusting to feedback. The only thing that doesn't change in this process is the vision, which, as mentioned in the video, keeps us from providing a 'bland, designed by committee, sludge'.
Thanks a lot for this video. I didn't know this is how Valve does it. Without knowing it, I did much of that myself on my current game - I religiously watched every streamer and youtuber and found SO much valuable feedback just watching other people play (and being unable to interrupt and correct them).
I discovered the importance of playtesting while working on my very own game. You can follow my development journey by watching the series "Developing" - ruclips.net/p/PLc38fcMFcV_uH3OK4sTa4bf-UXGk2NW2n
Knowing all of this, why did Artifact fail?
@@ZarrocLP because the testers didn't have to pay to play it lol
Imagine the anxiety when gabe stopped the presentation, then the whiplash when they all got jobs at valve, an absolute dream
"Whiplash" is such a strange word for me, since there's no real equivalent for it in my first language. So thanks for applying it here, giving me another chance of finally wrapping my head around it. (The movie "Whiplash" did very little in that regard for me, but that might have been on me.)
@@lonestarr1490 it's when you are in a car and stop suddenly, your body may be held in place, but your head whips forward,
@@lonestarr1490 whiplash is the very sudden changing of direction either physically or mentally. It's a specific kind of shock.
One would think that a job at valve is heaven, but wheeew. The unwritten part is that expertise and performance are largely irrelevant. Your productivity is controlled by what is allowed by the survivor show-modeled organizational structure. And that one’s career being the same as a contestant on survivor is extremely unhealthy for anyone but managers and their friends. But there are no managers? There absolutely are. Humans will always organize in a hierarchy, and the same is true at valve. Success is being in the right social circle, nothing else.
I just imagine him putting up his hand saying "That's enough." with the whole team just looking at each other confused before he continues with "You all start on Monday."
Playtesting is a Compass, not a Map. It is best used to tell if you are going in the right direction, not what that direction is.
That's a very good way of putting it!
Thanks. As a hobbiest game dev that spends nowhere near the amount of time I want on making the hundreds of incompatible and half formed ideas in my head into a real game, I spend a fair amount of time thinking about this stuff.
Kinda reminds me of the (probably fake but who cares) Henry Ford quote “If I would have asked people what they wanted, they would have said faster horses.”
Wouldn't that make it a map not a compass? A compass does tell you what direction you're going in.
My brother actually found out “players never look up” the hard way - as a player. There’s a temple in Twilight Princess where one of the last puzzles is simply looking up - super easy in hindsight, but it took him over an hour to solve it.
People never look up either
There's a prankster that hung $100 at a college campus free for grabs and it took the students a few hours to see and claim it
Can you elaborate please?
But @@Klipschrf35 is right - people never look up, even in real life. Because why would they? Above us is nothing than a blank sky. There were never real predators coming from above. Humans don't need to look up.
Which temple? I’ve played LoZTP a few times but it’s been awhile.
Water temple miniboss?
This type of development requires a very special style of leadership that doesn't punish their employees when they get negative feedback. If tests are done every week, and every week you hear what doesn't work with your current approach, your boss needs to be fully behind you to consider it a chance for improvement and not a "trick" to point out all of your mistakes.
Fully agree, a lot of leadership actually don’t like this type of stuff since they can’t hide their emotions when they know this is just part of the process since negative feedback create stress and usually make someone vulnerable and people will feel bad often.
@@pencilcheck Mostly insecure people who shouldn't be leaders. But yeah, humans are an anxious species.
@@drakesilmore3760 you are describing almost 99.9 of the current leadership today.
Shame that that version of Valve died after 2013. Since their pivot to free to play the company is defined by its incredible employee turnover.
Not just insecurity, stupidity and trauma projection. Most bosses will just scream at the devs, "I thought you said it was ready" and even when the devs are like "dude you gave us 3 days this is a 4 year dev cycle, we told you this would happen" the manager will hold that against them the rest of their careers, and say the devs have bad attitudes, and if they hired some new people the new people would "do it right". (The new people are Indian and get paid a fraction of what the old crew got paid.)
Kind of funny how Portal, a game all about testing and test chambers, would never have been as succesful as it was without its extreme amounts of player testing.
Y'know what they say, "Art imitates reality"
"Let's do another test... For Science!"
Beep bop... I'm the Philosophy Bot. Here, have a quote:
"For a man to conquer himself is the first and noblest of all victories"
~ Plato
I'm convinced they only made Portal the way it is (focus around testing) because they enjoyed watchig players playtesting their game and making it perfect much like what GLaDOS does in the game.
Ultrameta!
This “antagonist” concept is exactly why a lot of portal-inspired first person puzzle games fall flat and feel lifeless.
seriously I haven't found a single one that is any good :(
The Talos Principle is a game where the (somewhat secondary) story (with an antagonist) started to interest me a lot and become the main reason to keep playing for me
What are the odds I mention The Talos Principle and then immediately see the announcement to its sequel afterwards LETSGOOO
@@joshuam2263 I was actually the opposite: I loved the puzzles, but the story made me too sad to keep playing. That's a hell of an accomplishment!
@@joshuam2263 wait WHAAAAT ?
my new headcanon is that the "science" done in portal is just in-universe playtesting
Facts
And Glados is the game developers watching the playtesters
It would probably explain why both GlaDos and Wheatley can't help you solve tests. That seems like a sly nod to Valve's game designers painstakingly watching playtesters and can't help them 😂
@@TerranceBhS the euphoria from someone solving a test is just satisfaction that they got your puzzle
I guarantee that several of Glados' mean-spirited quips originated from playtester notes saying something similar. And then they developed her personality in that exact direction because the players LOVED it even more, each week, as the writing iterated through the same process.
Being one of those playtesting supervisors who remembered writing similar notes, and then having them show up as dialogue in the actual builds of the game, must have been a surreal but cathartic experience.
One of the best playtesting stories out there is when Shigeru Miyamoto's son first picked up and played Super Mario 64. Miyamoto was a little concerned with the fact that he wasn't showing interest in going into the paintings of Peach's Castle to go after the power stars, but spent over half an hour just running around and jumping and trying to run up and slide down slopes, walls, and attempt wall kick jumps outside Peach's Castle.
It ironically was a good sign, because as his son and so many other children pointed out in the post-playtest interviews for Super Mario 64, they *LOVED* how well Mario controlled. Miyamoto and his team spending a year focusing on Mario's controls, statistics, and animations really paid off if the simple act of running and jumping around in a 3D game could be fun.
I still remember my dad bringing the game home for an early Christmas present. He's not a gamer in any way, but when he saw someone playing the game on one of those Wal-Mart demo setups he thought it looked incredible. At 8 years old I was right around that perfect age where I was starting to develop the dexterity to play the game. For many years after that it was like a family tradition to bring the game out on Christmas and show off my skills and how they had improved over the year.
Oh man haha, I did the same thing as a kid! I genuinely have more memories about running around outside the castle than any actual map.
@@Cyax0kThat’s so cool man 😄
I heard Miyamoto called his son stupid, when he didn't like the feedback he was giving during feedback
Isn't one of the quirks of their design process that they basically won't work on content until they've stuck their platformer protagonist in a featureless room and polished the movement until it's fun even with nothing to interact with?
I have huge respect for the level designers of Portal
I just wish they would make a portal3 or a half life portal combined game where freeman and chel have to work together
Still impressive today 😄
@@IdentifiantE.S7 minutes ago
@@si2foo tbf, didn't Portal's story already end? I can understand Half Life 3 because of the cliffhanger, but Portal actually resolved its story. I don't really see how they could do much else. And a Half Life-Portal crossover just feels too fan service fan fiction-y for me.
@@cloakedhornets6515 pretty sure there's some theory on how they connect since they both open portals to other dimensions
For most of the first year of Portal's development, the "actual game" *was* the tutorial. The final game's sequence of test chambers were all initially concepted as being the tutorial section for a much larger game that they scaled back on significantly.
Ohh that explains why it always felt exactly as that lol. Except for the last proper chamber 18 which has every possible mechanic crammed together in a convoluted way, as if they were compensating for the rest of the game that never was :D
@@Tomlacko Chamber 18 is really the only real "graduation chamber" in the game (well, maybe chamber 19 as well.) Even the escape sequence is a tutorial, with escape_00 being a moving panel tutorial, escape_01 being a rocket turret tutorial, and escape_02 being... well the boss fight.
I replayed Portal a few months back and had forgotten you don't even get to place both portals until like two thirds of the way into the game. Which was surreal but it makes perfect sense if the whole game is effectively just a dolled-up antepiece for the final boss.
Yeah, to me it always felt like Portal is a tech demo for Portal 2, but calling it the tutorial for Portal 2 also feels right
@@annakrawczuk5221Except they designed Portal 2 to be playable by people who haven't played the first game. And introduced far more mechanics, which require time to teach. So Portal 2 is also very tutorial-y
14:03 This is especially shown in the climax of Portal 2.
Realistically, "shoot the moon" is the easiest puzzle solution of all time. It's basically the only interactive option on the screen. But seeing everything fall apart around you, tension at the highest point, and calling back to the source of portal surfaces; that made the climax incredibly satisfying without being hard.
I only just now connected that the ground up moon rocks mixed into a paste is what makes portal surfaces portalable.
@@FlamingZelda3 Bro how
@@FlamingZelda3 I mean, you are literally told that in the game, by Cave Johnson himself
@@LazyBuddyBan Yeah, that made it really cool!
@@LazyBuddyBan hence why my comment was worth making. because something that should've been common knowledge after beating the game twice only just recently clicked.
These Valve guys are pretty good, maybe they should make some more games
I know 😫
Right? This whole video makes me sad.
They've probably been working on CS2 after releasing Alyx, you know.
Yeah, like that Half-life series
@@Mate_Antal_Zoltan half life alyx is probably amazing but it’s also kind of lame that their biggest release recently isn’t able to be played by 99% of people
Pretty fitting that the developers of Portal also loves testing ;)
Seems like Valve realized this too since they love using Portal to showcase their new tech like VR and the Steam Deck.
It's basically their mascot nowadays
@@SimuLord in their defense, it is a good business strategy from an economic standpoint, making videogames is expensive and also very very risky economically these days, especially for AAA studios like Valve. So why risk the cost of making a new game when all they have to do is sit back relax and just take a cut from games people published on Steam and get a crap ton of money.
@@robbieaulia6462
Who becomes a game developer to not make games?
I just disagree with this take being the default "wow valve were so clever" moment. I think it's sad they lost their spirit, and it's a cautionary tale not one of some money focused triumph.
They lost to the suits.
@@SimuLord you know thats literally not true? just cus they have not released many games since then does not mean there was none in development, there was literally a lfd3 in development and got really far before it got canceled
Valve’s developer commentary is fascinating. Hearing about how they changed the puzzles to make it so players didn’t get stuck but also not easy enough for it to be a cake-walk (pun not intended) is really cool and gives a lot of insight into what it’s like to develop a game.
If only they still developed games.
They're releasing Counter-Strike 2 this summer! It's not really a new game, but they rebuilt CS:GO from the ground up to modernize it and enable its developers to make better updates over the next 10 years.
Well half life 3 should not be that far away, right? *RIGHT?!*
@@EthicalAllele isn't counter strike 2 already out?
Yeah they always blow my mind with their problem solving
@@IndicatedGoodLife I hope so lol
When Valve was busy developing CS2 which is coming out soon, they invited bunch of community figures and ex-pros to playtest it at Valve HQ. These people later said that there were dozens of Valve employees watching them play live in the same room. And all the time they were asked feedback and sometimes changes based on that were implemented the same day. This video gave me some background to that story.
SPUNJ also mentioned that the Valve employees would listen and watch, but if you asked them a question, they wouldn't really answer. That alligns perfectly with what was said at 11:10. Really interesting stuff, and knowing the masterpieces Valve has released in the past, this makes me very excited for CS2
Yeah that’s pretty standard procedure for user experience testing in general, try not to influence the result too much with your input. Customers playing at home won’t have access to an expert developer, so you need to pretend your playtesters don’t either.
CS2 sadly feels like they catered too much to those players lol
Testing early is also really important because people are much more willing to criticise something that doesn't look like it's received a ton of love and if it looks like it would be easy to make changes to it at its current stage. This doesn't only work with games mind you, it also works with websites, apps, and pretty much anything you can think of. In university I learned about wireframe designs in web design, where everything looks so barebones you sometimes are even just giving people things written on paper, it really helps the iteration process.
The best (and occasionally the most soul crushing) thing you can do after submitting your game to a jam is checking out streamers who are playing games from it. You can usually submit your game to their playlist and discover all the things that could go wrong, in realtime!
How can you sign up to playtest those submissions to the jam?
@@cantflyforshit Once it is submitted to itch anyone can playtest it. But they might resubmit a final version before the time is up, in which case it seems the old commentary is replaced.
If you want to playtest for a jam that's already over, many of them stay available on itch. I recently played games from GMTC's 2022 competition.
Mark, I cannot understate how much better your already phenomenally-good videos have gotten since you started making your magnet game.
I am a game designer and graduate of coincidentally relevant DigiPen. The moment you started working on your own project with the goal of not just creating for fun but "finishing" something, I started to recognize the genuine insights of development experience infect your videos. In the past, your videos have always been extremely well researched, well produced, and informative. Something to share with my friends who aren't in the industry but are interested in the topics at hand.
Now, it feels like your scripts are more... substantive is the wrong word. Accurate? Relatable, maybe. They feel like more than just talking points extracted from GDC talks, interviews, and postmortem articles. I wince and groan with empathy at the genuine struggles you face in your Developing series because I and everyone else I know who works in games has hit those same roadblocks. And the non-Developing videos feel like their topics and scripts reach so much deeper into the fundamentals of game design than perhaps they did in the past. A video about the single player level design can be really engaging, but it's not as critical to the job as Playtesting or Prototyping is. I have felt sometimes that topics may have come across more simplified or shallow than I think they could've been, even for a youtube audience full of non-developers. I felt less satisfied about some videos. I no longer feel that way.
You've been killing it recently. Keep it up.
I planned to go for digipen but circumstances changed, can i talk to you! I am going into game design, and want to make more contacts
We'll never stop talking about Portal, and that's beautiful. What a masterpiece.
glad you enjoyed it!
@@AlastairGames are you implying you were one of the people who made portal??
@@albingrahn5576 no, I'm just happy for them!
@@AlastairGames thats nice :)
I've been watching your videos for years and this might be your best work imo. You don't just "still have it", you have improved like crazy. Genuenly thank you for making videos like this.
Thanks!
This game has the most adorable sentry guns ever made.
It's actually kinda mad when that sentence makes people go: yes, yes indeed. You're absolutely right.
Who on earth decided to make sentry guns of all things, cute and lovable? And make you feel sorry when taking them out?
I fear Gabe adds something fishy to the office coffee maker every morning 😆
"Don't drink the coffee! They put something in it! To make ya forget! I don't even remember how I got here!"
Sentry guns were designed to be sold for consumer households according to Aperture
@@fojisan2398 Me when Gaben asks to pick up the can
@@tecinplace1316 It takes a special kind of mind to design those for civilian use instead of military lol. Whoever came up with the concept of Cave Johnson is a genius.
Cute? I found their flat tone and their kind invitations so they can kill you to be warnings of absolute psychopathy. Nothing is more terrifying than playing the game on your laptop at night, and hearing the dying sentrybot say, "I don't blame you..." because they want to devote their dying energy into making you feel guilty whilst pretending to do the opposite.
Maybe it's because I associate them with the 102 Dalmatians game's Water Gun Teddy Bear. "We're best friends! Let's play a game! You need a hug!" they say, but once you are in range, they rip out machine gun fire while yelling, "Yeeeeeeehaaaaaaaaw!"
A note for the very end, Viewfinder is insane. The amount of head spinning you initially got from Portal is there immediately in the demo. Everyone should try that if they even have a passing interest in the Portal series.
14:00 I thought this was incredible when it happened.
The entire time you're trained on clear delineated puzzles, with black and white panels on walls. I died in the fire a few times despite not having any hiccups playing the main game; the swap from slow cerebral puzzles with clear black and white panels, to a 5 second decision in a flaming garbage pit, felt like one of the best thematic shifts in a game I've ever experienced.
Everything after that _felt_ harder, not because the puzzles were more difficult, but because from exactly that point, on to the end of the game, you were trying to find a solution to a problem that you weren't meant to solve (Yes Valve had a solution in mind, but the theme was that GLaDOS no longer personally placed white tiles to lead you to a solution). All of the garbage cluttered the solutions significantly, and one time I stacked boxes to escape into an air duct rather than the correct solution which was to have the rocket-firing robot destroy the glass. But it matches the thematic shift.
Portal 2 continued right off the bat with that theme.
It's cool how a games style can build from playtesting and feedback, like Portal being clean and sterile to make puzzles more clear. Or for example Sonic 1's Green Hill Zone having checkered and striped patterns, to make running feel faster as the patterns fly by.
That was an amazing watch! Comin from the QA world myself I can fully understand their philosophy and it shows in their games.
Funny how you called the work in progress maps with the orange walls ugly, I personally love seeing old builds of finished games and the Hammer engine always feels clean with those height maps written all over the place, it has a strange almost liminal vibe.
Keep up the creativity!
Hi GMTK from Vietnam!
I have been a long time subscriber to you. I really love your channel and deep analysis of video game mechanics. You have inspired me to think more deeply about game designs and how much effort goes into making sure games convey the right experiences.
I have recently started a small channel to try to analyze game mechanics for Vietnamese. I am no where near as good or knowledgeable as you, but I am trying!
Thank you so much for your amazing videos, and for inspiring me. I wish you the very best!
Thanks for the kind words, good luck with your channel
Good luck
This must be the secret to their amazing writing too. They always have something for every possible thing you can do
Being in the process of re-designing a game ourselves, for the DBZ Budokai 4 Project, I can't thank you enough for all these informations.
Also, I can testify how much playtesting is needed in order to advance further in a Project: we got like a dozen of playtesters and they found out a lot of issues my friend and I would've never realized to begin with.
Playtest your games, dudes. It's all worth it.
What if valve made games?
The world isn't ready for that
What if heavy update?
Specially the ones that end in the number 3
The world is not longer what it was before.
Reality is gonna end itself
Yup, playtesting is super important. I was surprised myself how quickly people broke the game I was making and didn't pay attention to things that I thought were obvious. I remember putting a sign that explained how to play, but everyone kept missing it because it just blended with the background.
I love GLaDOS as a character, to the point that of all the characters I myself have come up with as a writer, the one inspired by her is one of my favorites. And heck, funny enough, I did actually have an experience "playtesting" her, where I found that what I was trying to go for with the personality didn't work. I ended up switching gears a bit, and I'm reasonably proud of where it's since ended up going.
As a person who is interested in writing interesting characters, would it be okay to ask how you exactly "play test" a character?
@@ConcavePgons have people read that character’s dialogue?
@@ConcavePgons Chuck them in a scenario and let your fingers dissociate feom your body so you watch them reacting according to their personality, in text form, live. A good character has escaped your control of them and will do what they do, and all you can do is try to organize them by controlling what the environment throes at them.
@@ConcavePgons playing them in a roleplaying-oriented TTRPG might also work
She gave me a bit of insight in what it might be like to be a woman.
Never had other men call me fat or sth.
I realise this comment will get buried, but GMTK is such a great channel/game jam organiser/studio. It’s the only channel that’s felt relevant at every step of learning game dev, from inspiring me to get into it in the first place, to giving me something to aspire to when I started the absolute torture that is trying to learn coding, to giving me experience in one of the GMTK game jams (can’t remember which one), and now as I get ready to release my first game this channel still feels relevant. Thank you guys, and see at GMTKGJ 2023.
Lovely comment, thank you! Best of luck with your game
What's your game? How can I follow it?
@@coffin7904 I've got a couple that I'm doing dual development on, and while both of them have their core game loops I want to get them slightly polished before I take to social media. I'll give you a shout when I'm at that stage, if that's the kinda thing you're interested in.
i found this super interesting because as a web developer if you asked me what the most important principles to usability testing are in my field i would tell you (almost word for word) the same things
turns out when you want to make sure your design or vision are actually working and not frustrating your users, there's only a few good ways to get that feedback
14:38 WTF I didn't realize it was you the entire video LMAO no wonder I'm loving this video so much. Omg you're literally the best youtuber ever
"You could almost think of the player as another designer." That is the best description of what my game design philosophy since studying it in college. A textbook I had emphasized and stressed the importance of regular playtesting, and I've stayed true to that philosophy whenever working on a game. Knowing that Valve does the same explains why I love their games so damn much.
Step one: teach the player c++.
I'm developing my own TTRPG, and it's kind of astounding and reassuring that I've come up with a playtest-focused philosophy very similar to Valve's. Being able to play the game twice a week gives me so much information on what systems need to be changed or rules need to be added. This is even more true when I'm playing under someone else as the GM.
If i was playtesting your comment i would be annoyed at not being able to figure out what ttrpg stands for without googling. Gm, isnt that a game moderator? 😏
Just a small rant. Read a 600 page book at work last week, filled with random acronyms i had to repeatedly google as i kept mixing up and forgetting them.
@@emmatitova2154 TTRPG = tabletop role-playing game
GM = Game Master (as in Dungeon Master, but not specific to Dungeons and Dragons)
@@emmatitova2154 Ya know fair enough, I throw around the acronyms so much I forget that people might not know what they mean lol.
@@mtfoxtrot5296 Also fair to talk to your audience. This is a GMTK video; the chances that a person reading your comment here knows what TTRPG stands for are pretty damn high.
Great video as per usual!
If I can humbly bring a little something I learned being a user researcher who runs playtest at a big studio, I think there's two important elements that might add to what you are saying.
1) One of the biggest issue reducing the usefulness of playtests is the culture/vision the devs have on it. Sadly, devs sometime develop an antagonistic relationship with GUR (Game User Research) due to the fact that most of what will come out of it is gonna be critical. We often refer to our job as having to tell someone its baby is ugly. Ego is at the core of creation and we often see defensivness as a primal response to the results of a playtest. What I see from your video is that Valve seems to have created a great culture around iterating where people don't see "bad reviews" from playtesters as an attack but rather as a problem to solve. In my experience its pretty rare to see this and it is quite commandable.
We also live with pressures for playtests to be marketing tools and it also hinders our process.
2) On the tip 4 "Designer should run playtests", I might push back here, not because I think they shoudln't be involved and that user researchers should be the only people working on this, but rather because I think designers should participate (as observers) but not be responsible of 1) the methodology employed, 2) running the actual playtest and 3) analyzing the data. These three steps demand a specific scientific background to not include biases and draw wrong conclusion who might harm the project. It's a really fine line to navigate and we often see devs who quickly observe without context jump to conclusion due to an anecdotal evidence.
All of this to say that the scientific method applied to make the best use of playtests is often conterintuitive and having specialist with research background is definitely a must if you have the means to pay such an expense.
Again thanks for your work, we often pass around your videos at our job (just so you know :) )!
This was really interesting to view as someone who predominantly plays CS:GO. As valve are currently play testing CS2, this added a lot of insight to their “limited access” approach to a beta. Great video!
Can't believe that a video about the two of my favorite games came out on my birthday. I'm not a game developer but your content is such an entertainment. Massive thank you
7:46 That's not the only reason, it didn't add much to puzzle making and acted weirdly with portals on walls
Having a little story while the credits play was brilliant.
I appreciate the addition of having content right until the end of the video. I always feel bad clicking off vids when Patreons are being shouted out but I cant sit through random usernames for a minute or more. Nice work.
I strongly believe that this video is one of those who contains the most valuable piece of advice a developer could ever get.
This video is a godsend! Im currently stuck with playtesting my game, and this is a best advice i could've asked for!
I like how some games have used Early Access as a way to get a lot of playtest information for design changes, like Hades did. Though not as early in development as Valve would like, Supergiant's approach to Early Access yielded great results in Hades, and this will likely be the case for Hades 2's Early Access later this year.
However, unlike Valve, they seem to base their early access impressions on freeform feedback in their discord server. This is wildly different from the sciency approach of silent observation, and definitely brings eskewed results, but Supergiant's team handled it very well.
They're my favourite game studio.
Yea I think different methods of gathering feedback isn't as important as Mark's final tip which is the devs themselves need to have a clear vision of what they want in the game and then use playtesting to validate it.
I've always wondered how Valve managed to avoid their games from feeling watered-down despite other games using similar testing methods can come out feeling very *"committee-designed"*...
Turns out blindly incorporating player feedback isn't enough.
Probably the best video on agile software development, without mentioning the term 'agile software development' even once.
I work in company software development, and I find the hardest part is to actually find testers… especially clients
the tips from the video:
Tip1: - test as early as possible (9:23)
Tip2: - test often (10:17)
Tip3: - shut up and watch (11:07)
Tip4: - designers should run play teats (12:08)
Tip5: - get the right people (12:53)
Tip6: - challenge your assumptions (13:43)
As you were talking about Glados boss fight, i was mixing it in my head with Wheatley boss fight at the end of portal 2.
And I was thinking, if the ending isnt just the test for if people were actually giving attention to the story, that they are moon stones that you shoot portals at
Not just that. I love the subtle details in that fight - I went in with no spoilers, and at that climatic scene, I was just shooting randomly. I hit the moon, and everything instantly clicked as the hints let me understand what happened.
But in the actual fight itself, I noticed (after re-playing) that to solve the previous puzzle, the player needs to have a portal at the center of the room. Afterwards, no matter which mouse button used, you never erase that portal. The moon is also the only bright object in the middle of the screen, making it an instinctive target to aim at. Wheatley is even screaming about the moon! All in all, I got the right answer because the game basically put me on rails, but I FELT like it was all me/my luck.
Yep when I saw the moon it clicked instantly what they wanted me to do and my jaw dropped in disbelief before I even fired a portal.
Originally, "shoot a portal to the moon" was a secret ending at the middle of the game but playtesting showed that pretty much no one could find it out so it was scrapped. It was only later in the development when faced with a dillema of how to end the game someone remembered this scrapped moon ending and suggested to re-implement it as an actual end.
@@Rhekke Are you just going to forget the Grey Goo that one sprays on a surface to make it portal-placement-ready?
So many streamers seem to completely miss the significance of the moon-rock-dust in the white paint (and its effect on Cave Johnson) because they are so busy being personalities, instead of playing the game.
Interesting to hear that Portal’s final boss design changed from an ultimate puzzle finale to a more story-driven finale. That’s exactly the change I’m thinking about for my game
What game are you making?
@@Dailyfiversorry, I didn’t see this at the time! I’m making a third person action adventure (kind of like breath of the wild, only with sci-fi rules). You play as a character who discovers a device able to unlock a buried smart kingdom, one with the power to lead the land’s indoctrinated people to improve their rationality skills
@@EmperorsNewWardrobe I’d love to follow this project If you have a steam page or social media!
@@Dailyfiver amazing! I don’t have either of those yet (I’m still deep in prototyping so not at any kind of marketing stage yet), but I can post back on here when I finally do have something. Out of interest, what specifically has made you want to follow this project?
@@EmperorsNewWardrobe I’m actually playing through tears of the kingdom right now and I love that type of gameplay loop. As an indie I don’t expect a massive scale like Nintendo has, but even a small game like that sounds super fun to me. Also Sci-fi is my absolute favorite.
I just published my first little game on steam recently just to learn the process. It’s a little game called “NugQuest” lol. Started off as a joke but once I finished college I decided to finish it up.
Honestly I think weekly play testing makes a lot of sense - especially since most software development has moved towards Agile.
I was one of the smooth-brains who didn't realize the rocket turret breaking the window in Portal was a hint that it could also break glass tubes that had cubes in them. I still got into the vent... by stacking hard drives and other junk items. Oof.
the devs over at id software and the creators of age of empires 2 were also obssessed with playtesting. Sandy, one of the level designers of Doom 2 and Quake, talks a lot about this on his youtube. It's a resource I highly recommend watching, not only as a historical testemony but also because he explains it so well.
2:23 As hinted at here, playtesting is also the reason we have the Companion Cube. Players didn't understand they were supposed to take it with them, and kept leaving it behind, so they gave it a different design and made GLaDOS talk about it a lot.
Portal 1 & 2 is an absolute masterpiece. The mechanics are easy to pick up, as the levels get harder they don't feel frustrating yet are challenging and feel rewarding to solve. I think this a the best game you can show to a beginner that's new to video games. I also love how big the portal universe feels because of the hidden lore sprinkled throughout the game with the secret rooms, easter eggs, and pre-recorded Cave Johnson messages, GLaDos backstory etc. It makes the game a little more eerie and really pulls you in because you if you pay close attention you notice there use to be actual people within the aperature labs but it is now abandoned even though on the surface it just looks like a mere puzzle game with a bunch of rooms, some robots and one human test subject.
14:12 This is absolutely true. The first time I played Portal, it was terrifying slowly approaching the pit of fire with seemingly no way to escape, then getting the chance to do so in the last few seconds. Practically, you just have to place two very easy portals. But the whole scene was visually so unexpected (the first and I think only time when fire appears in the game) and the pacing was so point on (the premise of slow, unavoidable death) that it already established the feel of difficulty. One minute, you are solving puzzles, one minute later, suddenly you are facing hell. When you replay Portal, you already know that you are going to escape and you just need to place two little Portals but the scene never ceases to hype you up and satisfy you in the end. Great design.
Valve rarely make games but when they do the game is perfect. It's a totally different feel playing a Valve game, they're just so satisfying
The entire thing really reminds me of what I've read about Agile development in the realm of software development.
That being: Part of it is to regularly sit down with the client/end users every two weeks or so, and show them the progress you've made, presumably via live demonstration of new features.
The feedback you get from that then informs you about how you're going to spend the next two weeks: new features, tweaks 'n' whatnot. Rinse and repeat.
That said, I've never seen it in practice. My experience so far is: You sit in a room for months on end, working on half-remembered information told to you by your boss, it's shipped out the door one dreary day, and you never meet the client.
Oh, and Valve's "cabals" also ring true for Agile: Multidisciplinary teams, basically.
The 'Agile' model is a little different to that... it's not about constantly engaging the client/end user in the feedback loop, it's about having a feedback/testing loop _between the teams working on the project_ which enables regular iterative improvements.
It's effective in software development because you can regularly test your code to check that it performs as expected, doesn't have downstream impacts, etc and then make more frequent, smaller changes based on everyone's results.
Your client/users are involved in regular testing & feedback, but far less frequently than the daily/half-weekly/weekly "buzz" meetings between colleagues (which are intended to get everyone on the same page, sharing ideas, tracking progress & determining next steps).
Speaking from experience, the Agile method of project management doesn't translate well into other industries for this exact reason... few have the ability to constantly test & iterate at the rate that software engineers can. When you're dealing with physical products or service industries you need a different (but still critically important) feedback & change management approach... usually with a longer change cycle.
@@medea27 I... am confused by your reply, on how to constructively reply, and I am just generally confused.
My mental curveball is the notion of "having a feedback/testing loop between the teams working on the project".
I am unaware of Agile being used as a discipline for inter-team coöperation on a project (except for the likes of Scrum@Scale, but then only more fleetingly than Scrum itself or Agile).
In fact, what ballpark number do you have in mind when talking about the total number of people working on a project? Mine was 8 +/- 3.
It just occurred to me, you could be thinking of projects that require ~100 people, which would then indeed be split up into many small teams.
I wholeheartedly agree with your third paragraph, and that was my main point: I'm surprised by the amount of end-user testing Valve does, and I draw parallels between this behaviour and that single component of Agile development; of very regular communication with the customer.
I love this kind of history and information! Great stuff! This also helps me out as a software developer to think about the subtleties of human experiences that you might not get from a simple "feedback form"
4:40 **looking at 'triple A' studios**: are you sure about that?
Valve is very unique and does alot of things right in my eyes. I hope they prevail through the ages and not give up on there own path.
I love that you include detailed sources in the descriptions! Please keep it up :D
11:30 that moment hits hard as a dm. Thinking you've done something clever with a puzzle just to realize it only makes sense to you sucks
This was my literal experience with portal: "What a cool tutorial, and a great segue from tutorial to game by making me reach the surface. I can't wait to apply this portal stuff combined with weapons here in the actual - ..." credits roll.
15:25 This reminds me of something I've praised Axiom Verge for - although as a metroidvania the core gameplay loop involves a lot of finding new places, meeting exotic lifeforms, and killing them, Trace spends a good deal of his dialogue trying to ask questions first and not shoot at all until he understands the situation from all sides. He starts every boss fight trying to talk his way out of it, and a good deal of his conversations with friendly (?) NPCs go the same way. So it feels *incredibly* well-earned when he finally concludes he's heard enough.
4:59 they’ve made a game since 2018?
They’ve deigned a game sense 2018
Mark, I am so impressed with how helpful and informative your videos are. Thank you for producing these! It's obvious that you put A LOT of preparation and work into them. I used to work at a well-known video game company *many* years ago and I've always been fascinated with the craft required to produce a well-made game. Your videos de-mystify so much of the theory and process that goes into that craft. Well done! You've just gained one more subscriber.
Valve sounds like such a great company to work at.
11:03 cut to 8 year old me playing HL2 on the orange box dying in the sewers to the helicopter for several days straight confused as fuck
I loved this. I really hope more game development companies can implement this creation method so we stop seeing games that are utter disasters like Redfall coming out.
What made Portal 1 and 2 such great games is that, despite being puzzle games, I was never once stuck on a level, neither did I breeze through, not even once did I have to turn to google for an answer, because I got sick of a puzzle and wanted to move on quick.
It has a really fantastic gameplay loop, and great boss fights that use everything you've learned during the game.
Play testing is so important to Valve that the plot of portal is literally that you are doing testing
That swing at GoW! :D i love it, and its so true. At the point they shipped it in, not having the puzzles would be just the same...
This sort of explains why it takes Valve a long time to make a game. This is a good thing, taking the time to make sure everything is polished to a mirror shine thanks to feedback and talent is what makes game studios like Valve great
What crazy idea to release this video before the GMTK Game Jam! It would surely have its fun influence on the Jam! I will be so excited to see many quirky games!
Legends says Gaben still playtesting HL3
It's so great to return to these videos - so packed with entertaining information - more to learn even on rewatching.
Great summary of something I didn't really see much: Valve! However, I'd like to stress one important point: While I believe designers might get more from running the playtests themselves, there are multiple reasons why this might be a bad idea (learned at my university and from following how studies are done today).
Firstly, it may impact the moral negatively. That's why you should have at least one person between you and potentially hurtful feedback. Especially if you run playtests online, with possibly anonymous players.
Secondly, it may impact the actual performance and behavior of the players. If they speak with the designers who are directly responsible for the level, they might say something different or think something is "not that bad of a deal" and leave it out. That's why in most playtests (and software tests in general) you have the standard phrase at the beginning of "I'm not a part of the design team." You actually have to stress you have no direct "investment" in this playtest.
On the other hand, designers might not get to see the actual recording of the playtests and I'm not sure that's a good thing (especially if it's muted). But that is a slightly different topic, I think.
that bit about game designers making wonky assumptions reminds me of the Subnautica devs.
in the early stages of development, the Aurora was just a static png plopped down to fill out space in the skybox. If you've played the game, you know it's an actual location you can visist.
playtesters would immediately go towards it (because it's a cool-ass space ship wreck. Why wouldn't you do that?), so the devs expanded it into a full area for the final game (though i think it was actually first added during an early access update).
Subnautica is a great example of how effective playtesting can be. They initially intended the game to be a relaxing adventure through an alien world where you could scan and read about different species, but during the playtesting they noticed everyone was genuinely terrified for 90% of the game so they leaned into it and added aggressive leviathans to contend with, which are now synonymous with Subnautica's identity.
Would love to see you tackle Dota 2 from a game dev perspective. It’s such a weird story in that it was basically just a purely player tested, unprofitable mod for 8-10 years before spawning one of the largest genres of video games. While League can work as an analogue for the moba genre, I would argue that Dota is far more ambitious from a game design perspective (even if to a fault, given how annoyingly high the skill floor is).
Really appreciate these videos you’re putting out and think Dota would be a good topic.
On a different note, recently playing through Signalis and this game is a gem! Really demonstrates how to do a lot with a little.
Wow definitely learned something new this video.
Had no idea Valve made games!
I love this "behind the scenes" type video youve been on
Please don't reference the "Offical handbook for employees" as it was manufactured by an employee to give the company a good image. Its content is mostly a sweetened lie for good PR. There is a reason it was "'accidently leaked", and it's because it's not real.
Do you have any sources for this?
10:58 so it is true that TF2 players never look up! And it doesn’t just apply to TF2! Thats crazy!
One caveat to keep in mind with that "Test Often" advice is that Valve has more money than god and can afford to slow down its production to a crawl to playtest everything continually. Testing your game every week makes it incredibly difficult for a team to work on big features that take way longer than a week to prototype and implement. It puts enormous amounts of pressure on devs to bugfix at the same time as prototyping. It eats up a good amount of time each week as well to recruit playtesters, schedule them, watch them play, have meetings about the session, etc. So that frequency is not viable if you're a small team working on tight deadlines.
However it's necessary, otherwise you may end up wasting time on mechanics and features, that will be removed later in development. As main cause of exceeding deadlines is lack of precise directions and goals.
@@ceu160193 No other company than Valve tests that frequently. I wouldn't call it necessary. My point is it's only viable under a strict set of circumstances.
@@Selestrielle Because other companies consider it acceptable, if they have to scrap pretty much finished mechanics and features.
@@ceu160193 They really don't. You have no idea what you're talking about.
I love the actual quote saying the process is repeated until its no longer “excruciatingly painful” to watch the playtests, because i can definitely relate to the pain of watching someone try to play a game that im fluent in, and being like “why dont they get that they just have to do this thing and then this will happen??” And i assume that goes double for the people who actually made the game
This reliance on playtester feedback can bite them in the ass though sometimes. Namely how they neutered the Director AI in the L4D games. Particularly the dynamic level pathing in L4D2. When initially designing the sequel they had a very robust AI pathing system where most levels the director could pick and choose from 3-4 paths through the level based on how well the team was doing(short for struggling teams, longer for pro teams). But apparently, lots of play testers found this very confusing on repeated play throughs so they scrapped almost the whole thing. Only remnants of it are in three levels that can change, but are not dynamic(it is randomly chosen when the map loads).
As somebody who played a fair bit on L4D2 I find the fact they removed something entirely based on feedback frustrating. They should have left it as an optional feature or even only usable by the AI on higher difficulties. shame really how that played out.
Similarly, the only reason they retconned the original ending to _Portal_ so that Chell doesn't really get to escape, is because when they were working on the sequel, their dumbass testers didn't remember that ending and were confused as to why they weren't playing as Chell anymore.
I needed this video. I got my brother to playtest a game I'm making a few days ago and I felt like crying lol. I thought the whole game was intuitive and it was impossible to get stuck. But no he did everything the most incorrect way possible. I'm glad it's not just me and it's just a part of game dev.
I guess they forgot to playtest during Artifact development.
Great video, buddy! I love learning about the behind the scenes of things that interest me like Valve/Portal.
Valve's secret weapon:
"We don't fucking make games anymore."
That's Valve's secret weapon.
Or update them.
HLA
@@EmptyMag a pointless tech demo for 1% of 1% who have VR gear.
Call me when it's made into a proper game that the full fanbase of HL can enjoy (note if I sound irritated, it's not at you, it's at Valve).
@@thedungeondelver It is a proper game
@@EmptyMag I respectfully disagree.
03:11 if that is not the most GIGACHAD move by Gabe, i dont know what is
The sad thing is that Valve doesn't make more games.
I remember finding out about viewfinder towards the start of college when i got into game dev. It was a really cool concept and to find out it was inspired by the game that was meant to be portal 2 makes it that much better
Valve been playtesting Half-life 3 for 20 years now, i see
They've done so much playtesting they haven't even gone past the first stages of development.
Amazingly well prepared episode. Thank you. It just made me think of the `Lean Startup` paradigm developed by Eric Ries and presented in the book of that title. It talks about `build-measure-learn` feedback loop as the primary tool for development of any product in any business. It was conceived to protect entrepreneurs from building a product that nobody wants by testing assumptions early and adjusting to feedback. The only thing that doesn't change in this process is the vision, which, as mentioned in the video, keeps us from providing a 'bland, designed by committee, sludge'.
Antidisestablishmentarianism
💀
Thanks a lot for this video. I didn't know this is how Valve does it. Without knowing it, I did much of that myself on my current game - I religiously watched every streamer and youtuber and found SO much valuable feedback just watching other people play (and being unable to interrupt and correct them).