The Hidden Cost Of GraphQL And NodeJS

Поделиться
HTML-код
  • Опубликовано: 19 окт 2023
  • Recorded live on twitch, GET IN
    / theprimeagen
    Reviewed article: www.softwareatscale.dev/p/the...
    MY MAIN YT CHANNEL: Has well edited engineering videos
    / theprimeagen
    Discord
    / discord
    Have something for me to read or react to?: / theprimeagenreact
    Kinesis Advantage 360: bit.ly/Prime-Kinesis
    Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))
    turso.tech/deeznuts
  • НаукаНаука

Комментарии • 407

  • @neociber24
    @neociber24 8 месяцев назад +476

    I mean, in an interview someone said that NodeJS use a single thread and the interviewer didn't believe it, a lot of devs think promises are multithreading

    • @Fs3i
      @Fs3i 8 месяцев назад +69

      That would be scary as heck. Imagine all the concurrency issues that would crop up, oh dear.

    • @sergisimon9620
      @sergisimon9620 8 месяцев назад +10

      ​@@Fs3i couldn't they? If functions do not alter a global state then they should be able to be parallel, right?

    • @chainingsolid
      @chainingsolid 8 месяцев назад +10

      If your going to go async you might as well go multi thread... so I would too expect it to be multi threaded.

    • @Fs3i
      @Fs3i 8 месяцев назад +18

      @@sergisimon9620 That's a big if. It's very easy to alter global / shared state, and then you deal with race conditions.

    • @aeggeska1
      @aeggeska1 8 месяцев назад +18

      Yep. I had that happen this week. A dev thought that "async / await" was multithreading. Scary.

  •  8 месяцев назад +225

    Senior JavaScript developer had resolved this already - he had callback hell, then rewrote everything to Promises. Await came out, and then rewrote back to Promises. Then rewrote everything back to callback. *Because it is just more comfortable* 😅

    • @0xDEAD_Inside
      @0xDEAD_Inside 8 месяцев назад +15

      The second sentence should have "await" at the end.

    • @juliocesar-fc1nm
      @juliocesar-fc1nm 8 месяцев назад

      @@nosferratu Eveeerythiiiiiing. I wish I could post the mem photo hahah

    • @kevinflorenzdaus
      @kevinflorenzdaus 8 месяцев назад +13

      haha sounds like the video from "Programmers are also human"

    • @wbtittle
      @wbtittle 8 месяцев назад +2

      Async/Await lesson curves.
      Make sure you put async in front of the function definition and AWAIT in front of where you want the code to stop until the await happens...
      Cool..
      Then you are debugging the transpiled code. HOLY F... Look at this... callback helll...

    • @rossbagley9015
      @rossbagley9015 8 месяцев назад +2

      Senior JavaScript developer learned one of Java, C#, Go, or Rust and implemented his server-side code in a language more appropriate for the problem.

  • @BlackAsLight448
    @BlackAsLight448 8 месяцев назад +107

    JavaScript will try and run async code as sync code until it hits a point that forces it to the back of the event loop. Meaning if you attach the async keyword to a function with no await keyword inside it, it will run in sync and never be placed at the back of the event loop.

    • @mattymerr701
      @mattymerr701 8 месяцев назад +10

      A different way to say it is that it is cooperative multitasking. It only switches when the user code tells it to switch. It won't pre-empt that switch.

    • @huge_letters
      @huge_letters 8 месяцев назад

      Not sure if I like it or hate it - it's probably good for perf but boy is it unintuitive sometimes.

    • @fardelian
      @fardelian 8 месяцев назад +10

      This is an extremely important detail, thanks for pointing it out. In the case of async functions which cache their response, the overhead is the JS engine creating a Promise object. Even if you write your code using async, if it can be executed in sequence without deferring to to the task queue, then it will be executed in sequence. Prime hates JavaScript, so he doesn't learn it and he doesn't understand it but he talks a lot about it...

    • @huge_letters
      @huge_letters 8 месяцев назад +20

      @@fardelian Prime understands JS very well and has been writing it professionally for years, in performance sensitive environments. He can still have misconceptions but saying that he doesn't (want to) learn it is absurd.

    • @spicynoodle7419
      @spicynoodle7419 8 месяцев назад +4

      Not quite. If you return a promise form a function and don't await it, the promise will be placed on the back of the queue. So the rest of your code can't depend on the result of the promise. If you don't await the promise it's going to be a pure side-effect. Kinda useless, might be better to use background workers that run in a separate thread

  • @user-iv5sl2dn3r
    @user-iv5sl2dn3r 8 месяцев назад +14

    Thanks man , this helps a lot . We always got level up after watching your videos and you provide very good quality content . Anyone can create a video reading a blogpost but you discuss about it very well , explains the topic by keeping beginners in mind and taking about pros and cons of it .
    I rarely write comment but this is worth it 👍

  • @tharsis
    @tharsis 8 месяцев назад +111

    To be fair, if SQL was actually SLQ, then it could be pronounced 'Slick' which I'd be a bit more tempted to use in speech than 'Squeal'

    • @0xDEAD_Inside
      @0xDEAD_Inside 8 месяцев назад +10

      *Sequel

    • @EricSampson
      @EricSampson 8 месяцев назад +1

      or “sleek”

    • @vcokltfre
      @vcokltfre 8 месяцев назад +3

      @@0xDEAD_Inside *esquill

    • @FlanPoirot
      @FlanPoirot 8 месяцев назад +4

      yeah it's sequel not squeal, idk why he pronounces it like that

    • @rafaelbordoni516
      @rafaelbordoni516 8 месяцев назад +1

      Call it squirrel 🐿️

  • @iulianbojinca7309
    @iulianbojinca7309 8 месяцев назад +10

    Bad programming is bad programming. Using callbacks can lead to callback hell with bugs and impossible refactoring strategies, while Promises lead to event loop starvation and slow performance. But honestly, I can deal with a misplaced async situation much more quickly than the callback hell.

  • @aotein
    @aotein 8 месяцев назад +4

    There's another cost: Not knowing computers or working together with other software ecosystems. I had to work together with another company, which based their GraphQL API onto a standard SQL server. They didn't know how to access the GraphQL-API without a framework (like the URL). They didn't know how documentation works at all. Their error handling was non-existing. Their database structuring was just one giant JSON and little thinking was done about how to access certain data points. I think the sole focus onto JS/TypeScript and resulting frameworks really hurts the knowledge of what computers actually do or how other software ecosystems should be able to interact with yours.

  • @jaysistar2711
    @jaysistar2711 8 месяцев назад +104

    Promises are not inheritly slower, just the implementation. C++ promises can be optimized away completly. Rust's futures are optimizable as well. If JavaScript is your problem, you shouldn't change styles to make your code unmaintainable; you should change your language.

    • @razhemo4191
      @razhemo4191 8 месяцев назад +22

      surprised this isnt the top comment, this place is too full of frontend developers 😅

    • @Dozer456123
      @Dozer456123 8 месяцев назад

      @@razhemo4191 You gotta make sure everyone's included

    • @asdqwe4427
      @asdqwe4427 8 месяцев назад +12

      The discussion was about JavaScript promises…

    • @razhemo4191
      @razhemo4191 8 месяцев назад

      @@asdqwe4427 why limit the discussion?

    • @dhanushc5744
      @dhanushc5744 8 месяцев назад

      @@asdqwe4427 " If JavaScript is your problem, you shouldn't change styles to make your code unmaintainable; you should change your language."

  • @tanmaytiwari2450
    @tanmaytiwari2450 8 месяцев назад

    Really good explanation we need to keep this in mind. I had one thought how are Observables and Subjects vs Promise

  • @LucasLocatelli
    @LucasLocatelli 8 месяцев назад +8

    Funny that most of the issues pointed were not about graphql...

  • @thewiirocks
    @thewiirocks 8 месяцев назад +1

    You are correct that writing SQL is better than ORMs. Ted Neward explained the problem pretty well in his article, "The Vietnam of Computer Science".

  • @aprilmintacpineda2713
    @aprilmintacpineda2713 8 месяцев назад +1

    13:00 Hmmm not sure if I remember it right but I’ve tried this before, async functions that doesn’t perform an await operation does not get pushed back the queue, they act as though they aren’t async, the only time when the function gets pushed back the queue is when they do await.

    • @SurenEnfiajyan
      @SurenEnfiajyan 8 месяцев назад +1

      But when they return, they wrap the return value in a promise, so at least one microtask is added. I think the morale is to avoid using async functions if you don't need to. Async functions are fine for slow IO operations such as db queries.

  • @grimm_gen
    @grimm_gen 8 месяцев назад +6

    Ultimatly it all depends on what project you are working on! I have a preference for GraphQL over Rest

    • @dhanushc5744
      @dhanushc5744 8 месяцев назад +2

      I think the point of video is graphql is slow doesn't matter your preference or the project. Its very easy to write bad code with gql/js.

    • @deidyomega
      @deidyomega 5 месяцев назад

      @@dhanushc5744 gql isn't slow, maybe apollo's implementation is slow, js is slow, but gql isn't slow. There are rust implementations of gql that are crazy fast, and don't have the same insane mem overhead.

  • @JeremyKolassa
    @JeremyKolassa 8 месяцев назад +4

    "When your tracking software slows down your entire universe, that's not good." Best description of the Heisenberg Uncertainty Principle?

  • @ES-eb6pb
    @ES-eb6pb 8 месяцев назад +40

    looks like 90% of people who learned graphql use it completely wrong! You should never do that for each loop and fetch each element of the array from IO! What you should do instead is Retrieve the query object the user sent, check if the array field is not undefined, and then build an SQL string that is performant and fetches what the user exactly requested. JUST CALL THE DB ONCE NOT A DOZEN TIMES

    • @asdqwe4427
      @asdqwe4427 8 месяцев назад +19

      And 99% don’t need to be using graphql at all

    • @bilbobaggins7075
      @bilbobaggins7075 8 месяцев назад +6

      @@asdqwe4427 that's like saying people shouldn't be using rest. it's just a standard, low understanding comment

    • @asdqwe4427
      @asdqwe4427 8 месяцев назад +9

      @@bilbobaggins7075 graphql is hard to get right. The flexibility is really cool if you have tones of different consumers. But most people have one frontend and then it’s way simpler to make simple api that caters to the use case the that you do have. Don’t make some generic solution that’s hard to make scale.

    • @svetlinzarev3453
      @svetlinzarev3453 8 месяцев назад

      Not possible when nested objects with paging is involved. That `... FETCH NEXT xyz ROWS ONLY` really kills it.

    • @gabrielspeed5464
      @gabrielspeed5464 8 месяцев назад +3

      ​@@bilbobaggins7075no it's not...
      GraphQL is not a standard, it's a spec. GraphQL does not conventionalize anything..
      There is no reason graph QL has it's own query language instead of just being a REST API accepting JSON ASTs..

  • @HrHaakon
    @HrHaakon 8 месяцев назад

    The two selects he's referring to at 9:51 are the SELECT stuff FROM users WHERE id = ?, and SELECT stuff FROM items WHERE userid = ?.
    So he's saying that you can do that roundtrip in < 100 ms, which is true, but if it's only 10 000 items, then the ping-ponging to and from the db is a significant source of time.
    I remember when I did 500 inserts and THEN committed, vs doing 500 insert + commits, and how the whole thing went from seconds to milliseconds just like that. In my defence, I tried, saw bottleneck, and THEN made the batch inserts, which are a bit more complicated in that the code will look a bit different.

    • @gwaptiva
      @gwaptiva 6 месяцев назад

      I wonder how big this application's access log is.. and how big the disk it lies on.

  • @BobFrTube
    @BobFrTube 8 месяцев назад +5

    Since I mostly write code that is not extremely performance-sensitive, I have no problem using JS (or TS). In the day, I was a big C# user and appreciated values (integers, structs, etc.) didn't need to use the heap. Struct, in particular, is a useful way to reduce GC. I'm keenly aware of the cost of promises because, before promises, I implemented my own versions (in JS and C#) by creating an object to hold the pending state). How much are today's programmers aware of how these mechanisms are implemented? And why, for much code, readability is more important, but not always.
    As go graphql -- I've found SQL simpler, but that's another conversation.

    • @Myself0094
      @Myself0094 8 месяцев назад +1

      Nowdays C# makes some kind of black magic with Task state machine and even more one with ValueTask. The latter is specifically optimised to have minimal perf impact on cases where value is returned synchronously.

    • @gro967
      @gro967 6 месяцев назад +1

      GraphQL and SQL have nothing in common, you can’t compare them…

  • @mosescosme8629
    @mosescosme8629 5 месяцев назад

    That last rant about that code snippet was one of the most important things I think I've ever heard.

  • @yowwn8614
    @yowwn8614 8 месяцев назад +16

    That one time I used graphQL (unwillingly), after 4 months into development, I resigned because stakeholders didn't believe it was the bottleneck.

    • @justADeni
      @justADeni 8 месяцев назад +5

      did you work at a startup?

    • @deidyomega
      @deidyomega 5 месяцев назад +1

      Graphql is only the bottleneck if you (or someone on your team) wrote a shitty gql server. Same with client side code.
      You can write a gql server in rust, with blazing fast speeds and none of the mem overhead. You can consume that graphql without any library, if all the libs are too slow. Or change your client lang. Write your gql client in rust, compile to wasm, then expose the entire thing to js as the glue if you need.
      gql is just a language, nothing in the language requires slow code.

  • @wbtittle
    @wbtittle 8 месяцев назад

    Thank you. This explains so many of my current issues...

    • @wbtittle
      @wbtittle 8 месяцев назад

      Cut and Paste might be ahead of this... But when people cut and paste async await with tracing all over the place...

  • @asoftyn4805
    @asoftyn4805 8 месяцев назад +2

    What if you loop the fields in the querry but do smth like a querry builder so we only get 1 promsise for all of the required data?

  • @IhsanMujdeci
    @IhsanMujdeci 8 месяцев назад +9

    I was conducting some interviews in my previous job. I had a few questions to go over to see how well they understood js. Most of them couldn't tell me how you can perform async tasks without promises. Callbacks are a forgotten technology for most devs in their 20s.

    • @JacobBogers
      @JacobBogers 7 месяцев назад +1

      Most coders are not coders, without a framework they are lost

    • @gabrielspeed5464
      @gabrielspeed5464 7 месяцев назад +3

      @@JacobBogers you're using Dart complaining about frameworks on a topic where frameworks aren't even involved (promises are native in JS)
      You don't even want to use Kotlin and Swift directly. The irony here is so sweet

    • @JacobBogers
      @JacobBogers 7 месяцев назад

      ​@@gabrielspeed5464 "you're using Dart complaining about frameworks"
      That is very trolly and dishonest claiming things I never even implied let alone said. But what to expect from an arrogant lazy POS, who cant even code.

  • @PwrXenon
    @PwrXenon 8 месяцев назад +6

    JavaScript on the backend is only cost

  • @dabbopabblo
    @dabbopabblo 8 месяцев назад +34

    I avoid chatty protocols by just anticipating every action a users going to take before they even enter my site and serve them everything in the database in one request and make every possible change they might make at once before returning. Can't have a chatty system if you only use 1 request.

    • @deidyomega
      @deidyomega 5 месяцев назад

      No joke we actually did this for a client once. On site load, download the entire user's data locally. For our very weird use case, it actually ended up being faster.

    • @chunkspiggle3916
      @chunkspiggle3916 5 месяцев назад

      ​@@deidyomegaThat's crazy, I'm rewriting an app that did not only exactly that but also leaked every other user's full data in every response from the backend (including their [encrypted] passwords!!! You could see it all by just hitting F12!) Changing that, on top of the UI changes my client wants, would require a rewrite of 90% of the codebase anyways so I'm redoing this horseshit in go+htmx+templ now lmfao

  • @CarlosEstebanLopezJaramillo
    @CarlosEstebanLopezJaramillo 8 месяцев назад

    These issues can be addressed based on the database, for example GraphQL has a graph structure, Neo4j is a graph DB, the Neo4j-GraphQL library transforms GrapQL queries into Cypher (SQL for graph DBs) queries, since Neo4j takes care of the whole query instead of GraphQL resolvers chatting, it removes the "chatty protocol" part, or the "n+1" issue common in GraphQL, though not sure how this could be fixed in non graph DBs...

    • @blender_wiki
      @blender_wiki 6 месяцев назад

      N+1 issue can be solved easily by hiring a descend developer or using a good ORM if you can't find one.

    • @CarlosEstebanLopezJaramillo
      @CarlosEstebanLopezJaramillo 6 месяцев назад

      @@blender_wikiIndeed, just saying Neo4j has the solution embedded in their GraphQL library, so I don't even need to think about it :)

  • @Kurimson
    @Kurimson 8 месяцев назад +2

    What library would you suggest over express? I've always thought of express as the standard go to and never questioned its performance. I mean everything I've ever used it for seemed fast.

    • @maltbiscuit9995
      @maltbiscuit9995 7 месяцев назад

      Yeah that's what I am thinking. I got a nice React, Apollo and Mongo stack going on here. It's a great package to bring stuff to market and try things out.

    • @jeffreysmith9837
      @jeffreysmith9837 7 месяцев назад

      It is the standard for node but node sucks. Poor performance, fake types, and has no backend ecosystem.

    • @AmatuerHourCoding
      @AmatuerHourCoding 4 месяца назад +1

      Nestjs with fastify. If performance is that serious, golang.
      If you aren't having performance issues then don't worry about it 🤷
      Instagram runs on Python so its not that serious

  • @AqgvP07r-hq3vu
    @AqgvP07r-hq3vu 8 месяцев назад +4

    "Are callbacks really async if your date never does them?"
    He got me

    • @ark_knight
      @ark_knight 8 месяцев назад

      I mean thats definitely not in sync, or else there would be a callback. from your date.

  • @LurajerLP
    @LurajerLP 8 месяцев назад +5

    Graph Kwil - You're a mad man

  • @rabbitcreative
    @rabbitcreative 8 месяцев назад +2

    20-years in the industry and all I have to say is, "I told you so".

  • @gonzalomunoz2767
    @gonzalomunoz2767 8 месяцев назад +3

    Maybe I'm misunderstanding something basic here, but isn't all async code basically the same under the hood? Callbacks will also get pushed to the tail of the event loop and actually get executed whenever is their turn, won't they?

    • @jeffreysmith9837
      @jeffreysmith9837 7 месяцев назад

      Promise object has an overhead, idk the details

  • @rodrigolj
    @rodrigolj 8 месяцев назад +2

    “My favorite type of tweet is the one where everybody loses.”

  • @jimhrelb2135
    @jimhrelb2135 8 месяцев назад

    Callbacks as continuation-passing style (CPS) + sane compiler = Callback heaven (ref: tail call optimization and cps basically turns unbounded recursion -> iteration)
    Callback as CPS mechanism + no actual compiler = callback hell

  • @user-vl4gh7pt3d
    @user-vl4gh7pt3d 8 месяцев назад

    What is the name of the drawing software he was using?

  • @neptronix
    @neptronix 8 месяцев назад

    Interesting, i was wondering why nodeJS is 3x slower than PHP with swoole, despite the fact that it was built for async first, and async behavior in PHP was very after the fact.
    That's some serious computational pork!

  • @Elkemper
    @Elkemper 7 месяцев назад +3

    You man, i discovered your videos just few weeks ago, and i wanna thank you. I'm quite hardened SDET and working with JS|TS more than 5 years (and few years on other langs before), but most of the time i worked as a lead on the-only sdet on a project, and i never had a buddy, with whom i can discuss js-related stuff, more to say, really deep-knowledged buddy. And your videos for me - it's a great pleasure to just hang out, learn very interesting stuff which i presumably will never use in my work , but it expands my view and understanding of the lang and tools.
    i hate reading tech literature, so everything that i've learned - is just experimenting and trying to code what is in my head.
    and i think i'm not the only one in this situation, and i hope more of someone like me will find your content helpful
    So please, explain more, show more, rant, but show how do better.
    Thank you.

  • @SlawomirBudzynski
    @SlawomirBudzynski 7 месяцев назад

    If it's about optional async function it doesn't work as you've mentioned at least in Chrome browser. I was a little surprised this some time ago when I've found that async doesn't equal setTimeout(cb, 0).
    async function test() {
    console.log(1);
    };
    test();
    console.log(2);
    Will return:
    1
    2

  • @TheRcfrias
    @TheRcfrias 3 месяца назад +1

    is there any evidence which confirms that promises-async/await are slower the callbacks?

  • @jerrygreenest
    @jerrygreenest 3 месяца назад +1

    «GraphQL gives you a promise of easy...»
    But the promise part is hard, it adds some CPU overhead, it seems.

  • @anlumo1
    @anlumo1 8 месяцев назад +22

    If you care a lick about performance, why the hell would you use JavaScript in the first place?

    • @etherweb6796
      @etherweb6796 8 месяцев назад +3

      Combine that with object prototype pollution and eval exploits it is just not a good language for server side at all. (And I like JS in the browser) It might even be worse than using PHP - if it has filesystem access.

  • @maplepenguin8568
    @maplepenguin8568 8 месяцев назад +1

    Are Promises without using the async and await keywords still problematic?

    • @pkop4
      @pkop4 8 месяцев назад +2

      Yes, async await is just syntax sugar for promises. As he described every ".then()" is incurring the costs he's describing as well

  • @jfftck
    @jfftck 8 месяцев назад +1

    So, the fix is to implement a new task keyword that does real threading to functions in JavaScript.

  • @pedroaugusto656
    @pedroaugusto656 8 месяцев назад +1

    Sorry but how aren't promises just callbscks ?
    I always thought they were the same kust written different

  • @mattius17
    @mattius17 7 месяцев назад

    The rant about "x" reminds me of Yahzee (of Zero Punctuation) calling tweets "excrements" now 😅. Epic video Prime

  • @hanabimock5193
    @hanabimock5193 8 месяцев назад +3

    it is incredible that this channel monetises reading blogs.

  • @JarrodNotJared
    @JarrodNotJared 8 месяцев назад +1

    do a vid where you optimize the code you showed at the end

  • @hannessteffenhagen61
    @hannessteffenhagen61 7 месяцев назад +1

    With the memory thing, this is why garbage collection gets such a bad rep. It's not that garbage collection is slow. It's creating all that garbage in the first place that's making things slow! The garbage collection part is just where you end up realising the cost. Kinda like blaming some code for dereferencing a nullpointer when it shouldn't have been passed one in the first place.

  • @alex_nita
    @alex_nita 7 месяцев назад

    14:35 what site is that?

  • @lionbryce10101
    @lionbryce10101 8 месяцев назад +3

    Aren't Promises just roided out callbacks? Like Object-Oriented callbacks.
    Except roided out ends up meaning way slower

  • @freeideas
    @freeideas 8 месяцев назад +1

    Or you can use java virtual threads, which are simple -- i.e. no promises, no callbacks; no async/await, and get all the scalability benefits of microthreading. Instead of constructing C so can give it to B as a promise or callback, then give B to A; in java you just do A,B,C.

  • @Ked_gaming
    @Ked_gaming 8 месяцев назад +1

    There's so many things wrong with the approach I don't even know where to start

  • @renatocustodio1000
    @renatocustodio1000 8 месяцев назад +1

    Isn't this basically the n+1 problem at an API level?

  • @fredrik8737
    @fredrik8737 8 месяцев назад

    NodeJS threads comes from making a another request to another NodeJS instance. Don’t do all the the work in GraphQL resolvers if that’s slow, feature flag, auth etc

  • @StingSting844
    @StingSting844 8 месяцев назад +2

    Isn't this solvable using Dataloader?

  • @adesanoyesamson668
    @adesanoyesamson668 6 месяцев назад

    Am on the team write SQL instead of using ORM. use ORM if performance not involved

  • @jackdanielson1997
    @jackdanielson1997 8 месяцев назад +2

    Oh no, I might get a 5ms delay if I'm getting nested data for 100 users. Shucks, I guess I should optimize that when the db queries are taking up 400ms and the network latency is 100ms

  • @aelsayed313
    @aelsayed313 8 месяцев назад +1

    Arent promises resolution microtasks? In the video he illustrates that they are tasks

  • @Saiphes
    @Saiphes 8 месяцев назад

    "Functions are people" -- primeagen, probably

  • @magfal
    @magfal 8 месяцев назад

    6:46 i wonder how Graphile would perform in comparison

  • @Brian-ro7st
    @Brian-ro7st 3 месяца назад

    As a node dev, I don’t claim anyone who prefers promises to async/await

  • @jordixboy
    @jordixboy 8 месяцев назад

    what would be the criteria to make a function async?

    • @dhanushc5744
      @dhanushc5744 8 месяцев назад

      when you want to run a blocking code like Db/network calls etc

  • @georgehelyar
    @georgehelyar 8 месяцев назад

    Do JS promises really continue asynchronously even when the promise is resolved immediately?
    I'm not a js dev (thankfully) but when I've actually needed that behaviour in the past I've had to use a setTimeout(0), even with promises.

    • @SurenEnfiajyan
      @SurenEnfiajyan 8 месяцев назад +2

      When you resolve the promise immediately, the callback passed to then() method is not called immediately.
      Promise.resolve('async').then((v) => console.log(v));
      console.log('sync');
      this code will first print 'sync' and then 'async'

    • @outwithrealitytoo
      @outwithrealitytoo 4 месяца назад

      They don't "continue" they pause and cede control to the main loop, before being restarted when they hit the top of the queue. It has the advantage of stopping the processing getting lumpy at the cost of performance. i.e. on average responses take a little longer, but none egregeously so. Lumpy (jitter) data processing is painful because it shows with the frequent short messages taking way too long, rather than with the occasional long running culprits.
      ( it's analogous to a sleep(0) if you are using a real-time micro-kernel.)

  • @felipedidio4698
    @felipedidio4698 8 месяцев назад +2

    Oh, I get it. If you write garbage code, the garbage collector has to work harder.

  • @voidreact
    @voidreact 8 месяцев назад

    you gotta stream an sc2 session sometime dude

  • @Pekz00r
    @Pekz00r 7 месяцев назад

    Why would you ever return more than 100 items to the client from an API?

  • @huge_letters
    @huge_letters 8 месяцев назад

    13:27 - is that true tho? When file write is done the callback would still get added to the task queue, no? Otherwise it would break code execution if there're other tasks running at the moment - it still has to wait until at least the current task is finished until executing a callback. Otherwise if say I'm executing task1, register a callback for my file write, task1 gets finished, file write is still executing - does that mean now it won't just start executing task2 and will wait until write finishes to call my callback?
    So I don't really understand the difference here between a promise and a callback - unless what you mean is that callback will get added to the start of a queue and will be executed as a next task but the promise will get added to the end of queue and might wait for quite a while until it gets its turn.

    • @dealloc
      @dealloc 8 месяцев назад

      The difference comes down to the user agent. Promises are microtasks, just like callbacks. But callbacks can be queued differently, depending on what kind of callback it is.
      In Node there are callbacks for different phases of the event loop; timers, pending callbacks, poll, check and close callbacks ( socket.on('close', () => {})
      As you can see there isn't one place where a callback is called, but it depends on the callback and the current phase that the event loop is in. This is the macrotask queue.
      Promises (and process.nextTick(), queueMicrotask) are executed in between these phases. This is called the "microtask" queue. Microtasks have higher priority and are always executed before the next macrotask in the queue.

    • @huge_letters
      @huge_letters 8 месяцев назад

      @@dealloc I see, but since microtasks are executed before the next macrorask and promises are microtasks - wouldn't that mean they would be executed before callbacks(which would go to macrorask)?

  • @W1TEA
    @W1TEA 8 месяцев назад +7

    yet another 'I built a benchmark that shows.....writing with chisels on stone tablets is faster than Rust, Go, C, and Commodore Basic"

  • @chauchau0825
    @chauchau0825 15 дней назад

    I feel sad it took that many years for people to figure this out

  • @rossbagley9015
    @rossbagley9015 8 месяцев назад +1

    So, isn't the conclusion to use a more suitable server-side language for GraphQL/REST/complex business logic/everything on the server side? Unless you're trying to bang out some sort of prototype or tiny one-off site...

    • @outwithrealitytoo
      @outwithrealitytoo 4 месяца назад

      The lesson is, I believe, use technologies that allow you to do more work in the database. What is killing Node here is the chatty protocol, but the chatty protocol is the primary problem. This leads Node to bow out early in the scaling game but alternative solutions based on the same protocols only scale in the sense that "you can pay for double the compute and ten times the network capacity than you could if you didn't use naive database abstraction libraries, but it will at least work (unlike Node)". Node is a canary in the coal mine for the underlying issue.

  • @flamendless
    @flamendless 8 месяцев назад +1

    I wonder if this dataloader promises issue also affects django + graphene

    • @pbkobold
      @pbkobold 8 месяцев назад +1

      Oh it does. Badly. Every time I have to debug the implementation’s ‘promises’ port to Python, it feels like getting hit in the head.

    • @flamendless
      @flamendless 8 месяцев назад

      @@pbkobold so more or less it has something to do with the implementation of async (event loop)? In my case we use gunicorn. I'll read about the implementation of that

  • @idkidk9204
    @idkidk9204 8 месяцев назад +1

    Meanwhile me happily using threads, async and such noice stiff like this in c# without any drama

  • @lulagoodwin5372
    @lulagoodwin5372 8 месяцев назад

    by the time they fix this there'll be another framework out where this isn't even a problem

  • @mateogalic7128
    @mateogalic7128 8 месяцев назад

    Squeal got me every time 😂

  • @MuhammadTajammulZia
    @MuhammadTajammulZia 5 месяцев назад

    Sarcasm so strong I can never surely say which technologies he works with..

  • @elomatsouagboka6844
    @elomatsouagboka6844 7 месяцев назад

    If express is that bad as a framework, what do you suggest us to use instead? In node.js ecosystem of course.

  • @IS2511_watcher
    @IS2511_watcher 8 месяцев назад +1

    The hidden cost of NodeJS? Your soul.

  • @mileselam641
    @mileselam641 8 месяцев назад +10

    "But GraphQL is complex!"
    Use Postgraphile
    "But the SQL it generates is slow!"
    Use Postgraphile
    "And avoiding N+1 in resolvers is hard!"
    Just use Postgraphile

    • @varshard0
      @varshard0 8 месяцев назад +1

      But then you would be programming everything in SQL with Postgraphile and putting business logic in SQL.
      Also anytime you want to make a tiny change to your function, you would have to create a new migration file and make it hard for a version control to show what's actually changed.

    • @mileselam641
      @mileselam641 8 месяцев назад

      @@varshard0 Postgres supports idempotency in DDL.
      CREATE OR REPLACE FUNCTION foo() …
      Put that in an SQL file and re-run as often as you need to. Always ends up in the same state. Structural changes to tables is harder, but that's no free lunch in any workflow.
      ALTER TABLE IF EXISTS foo ADD COLUMN IF NOT EXISTS bar int4 NOT NULL DEFAULT 0;
      Fast for adding the new column. Free when the column already exists. Idempotent SQL. There's your diffs and version control. DB schema *is* business logic.
      Algorithms have always taken a lower priority to data structures among top coders, and databases are nothing if not data structures.
      "But the API shouldn't leak the DB schema."
      1. 99.9% of the time, the API and schema are the same, especially with most ORMs.
      2. You can always just make a new schema and define views to reflect your API.
      3. Defining the DB schema bubbles up so you don't have to redefine records and types again at the API server level and yet again at the UI consumer layer.
      Starting in the middle of your stack compromises the whole stack; your DB structure gets compromised by the impedance mismatch, and your development cycle gets compromised by writing all your types manually in triplicate. You're right that Postgraphile introduces some complications to typical dev workflows, but let's not pretend our current dev workflows are smooth and seamless as they are. Better to err on the side of clean and uncompromised data structures.
      But if Postgraphile isn't to your liking, there's also Hasura. Point is, life's too short to be writing object definitions in triplicate only to hamstring you DB's performance. Focus on the DB structure, and you've fixed most performance and workflow problems before they even start.

    • @mileselam641
      @mileselam641 8 месяцев назад

      @@varshard0 "Bad programmers worry about the code. Good programmers worry about data structures and their relationships." - Linus Torvalds

  • @natedavidoff668
    @natedavidoff668 8 месяцев назад +1

    At this point programmers don’t really know how anything they are coding works behind the scenes.

  • @AshtonMotana
    @AshtonMotana 8 месяцев назад +2

    Maybe stop using foreign keys to join nested items with a parent data model. Instead, capture the parentID in the item and find all items by the patentID field, and directly query the items. No need to query the user just to get the users' items. Doing this then removing .populate('userItems') from the user resolver will speed up the server respons. My limited experience and this observation tells me to avoid joining on nested lists.
    I had a query contacting a resolver that had. populate('userItems') the GraphQl query wasnt requesting the items and ther server wasn't sending the item. The .populate still causes your server traverse the data and smow its response to the client.

  • @pedropertino7324
    @pedropertino7324 8 месяцев назад +2

    hey man i've always loved your content, but i think your reasoning about promises is wrong, that is the bases of concurrency (sharing the resources instead of hoarding them), you would have the same issue with callbacks (even if things are not actually async, you would stil steal the resource while doing your sync stuff and other requests would pay the price, you just moved the issue to somewhere else, not fixed it).
    my guess is (from what i've notices) that when promises are already resolved (the cache you mentioned, but also it can happen when reading a huge directory without throttling ) the list (which im pretty sure is an array) becomes too big and inefficient for pushing/popping elements from it.
    await also counts as 2 promises either way (each await), thats why `await null` does not throw, because you are technically doing `await Promise.resolve(null)`

  • @worgenzwithm14z
    @worgenzwithm14z 8 месяцев назад +12

    I think the problem here is using js on the backend, not as much graphql

  • @andreicalegari1837
    @andreicalegari1837 8 месяцев назад +1

    Even the people who made Express know how bad it is

  • @overbyte
    @overbyte 4 месяца назад

    Orms are like graphql - fast to get a prototype running but with massive tech debt later on in mature software

    • @outwithrealitytoo
      @outwithrealitytoo 4 месяца назад

      Controversially I'm going to say "it's not a speed thing, it's a skill thing". With ORMs/Graphsql you can get a naive prototype running with people with a light understanding of data and no SQL. However, as the transaction number and data volume increases, and as requirements change requiring data migrations occur; so it is that staff with greater knowledge of data are required.
      When it is time to scale, if you start with skilled developers and don't go down the data-framework rabbit hole then all you need is a bolt on DBA.
      If you went down the rabbithole then a) your devs have probably taken flight b) you're new staff are going to have the joy of migration c) a DBA won't be able to help much until the code is sorted.
      Sometimes "time to scale" is just moving from dev to prod ... insert burning house meme here

  • @thekwoka4707
    @thekwoka4707 8 месяцев назад

    The individual promise thing is definitely not sensible in the graphql context.
    It would be a bit different if it was like a component system where you want state for something as low as possible, so you might have many components making similar calls for different data that can be batched into a request. Where doing the extra request in a single promise from the top could be impractical or increase dataloading.

  • @fu886
    @fu886 8 месяцев назад

    i miss flash/java applets

  • @satyayuga0
    @satyayuga0 7 месяцев назад

    Wouldn't be EFCore (ORM) with LINQ be a 'slq'???

  • @clamhammer2463
    @clamhammer2463 8 месяцев назад +2

    I jumped on the async bandwagon and was ALL about it. EVERYTHING was async. Then my project got large, and the mental debt was so significant. ONLY use it when you truly need asynchronous code. It doesn't need to be and shouldn't be all or nothing. For example, if you are writing logs to a file, that can be async to not block the app's execution, which is a good practice. However, using async for every operation, especially in larger projects, can lead to issues like race conditions and increased mental overhead. Therefore, it's essential to use asynchronous code judiciously, where it provides a real benefit, and not make everything async 'just because.' This balanced approach ensures a more maintainable and efficient codebase.

  • @malcolmhutchison
    @malcolmhutchison 8 месяцев назад

    The Structured Languages for Queries (SLQ) should of course be pronounced as "slick" ;)

  • @u9vata
    @u9vata 3 месяца назад

    Omg, they lost me at not wanting to use JOIN in SQL by default but separate calls. I honestly start to think abstractions, styles and such were all bad ideas as they are "guidelines over thinking"... but one cannot really avoid thinking otherwise result will be bad....

  • @mattymerr701
    @mattymerr701 8 месяцев назад

    Mercurious. People who are interested in merpeople but not fully sure about it yet

  • @unforkableonion5081
    @unforkableonion5081 8 месяцев назад

    uh oh, did i hear linked list?

  • @leaf1208
    @leaf1208 8 месяцев назад

    If I hear this man pronounce "GraphQuel" one more time istg

  • @aybmab2
    @aybmab2 8 месяцев назад

    I missed the reference/joke about asking javascript devs to not write promises and them being mind blown 😬 can anyone explain please

  • @seancooper5007
    @seancooper5007 7 месяцев назад

    What's hidden about it?

  • @dickheadrecs
    @dickheadrecs 8 месяцев назад

    chroot, chroot, chroot is on fire

  • @hipertracker
    @hipertracker 8 месяцев назад

    The resolver doen't need to run several call per field. It can one query instead. Building own GraphQL endpoint is a waste of time. It's simpler and faster to use Hasura GraphQL server.

  • @sideone3581
    @sideone3581 5 месяцев назад

    Wait I wanted to use NodeJs and Graphql in my project
    but
    I am confused now
    can anyone tell me how you supposed and prefer to develop a large maintainable and easily scalable backend
    for something like social media platform like facebook ( for very start )

    • @fcnealvillangca7943
      @fcnealvillangca7943 5 месяцев назад

      Rest API first finish your app. Slowly migrate to graphql if necessary if not stay in rest API

    • @Nassifeh
      @Nassifeh 4 месяца назад

      You don't. Nobody builds the app that will eventually have 100 million users from day 1. Use whatever gets it done until you have money coming in. Facebook employs literally thousands of engineers; you can't just have what they have from getting one weird trick off the internet. "Easily scalable" is for teams working on new products for existing large companies.

    • @sideone3581
      @sideone3581 4 месяца назад

      @@fcnealvillangca7943 I am thinking about gRPC now , I am a dumb : I don't know any

  • @chirale
    @chirale 7 месяцев назад

    I'm concerned about NodeJS projects maintainability in the long run.

    • @dwiatmika9563
      @dwiatmika9563 7 месяцев назад

      Yes, it has the worst ecosystem by miles

  • @Tom-jy3in
    @Tom-jy3in 8 месяцев назад +1

    As a fullstack developer who writes backend code in Java, it baffles me that something that is supposed to run async is processed synchronously and single threaded in node...

    • @jeffreysmith9837
      @jeffreysmith9837 7 месяцев назад

      Isn't that normal? The event loop is single thread but async operations do work on worker threads.

  • @mble
    @mble 8 месяцев назад

    1:00 I wanna learn how to make async without promises! Can anyone recommend me some article about this?

    • @dhanushc5744
      @dhanushc5744 8 месяцев назад

      btw async functions implicitly return promises. so you still do .then or whatever

  • @maxlife459
    @maxlife459 8 месяцев назад +1

    19:30 heh, I typed that :P