What event queue would you recommend? We are currently doing a polling REST API for our longer running micro services and it feels "wrong" to continue this pattern while we are growing.
You're talking about reading data from different microservices so HTTP and gRPC are the correct way to go (if anybody is trying to read data by passing messages on queues then they're beyond help). I would also argue that gRPC isn't worth the complexity overhead if you're not transferring large amounts of data.
Also curious on what event queues you would recommend. Some I'm familiar with are Kafka for streaming data and Rabbit MQ. Totally based on use case. A video breaking down which queues to use when would be awesome.
I think it quickly becomes a mess when microservices talk to each other. You could do that with a message queue instead. Microservices can put messages into this queue and read/listen to messages, and suddenly your architecture's dataflow is way more understandable. If you think about it long and hard, the microservices talking to each other problem is similar to two-way data binding, and the solution I'm proposing is similar to what Redux does to solve the issue in a sense. Unidirectional Data Flows on the server^^
Oh I like that framing! It was a big ah-ha for me when I read the react-query docs (and reinforced by some of Thea’s videos) and shifted my thinking away from querying and to “state sync”. It’s a little more of a stretch to see an event bus that way, but could be very useful!
@@efkastner well yeah it's a stretch :D but if you have a service which is dedicated to reading the queue and mutating the database, then it could actually look like a reducer, that gets the current state from the db, and the "action" is the message. What you'd get out of this is very easy swapping of microservices, and replayability? Probably there are no race conditions anymore as well while writing the db. Never tried, though I'm just thinking in theory. (It feels like a bunch of unnecessary complexity, so maybe it doesn't make sense)
I think that gap that you show between "ServerClient" is a really good way to visualize that use case, and how useful GraphQL will be in comparison to the opposite use case when the server + client are "touching". I went down the path of using GraphQL a couple of years ago, and I can see it being great when dealing with multiple languages and/or dev teams, or with 3rd party APIs where I'm only the client. But for my big project where I'm the solo fullstack dev, and using TypeScript on both the nodejs backend + also the frontend... over the last couple of years I've slowly realized that I was just making heaps of extra work for myself dealing with 2 typing systems (TS + GraphQL) instead of just the 1 (TS). Plus all the extra complexity of having to write queries in another language, and all the extra tooling needed there. Overall it's slowed the whole project down a lot, without really any benefit for this project in particular. My use case in this instance was pretty much the opposite of when GraphQL shines. I'm just about to start removing GraphQL from this project. It's my own system (not for a client), and the frontend is still in its infancy... somewhat due to graphql slowing me down. I'll be taking a look at tRPC, and either using that, or building something customized for myself that has some similarities. Either way, my project isn't going to be any less typesafe than graphql in the end.
There’s a lot of tools that fix a lot of your problems. Such as nexus or type graphql to completely remove writing your code in the graphql schema language. Everything is kept in JavaScript with these tools. Which means your typescript is automatically created for you, and is the main thing you’re using
Thanks for making this video! I've been a graphql dev for 1 year and while I've gotten used to the mechanics of the system, I've never really understood the "why" behind it all. I work at an online retail company and this "middle tier" project was started more than two years ago and changed hands a couple times, so now people like me have no idea what the original point even was. This video helped me form a better idea of what it's for, and what my company is still missing in order to better leverage this technology. I might even be able to impress my manager by formulating a well-defined plan for improvements.
BTW, I am in a scrum group with front end developers and I hate it. I feel like team meetings are a complete waste of my time because they are only talking about React 99% of the time and I work on the Graphql server and resolvers. I'm going to make a new push to break up the stupid "full stack team" pattern we're using. It's stupid, at least for the way this company functions.
the between server communication is likely better dealt with as an event, posted to an event queue / log rather than a realtime connection that could / will fail.
Will repeat myself. Yes, you may use gRPC for microservices, there's no evil in it. Use event-driven architecture if you have very good reasons for it (having proven that synchronous doesn't work in practice *for your specific problem*, rather than just assuming it wouldn't or that it's somehow "not in the spirit of microservices"), if you have a damn good observability infrastructure and a strategy for dealing with transactions. Don't do event-driven because you've read it in a popular microservices book that takes a dogmatic stance on it, ignoring the enormous cost in added complexity and maintainability.
we use Hasura in production its is actually quite amazing... our backend is written in golang using the gqlgen library and we auto generate around 5M lines of compliant golang code because of the graphql schema Hasura provides... Theo you fail to understand that companies operate in different levels. If my company is worth 1B+ do you really think we would use Hasura? Of course not, its better to build your own infrastructure because you have the cash/manpower to do so. Hasura is fundamentally game-changing for startups unders ~25M and can easily handles thousands of concurrent subscriptions through Hasura Cloud. There is a reason Hasura is worth so much these days, and you fail to understand Hasura for what it truly is. You can easily eject out of Hasura because it does NOT interfere with your postgres database whatsoever
What even is the argument against Hasura? Or even supabase or whoever put gql straight into postgress? There's so many applications where 95% of the endpoints just pass data straight to/from the db. I genuinely don't understand why Hasura or some BaaS would be bad here
He called out Hasura for using GraphQL as SQL replacement on the backend, but then everything he described as the appropriate use of GraphQL can be done by Hasura. So Hasura is not the problem, only the problem is misuse of tools.
It feels good that the same arguments I had some years ago during an internal discussion, that GraphQL is probably best utilized as a gateway to your whole microservice clusterfuck rather than talking to the db's itself. The gateway also prevents the frontend developer to know about the slight differences in naming between the different microservices, as they only need to care about the contract GraphQL schema defines./
In case of distributing requests between microservices, GraphQL can address this on it's own via GraphQL Federation. The same applies for requests initiate from one service to another. Instead of calling the service directly it calls the federation. This removes the requirement for the service to know about each other. The only information that the service needs is where the federation gateway is. This allows you to easily merge and split services without need for application wide refactoring.
Hasura is literally a server deployed as a microservice. It IS the server between the database and the client so it falls right in line with your "acceptable use cases for GraphQL" diagram. Memes and hot takes that paint the whole world black and white are great for driving Twitter and RUclips engagement (here I am, engaging with your content) but at the very least you could try to do an honest review that adheres to your own "rules". If your take is that systems that have any kind of GraphQL -> SQL translation layer in their stack is evil at least be honest with yourself and viewers and say that instead of this nonsense about requiring a server in the middle.
This is a surprisingly good video given that I disagree with the two contentious points you make: 1. GraphQL shouldn’t be used directly in front of DBs. 2. GraphQL shouldn’t be used for inter-service queries. And here’s why: 1. The case for DB backed GraphQL APIs (e.g. Hasura) I think you're arguing against the idea of having micro-services use GraphQL to query their own data, and I 100% agree with that. However, the value of this tech when used correctly is as a rapid project accelerator that won't tie you into loads of tech debt down the line. Creating a schema directly in front of a DB is fine provided that the schema is a completely logical view of your domain, since in that case you’re free to change the implementation details later. Hasura makes this possible to do since you can create mapping rules that allow you to present your DB logically. Given how fast it is to create very performant services this way, it’s actually a really good option for start-ups and green-fields, and I’d particularly recommend it, since although it’ll allow you to get your tech. stack implemented quickly, the logical schema you (with care) end up, will mean you don’t get stuck with a monolith when it’s time to scale, and you have a really simple path to moving to something more scalable on the back-end as needed. You even say as much yourself, and I quote: > This is what I love about GraphQL. None of this back-end implementation detail exists to me anymore. 2. The case for using GraphQL for inter-service queries If the thing you really love about GraphQL is that none of the implementation detail need concern you as a front-end dev, then isn’t that an equally useful trait for other services on the back-end? Or, stated another way, if the benefit to this formal interface is the ability to change implementation detail provided queries flow through this interface, then isn’t it better that more of the traffic flows through it? You don't explicitly say why you disliked this pattern, but perhaps your objection is the performance hit from going through some additional resolvers. If so, then it’s worth noting that this doesn’t apply in the case that services are implemented as subgraphs, since in that case you can then either query them directly, or query them via a low latency router that adds no perceivable overhead. Thoughts?
Good video, good points. Ever since I learned GraphQL, it made me appreciate the art of crafting API's, and made me like to *start* with GraphQL schema's when designing new object-oriented systems. However, *my* very favorite feature of GraphQL is its introspection capabilities. I do wish the spec would "require" it to always be public, so that clients could reasonably employ tools that could dynamically generate UI's, or even new machine-to-machine processes, based solely on schemas..
I'd argue that the GraphQL client should always be considered as part of the frontend domain as most UIs are data-driven and frontenders knows best what data dependencies each component has. Over time frontenders needs to make changes to the frontend code, and then it'll lead to a much more streamlined and frictionless process if they adjust the graphql schema / resolver at the same time. If the frontenders needs to align and communicate with the backenders every time they need to make a change impacting the graphql sschema, it may risk leading to technical debt instead, as there's a big chance that a workaround will be taken instead if time is an issue. The graphql server as I see it is a BFF (Backend For Frontend), meaning its for moving complexity and business logic from the frontend/client/browser to instead handle and prepare it in the graphql server. Therefore there are very strong arguments for writing the graphql server in node.js (javascript, typescript, rescript, purescript or some other compile-to-js language). Other than that I agree about server to server comm should be gRPC, and that is is only intended as a thin interface/proxy without direct db connectivity... but the split between FE and BE is where I disagree. This is based on my experience from working this way.
I was told GraphQL is used over REST primary because you can tell it to "filter" the response and thus you get exactly what you want. The problem is, you can emulate this with REST too if you really wanted. Either in the query params or inside the body you can tell the server what you want then your REST api will filter it based of your request.
If you design your graphQl schema you can literally fetch all the data in your database or just a single scalar value or anything in between in ONE REQUEST. It took me a while to understand how powerful this ability is but once I saw it, I can't go back to REST! The key is all in the design of you graphQL Schema plus custom DataFetchers to solve N+1 problems!
A nice thing that you brought up here Theo is Data contract which is probably the most common problem that any software solution without proper planning face. Explicit schemas as contracts is a must. For the most complex systems, we even have dedicated versioned schema repositories nowadays thank god. GraphQL was one of the moves towards this in a more narrowed sense than when talking about architectures. Info near the definition, flexible and extensible metadata, all come very handy. Also a very bad practice I see a lot is to try to come up from the beginning with an ubiquitous data model that will not be needing updates any sooner (that is an illusion and probably it will become harder to cleanup those unwanted attrs and add useful ones ). My tip is, unless it is a very fundamental attribute of the particular entity wait until you implement the feature that will need the attribute. I get the creeps when i see those humongous entities that look like they came from an enterprise software of the 90s.
You should look into federated graphql. With that at the gateway each service maintains it's own schema. This provides a consistent interface for inter service requests.
Came here to say this. The use case he made where the payment service needs to update the user is perfect for a combination of stitching and federation. You make the payment, that mutation returns the user, and you just spread the user, which, at least in Relay, updates that state.
With time and missions done, I realize that not so many people are understanding GraphQL. Do you think it's linked to its "not well explained" documentation?
At work our app uses GraphQL stitching to combine Hasura with some custom NestJS APIs. We're in the process of porting to a single tRPC server and it is SHOCKING how much code we're removing. To be fair, a lot of that code is generated. However, the more significant change is to our dev process where everything is 10x easier.
Yeah I fell into the “wow Hasura is amazing” hype… but ended up having the same experience in the end: it just sugar coated the complexity, didn’t really deal with it. But it’s not bad! Just somehow I failed to find a really good use case for it even though I tried.
I went through Apollo’s free courses, which are super high quality, and learned GraphQL was not for the project I was working on. I’m really glad I went through the courses still, because now I’ll know when to use it.
With regards to microservices, GraphQL Federation can be used to provide a unified API while keeping services isolated; services define their schemas and ownerships. As for using GraphQL to communicate _between_ services, you'd be better off with using something like gRPC, Protobuf or other event queue systems alike. GraphQL only make sense when you want to expose APIs to one or more clients.
GraphQL for server to server is perfectly valid use case. I have used it in production with 7 microservice and it was a pleasure to use. Each service just needs to expose an internal graphql schema at say /internal/graphql that can be consumed by other microservices.
@@82TheKnocKY Yes exactly. This makes everything type safe. You can use graphql-codegen to make sure that all the queries are valid in CI so the contract enforcement can be automated. As far as GRPC is concerned, GraphQL can be just as fast if you use HTTP2 on server side (easy using a service mesh). The only saving using GRPC is that it has a smaller payload because of Protobuf encoding but the difference won't be significant.
@@gt4adolfo Event driven is not always good. For instance, if order service wants to make sure that the customer has a valid email, it should call customer service using HTTP and not Event. Event Driven is good for async communication.
@@quant-daddy is possible to achieve it using event driven... youre coupling microservices... for this reason exists Aggregation events. the same microservice have to check for example the "email". is not necesary coupling.
hey @Theo, heard about federated graphql? Have a look, it does let you have individual services also speak graphql, then use apollo router as your gateway (no code, just deploy), then your individual services can talk to other services through the router as well.
You didn’t give any real reasons why graphql server is bad. I recently started a project with aws and when looking for resources to connect my dynamodb tables to my front-end, aws-amplify came up among the first solutions with graphql the default. I think I agree it’s probably not the best route, but I like to know why.
Isn’t Hasura just auto-generating CRUD GraphQL (and recently REST) API, instead of querying DB’s using GraphQL? Also, can you share your thoughts on public GraphQL APIs?
I don't really agree with theo's position on this either, hasura has been all about having a low to no code backend in minutes for me. It solves the boilerplate nonsense of adding CRUD to your backend, and you can use actions for complicated logic. It's unbeatable when it comes to building something quickly You just define queries in your frontend, run graphql-codegen and use apollo-hooks, there, you're done One of the argument I've seen is that you should never expose your database schema "directly" to your client, without much explanation as to why. Well first of all you can use SQL views and functions and build your hasura graphql schema around that, and second: why not ? permissions in hasura are made to make sure your users see only what they're allowed to ... Anyway, hasura rocks, it will make your app building so much faster and it is ridiculous to dismiss it over some ideology, we've used and hacked technologies to use them where they were not supposed to for ever, won't stop now
Hasura is great for base CRUDs, an events driver to fire off serverless fns, while adding your own apis (more serverless fns) for business logic in highly efficient and no boilerplate ways. Did I mention serverless? And Hasura is the (graphql) gateway. Theo is right: He doesn't know what he is talking about. He like most are too stuck on microservices. Go serverless and event driven, then Hasura makes sense. PS. You also must hate Firebase. Me too, Firebase is a trap. Hasura is conceptually like Firebase as a BEaaS, but better in every way: complexity and performance scaling, transparent and open source, graphql ecosystem
Another thing that I find GraphQL helpful is with defined schema, graphql client (apollo) can automatically merge same objects from different response which help consistency in the UI without manually doing in that in reducer or in react-query callback. How do you normally deal with this when you are using trpc/rest
What sort of a question is this? REST APIs is a staple and it won't go away ... ever. Not until we remove HTTP. They are simple to consume and fast to build. IMHO most new projects should start with a REST API and if they have some scaling issues then they should consider GraphQL, tRPC. REST is the perfect fit for small teams. Just don't overengineer it with dumb HTTP semantics. I've seen overengineered REST APIs and they are not fun to work with. Also keep in mind that REST doesn't mean JSON. You can use REST APIs and pass XMLs to keep extra object information. The downside of JSON is that it loses a lot of context. Granted most of the time you don't need the said context. But it just goes without saying.
Graphql uses schemas, which defines a standardized contract between front-end en back-end. In REST you can also create schemas but you have to implement your own or use tools like OpenAPI. Another benefit of graphql is reducing data overfetching. In REST you might need to do several request to get data you need but in Graphql you just define the fields you need and make a single request
Theo can you explain why having graphql directly on a database is bad? Isn’t it just like a serverless implementation of graphql? I mean ultimately with hasura o assume there’s still a server in between the database and the client, or do I assume wrong?
I have been using Wordpress as a headless CMS, and using WPGraphQL gives you real power, and its so superior to the WP Rest API, in a lot of ways. It allows you to just don’t really care about API routes defined by Wordpress, and just fetch your things (with a proper caching solution, which can be a problem using GQL…).
We use javascript for things it's not meant for. Here's the thing, if the tool works well for your use case and there's no much bad trade-off, then you should ideal it.
Absolutely agree David. I am doing this since some time now: if it works i'll do it. It's not only a question of do I need to use GraphQL or not? The question is: what's the best tool I am confortable with to work with faster !
So wait, i am conflicted now, am i an idiot for using graphql for storing data i need from a graphql api into my database, and then also for fetching the data i need for my frontend ? I am very confused by this video
I don’t often comment on videos but those I do are ones that help guide me in the right direction, light bulb moments if you will. It’s been a while since I’ve played around with programming and naturally things have changed a bit. I’ve been researching tools & tooling to possibly build small apps to manage my seemingly unmanageable life. TLDR: this was insightful
Awesome that you spent the time explaining this. I've seen crazy usages of GraphQL, even all the information on the client managing a client side database 🥺
I agree 100%. Have you used federation? (E.g. Apollo's solution for joining multiple graphql endpoints into one) would be curious to see your take on that pattern :-)
Yeah the "GraphQL Resolver Gateway" should surely be "GraphQL Federation Server" and then GraphQL totally does handle the call end to end back to each micro-service. You also get the added benefit of writing queries that can cross over micro-service boundaries which all get resolved seamlessly.
But why? GraphQL makes trade-offs, and personally I find the trade-offs worth when communicating between client/server, not so much with server/server, cause now each microservice needs to also be written with GQL in mind.
@@kefkef5 good question. The reason is: say I want a "global" profile service for my users. But, depending on the department / apps they've interacted with in our ecosystem, the metadata is subject to change from a wholistic standpoint. So, federation would help us build a collective graph for the profile of any given user.
@@kefkef5 There are some situations where microservice1 { field: microservice2 { data } } is insanely powerful, and I'm not aware of too many alternatives. The caller doesn't need to know or care where any data is and the GraphQL Federation server glues everything together seamlessly with no significant performance penalty that my APM can detect.
Can't you do that anyway when you have your GQL service calling the rest services to ask for the data and glue it together on the GraphQL? Now the backend is also forced to provide you a GraphQL endpoint, which they might think it's not worth the time investment compared to exposing the data through rest.
The critique that Hasura is an anti-pattern is based on the idea that you don't want to expose your db schema to public clients. It's valid. However Hasura has features to limit exposure of fields and modify permissions. The cost/benefit for small time devs is huge with Hasura. If you are in a highly-resourced company you can use Hasura and then put another GraphQL Yoga engine in front of you want another layer of abstraction and employ more devs.
Great video. I just signed on to use a data company who uses GraphQL to access their data. I have many years of SQL and so far I am not a fan of GraphQL. No default aggregate / Count?? Because it is like you said for the UI and not the database. I need to ask this company on why they are using GraphQL as their interface to their database.
Maybe you are not actually hitting their database directly, but rather are making a bulk entity request to one or more services? They may be considering all client requests equal, regardless of whether they are coming from an individual-entity record-viewing UI, or from a batch reporting job. But yeah, if you're trying to aggregate normalized, relational data, SQL would be better. But that doesn't mean they will be ok with giving access to query their database directly.
I was thinking about the graphql between microservices - is it good idea to use GraphQL API for between microservices just to have a single API implementation over all infrastructure ? Then you can expose all these graph ql apis using the apollo federation/proxy service. The key thing to mention is, that microservices themselves will not communicate directly between each other but instead using kafka topics. What do you think about that Theo ?
Very good explanation on how to use GraphQL. I have a question, what if we use GraphQL between each service and the Gateway? So, each Microservice exposes its own schema and the Gateway is in charge of resolving it according to the client's requests, in other words what I mean is, it is a bad idea to have graphql between the client and the server, but also to have it between each microservice and the gateway. Thanks
So you have multiple microservices that call the same database? Congrats! you have a distributed monolith) One of the biggest advantages of microservices is the ability to scale them independently. You just multiply it by 0 when you join everything in DB. With a schema like this you will end up using 600$ RDS DB instance on AWS (without traffic). I know this because I did this)
@@zzzz6538 In microservice architecture, every service should own its own data. If that requires a database, then the service should have its own database. This allows the service to define its own data models and scale its infra independently. Say you have 10 different services using the same DB. Most of the time each service needs 10 units of power (simplification for brevity), but occasionally each service will need up to 30 units of power. You could have 1 DB that provides 300 units of power all the time ( 10 services * 30 max units of power) or 10 DB that provide 30 units of power. 10 small DBs @ 30 units of power will be cheaper than 1 DB @ 300 units of power. Using current EC2 prices, 10 machines with 4 vCPU and 16GiB ram would cost $650/mo. vs. 1 machine with 40 vCPU and 160 GiB ram would cost $850/mo. It's about 25% cheaper splitting to multiple instances. Far more important than price though is independent data modeling and reducing complexity. If you outgrow a single DB instance and have to shard your data, determining how to shard data across 1000 tables using a single identifier for consistent hashing while maintaining foreign key integrity per sharded instance would be an absolute nightmare. My head is spinning just thinking about how to do it. If, instead, each of your services use their own DB: 1.) you're less likely to reach a scale where sharding is required. 2.) the relationship complexity between the few tables that belong to the service is going to be much easier to manage and determine how data should be split and which identifier is best for hashing while keeping related entities in the same shard.
I don't understand the reference to Hasura, when it is interpreting postgres schema, plus reading some other metadata to generate an API (runs on a server). This API is communicated with via graphql to the client. I hardly expose any of the underlying database via Hasura, and mostly use more advanced postgres views and Hasura actions to define my full set of backend services, which then is all stitched together so my client can use it. Note, I am using Hasura for prototypes and getting into front end (which it has allowed me to do, while at the same time up skilling in postgres and getting my data model right. Am I misunderstanding how Hasura works? For me all it does is take the heavy backend effort away from maintaining a graphql able server. Definitely not going to attempt running it in my usual backend language, Python. However I know I could serve a model in Python and use Hasura actions or Remote schemas to integrate it (that part is not done using graphql)
To summarize what I am trying to say (Use case of chatGPT to phrase things more methodically, haha)... It's worth noting that the criticism you mentioned refers to the idea of using GraphQL as an internal communication protocol between the server and the database, which is not the intended use case for GraphQL. GraphQL is a query language, it can be used for flexible and efficient communication between client and server. Hasura, on the other hand, does not use GraphQL as an internal communication protocol between the server and the database. It uses GraphQL only as a query language for client-server communication. Hasura reads the schema of the database and generates the GraphQL API based on that schema. Once the API is generated, the client can communicate with the Hasura server using GraphQL queries and mutations, and the Hasura server will handle the communication with the database on the backend. In summary, Hasura sits between the client and the database, using GraphQL as the query language for client-server communication, and it does not use GraphQL as an internal communication protocol between the server and the database.
Hi Theo, I've only worked on the front end so far and now I just begun learning React for almost 1 week now learning useState and useEffect at the moment. What are the next things would you recommend I should focus on after React? I think Redux would be the next step? Maybe GraphQL after Redux? There is lots to learn and work on. At the moment I am focused on trying to create a scheduler App for myself. Kind of like a journal mixed with dates and goals you could set for yourself, not a to-do App. But my end goal is to land a job as a full stack developer with JavaScript as the client side and Java as the backend. I would appreciate your honest and brutal opinion, thank you Theo.
Have you watched my "your goals kinda suck" video yet? Think that will be helpful. Also don't learn graphql or redux if you don't immediately need them, both are SUPER heavy to take on w/ a solo project. Best learned in a big codebase that you don't run lol
"when graphql happened for the first time ever there was a real line drawn between the backend and the frontend, that i didn't have to worry about the other side". That's not because of graphql, but the lack of basic knowledge about design patterns, SOLID principles, etc.
I still don't understand how gql is better than trpc for communication (assuming you use TS for your server ofc)... You could define typescript types for your BE-FE contract just as easily as you define a gql schema....only you don't need to use a different syntax for it....
My company attempted to adopt GQL for the exact thing you say not to adopt it for in this video ("hey I read somewhere that GQL is good for handling complex datasets or whatever, let's use it") for 5 disastrous months in 2021 before going back to REST. Guys, he is right... GraphQL is NOT an API/DB gobetween. Trust me on this.
I was wondering why is what hasura or platformatic do so wrong, yes ofc depends on the scope of the project, some are out of scope of what those frameworks can do, but for simple stuff why not?
I am planning to study graphql and this exactly what I am looking. But I have a question does the server built by REST API then graphql consumes it? please help me with this
Thank you Theo, reading comments I realize i'm not alone, I've worked with it before but never really understood it. I finally get why and where GraphQL is valuable.
I wondered about the title of this video, so I went to google trends and guess what? For the last 5 years, searchs related to GraphQL are going well, especially on 2022. People is indeed very interested on GraphQL. So there you have it.
So the main thing that GraphQL solves, being the hard separation between FE and BE, is based on a middle contract layer. Doesn't Swagger/Openapi do this as well?
It would be interesting if you could properly address your beef with hasura. Perhaps a dedicated post explaining exactly why you don't like it? Or even discussion panel? I'll be there to meet your objections in a constructive way
What Im really interested in ist the difference between graphql and odata. OData manages to solve the classical REST Problem of Over- and Underfetching by implementing standardised funktions. So why not use OData instead of graphql when both share the Same funktionality, while OData simplifies a Lot of bloatware
Odata still fetches the data from the database, and it doesn't really solve the asking for exactly what you need, you can only query about a user for example, but how about a user's shopping cart? That's not solved with odata, what about user's saved cards? unless you define 1 endpoint which returns *everything* about a user, odata doesn't really give you exactly what you want. On top of that, odata does make you call database for all that information, it just removes fields on network layer. You really don't want that.
@@NishchalGautam Please correct me if I'm wrong, but isn't that solved by the $expand and $select query? With those you can define JOIN operations and also define the exact attributes which are return. With a good optimizer (the SAP CAP framework uses one), this can also be translated to the SQL statements to only query for the required data. I get that sich an optimizer has its limits, but in general it is still possible.
@@rediate2721 now do the same thing, but across multiple microservices, remember, you don't join in graphql servers, unless it's a special circumstances, you can't use odata (unless I'm missing something) across multiple microservices which will allow you to query from multiple services. With GraphQL, you can, it's called federated microservices
Would this mean that you would not recommend something like Hasura? We benefit greatly from shipping our new features faster, with some limitations of Hasura of course.
We actually use multiple approaches, whereas Hasura is our main GraphQL gateway and simpler queries and subscriptions are handled by Hasura directly and complex queries and all mutations where validation and business logic is required is handled by a subschema and our API’s. Our entire AuthZ layer is custom and for Hasura handled through webhooks.
A minute and thirty in I had to explain to my OH why I yelled 'fuck you Theo'. In all seriousness, this is great content. The misuse of GraphQL is a mess I've inherited from the version of me from last year who didn't really know what he was installing, and I wake up everyday thinking about pulling the trigger on a fat rewrite
I'm using graphql for a new SAAS project and we designed it as a supergraph/gateway for all rest microservices so client all access the same graphql api and the rest API remain private into the cluster. sure you won't add logic on the database nor creating any kind of monolithic api for transforming to sql
Scoping this question to the 'fullstack dev working on a big solo project' Does having application data that can be modeled like an actual graph data structure come into play when thinking about using graphql? Or I am conflating the concepts like someone would with assuming Java and JavaScript being named similar means they are closely related?
I've never actually used it, so take this with a grain of salt; the graph part is taken out of thin air from what I've understood. Graph data structures should therefore not have anything to do with it. Feels more like they just went with what sounded good, more of a sales pitch than anything else.
what about schema generation in something like MongoDB Atlas distributed app data DB. It would reach that boundary to talk to a db, but than other dbs can process data from it. and in this process you have a schema that represents data in a persistation layer, in the GraphQL API layer, and the UI Layer with react hooks for sofisticated data interaction and CRUD. anyway, just a train of thought, still a lot to be understood about these technologies.
Look kids. There is no reason you can't use GraphQL directly on top of a DB, but if you team is bigger than a handful of people you'll have a hard time not regressing something somewhere. Basically Theo is saying separation of concerns is good. And basically necessary as your team scales. There is not technical limitation here.
In this case, AppSync is the resolver layer, and you use VTL code to handle logic. So you're not directly accessing the database, it's just a low code backend service for your app.
GraphQL can actually be a good solution of querying graphDBs. Neo4j with AuraDB has a library that autogenerates GraphQL from your graph schema your use for CRUD operations to your database (as a functional alternative to cypherQL)
@@t3dotgg I think that opinion comes from your focus on SQL databases. Plenty of graphDBs, like Neo4j, support the use of GraphQL for CRUD operations. Neo4j has a library where it auto-generates GraphQL from your schema. Under the hood this GraohQL calls CipherQL to write to the database, but this allows the user to use GraphQL’s often better syntax over CipherQL depending on the use case. Neo4j has an article about the library on their website.
YES EVENT QUEUES ARE BETTER FOR MICROSERVICE COMMS I SHOULD HAVE MENTIONED THAT I AM SORRY
What event queue would you recommend? We are currently doing a polling REST API for our longer running micro services and it feels "wrong" to continue this pattern while we are growing.
Do you mean event driven architecture?
You're talking about reading data from different microservices so HTTP and gRPC are the correct way to go (if anybody is trying to read data by passing messages on queues then they're beyond help). I would also argue that gRPC isn't worth the complexity overhead if you're not transferring large amounts of data.
Also curious on what event queues you would recommend. Some I'm familiar with are Kafka for streaming data and Rabbit MQ. Totally based on use case. A video breaking down which queues to use when would be awesome.
Haha thanks was thinking that myself. That's what we did at Best Buy.
Thank for creating content ❤
For anyone wondering what the actually name for "GraphQL Resolver Gateway" is, its called GraphQL Mesh, and that mixed with GraphQL Hive is magic
As someone using Mesh in a large project, get as far as you can from Mesh 😅
I think it quickly becomes a mess when microservices talk to each other. You could do that with a message queue instead. Microservices can put messages into this queue and read/listen to messages, and suddenly your architecture's dataflow is way more understandable.
If you think about it long and hard, the microservices talking to each other problem is similar to two-way data binding, and the solution I'm proposing is similar to what Redux does to solve the issue in a sense. Unidirectional Data Flows on the server^^
Oh I like that framing! It was a big ah-ha for me when I read the react-query docs (and reinforced by some of Thea’s videos) and shifted my thinking away from querying and to “state sync”. It’s a little more of a stretch to see an event bus that way, but could be very useful!
@@efkastner well yeah it's a stretch :D but if you have a service which is dedicated to reading the queue and mutating the database, then it could actually look like a reducer, that gets the current state from the db, and the "action" is the message.
What you'd get out of this is very easy swapping of microservices, and replayability? Probably there are no race conditions anymore as well while writing the db.
Never tried, though I'm just thinking in theory. (It feels like a bunch of unnecessary complexity, so maybe it doesn't make sense)
Kafka is you friend
I think that gap that you show between "ServerClient" is a really good way to visualize that use case, and how useful GraphQL will be in comparison to the opposite use case when the server + client are "touching".
I went down the path of using GraphQL a couple of years ago, and I can see it being great when dealing with multiple languages and/or dev teams, or with 3rd party APIs where I'm only the client.
But for my big project where I'm the solo fullstack dev, and using TypeScript on both the nodejs backend + also the frontend... over the last couple of years I've slowly realized that I was just making heaps of extra work for myself dealing with 2 typing systems (TS + GraphQL) instead of just the 1 (TS).
Plus all the extra complexity of having to write queries in another language, and all the extra tooling needed there.
Overall it's slowed the whole project down a lot, without really any benefit for this project in particular. My use case in this instance was pretty much the opposite of when GraphQL shines.
I'm just about to start removing GraphQL from this project. It's my own system (not for a client), and the frontend is still in its infancy... somewhat due to graphql slowing me down.
I'll be taking a look at tRPC, and either using that, or building something customized for myself that has some similarities. Either way, my project isn't going to be any less typesafe than graphql in the end.
I have a feeling GraphQL will be perfect for you 🙏
@@t3dotgg did you read his whole comment?
@@succatash lmao
There’s a lot of tools that fix a lot of your problems. Such as nexus or type graphql to completely remove writing your code in the graphql schema language. Everything is kept in JavaScript with these tools. Which means your typescript is automatically created for you, and is the main thing you’re using
@@succatash prolly meant to type tRPC instead of GraphQL lol
Thanks for making this video! I've been a graphql dev for 1 year and while I've gotten used to the mechanics of the system, I've never really understood the "why" behind it all. I work at an online retail company and this "middle tier" project was started more than two years ago and changed hands a couple times, so now people like me have no idea what the original point even was. This video helped me form a better idea of what it's for, and what my company is still missing in order to better leverage this technology. I might even be able to impress my manager by formulating a well-defined plan for improvements.
BTW, I am in a scrum group with front end developers and I hate it. I feel like team meetings are a complete waste of my time because they are only talking about React 99% of the time and I work on the Graphql server and resolvers. I'm going to make a new push to break up the stupid "full stack team" pattern we're using. It's stupid, at least for the way this company functions.
the between server communication is likely better dealt with as an event, posted to an event queue / log rather than a realtime connection that could / will fail.
Will repeat myself. Yes, you may use gRPC for microservices, there's no evil in it. Use event-driven architecture if you have very good reasons for it (having proven that synchronous doesn't work in practice *for your specific problem*, rather than just assuming it wouldn't or that it's somehow "not in the spirit of microservices"), if you have a damn good observability infrastructure and a strategy for dealing with transactions. Don't do event-driven because you've read it in a popular microservices book that takes a dogmatic stance on it, ignoring the enormous cost in added complexity and maintainability.
we use Hasura in production its is actually quite amazing... our backend is written in golang using the gqlgen library and we auto generate around 5M lines of compliant golang code because of the graphql schema Hasura provides...
Theo you fail to understand that companies operate in different levels. If my company is worth 1B+ do you really think we would use Hasura? Of course not, its better to build your own infrastructure because you have the cash/manpower to do so. Hasura is fundamentally game-changing for startups unders ~25M and can easily handles thousands of concurrent subscriptions through Hasura Cloud. There is a reason Hasura is worth so much these days, and you fail to understand Hasura for what it truly is. You can easily eject out of Hasura because it does NOT interfere with your postgres database whatsoever
Hasura is great honestly. Has its limitations and kinks but definitely useful and great for getting started quick and reliably.
What even is the argument against Hasura? Or even supabase or whoever put gql straight into postgress? There's so many applications where 95% of the endpoints just pass data straight to/from the db. I genuinely don't understand why Hasura or some BaaS would be bad here
He called out Hasura for using GraphQL as SQL replacement on the backend, but then everything he described as the appropriate use of GraphQL can be done by Hasura. So Hasura is not the problem, only the problem is misuse of tools.
It feels good that the same arguments I had some years ago during an internal discussion, that GraphQL is probably best utilized as a gateway to your whole microservice clusterfuck rather than talking to the db's itself. The gateway also prevents the frontend developer to know about the slight differences in naming between the different microservices, as they only need to care about the contract GraphQL schema defines./
In case of distributing requests between microservices, GraphQL can address this on it's own via GraphQL Federation. The same applies for requests initiate from one service to another. Instead of calling the service directly it calls the federation. This removes the requirement for the service to know about each other. The only information that the service needs is where the federation gateway is. This allows you to easily merge and split services without need for application wide refactoring.
Hasura is literally a server deployed as a microservice. It IS the server between the database and the client so it falls right in line with your "acceptable use cases for GraphQL" diagram. Memes and hot takes that paint the whole world black and white are great for driving Twitter and RUclips engagement (here I am, engaging with your content) but at the very least you could try to do an honest review that adheres to your own "rules".
If your take is that systems that have any kind of GraphQL -> SQL translation layer in their stack is evil at least be honest with yourself and viewers and say that instead of this nonsense about requiring a server in the middle.
This is what I was thinking
I agree. I would genuinely like to hear Theo's thoughts on Hasura beyond "No. Bad. Bad."
This is a surprisingly good video given that I disagree with the two contentious points you make:
1. GraphQL shouldn’t be used directly in front of DBs.
2. GraphQL shouldn’t be used for inter-service queries.
And here’s why:
1. The case for DB backed GraphQL APIs (e.g. Hasura)
I think you're arguing against the idea of having micro-services use GraphQL to query their own data, and I 100% agree with that. However, the value of this tech when used correctly is as a rapid project accelerator that won't tie you into loads of tech debt down the line.
Creating a schema directly in front of a DB is fine provided that the schema is a completely logical view of your domain, since in that case you’re free to change the implementation details later. Hasura makes this possible to do since you can create mapping rules that allow you to present your DB logically.
Given how fast it is to create very performant services this way, it’s actually a really good option for start-ups and green-fields, and I’d particularly recommend it, since although it’ll allow you to get your tech. stack implemented quickly, the logical schema you (with care) end up, will mean you don’t get stuck with a monolith when it’s time to scale, and you have a really simple path to moving to something more scalable on the back-end as needed.
You even say as much yourself, and I quote:
> This is what I love about GraphQL. None of this back-end implementation detail exists to me anymore.
2. The case for using GraphQL for inter-service queries
If the thing you really love about GraphQL is that none of the implementation detail need concern you as a front-end dev, then isn’t that an equally useful trait for other services on the back-end? Or, stated another way, if the benefit to this formal interface is the ability to change implementation detail provided queries flow through this interface, then isn’t it better that more of the traffic flows through it?
You don't explicitly say why you disliked this pattern, but perhaps your objection is the performance hit from going through some additional resolvers. If so, then it’s worth noting that this doesn’t apply in the case that services are implemented as subgraphs, since in that case you can then either query them directly, or query them via a low latency router that adds no perceivable overhead.
Thoughts?
Thanks!
Good video, good points. Ever since I learned GraphQL, it made me appreciate the art of crafting API's, and made me like to *start* with GraphQL schema's when designing new object-oriented systems. However, *my* very favorite feature of GraphQL is its introspection capabilities. I do wish the spec would "require" it to always be public, so that clients could reasonably employ tools that could dynamically generate UI's, or even new machine-to-machine processes, based solely on schemas..
I'd argue that the GraphQL client should always be considered as part of the frontend domain as most UIs are data-driven and frontenders knows best what data dependencies each component has. Over time frontenders needs to make changes to the frontend code, and then it'll lead to a much more streamlined and frictionless process if they adjust the graphql schema / resolver at the same time. If the frontenders needs to align and communicate with the backenders every time they need to make a change impacting the graphql sschema, it may risk leading to technical debt instead, as there's a big chance that a workaround will be taken instead if time is an issue. The graphql server as I see it is a BFF (Backend For Frontend), meaning its for moving complexity and business logic from the frontend/client/browser to instead handle and prepare it in the graphql server. Therefore there are very strong arguments for writing the graphql server in node.js (javascript, typescript, rescript, purescript or some other compile-to-js language). Other than that I agree about server to server comm should be gRPC, and that is is only intended as a thin interface/proxy without direct db connectivity... but the split between FE and BE is where I disagree. This is based on my experience from working this way.
I was told GraphQL is used over REST primary because you can tell it to "filter" the response and thus you get exactly what you want. The problem is, you can emulate this with REST too if you really wanted. Either in the query params or inside the body you can tell the server what you want then your REST api will filter it based of your request.
Graphql sounds like another useless tech for most applications
If you design your graphQl schema you can literally fetch all the data in your database or just a single scalar value or anything in between in ONE REQUEST. It took me a while to understand how powerful this ability is but once I saw it, I can't go back to REST! The key is all in the design of you graphQL Schema plus custom DataFetchers to solve N+1 problems!
A nice thing that you brought up here Theo is Data contract which is probably the most common problem that any software solution without proper planning face.
Explicit schemas as contracts is a must.
For the most complex systems, we even have dedicated versioned schema repositories nowadays thank god.
GraphQL was one of the moves towards this in a more narrowed sense than when talking about architectures.
Info near the definition, flexible and extensible metadata, all come very handy.
Also a very bad practice I see a lot is to try to come up from the beginning with an ubiquitous data model that will not be needing updates any sooner (that is an illusion and probably it will become harder to cleanup those unwanted attrs and add useful ones ).
My tip is, unless it is a very fundamental attribute of the particular entity wait until you implement the feature that will need the attribute. I get the creeps when i see those humongous entities that look like they came from an enterprise software of the 90s.
You should look into federated graphql. With that at the gateway each service maintains it's own schema. This provides a consistent interface for inter service requests.
Came here to say this.
The use case he made where the payment service needs to update the user is perfect for a combination of stitching and federation.
You make the payment, that mutation returns the user, and you just spread the user, which, at least in Relay, updates that state.
Theo, that was exactly the question I asked myself yesterday.
With time and missions done, I realize that not so many people are understanding GraphQL. Do you think it's linked to its "not well explained" documentation?
At work our app uses GraphQL stitching to combine Hasura with some custom NestJS APIs. We're in the process of porting to a single tRPC server and it is SHOCKING how much code we're removing. To be fair, a lot of that code is generated. However, the more significant change is to our dev process where everything is 10x easier.
Yeah I fell into the “wow Hasura is amazing” hype… but ended up having the same experience in the end: it just sugar coated the complexity, didn’t really deal with it.
But it’s not bad! Just somehow I failed to find a really good use case for it even though I tried.
I went through Apollo’s free courses, which are super high quality, and learned GraphQL was not for the project I was working on. I’m really glad I went through the courses still, because now I’ll know when to use it.
With regards to microservices, GraphQL Federation can be used to provide a unified API while keeping services isolated; services define their schemas and ownerships.
As for using GraphQL to communicate _between_ services, you'd be better off with using something like gRPC, Protobuf or other event queue systems alike. GraphQL only make sense when you want to expose APIs to one or more clients.
Between services you own? Sure. If you don't own those services gRPC can be hell to debug
@@KManAbout Absolutely. gRPC is not suitable for public APIs imo.
@@dealloc yup glad we agree. It is super good for private one as you said. I should use it more tbh
With http3 things are going far
GraphQL for server to server is perfectly valid use case. I have used it in production with 7 microservice and it was a pleasure to use. Each service just needs to expose an internal graphql schema at say /internal/graphql that can be consumed by other microservices.
What specifically did you like over Rest? Did you manage to make the gql schema work as a 'contract' between the different teams?
@@82TheKnocKY Yes exactly. This makes everything type safe. You can use graphql-codegen to make sure that all the queries are valid in CI so the contract enforcement can be automated. As far as GRPC is concerned, GraphQL can be just as fast if you use HTTP2 on server side (easy using a service mesh). The only saving using GRPC is that it has a smaller payload because of Protobuf encoding but the difference won't be significant.
Antipattern avoid coupling when you re talking about MS architecture use Event Driven or Event Sourcing instead
@@gt4adolfo Event driven is not always good. For instance, if order service wants to make sure that the customer has a valid email, it should call customer service using HTTP and not Event. Event Driven is good for async communication.
@@quant-daddy is possible to achieve it using event driven... youre coupling microservices... for this reason exists Aggregation events. the same microservice have to check for example the "email". is not necesary coupling.
Fun to watch this older video after coming to know of you more recently.
I really like Prisma lately. They seem to fix and work on the various issues.
"we define a schema and don't have to talk to each other anymore" - remembers me the time we used SOAP
hey @Theo, heard about federated graphql? Have a look, it does let you have individual services also speak graphql, then use apollo router as your gateway (no code, just deploy), then your individual services can talk to other services through the router as well.
You didn’t give any real reasons why graphql server is bad. I recently started a project with aws and when looking for resources to connect my dynamodb tables to my front-end, aws-amplify came up among the first solutions with graphql the default. I think I agree it’s probably not the best route, but I like to know why.
Isn’t Hasura just auto-generating CRUD GraphQL (and recently REST) API, instead of querying DB’s using GraphQL?
Also, can you share your thoughts on public GraphQL APIs?
I don't really agree with theo's position on this either, hasura has been all about having a low to no code backend in minutes for me. It solves the boilerplate nonsense of adding CRUD to your backend, and you can use actions for complicated logic. It's unbeatable when it comes to building something quickly
You just define queries in your frontend, run graphql-codegen and use apollo-hooks, there, you're done
One of the argument I've seen is that you should never expose your database schema "directly" to your client, without much explanation as to why. Well first of all you can use SQL views and functions and build your hasura graphql schema around that, and second: why not ? permissions in hasura are made to make sure your users see only what they're allowed to ...
Anyway, hasura rocks, it will make your app building so much faster and it is ridiculous to dismiss it over some ideology, we've used and hacked technologies to use them where they were not supposed to for ever, won't stop now
Yeah, I'm not sure why he is hating on Hasura. It's an incredible tool for generating an api.
Hasura is great for base CRUDs, an events driver to fire off serverless fns, while adding your own apis (more serverless fns) for business logic in highly efficient and no boilerplate ways. Did I mention serverless? And Hasura is the (graphql) gateway.
Theo is right: He doesn't know what he is talking about.
He like most are too stuck on microservices. Go serverless and event driven, then Hasura makes sense.
PS. You also must hate Firebase. Me too, Firebase is a trap. Hasura is conceptually like Firebase as a BEaaS, but better in every way: complexity and performance scaling, transparent and open source, graphql ecosystem
What's a problem to use gql as api gateway instead of REST(HTTP/JSON) API? For example to use `GraphQL Mesh` instead of Kong/Tyk/Ocelot/KrakenD etc.?
Another thing that I find GraphQL helpful is with defined schema, graphql client (apollo) can automatically merge same objects from different response which help consistency in the UI without manually doing in that in reducer or in react-query callback. How do you normally deal with this when you are using trpc/rest
does traditional REST APIs make sense in your experience?
What sort of a question is this? REST APIs is a staple and it won't go away ... ever. Not until we remove HTTP. They are simple to consume and fast to build. IMHO most new projects should start with a REST API and if they have some scaling issues then they should consider GraphQL, tRPC. REST is the perfect fit for small teams. Just don't overengineer it with dumb HTTP semantics. I've seen overengineered REST APIs and they are not fun to work with. Also keep in mind that REST doesn't mean JSON. You can use REST APIs and pass XMLs to keep extra object information. The downside of JSON is that it loses a lot of context. Granted most of the time you don't need the said context. But it just goes without saying.
@@xtremescript Well, that was kind of my point, why isn't REST mentioned in the video?
@@xtremescript rest is more suited for scalability than even gql imo
"Don't use stuff for things it was not designed to do" - wears a baseballcap while sitting at his desk 🤨
Not always true about databases, there’s graph database called dgraph, and it looks really nice there
Did I miss something or was standard REST not even mentioned? Isn't that also a way to separate Front and Back people?
Graphql uses schemas, which defines a standardized contract between front-end en back-end. In REST you can also create schemas but you have to implement your own or use tools like OpenAPI. Another benefit of graphql is reducing data overfetching. In REST you might need to do several request to get data you need but in Graphql you just define the fields you need and make a single request
One of the few channels where I don't mind the clickbait titles, the content is always good.
What are your thoughts on Postgraphile?
Thanks for the great explanations.
Theo can you explain why having graphql directly on a database is bad?
Isn’t it just like a serverless implementation of graphql? I mean ultimately with hasura o assume there’s still a server in between the database and the client, or do I assume wrong?
I have been using Wordpress as a headless CMS, and using WPGraphQL gives you real power, and its so superior to the WP Rest API, in a lot of ways. It allows you to just don’t really care about API routes defined by Wordpress, and just fetch your things (with a proper caching solution, which can be a problem using GQL…).
We use javascript for things it's not meant for. Here's the thing, if the tool works well for your use case and there's no much bad trade-off, then you should ideal it.
Absolutely agree David. I am doing this since some time now: if it works i'll do it. It's not only a question of do I need to use GraphQL or not? The question is: what's the best tool I am confortable with to work with faster !
So wait, i am conflicted now, am i an idiot for using graphql for storing data i need from a graphql api into my database, and then also for fetching the data i need for my frontend ?
I am very confused by this video
I don’t often comment on videos but those I do are ones that help guide me in the right direction, light bulb moments if you will.
It’s been a while since I’ve played around with programming and naturally things have changed a bit.
I’ve been researching tools & tooling to possibly build small apps to manage my seemingly unmanageable life.
TLDR: this was insightful
Awesome that you spent the time explaining this. I've seen crazy usages of GraphQL, even all the information on the client managing a client side database 🥺
Awesome explanation. I finally get it. Thanks theo!
I agree 100%. Have you used federation? (E.g. Apollo's solution for joining multiple graphql endpoints into one) would be curious to see your take on that pattern :-)
Yeah the "GraphQL Resolver Gateway" should surely be "GraphQL Federation Server" and then GraphQL totally does handle the call end to end back to each micro-service. You also get the added benefit of writing queries that can cross over micro-service boundaries which all get resolved seamlessly.
But why?
GraphQL makes trade-offs, and personally I find the trade-offs worth when communicating between client/server, not so much with server/server, cause now each microservice needs to also be written with GQL in mind.
@@kefkef5 good question. The reason is: say I want a "global" profile service for my users. But, depending on the department / apps they've interacted with in our ecosystem, the metadata is subject to change from a wholistic standpoint. So, federation would help us build a collective graph for the profile of any given user.
@@kefkef5 There are some situations where microservice1 { field: microservice2 { data } } is insanely powerful, and I'm not aware of too many alternatives. The caller doesn't need to know or care where any data is and the GraphQL Federation server glues everything together seamlessly with no significant performance penalty that my APM can detect.
Can't you do that anyway when you have your GQL service calling the rest services to ask for the data and glue it together on the GraphQL?
Now the backend is also forced to provide you a GraphQL endpoint, which they might think it's not worth the time investment compared to exposing the data through rest.
Please have a look on hasura for graphql
Hasura is an anti-pattern lol
The critique that Hasura is an anti-pattern is based on the idea that you don't want to expose your db schema to public clients. It's valid. However Hasura has features to limit exposure of fields and modify permissions. The cost/benefit for small time devs is huge with Hasura. If you are in a highly-resourced company you can use Hasura and then put another GraphQL Yoga engine in front of you want another layer of abstraction and employ more devs.
This video really helped me understand the theoretical perspective of GraphQL. I was indeed viewing the whole concept like Hasura has designed..
Could you please explain why querying DB from GQL server is bad idea, and what's wrong with Hasura?
Great video. I just signed on to use a data company who uses GraphQL to access their data. I have many years of SQL and so far I am not a fan of GraphQL. No default aggregate / Count?? Because it is like you said for the UI and not the database. I need to ask this company on why they are using GraphQL as their interface to their database.
Maybe you are not actually hitting their database directly, but rather are making a bulk entity request to one or more services? They may be considering all client requests equal, regardless of whether they are coming from an individual-entity record-viewing UI, or from a batch reporting job. But yeah, if you're trying to aggregate normalized, relational data, SQL would be better. But that doesn't mean they will be ok with giving access to query their database directly.
I was thinking about the graphql between microservices - is it good idea to use GraphQL API for between microservices just to have a single API implementation over all infrastructure ? Then you can expose all these graph ql apis using the apollo federation/proxy service. The key thing to mention is, that microservices themselves will not communicate directly between each other but instead using kafka topics. What do you think about that Theo ?
Very good explanation on how to use GraphQL. I have a question, what if we use GraphQL between each service and the Gateway? So, each Microservice exposes its own schema and the Gateway is in charge of resolving it according to the client's requests, in other words what I mean is, it is a bad idea to have graphql between the client and the server, but also to have it between each microservice and the gateway. Thanks
So you have multiple microservices that call the same database? Congrats! you have a distributed monolith) One of the biggest advantages of microservices is the ability to scale them independently. You just multiply it by 0 when you join everything in DB.
With a schema like this you will end up using 600$ RDS DB instance on AWS (without traffic). I know this because I did this)
Care to explain. I like to know more about that
@@zzzz6538 In microservice architecture, every service should own its own data. If that requires a database, then the service should have its own database. This allows the service to define its own data models and scale its infra independently.
Say you have 10 different services using the same DB. Most of the time each service needs 10 units of power (simplification for brevity), but occasionally each service will need up to 30 units of power. You could have 1 DB that provides 300 units of power all the time ( 10 services * 30 max units of power) or 10 DB that provide 30 units of power. 10 small DBs @ 30 units of power will be cheaper than 1 DB @ 300 units of power. Using current EC2 prices, 10 machines with 4 vCPU and 16GiB ram would cost $650/mo. vs. 1 machine with 40 vCPU and 160 GiB ram would cost $850/mo. It's about 25% cheaper splitting to multiple instances.
Far more important than price though is independent data modeling and reducing complexity. If you outgrow a single DB instance and have to shard your data, determining how to shard data across 1000 tables using a single identifier for consistent hashing while maintaining foreign key integrity per sharded instance would be an absolute nightmare. My head is spinning just thinking about how to do it. If, instead, each of your services use their own DB: 1.) you're less likely to reach a scale where sharding is required. 2.) the relationship complexity between the few tables that belong to the service is going to be much easier to manage and determine how data should be split and which identifier is best for hashing while keeping related entities in the same shard.
What is about Hasura?
I don't understand the reference to Hasura, when it is interpreting postgres schema, plus reading some other metadata to generate an API (runs on a server). This API is communicated with via graphql to the client.
I hardly expose any of the underlying database via Hasura, and mostly use more advanced postgres views and Hasura actions to define my full set of backend services, which then is all stitched together so my client can use it.
Note, I am using Hasura for prototypes and getting into front end (which it has allowed me to do, while at the same time up skilling in postgres and getting my data model right.
Am I misunderstanding how Hasura works? For me all it does is take the heavy backend effort away from maintaining a graphql able server. Definitely not going to attempt running it in my usual backend language, Python. However I know I could serve a model in Python and use Hasura actions or Remote schemas to integrate it (that part is not done using graphql)
To summarize what I am trying to say (Use case of chatGPT to phrase things more methodically, haha)...
It's worth noting that the criticism you mentioned refers to the idea of using GraphQL as an internal communication protocol between the server and the database, which is not the intended use case for GraphQL. GraphQL is a query language, it can be used for flexible and efficient communication between client and server. Hasura, on the other hand, does not use GraphQL as an internal communication protocol between the server and the database. It uses GraphQL only as a query language for client-server communication.
Hasura reads the schema of the database and generates the GraphQL API based on that schema. Once the API is generated, the client can communicate with the Hasura server using GraphQL queries and mutations, and the Hasura server will handle the communication with the database on the backend.
In summary, Hasura sits between the client and the database, using GraphQL as the query language for client-server communication, and it does not use GraphQL as an internal communication protocol between the server and the database.
Hi Theo, I've only worked on the front end so far and now I just begun learning React for almost 1 week now learning useState and useEffect at the moment. What are the next things would you recommend I should focus on after React? I think Redux would be the next step? Maybe GraphQL after Redux? There is lots to learn and work on. At the moment I am focused on trying to create a scheduler App for myself. Kind of like a journal mixed with dates and goals you could set for yourself, not a to-do App. But my end goal is to land a job as a full stack developer with JavaScript as the client side and Java as the backend. I would appreciate your honest and brutal opinion, thank you Theo.
Have you watched my "your goals kinda suck" video yet? Think that will be helpful.
Also don't learn graphql or redux if you don't immediately need them, both are SUPER heavy to take on w/ a solo project. Best learned in a big codebase that you don't run lol
@@t3dotgg Thank you Theo, I will look for the video you recommended, and I appreciate the guidance.
"when graphql happened for the first time ever there was a real line drawn between the backend and the frontend, that i didn't have to worry about the other side". That's not because of graphql, but the lack of basic knowledge about design patterns, SOLID principles, etc.
What does it mean 'as long as its I/O performant' ?
I still don't understand how gql is better than trpc for communication (assuming you use TS for your server ofc)... You could define typescript types for your BE-FE contract just as easily as you define a gql schema....only you don't need to use a different syntax for it....
Does Apollo Federation change anything about GraphQL
My company attempted to adopt GQL for the exact thing you say not to adopt it for in this video ("hey I read somewhere that GQL is good for handling complex datasets or whatever, let's use it") for 5 disastrous months in 2021 before going back to REST.
Guys, he is right... GraphQL is NOT an API/DB gobetween. Trust me on this.
I was wondering why is what hasura or platformatic do so wrong, yes ofc depends on the scope of the project, some are out of scope of what those frameworks can do, but for simple stuff why not?
why is the diagram backwards?!?!? :)
What is GQL solving better than JSON schema as a contract between teams (BE/FE) and generating TS types from those for clients to use?
with a json schema + http you have to know the endpoint. graphql abstracts that.
@@jmann277 yes, but those endpoints can be typed routes which in turn are your named queries, right?
Senor, yo no speako English good. Do I understand correct: I use graphQL for server, and SQL for frontend si? Thanks!
RDBMS is a server, and User Service is a DB client application, so why, you say, shouldn't GQL be put between a client and a server?
Why can't you use a rest API to do this though? Can't you replace the graphQL server in this architecture with a rest API to do the same thing?
Best explanation about GraphQL
How is it even possible to use GraphQL between the server and the DB? What database is supporting GraphQL queries??
I am planning to study graphql and this exactly what I am looking. But I have a question does the server built by REST API then graphql consumes it? please help me with this
8:27 so graphql client is not good for real time and rapidly changing data?
Thank you Theo, reading comments I realize i'm not alone, I've worked with it before but never really understood it.
I finally get why and where GraphQL is valuable.
why micros ervices swagger cannot be contract between backend and forentend? If I don't has too many micro services..
This confirms my thought. I currently can't see any value in Graphql if all it does is describe a schema.
I hope it is
I wondered about the title of this video, so I went to google trends and guess what? For the last 5 years, searchs related to GraphQL are going well, especially on 2022. People is indeed very interested on GraphQL. So there you have it.
Okay, you are hands down the best tech RUclipsr.
Dude this video is gold! And you are a star!
So the main thing that GraphQL solves, being the hard separation between FE and BE, is based on a middle contract layer. Doesn't Swagger/Openapi do this as well?
I fail to see why it is "bad" at server to server communication.
It would be interesting if you could properly address your beef with hasura. Perhaps a dedicated post explaining exactly why you don't like it? Or even discussion panel? I'll be there to meet your objections in a constructive way
What Im really interested in ist the difference between graphql and odata. OData manages to solve the classical REST Problem of Over- and Underfetching by implementing standardised funktions. So why not use OData instead of graphql when both share the Same funktionality, while OData simplifies a Lot of bloatware
Odata still fetches the data from the database, and it doesn't really solve the asking for exactly what you need, you can only query about a user for example, but how about a user's shopping cart? That's not solved with odata, what about user's saved cards? unless you define 1 endpoint which returns *everything* about a user, odata doesn't really give you exactly what you want.
On top of that, odata does make you call database for all that information, it just removes fields on network layer. You really don't want that.
@@NishchalGautam Please correct me if I'm wrong, but isn't that solved by the $expand and $select query? With those you can define JOIN operations and also define the exact attributes which are return. With a good optimizer (the SAP CAP framework uses one), this can also be translated to the SQL statements to only query for the required data. I get that sich an optimizer has its limits, but in general it is still possible.
@@rediate2721 now do the same thing, but across multiple microservices, remember, you don't join in graphql servers, unless it's a special circumstances, you can't use odata (unless I'm missing something) across multiple microservices which will allow you to query from multiple services.
With GraphQL, you can, it's called federated microservices
I just wish they have the "select *" operator. Having to write out / link the primitives each time is silly.
You can use GraphQL snippets depending on your use case
Aww what's wrong with Hasura?
Would this mean that you would not recommend something like Hasura?
We benefit greatly from shipping our new features faster, with some limitations of Hasura of course.
I bet I can ship faster on the T3 Stack w/ SQL than you can on Hasura. Watch my database video if you want more thoughts on this in particular
We actually use multiple approaches, whereas Hasura is our main GraphQL gateway and simpler queries and subscriptions are handled by Hasura directly and complex queries and all mutations where validation and business logic is required is handled by a subschema and our API’s.
Our entire AuthZ layer is custom and for Hasura handled through webhooks.
A minute and thirty in I had to explain to my OH why I yelled 'fuck you Theo'. In all seriousness, this is great content. The misuse of GraphQL is a mess I've inherited from the version of me from last year who didn't really know what he was installing, and I wake up everyday thinking about pulling the trigger on a fat rewrite
I'm using graphql for a new SAAS project and we designed it as a supergraph/gateway for all rest microservices so client all access the same graphql api and the rest API remain private into the cluster. sure you won't add logic on the database nor creating any kind of monolithic api for transforming to sql
Scoping this question to the 'fullstack dev working on a big solo project'
Does having application data that can be modeled like an actual graph data structure come into play when thinking about using graphql?
Or I am conflating the concepts like someone would with assuming Java and JavaScript being named similar means they are closely related?
I've never actually used it, so take this with a grain of salt; the graph part is taken out of thin air from what I've understood. Graph data structures should therefore not have anything to do with it. Feels more like they just went with what sounded good, more of a sales pitch than anything else.
what about schema generation in something like MongoDB Atlas distributed app data DB. It would reach that boundary to talk to a db, but than other dbs can process data from it. and in this process you have a schema that represents data in a persistation layer, in the GraphQL API layer, and the UI Layer with react hooks for sofisticated data interaction and CRUD. anyway, just a train of thought, still a lot to be understood about these technologies.
Look kids. There is no reason you can't use GraphQL directly on top of a DB, but if you team is bigger than a handful of people you'll have a hard time not regressing something somewhere. Basically Theo is saying separation of concerns is good. And basically necessary as your team scales. There is not technical limitation here.
How to solve the N+1 problem
GatsbyJS approves this message 👍🏼
HAHAHAHAHAHAHAHAHAHAHAHAHA
What do you think about graphql(appsync) used in AWS stack to implement crud access to dynamodb nosql table?
In this case, AppSync is the resolver layer, and you use VTL code to handle logic. So you're not directly accessing the database, it's just a low code backend service for your app.
As soon as you said "do not use graphql for you database", the first thing I did was check to see if thats exactly what I did😅
Why Gatsby use graphql see the template use with wordpress, and wordpress use php under the hood.
Mustache or must cache?
Great info. New subscriber.
Instructions unclear. Now using GraphQL for everything except client server state transfer
GraphQL can actually be a good solution of querying graphDBs. Neo4j with AuraDB has a library that autogenerates GraphQL from your graph schema your use for CRUD operations to your database (as a functional alternative to cypherQL)
If your gql is being "auto-generated", you're using gql wrong.
@@t3dotgg I think that opinion comes from your focus on SQL databases. Plenty of graphDBs, like Neo4j, support the use of GraphQL for CRUD operations. Neo4j has a library where it auto-generates GraphQL from your schema. Under the hood this GraohQL calls CipherQL to write to the database, but this allows the user to use GraphQL’s often better syntax over CipherQL depending on the use case. Neo4j has an article about the library on their website.