Devon, I am a 44 year-old software developer in the Enterprise world. I have to say your videos are some of the best I've seen for what you seem to want to accomplish. You're showing the people who watch this video the fact that you have no fear of digging through documentation, putting in lots of hours, overcoming frustration. So many people wanting to learn how to code just assume that "experts" just go on their keyboard and just start typing away without frustration. But you seem to show the reality of it. I would love to see you talk more about just the mental process you go through from square one to finish, the habits/practices you follow and the mental challenges you have to overcome to convince yourself that you can do this stuff. I think that's the biggest hurdle for a lot of people is that feeling of imposter syndrome. You were going to do really well in your career. Keep up the good work man.
I want you to know as a 31 year old developer currently struggling with doing new things out of my comfort zone this line "people wanting to learn how to code just assume that "experts" just go on their keyboard and just start typing away without frustration. But you seem to show the reality of it." is what I needed to hear right now, thanks
I feel your comment is very important for people who know nothing in programming mainly this part "You're showing the people who watch this video the fact that you have no fear of digging through documentation, putting in lots of hours, overcoming frustration" many people fail to realize even the most competent of programmers sit reading through documentation to make stuff work. You may know every piece of syntax and code for a given language but making it work how you need is not always going to be easy without documentation :) I feel there is often too much feeling of imposter syndrome for new coders and sometimes are afraid to read documentation feeling they are cheating however it could not be much further from the truth :) the documentation is there for people to learn from, the original library creators give the documentation to aid others in working with their library and such. Do not be afraid to read documentation and watch endless videos it will make you more competent in your goals and project. I myself am stuck deciding on what project to even embark on to expand my basic c# knowledge after learning from unity making games I now want to expand into software and other things ! The key here is to read and learn and read some more :) happy learning !
kzm 夢を I probably do that same kind of stuff in microsoft word and it is easier because in word, you can snap the boxes to be in line with each other and make sure that the boxes are in the same dimensions. Also switch from portrait to landscape 🤣
I would do it just like this, but put each of the services in a docker container. That way you can have the container restart itself if the service crashes. Load balancing between containers instead of ports.
heads up, docker containers don't automagically restart, there's some further configuration and setup needed to catch process ending errors and restart (pm2, forever, etc).
All developers should do this every so often, in any field. This not only gives other people insight in your way of thinking, but it also serves the 'rubber ducky' purpose. Because you're explaining it to others, your mindset changes about the problem and you get additional different views as well. Good practise man, awesome!
Amazing video with an amazing ad, just amazing! You don't know how much you help me with these videos each time you upload, not just because of the knowlegde, but also the motivation to graduate and learn so much more! Thanks a lot man.
I'm a final year computer science student this year, and this is the most informative channel for me on RUclips. This is exactly the same topics i'm a struggling with and learning to myself at the moment. Keep up with the good videos dude, there are people out there who really benefit from them.
yoo i watched you since minecraft and it’s crazy how you grew this channel so fast and so big! I remember when this channel was at 1K :D. I’m learning so much from these video’s and this is the kind of stuff i wanna do later so this is really fun to watch and you make it even more entertaining. You really motivate me to do this kind of stuff like coding. Thanks for this amazing content! :)
Don’t know you at all, but this type of video is so valuable. It’s a piece that is missing from the “self taught” coding world. It’s like there are the basics courses and then there are completed project courses and then there are “watch expert live coders do amazing things”. This right here is the messy middle of “how do you Think about a project”. Thank you for this “design meeting”. I’ll be following you.
I'm learning more and more programming right now, and I'm building up to deploy a website. I keep coming back to this video every 2 days or so to see how much more I understand of what's going on.
This is a pretty good start. As long as you have your ports in order, and you're only working with one server, this is a pretty solid plan. Though I would just call this application organization, scalability kinda refers to multiple servers, load balancing, high availability, and managing requests with idempotency
I honestly watch these having no idea what your talking about but somehow am just genuinely interested in what your talking about 🤷🏽♂️😂. It’s like I’m in a class that I don’t need for my degree but your a really cool professor so I automatically get sucked in to paying attention. Idk if that makes sense but keep up the good work!!! Been subbed to you for a good min.!
I've seen a few others mention this already but definitely look into Docker and with it Docker Compose. Using Docker Compose you can not only automate the process of booting all of these services at once but also save yourself work in the future when deploying individual services becomes as easy as running a simple docker command. I use Docker for work and my personal server which is set up much like your proposal, and it works beautifully. Keep up the good work though man, love seeing how others approach the same problem.
You make me get excited to get back to programming and learning new stuff Such an inspiration , these new type of youtube channels are amazing and you are on the top of them in my opinion KEEP IT UP
Another step to make your API scalable is versioning. If you have multiple apps talking to the same endpoints, changing the endpoints (either response/request or route) without versioning would require you to update all applications to understand the changes. With versioning, however you can create a new version of an endpoint and avoid breaking other apps the uses the older version. This allows for incremental updates without the need to break anything.
I wouldn't say to just start coding, you could be diving in without the necessary tools. Since you're a student, you probably are learning the foundations of computer science - how to code a basic program, concepts like object-oriented programming, data structures, algorithms (etc). That's computer science. A lot of what this video deals with is software engineering, which you will start to learn but it's also its own thing. I wouldn't expect a computer science curriculum to immediately teach you full stack web application design early on; confusion would be normal. Stuff like designing large scale web applications, client-server architecture, and some computer networking (what's kinda happening in this video) typically come after there being a foundation. Best to understand where you are before "just coding". If you do want to dive head first, there are a lot of tutorials on projects that will help build up to something like this: build a personal website, start an Android/iOS app, etc.
No worries, like Yusuf Ahmed said.. I've only dived into this type of development in the past few months. The past 3-4 years for me have mostly been computer science fundamentals
Ohh, how awesome! I am also getting started with NodeJS etc, you are basically doing exactly what I want as well. But haven't got the time to actually get my head wrapped around it due to work. Looking forward to the next videos, I hope I can learn a lot of stuff from that already!
I've just completed my AWS Developer Certification, AWS does this all for you. Use an Auto scaling group with a min and max amout of instances/servers, use an Elastic Load Balancer to assist with routing and load of incoming traffic. You could also go serverless and use AWS lambda, API Gateway and Dynamo for your backend calls, then use S3 to host your static files.
If you want to build a big website or just make a small website super performant, i'd recommend next.js for the front-end (which you already use) and making the back-end using Go since it's super fast and faster than NodeJS.
Jesus Christ Devon! What the heck happened?! You grew so much! Hope you remember me haha, your first fan you have ever met, sort of by accident. But I’m happy for you man. You grew more than I thought and I’m extremely impressed. Love the content and I’m gonna binge to catch up - nick, that weird dude from X15
It's amazing how I spent a full week researching this. My goal is the same, to separate my project into multiple servers. One serves the frontend, another serves the backend. From there every application can be dedicated and/or bundled up somewhere, ideally organized to keep maintenance seamless when interacting with applications. I'm going to look into what you recommended to do and I thank you for finding out all of this. I would love to see your project files (excluding your applications of course) or if you made a boiler template out this, but that's a lot to ask for. Regardless. Thank you.
Docker container per app + Nginx on the main machine to route to each one. If anything grows big and consumes a lot of resources, you can extract it to separate server without any efforts.
I'm currently interning at a bank in Australia and what you said about the complexity is true. We also use Nginx to route our different applications/API's we build.
If you want it to be even more scalable, you could create a fully static frontend (using something like GatsbyJS or react-snap for the frontend. This lets you use React while still getting some SEO built in at compile time). The problem with Next.js is that it requires you to run on an actual server with the NodeJS runtime installed, whereas if you built your app into static HTML/JS/CSS, you can serve the files from anywhere, basically for free. This makes your frontend extremely scalable and cheap to serve. The only tradeoff is that you don't get the awesome SSR that comes from using Next.js -- however, you can still have some good SEO optimization by computing the render() functions at compile time (that's what GatsbyJS or react-snap do for you). Anyways, great video as usual! I really enjoy watching your stuff.
I dont understand much as Im just at the start of my CS schooling, but this kind of stuff is so interesting to see how it's actually done behind the scenes. Also 10/10 music selection at the paint part
Well, what you described is literally the reason why microservice-based architecture design became relevant. I'd strongly advice using a Kubernetes cluster with each microservice deployed as a different, separate service (i.e. frontend, backend, each webapp, etc.). Something like Google Cloud Platform Kubernetes Engine should be fairly easy and straightforward to get into and would give you an insanely scalable and reliable solution on the platform (and infrastructure) side.
man I really appreciate your work in making these videos. love watching them! Maybe you can even add some caching tools as well like Redis or Varnish to make it even cooler! keep up to good work :)
I personally have some good experiences with Azure Service Fabric. It's comparable to Docker or Kubernetes but allows you to create your applications in microservices. With these services it's much easier to make scalable applications because Service Fabric can automatically start or remove services based on your load.
Any specific reason why you wanna go with server side rendering? I would go for client side rendering. Especially for the web application, which probably will need some form of rendering after the initial load. So it prevents you from always reloading the complete page when you want to fetch some new data.
Most search engines already address this issue. You can also implement caching and ld+json to fix any other issues you have in particular. I mean, if you want to rank high on duck duck go, i guess you have no choice.
SEO is a good point, although I think you can still achieve acceptable SEO fairly easy with client-side rendering. In the end it's up to Devon. I just wanted to question his idea, that he tires to get to the bottom of it, before deciding for good.
And before I forget :) regarding scalability (because it was like the main point of this "design meeting"): Most of the time server-side rendering will be more performance heavy on the server. Of course this varies from framework to framework and how you structure your code. This doesn't mean that when using client-side rendering the performance of the site (as seen by the user) will be better. Usually quite the contrary. But like the name already says: The server does all the rendering. So you might run into problems if a lot of users try to access your page at the same time, when having a weak server.
Wow, awesome man. I wish you did more videos, I'm starting to study Computer Science this semester and I like your videos very much! (although I hardly ever understand what you're doing) hahaha keep doing them! hugs from Brazil! ;)
This is exactly how I do it. I'd suggest vue over react . I've used react, angular, and vue and vue is really great, especially if you're new. I've never tried next.
Love your videos! I just finished building a web app with scalability in mind as part of a capstone project. Looks like you're taking the right approach at tackling this and I'm excited to see the progression of your solution. I'm not sure if you mentioned it but look into docker. Containerization definitely helps with scalability and easy/fast deployments. What are you using to host your server? You can look into launching each app as independent microservices on their own AWS instances which would allow you to future scale them independently as they grow. Amazon offers great pricing on server hosting as you only pay for what you use.
Dude I'm been watching for a while and been using the same frameworks as you even before you did and now you're using next.js and react and I'm using them as well, what a coincidence lol
i love how he has a sock over his microphone. When you are a student, you have to be practical. Reminds me of the good old days of been in college but poor.
I don't really have any experience with what you're doing, but I think I understand the basics. I probably don't know what I'm talking about, but I really like the idea of trying out IPFS and neo4j, no idea what I could do with them until I try though! I do really like what you're doing, maybe I should dive in and try out a bit of everything, see what I like! Thanks for insiring me to try new things! :D
@Scott I agree, but also it would beneficial to leverage a cloud provider. Google Cloud has a very good Kubernetes Service if you want to go the container route. I personally would use AWS API Gateway and EC2 for scalable backend servers as well as their Application load balancers instead of Nginx for setting up routing paths. Static resources can be stored in AWS S3 and Cloudfront for CDN. Frontend can also be hosted on EC2.
Do you think you can post some kind of list of all the music you listen to? I enjoy all your music choices, specifically the one at 4:30, i tried to find it on soundcloud but could not, either way a playlist of all your music you use in your videos and or one of all the music you listen to in general (like a public soundcloud playlist?) would be really awesome, love all your vids!
Devon, it’s interesting to see a developer solve what is usually left to system admins and infrastructure architects. The solution you present is a nice illustration of breaking a system into discrete components on ONE server. How are your clients maintain state ?
You an use Vmware ESXI on a single server, then run multiple servers on there with pfsense as a firewall. Then you can use haproxy on the pfsense to load balance your front end to different front end servers. All on one server. Thanks for the insight on the web side.
Let me start by saying that for what.you are trying to achieve your setup is probably good enough. My suggestion below is if you wanted to create a production ready system. You can setup each of your services as a docker container, including your nginx instance. Use docker compose to create an internal network on the server. This has to advantages: 1.) Each service runs in it's own container meaning it can maintain it's own dependencies 2.) Since your services are containairzed you can easily throw them on kubernetes or aws if you need to with minimal work.
oef spicy code I'd first of all recommend using editorConfig and eslint for styling and linting. A lot of let variables can be converted to const, which is probably the easiest and fastest fix I always recommend. Next I'd recommend splitting your app.js into several routes and components. Your app.js is your starting point but shouldn't be your only point. What I did in my boilerplates for work is create a app.js that loads all global packages that you'd need, for example cookie parser or body parser. Once all those are loaded I run a function from a different file called "initAPi()", I keep it in the same directory but it can be wherever you want. In this file I handle the route logging (for debug mode), cors header handling, authentication en finally the routes themselves. It's just a list of app.use('/api/', somethingRoutes) in which the somethingRoutes are the routes of 1 specific REST module. The rest modules I group under a directory called REST, there I create groups for example user, items, admin, etc. each of these modules has 2 files: itemController.js and itemRoutes.js in the itemController I module.exports each function that I need for that specific module in the itemRoutes I create a new router variable from express. I then decorate that router with the route and the function from the controller itemRouter.get('/items', itemController.getItems) You can also prefix them with authentication functions from other controllers: itemRouter.get('/items', authenticateUser, itemController.getItems) I then exort the router. Back in my initApi file I simply import each router from the REST folder (you can do that manually or write a function that retrieves all file names based on a 'Routes.js' substring, so you never miss/forget one), and then put them in the list of app.use Lastly, I export the initApi function (but not with a trailing () because that will execute the function each time you import it). In most cases the app.js file will then only be about 50 lines or less. If you want to work on one specific REST module, for example your email, you can be sure that all other stuff won't change. You can even bring it up a notch by turning the controller functions in async functions and use custom reusable request and response handlers, which will give a perfect request/response or an error object. I can go on for days :D I love nodejs/typescript backend boilerplating!
I think you'd enjoy researching how to do a web architecture using micro services, I attempted this once for a toy live streaming site that used webtorrent to offload video and it's pretty cool if you can get it done right. the key for scaling is to never scale vertically, always try to scale horizontally. architecture the solution so you just need to spin up more servers to handle load dynamically and then spin down ones when the load is no longer there. Also, that feeling of not knowing wtf you're doing is normal and goes away with time.
wow! again! You know i was thinking about some programm which visualise the music from comp to a LED /RGB tape. And i thought it is easy. Bro. Love you that you showed us that none of the products are easy to do. Hope for new vids
You might want to have a look at Docker + jwilder/nginx-proxy. It's a reverse proxy as well, but takes a domain-name-based approach instead of a route-based approach and I've come to appreciate it a lot since it requires virtually no configuration (that is, if your Docker environment is already up and running).
Really like this type of video dude, it's nice to see someone else go through their thought process. This seems like the best way to host on a single server. What would be the process if an app crashes or something?
Devon you are doing a wonderful job by making such videos that are inspiring as well as motivating for our young programmers. You are an inspiration and motivating factor for me. Wish you best of luck for your future hope you have a wonderful future ahead. Keep up the good work!
Thanks for making this video! I'm a pretty junior programmer, and I get intimidated and overwhelmed easily with all the stuff that's available out there. (Do I use restify? express? what's the difference? Apply that to all the different things out there... ) Your videos really help me remember about how programming is really fun and useful, and makes me more motivated when I get back to the cubicle again tomorrow. :)
Hi, I separate my programs and deploy them on separated docker containers, so every frontends and backends are served on their own thread and with their own resources. I'm learning how to use kubernetes, that allows to create a "pod" which is a group that has containers and you can expose that pod as service and you get a single link to consume your backend and by default you get load balancing through the containers that are part of the pod
This is awesome, I've also been learning about scalable server architecture for the same reasons, so I'm excited to see what you come up with. Can't wait for the next video.
I wouldn't call the 8081 port the 'back-end' but rather your (restful?) (named) api. When you do a request to your 'front-end' port 8080 you are actually first going through a back-end which works like this: bootstrap the back-end application, route (with parameters? but handled by nginx), fetch the data and serve it to the front-end either by a templating engine (by rendering a template) (ex: twig) or through data output (ex: json). If you are just to render the homepage and do synchronous and asynchronous http requests then you are effectively creating a single page application. A service architecture per port is interesting, going to ask my devops how they locate the microservices.
Only 45 seconds in, but I noticed you had all your code in a single file which is not a good idea. You should use express middleware and use const router = require('express').Router(); and then module.exports = router; then require it in the main file, and use app.use('/routeHere', configuredRouterThatYouImported); I'm bad at explaining just look up 'Express Router', you'll figure it out. So now you can break your back-end up, and not have everything clustered into one big ass file. A little further in I notice that you're over complicating things, you could setup nodejs to just send back index.html always, then in React you handle the routes/and components should be loaded, and you can lazy-load/code-splitting stuff for performance. Then based on each route you can make a certain api request for resources.
I was first very skeptical of your comment, because I thought that you were speaking of NGINX or something else and it didnt make sense to me that you're talking about express routers because I didnt see the `app.js`. Looked back at 00:45 , and app.js is really messy indeed. Good catch. Although, the rest of the video is pretty solid, if I havent missed something. At least the overall content.
Your application may need to spin up additional servers to handle increases in traffic to your application. You should check out docker while you're still researching. It would allow to use a microservice architecture that would be way easier to scale on multiple servers.
I miss modular and mininal in the opening. Express is a great choice. I trimmed that down, reducing depencies to absolute basics, like cURL and gettext. You need to redesign the stuff with Emascript spec. as baseline, like the browsers. The rendering stuff, which is also greatly trimmed and worked on in the GNU/Linux & FOSS world. The Mesa/GBM/KVM/DrI3/opengles/vulkan path is changing the world and the preferred languages are C/C++, Javascipt and Python. Then comes packaging..
![Coding a YouTube Descriptions Updater [ Part 1 ]](/img/1.gif)
Devon, I am a 44 year-old software developer in the Enterprise world. I have to say your videos are some of the best I've seen for what you seem to want to accomplish.
You're showing the people who watch this video the fact that you have no fear of digging through documentation, putting in lots of hours, overcoming frustration.
So many people wanting to learn how to code just assume that "experts" just go on their keyboard and just start typing away without frustration. But you seem to show the reality of it.
I would love to see you talk more about just the mental process you go through from square one to finish, the habits/practices you follow and the mental challenges you have to overcome to convince yourself that you can do this stuff.
I think that's the biggest hurdle for a lot of people is that feeling of imposter syndrome.
You were going to do really well in your career. Keep up the good work man.
I watch devon's videos too Can I get a job please? *.*
@@XenoContact im confused now.
I want you to know as a 31 year old developer currently struggling with doing new things out of my comfort zone this line "people wanting to learn how to code just assume that "experts" just go on their keyboard and just start typing away without frustration. But you seem to show the reality of it." is what I needed to hear right now, thanks
@@germanhoyos4422 You're never too old to learn something new - go get 'em :-)
I feel your comment is very important for people who know nothing in programming mainly this part "You're showing the people who watch this video the fact that you have no fear of digging through documentation, putting in lots of hours, overcoming frustration" many people fail to realize even the most competent of programmers sit reading through documentation to make stuff work. You may know every piece of syntax and code for a given language but making it work how you need is not always going to be easy without documentation :) I feel there is often too much feeling of imposter syndrome for new coders and sometimes are afraid to read documentation feeling they are cheating however it could not be much further from the truth :) the documentation is there for people to learn from, the original library creators give the documentation to aid others in working with their library and such. Do not be afraid to read documentation and watch endless videos it will make you more competent in your goals and project. I myself am stuck deciding on what project to even embark on to expand my basic c# knowledge after learning from unity making games I now want to expand into software and other things ! The key here is to read and learn and read some more :) happy learning !
> have photoshop in the taskbar
> launches Paint to draw a workflow
kzm 夢を I probably do that same kind of stuff in microsoft word and it is easier because in word, you can snap the boxes to be in line with each other and make sure that the boxes are in the same dimensions. Also switch from portrait to landscape 🤣
Doug Cannon man you can do the same in photoshop once you're used to it and it's easier imo
just a matter of habit i guess
this is galaxy brain
Hahahaa
first thing I did when I installed my first Linux distro - googled "alternative to MSPaint".
Pinta is great btw :D
Screw Python, programming in paint is the real deal in 2018
Teodor Bajenaru but...
“Python’s where it’s at”
(According to ads I get at least)
Its_ Me lol haha, I get that ad too 😂
So funny! I see that add every single day, it pisses me off lol
Someone has built a working program by making a bitmap through paint.
I also get the Unity C# course too and I did end up buying it. It is actually pretty great.
I would do it just like this, but put each of the services in a docker container. That way you can have the container restart itself if the service crashes. Load balancing between containers instead of ports.
I also vote for this idea. It really is a step up.
More work eventually ends up in more convenience.
And run the containers with Kubernetes!
Another thing that might consider, is to use javascript in asynchronous way.
Also, docker swarms could be used if you happen to have more machines available to run your apps at some point.
heads up, docker containers don't automagically restart, there's some further configuration and setup needed to catch process ending errors and restart (pm2, forever, etc).
All developers should do this every so often, in any field. This not only gives other people insight in your way of thinking, but it also serves the 'rubber ducky' purpose. Because you're explaining it to others, your mindset changes about the problem and you get additional different views as well. Good practise man, awesome!
Amazing video with an amazing ad, just amazing! You don't know how much you help me with these videos each time you upload, not just because of the knowlegde, but also the motivation to graduate and learn so much more! Thanks a lot man.
I’m a photographer and have no knowledge what so ever in coding, I just enjoy learning from you man. Keep it up.
I'm a final year computer science student this year, and this is the most informative channel for me on RUclips. This is exactly the same topics i'm a struggling with and learning to myself at the moment. Keep up with the good videos dude, there are people out there who really benefit from them.
yoo i watched you since minecraft and it’s crazy how you grew this channel so fast and so big! I remember when this channel was at 1K :D.
I’m learning so much from these video’s and this is the kind of stuff i wanna do later so this is really fun to watch and you make it even more entertaining. You really motivate me to do this kind of stuff like coding. Thanks for this amazing content! :)
I like your approach to programming related topics. It's fluid. You sketch first then dive deeper later. Seems very productive
Don’t know you at all, but this type of video is so valuable. It’s a piece that is missing from the “self taught” coding world. It’s like there are the basics courses and then there are completed project courses and then there are “watch expert live coders do amazing things”. This right here is the messy middle of “how do you Think about a project”. Thank you for this “design meeting”. I’ll be following you.
Where have you been?? I'm excited for this video
Same
bruh
I'm learning more and more programming right now, and I'm building up to deploy a website. I keep coming back to this video every 2 days or so to see how much more I understand of what's going on.
Great stuff, Devon. Your coding channel always delves into new tech and I want to check out next.js after this. Cheers for another great vid!
This is a pretty good start. As long as you have your ports in order, and you're only working with one server, this is a pretty solid plan. Though I would just call this application organization, scalability kinda refers to multiple servers, load balancing, high availability, and managing requests with idempotency
I honestly watch these having no idea what your talking about but somehow am just genuinely interested in what your talking about 🤷🏽♂️😂. It’s like I’m in a class that I don’t need for my degree but your a really cool professor so I automatically get sucked in to paying attention. Idk if that makes sense but keep up the good work!!! Been subbed to you for a good min.!
I've seen a few others mention this already but definitely look into Docker and with it Docker Compose. Using Docker Compose you can not only automate the process of booting all of these services at once but also save yourself work in the future when deploying individual services becomes as easy as running a simple docker command. I use Docker for work and my personal server which is set up much like your proposal, and it works beautifully. Keep up the good work though man, love seeing how others approach the same problem.
I like how you shoot your videos and edit them, inspires more developers like me to do the same with more creative ways. Keep up the good work!
You make me get excited to get back to programming and learning new stuff
Such an inspiration , these new type of youtube channels are amazing
and you are on the top of them in my opinion
KEEP IT UP
Dude.. your videos are so unique and fun, please keep doing what you doing! Great vid!
Another step to make your API scalable is versioning. If you have multiple apps talking to the same endpoints, changing the endpoints (either response/request or route) without versioning would require you to update all applications to understand the changes.
With versioning, however you can create a new version of an endpoint and avoid breaking other apps the uses the older version. This allows for incremental updates without the need to break anything.
I don't understand anything.
I do code haha. I'm a sophomore studying CS. This just is really complicated.
I wouldn't say to just start coding, you could be diving in without the necessary tools. Since you're a student, you probably are learning the foundations of computer science - how to code a basic program, concepts like object-oriented programming, data structures, algorithms (etc). That's computer science.
A lot of what this video deals with is software engineering, which you will start to learn but it's also its own thing. I wouldn't expect a computer science curriculum to immediately teach you full stack web application design early on; confusion would be normal. Stuff like designing large scale web applications, client-server architecture, and some computer networking (what's kinda happening in this video) typically come after there being a foundation.
Best to understand where you are before "just coding". If you do want to dive head first, there are a lot of tutorials on projects that will help build up to something like this: build a personal website, start an Android/iOS app, etc.
@Ruben You're so helpful
@Ruben 10/10 best fucking help
No worries, like Yusuf Ahmed said.. I've only dived into this type of development in the past few months. The past 3-4 years for me have mostly been computer science fundamentals
Ohh, how awesome! I am also getting started with NodeJS etc, you are basically doing exactly what I want as well. But haven't got the time to actually get my head wrapped around it due to work.
Looking forward to the next videos, I hope I can learn a lot of stuff from that already!
I love seeing other people working through this kind of thing. Good luck.
I've just completed my AWS Developer Certification, AWS does this all for you. Use an Auto scaling group with a min and max amout of instances/servers, use an Elastic Load Balancer to assist with routing and load of incoming traffic. You could also go serverless and use AWS lambda, API Gateway and Dynamo for your backend calls, then use S3 to host your static files.
dude, you are insane! don't stop doing ur vids
oh jeeeeeeeesus he liked ;0
Congrats on such a cool sponsor, keep it going man! You do inspire me, спасибо
This was awesome. I've been pissed that I could not find a video like this sooner. I wasn't even sure where to begin.
i have no idea what hes saying but he presents all this info in such an entertaining way.
If you want to build a big website or just make a small website super performant, i'd recommend next.js for the front-end (which you already use) and making the back-end using Go since it's super fast and faster than NodeJS.
Jesus Christ Devon! What the heck happened?! You grew so much! Hope you remember me haha, your first fan you have ever met, sort of by accident. But I’m happy for you man. You grew more than I thought and I’m extremely impressed. Love the content and I’m gonna binge to catch up
- nick, that weird dude from X15
Awesome videos where you give a great overview of what you need to do/learn to implement something! Keep doing these videos!
It's amazing how I spent a full week researching this.
My goal is the same, to separate my project into multiple servers. One serves the frontend, another serves the backend. From there every application can be dedicated and/or bundled up somewhere, ideally organized to keep maintenance seamless when interacting with applications.
I'm going to look into what you recommended to do and I thank you for finding out all of this.
I would love to see your project files (excluding your applications of course) or if you made a boiler template out this, but that's a lot to ask for. Regardless. Thank you.
Half video it's me actually understanding your code, the other half it's me looking your beautiful face, admiring how smart you are.
I am beyond happy that I was able to keep up with this. Thank you!
Best Computer Science channel in all of RUclips!
Man, you're videos are just awesome! You inspire me to start a project without knowing anything ✌🏼
Sounds like a docker problem
Exactly
Honestly this may make my current project work even better, but I do recommend using docker with the same setup you beautifully drew in paint.
Hey Devon. I really appreciate posts like this. Thank you.
I always learn some more in each of your video so thanks Crawford
Docker container per app + Nginx on the main machine to route to each one. If anything grows big and consumes a lot of resources, you can extract it to separate server without any efforts.
Every time I watch one of your videos I get so amped on programming and geeky computer stuff. So amped that I actually do my homework
I'm currently interning at a bank in Australia and what you said about the complexity is true. We also use Nginx to route our different applications/API's we build.
If you want it to be even more scalable, you could create a fully static frontend (using something like GatsbyJS or react-snap for the frontend. This lets you use React while still getting some SEO built in at compile time). The problem with Next.js is that it requires you to run on an actual server with the NodeJS runtime installed, whereas if you built your app into static HTML/JS/CSS, you can serve the files from anywhere, basically for free. This makes your frontend extremely scalable and cheap to serve.
The only tradeoff is that you don't get the awesome SSR that comes from using Next.js -- however, you can still have some good SEO optimization by computing the render() functions at compile time (that's what GatsbyJS or react-snap do for you).
Anyways, great video as usual! I really enjoy watching your stuff.
Just looked more into Next.js and realized they have a static HTML export feature. You can disregard my comment :D
Digging, hope you continue to architect and build it and take us along the journey. Loved this and the CS degree video.
I dont understand much as Im just at the start of my CS schooling, but this kind of stuff is so interesting to see how it's actually done behind the scenes. Also 10/10 music selection at the paint part
Well, what you described is literally the reason why microservice-based architecture design became relevant. I'd strongly advice using a Kubernetes cluster with each microservice deployed as a different, separate service (i.e. frontend, backend, each webapp, etc.). Something like Google Cloud Platform Kubernetes Engine should be fairly easy and straightforward to get into and would give you an insanely scalable and reliable solution on the platform (and infrastructure) side.
dude love this kind of video of yours no one does this except you , i get a grasp (somehow lol) on some web apps .
man I really appreciate your work in making these videos. love watching them! Maybe you can even add some caching tools as well like Redis or Varnish to make it even cooler! keep up to good work :)
I personally have some good experiences with Azure Service Fabric. It's comparable to Docker or Kubernetes but allows you to create your applications in microservices. With these services it's much easier to make scalable applications because Service Fabric can automatically start or remove services based on your load.
Any specific reason why you wanna go with server side rendering? I would go for client side rendering. Especially for the web application, which probably will need some form of rendering after the initial load. So it prevents you from always reloading the complete page when you want to fetch some new data.
SEO. If you want your SPA to rank on a search engine without various hacks, server side rendering with Next.js is painless.
Most search engines already address this issue. You can also implement caching and ld+json to fix any other issues you have in particular.
I mean, if you want to rank high on duck duck go, i guess you have no choice.
SEO is a good point, although I think you can still achieve acceptable SEO fairly easy with client-side rendering.
In the end it's up to Devon. I just wanted to question his idea, that he tires to get to the bottom of it, before deciding for good.
And before I forget :) regarding scalability (because it was like the main point of this "design meeting"):
Most of the time server-side rendering will be more performance heavy on the server. Of course this varies from framework to framework and how you structure your code.
This doesn't mean that when using client-side rendering the performance of the site (as seen by the user) will be better. Usually quite the contrary. But like the name already says: The server does all the rendering. So you might run into problems if a lot of users try to access your page at the same time, when having a weak server.
@@Anony584 SEO isn't a problem anymore for client side rendering. Google has confirmed that it waits for JS loads when indexing pages.
Keep this style of videos coming, they ROCK!
Wow, awesome man.
I wish you did more videos, I'm starting to study Computer Science this semester and I like your videos very much! (although I hardly ever understand what you're doing) hahaha
keep doing them!
hugs from Brazil! ;)
I've checking everyday ur channel and ur videos makes my day better Thanks!!
This is exactly how I do it. I'd suggest vue over react . I've used react, angular, and vue and vue is really great, especially if you're new. I've never tried next.
You are just amazing. I really love your videos man. You are my inspiration.
Love your videos! I just finished building a web app with scalability in mind as part of a capstone project. Looks like you're taking the right approach at tackling this and I'm excited to see the progression of your solution. I'm not sure if you mentioned it but look into docker. Containerization definitely helps with scalability and easy/fast deployments. What are you using to host your server? You can look into launching each app as independent microservices on their own AWS instances which would allow you to future scale them independently as they grow. Amazon offers great pricing on server hosting as you only pay for what you use.
Btw I used the same exact tech stack as you, so if you have any questions feel free to ask!
Dude I'm been watching for a while and been using the same frameworks as you even before you did and now you're using next.js and react and I'm using them as well, what a coincidence lol
I know fuck all about programming but I just love to watch this dude mainly cuz he is just so damn enthusiastic about what he does
This is so interesting. I would watch your videos every day! Thank you for sharing :)
i love how he has a sock over his microphone. When you are a student, you have to be practical. Reminds me of the good old days of been in college but poor.
This is literally what I’m trying to learn rn thank you so much
I don't really have any experience with what you're doing, but I think I understand the basics.
I probably don't know what I'm talking about, but I really like the idea of trying out IPFS and neo4j, no idea what I could do with them until I try though! I do really like what you're doing, maybe I should dive in and try out a bit of everything, see what I like! Thanks for insiring me to try new things! :D
@Scott I agree, but also it would beneficial to leverage a cloud provider. Google Cloud has a very good Kubernetes Service if you want to go the container route. I personally would use AWS API Gateway and EC2 for scalable backend servers as well as their Application load balancers instead of Nginx for setting up routing paths. Static resources can be stored in AWS S3 and Cloudfront for CDN. Frontend can also be hosted on EC2.
Do you think you can post some kind of list of all the music you listen to? I enjoy all your music choices, specifically the one at 4:30, i tried to find it on soundcloud but could not, either way a playlist of all your music you use in your videos and or one of all the music you listen to in general (like a public soundcloud playlist?) would be really awesome, love all your vids!
Can't wait to see progress on this!
I'm just some high school kid and I'm trying to learn more about back end dev and stuff like this is really useful. thanks man :)
Devon, it’s interesting to see a developer solve what is usually left to system admins and infrastructure architects. The solution you present is a nice illustration of breaking a system into discrete components on ONE server. How are your clients maintain state ?
love the vids, please dont stop motivating me through college :) engineering student loans are insane
You an use Vmware ESXI on a single server, then run multiple servers on there with pfsense as a firewall. Then you can use haproxy on the pfsense to load balance your front end to different front end servers. All on one server. Thanks for the insight on the web side.
You should definitely version your API. This way you can still support services that use your old API.
Let me start by saying that for what.you are trying to achieve your setup is probably good enough. My suggestion below is if you wanted to create a production ready system.
You can setup each of your services as a docker container, including your nginx instance. Use docker compose to create an internal network on the server.
This has to advantages:
1.) Each service runs in it's own container meaning it can maintain it's own dependencies
2.) Since your services are containairzed you can easily throw them on kubernetes or aws if you need to with minimal work.
oef spicy code
I'd first of all recommend using editorConfig and eslint for styling and linting. A lot of let variables can be converted to const, which is probably the easiest and fastest fix I always recommend.
Next I'd recommend splitting your app.js into several routes and components. Your app.js is your starting point but shouldn't be your only point.
What I did in my boilerplates for work is create a app.js that loads all global packages that you'd need, for example cookie parser or body parser. Once all those are loaded I run a function from a different file called "initAPi()", I keep it in the same directory but it can be wherever you want. In this file I handle the route logging (for debug mode), cors header handling, authentication en finally the routes themselves. It's just a list of app.use('/api/', somethingRoutes) in which the somethingRoutes are the routes of 1 specific REST module.
The rest modules I group under a directory called REST, there I create groups for example user, items, admin, etc.
each of these modules has 2 files: itemController.js and itemRoutes.js
in the itemController I module.exports each function that I need for that specific module
in the itemRoutes I create a new router variable from express. I then decorate that router with the route and the function from the controller
itemRouter.get('/items', itemController.getItems)
You can also prefix them with authentication functions from other controllers:
itemRouter.get('/items', authenticateUser, itemController.getItems)
I then exort the router.
Back in my initApi file I simply import each router from the REST folder (you can do that manually or write a function that retrieves all file names based on a 'Routes.js' substring, so you never miss/forget one), and then put them in the list of app.use
Lastly, I export the initApi function (but not with a trailing () because that will execute the function each time you import it).
In most cases the app.js file will then only be about 50 lines or less. If you want to work on one specific REST module, for example your email, you can be sure that all other stuff won't change.
You can even bring it up a notch by turning the controller functions in async functions and use custom reusable request and response handlers, which will give a perfect request/response or an error object.
I can go on for days :D I love nodejs/typescript backend boilerplating!
Yep, that's architecture is pretty good. If you ever code a load balancer in one of your vids I'll be the first to watch. Well done, Devon.
Glad you switched the topic to programming. Looking up to more videos!
I think you'd enjoy researching how to do a web architecture using micro services, I attempted this once for a toy live streaming site that used webtorrent to offload video and it's pretty cool if you can get it done right.
the key for scaling is to never scale vertically, always try to scale horizontally. architecture the solution so you just need to spin up more servers to handle load dynamically and then spin down ones when the load is no longer there.
Also, that feeling of not knowing wtf you're doing is normal and goes away with time.
Definetly look into docker (compose), gonna make your life way easier, especially when it comes to the deployment of the whole system!
wow! again! You know i was thinking about some programm which visualise the music from comp to a LED /RGB tape. And i thought it is easy. Bro. Love you that you showed us that none of the products are easy to do. Hope for new vids
You might want to have a look at Docker + jwilder/nginx-proxy. It's a reverse proxy as well, but takes a domain-name-based approach instead of a route-based approach and I've come to appreciate it a lot since it requires virtually no configuration (that is, if your Docker environment is already up and running).
Really like this type of video dude, it's nice to see someone else go through their thought process.
This seems like the best way to host on a single server. What would be the process if an app crashes or something?
Devon you are doing a wonderful job by making such videos that are inspiring as well as motivating for our young programmers. You are an inspiration and motivating factor for me. Wish you best of luck for your future hope you have a wonderful future ahead. Keep up the good work!
Thanks for making this video! I'm a pretty junior programmer, and I get intimidated and overwhelmed easily with all the stuff that's available out there. (Do I use restify? express? what's the difference? Apply that to all the different things out there... ) Your videos really help me remember about how programming is really fun and useful, and makes me more motivated when I get back to the cubicle again tomorrow. :)
you made me take on a whole new perspective on programming... that it should be a fun learning experience :)
You should do some "what's inside my windows and linux" distributions video.. like what you prefer using for programming etc etc..
I laughed so hard when you captioned "Some High Quality Documentation", then I know what's going to happen next. Legendary MS Paint drawings.
Hi, I separate my programs and deploy them on separated docker containers, so every frontends and backends are served on their own thread and with their own resources. I'm learning how to use kubernetes, that allows to create a "pod" which is a group that has containers and you can expose that pod as service and you get a single link to consume your backend and by default you get load balancing through the containers that are part of the pod
I am so excited.. Please be more regular with videos. It is soooo awesome !!! Please try to be regular... Thanks !!!
This is awesome, I've also been learning about scalable server architecture for the same reasons, so I'm excited to see what you come up with. Can't wait for the next video.
I wouldn't call the 8081 port the 'back-end' but rather your (restful?) (named) api.
When you do a request to your 'front-end' port 8080 you are actually first going through a back-end which works like this: bootstrap the back-end application, route (with parameters? but handled by nginx), fetch the data and serve it to the front-end either by a templating engine (by rendering a template) (ex: twig) or through data output (ex: json). If you are just to render the homepage and do synchronous and asynchronous http requests then you are effectively creating a single page application.
A service architecture per port is interesting, going to ask my devops how they locate the microservices.
Only 45 seconds in, but I noticed you had all your code in a single file which is not a good idea. You should use express middleware and use const router = require('express').Router(); and then module.exports = router; then require it in the main file, and use app.use('/routeHere', configuredRouterThatYouImported);
I'm bad at explaining just look up 'Express Router', you'll figure it out.
So now you can break your back-end up, and not have everything clustered into one big ass file.
A little further in I notice that you're over complicating things, you could setup nodejs to just send back index.html always, then in React you handle the routes/and components should be loaded, and you can lazy-load/code-splitting stuff for performance. Then based on each route you can make a certain api request for resources.
I was first very skeptical of your comment, because I thought that you were speaking of NGINX or something else and it didnt make sense to me that you're talking about express routers because I didnt see the `app.js`.
Looked back at 00:45 , and app.js is really messy indeed. Good catch.
Although, the rest of the video is pretty solid, if I havent missed something. At least the overall content.
Hi, I from Brazil, I doing software engineering, I love watching your videos, it always inspires me to study more
Dude you really go crazy with that Paint workflow xD
Your application may need to spin up additional servers to handle increases in traffic to your application. You should check out docker while you're still researching. It would allow to use a microservice architecture that would be way easier to scale on multiple servers.
I setup something similiar with docker containers and I used haproxy to send traffic to the correct applications. It's pretty sweet
I miss modular and mininal in the opening.
Express is a great choice. I trimmed that down, reducing depencies to absolute basics, like cURL and gettext.
You need to redesign the stuff with Emascript spec. as baseline, like the browsers.
The rendering stuff, which is also greatly trimmed and worked on in the GNU/Linux & FOSS world. The Mesa/GBM/KVM/DrI3/opengles/vulkan path is changing the world and the preferred languages are C/C++, Javascipt and Python.
Then comes packaging..
This is the pattern that I follow for my apps (Nginx). It's perfect.
I love when you upload these kinds of vids!!! 👍👍👍👍
Yeeeeey he's back!!! Hyped :)
Does data not move much slower communicating between servers/ports?