jokes aside, he's not a bad looking dude at all, getting jacked wouldn't hurt tho, as long as he doesn't covert to a life coach and talk about it non-stop like that other coder youtuber John Sonmez.
@@TechdubberStudios It may pay less than ads, but it's many times better. I support websites that responsibly use cryptomining, and I block ads. Please, don't say that ads are better. They have never been any good to anybody's web browsing experience. Oh, and you can use cryptomining along with Arc, another earning method that does not involve ads. I'm done with Google's creepy trackers. Cryptocurrency mining is the future.
@@ezshroom I am genuinely 100% with you on the crypto movement. I hate ads. Always have hated them. But there are at least 2....3 big corporations that come to mind that were built on the ads business model, but with crypto mining... can't find one. And browser-crypto-mining is not exactly a new technology. I really want it to replace ads. I really do. Hate the pop-ups, spying, tracking, that's going on. And the first corpo that comes to mind would be Netflix, when considering whom should adopt the crypto model. Because the users stay on netflix and binge-watch hours and hours!
@@ezshroom also, do you happen to know any website/forum/subreddit focusing on browser-based mining? I would really like to join and dig in more into this subject.
Lol just killed the whole argument. Never heard of it before. Just goes to say that tech is exponential. Wonder if it will cause the cosmic crash eventually.
Fantastic job explaining this! As always, the hilarious dry humor and "next level" metaphors help drive home points and keep things entertaining. Really helped clear up a bunch of stuff and get me pointed in the right direction. Many thanks!
Also check out the create-exact-app npm (that's exact not react). Like NextJS but Express-forward design, full control at the server side level of what's going on.
@@jpsimons Just FYI, next.js also gives you full server side control. You can just run next as a library within an express server. In my experience, it's super ergonomic while preserving the state-of-the-art benefits of next (code splitting, automatic static optimization, incremental static generation, etc.). Having said that, I have not yet checked out create-exact-app, and am not sure how it differs from nextjs.
I avoided serverside rendering a meta tag by registering a sub-domain, doing the serverside-rendering there and making my app only compatible with a set number of user-agents. Brilliant!
This is not true. You can have Client Side Rendering with Server-side routing. You send a different HTML page with some meta tags to describe your page for a different route This is my project following CSR and I don't have this problem github.com/polavi-project/ecommerce/
this feels like server side rendering. If you click anything at the ecommerce its loading all again... I didn't study your code, but this feels like classic php rendered eshop :D
Greate video! I use Laravel on the server side to serve up everything. Static html pages and React apps or a combo of both. It's easy to embed a react app within a .blade template file. Meanwhile Laravel takes care of everything else, like API services, user registration and authentication, etc. Best of both worlds.
I just put my meta tags with variables like %PAGE_NAME% %PAGE_IMAGE% and replace these later while serving the page with express. doesn't work while client-side routing but It works for link previews.
I think your points on pre-rendering are slightly off. Not mentioning tools like Gatsby, Scully, or Gridsome is a miss as they can be used to render those dynamic routes you say cannot be pre-rendered. It's worth mentioning that JAMStack options are becoming really incredible for developers and give you the same benefits as server side rendering out of the box.
Yep, that was a huge relief after realizing this problem when I was almost done with my first website. Easy to set up too. Never tried Next.JS, but I'm interested. Is it worth it for multi page apps?
7:20 would it be possible to use the same url but check the header value on the for example nginx server? like is this user agent a bot (twitter, fb, etc.) => proxy to your slim API for only the meta data response and if its a real user (mac, windows, chrome, firefox user agent etc.) => proxy to your real page / default response / SSR page. maybe im forgetting something. i dont know if this could work.
I had to do that once, I used a Lambda function since it was hosted on AWS, and the function intercepts the CloudFront distribution request and updates the HTML if the request comes from a robot, adding the OpenGraph tags.
Why do you want another domain for that. The server that delivers the app index.html could also deliver the meta response instead, based on user agent. One problem in both situations you could get (separate service or combined), is that the bots make checks whether the content they see is different to a regular user. Whether any of them does it, I don't know, but I would check this possibility.
Our course -- I followed and understood all the way to the end. This is because I'm an unemployed ex-Tech Lead [who has never worked at a FANG company], and a thousandaire.
there is a workaround : just add conditional tag in the small server that builds your page. you can still use client side rendering except for meta tags
Hey Ben, I'm starting a new react web app for something like delivery.com and doordash.com where SEO is a major thing. Like rich structured snippets of stars etc in google results. Is SSR the only option?
In that case Gatsby or NextJS or any other you recommend? I don't know if Gatsby will do but their web.dev score hits 💯 so was thinking that but in a dilemma
@@bawad do you know an easy way to migrate from create react app to next.js? I've found problems with REDUX, svg images and react router active route (NavLink). Great video btw ;)))
A solution to your problem could be to build a single page application, with each end point for the app being pre rendered. It's basically jamstack. Once a user loads one page, the others do not need to be loaded.
There are benefits to client side rendering? Btw rather than checking the user agent, just include a client side redirect location="whatever.com" Because the bots dont have javascript they wont be redirected, but users will
I made my "yarn deploy" script change react's build/index.html to build/index.php, then it throws in a little php script that grabs some info from the good ole' mongoDB and 'echo's the metatags into the , only when that url is directly accessed. (You can get the URL params in PHP) It's only a few lines of code. The hardest part was figuring out how to configure Nginx's reverse proxy to render PHP. O_O
If you manage the web server, you could use the web server's router to do the same exact hack you described without the need for a different subdomain, just a route that checks the user-agent of the client and returns different HTML based on it.
I started out doing server side rendering using Java/PHP spitting out html. Now that I'm researching client side rendering the first limitation I found is I have no f*cking access to any server resources. All server resources a page needs have to be directly accessible from the client. BAD.
I'm not a fan of server side unless it's to generate a HTML based email. Just seems so cluttered when doing work. Even the emails I generate on the server side are dreadful to work with but barrable since there aren't many.
In your example of the sub domain. I think that’s not needed / don’t need to redirect. If you are using a server side program it can decide to return the react app or the preview on the normal domain using the same user agent logic or an IP address list of known Facebook servers.
That's what I was thinking. Maybe something like this to handle the routing internally to an SSR-generated page: www.nginx.com/resources/wiki/modules/user_agent/ or this: serverfault.com/questions/775463/nginx-redirect-based-on-user-agent I'm sure there are other reverse proxies that support this, but this doesn't seem to be the most portable solution. Edit: added an example of a similar implementation from ServerFault
Don’t build a SPA if you are building a website. The A in SPA is for application. Use SSG or SSR. Pre-rendering the page for web crawlers is a hack especially if you have a page that needs to display dynamic data like product price. You need to constantly update you pre-rendered pages and it’s not everyday that your pages will be crawled by SEO engines. Lastly, don’t ever mess with SEO and meta tags if you don’t know what you’re doing. Always check with an SEO expert or use a good SEO tool. SEO is not just some random buzz word that web developers need to learn quickly. SEO is a long and sometimes painful game.
lol dude just make a request to your server to update it for you. complicated sure, but who the hell cares people just want it to work. what's interesting that I also haven't considered in my application is that users might want to visit the site if someone shares a link, if user makes a request all i can do is spit json back at them, i have no template for them to visit, and only users can see if they are logged in, i'll need to make guest template for non logged in users
You can write normal HTML with some normal CSS just to make the preview template, imagine that this is the loading page, and only once the client-side rendering is ready, you can show the content of that on top of that base temporary HTML template. I've never done client-side rendering before, and I've never used React, but I think this should work. Why are you saying that it's not possible?
Detecting the user agent and serving different content is called "dynamic rendering": developers.google.com/search/docs/guides/dynamic-rendering I've actually been wondering about this lately... whether to use Next.js or set up dynamic rendering so that only bots get the SSR version. But I don't get why you're talking about adding the "share." subdomain to serve different content? Why not just use dynamic rendering on the regular URLs?
I done this in a similar way. Except the share link page didn't check any user agent headers. It just featured a JavaScript redirect to the normal URL. Bots don't run JavaScript, don't redirect, they get the meta tags they need. User does run JS, redirects to the main SPA. And, for good measure, a fallback clickable link on the share page shows in case redirecting is blocked.
I'm sure someone has already pointed out, but if someone copies the url from the browser, they won't have the `share.` prefix. Probably need a global UA interceptor that results in pages being rendered server-side. Year 2030: Browsers introduce a feature to request server-rendered pages by pretending to be a facebook/twitter bot.
It works, that's what I do too. But forget about the subdomain solution. Lots of people don't care about some button "sharable link". They grab the url from the browser and send it to their Friends via FB or Whatsapp. I detect the crawler and redirect to a subdomain. When it comes to SEO - according to google they are fine with that, as long as the content of the redirect is not that different from the non-redirected version. But it's all hacky and not elegant. Facebook should recognize, how the web actually works nowadays and update their crawlers to execute javascript and allow to make xhr requests.
when you share a deep link, the deep link shouldn't point to the webapp, but to server-generated pages with proper meta tags. The generated pages are dummy pages that redirects to the webapp.
Isn't this what client side routing or search queries are for? On document load before the window. Try some JSON data from thread, clone or populate a document fragment with your html and finally render the fragment using the query parameter and its prefix/suffix..
The User-Agent hack will work, but you risk getting penalized by search engines - some of them test with a real browser string to see if you're trying to manipulate the search engine. You might be able to look more "honest" to a bot by serving a Vary: User-Agent header and being up front about it, but then your caching is in the toilet. I'm afraid SSR is pretty much the only way to go, though personally I wouldn't use a framework to render the whole page on the server side (which is only an option if your server side is JS anyhow) but just render the non-interactive layout and content, then inject the interactive parts on the client side. The content that's interesting in terms of SEO tends to be non-interactive, and interactive elements generally aren't SEO relevant, so this tends to work out quite well.
So everyone knows Javascript is slow, C++ is fast... What if you want to create a browser game where performance is important? Simple, code your back end in C++ and have it render each frame and then send the result to the client. Problem solved!
Had this issue while using a MeteorJS website running with ReactJs as the client side is that we created a crawler ( i think there is an npm project for this) that would go to every page, render it and save it on our DB. When a non-human (Google) would access the site it would served this rendered HTML, making it SEO friendly. Basically our server would use the User Agent to define what type of content the user would get served. Hope this help
can you please explain what do you mean by 'render it and save it on db' , do you mean like render the dom elements and attach to html and store it in some link and add that link in db , or what exactly are you storing in db, if this is the case wouldn't it be too much for pages which are dynamic like /rob/photos, /sam/photos and likewise to be stored in db or am i missing something
Your solution is nice, but people copy the url from the browser, and paste it where they wanna share it, and that won't work (they link will not have the "share" subdomain.. Or am I missing something? Anyway I use angular, and it has server side rendering so I think i will just use that.. You can also just switch to angular.. I know you like it
Don't look at the User-Agent: that way lies madness. Instead, generate share.careerexpectations.com/reactjs with all the appropriate meta tags - and a line like this: window.location = "careerexpectations.com/reactjs". If a bot or spider or search engine or whatnot visits that page, the script tag will be ignored; if a browser does, the meta tags will be (mostly) ignored but the script will be executed and the user will be redirected appropriately. Alternatively, of course, drop the share subdomain, and just generate every page. It might be a little slower, since you'll be generating all this meta-data for browsers that don't care about it, every time the user reloads from a page other than the root, but it's simpler and you don't have to worry about what happens if the user shares the URL from the location bar instead of pressing the Share button.
Ngl I'm pretty sure the implementations are for security reasons. It wouldn't be fun if someone can simply inject arbitrary javascripts through link previews.
Why not send the proper meta tags according to the route by default in html and leave it up to scripts to overwrite them for the web client SPA. This way they web crawlers get their static tags because they never render with the scripts and SPAs get their correct content. Maybe I need to elaborate more but I hope you get the gist. It's possible this is how some of the SPA framework static rendering frameworks (Vue>NUXT, etc) operate as this doesn't seem like that crazy of an idea
Ok sure, you wont get the meta thing when you share a link. But what about the hard refresh on all pages and the form validations you need a server to be active on all your calls. Submit a simple form, wait for the server to respond. Nope, did not work, show the errors to the client. And repeat. (please do not say that you will send validation function alongside the page). How would you load a huge data with a spinner? Serve a page with a loader, and when the data is loaded, respond back again with the data?! (WebSocket) Or would you rather serve a page, that does another API call to get the data. How would you manage all that?!
Nice post. However I am curious as to why these FB bots can't read a client rendered app? I am sure that these bots are ultimately rendering the app on a web engine/browser, because they need to generate the preview of the rendered content. The static html will also most likely load some resources like CSS and JS over the network (unless the bot expects an inlined css/js), so it's not going to be an instant preview. So maybe the limitation is not technical but rather functional, in that the bot is probably not willing to wait too long for the page to load (which is the case with most client rendered apps)?
This isnt a problem with client side rendering. It's a problem with the places building these link previews. They could just as easily run a bit of js to get an accurate preview. Google already does this afaik. But facebook tends to be a bit behind on this sort of thing.
For some wierd cases like mine where only /particularRoute need to work like SSR what i actually tried was hosted gatsby project in particular route of CRA project and it worked just need to handle few re-routing cases
Look into Rendertron, and do your user-agent stuff in your web server, if it's a bot send it to Rendertron that generates a static html from the current route by visiting your site first and evaluating the Javascript. Or if you are going todo a special service for this, skip the subdomain business? Just proxy to you service if it's a bot. Prerender.io has a nginx setup you can't use for this.
Is this really a hack? This makes a lot of sense. Also, you could add some redirect rules for that bot request, that way none of your external links have to be to a different domain. Pretty sure something like S3 supports this for your static files.
"I spent my time building a house, and then decided I wanted this house to fly" This is exactly what I did and why I'm here lol. Goodbye React, hello Nextjs.
This might impact Google rankings down the line for you, as in you would not benefit from external signals generated by your users. If you don't care about that then I think it's a solid way to go!
This guy tries to sound like he knows what he's talking about but he either overcomplicates things or over simplifies things and I can't follow his train of thought. The mark of an expert is to be able to explain really complicated ideas to a toddler. Also, the fact that he thinks Javascript is the best language there is exposes his level of knowledge in this field even more.
The solution to the preview links would be to have a back-end endpoint that fetches that links metadata and pass in a URL we want to preview. This isn't something that is unsolvable in CSR.
I would suggest using prerender.io and then you don't have to mess around the SSR. You can run it yourself using this docker image github.com/tvanro/prerender-alpine. I have it running on a site that has over 100k pages.
Looking for exactly this solution. Was thinking of using aws api gateway to check the routes and then redirect based on headers. Social og links are super important for shareability..
These are some very irritating drawbacks this guy just mentioned. The same I faced with seo related tags like keyword that don't appear in view source code
been struggling for two days to understand what are SSR and SSG and CSR and i literally just got every thing on a 10 min video about big mac and ice creem
There's some services that can help, prerender.io got popular for it but now there's other alternatives www.prerender.cloud/docs/alternatives, including the one built into Netlify docs.netlify.com/site-deploys/post-processing/prerendering/
This video is 2 years old, I don't know if it existed when you made it but today there is a tool called puppeteer that uses a headless Chrome browser to actually pre-render your HTML that you can feed to bots and still feed the client side rendered to humans
Ben awad, you are on your grin so much that when you get to be a millionaire-you sound like your on your purpose. Also can I get you you coach me in programming? Do you have a discord?
my solution for this is: 1. set a lambda as an index 2. on the index lambda, it should run a headless tool (like a headless chrome) to render the page if the request is a bot, else just serve the raw js
The problem I see with the magic link is that when a user just copies the url instead of pressing a share button to get the special share.* link that there will not be meta tags and when you do use the special share.* then there are meta tags. And I think most users will probably just copy the url they are on now instead of looking for a share button
Nice solution. Only downside guess would be that Google wants you to show the same content to their bot as the user. But probably doesn't matter if you don't want to index those pages.
Ben's got 99 problems, but a girlfriend ain't one.
Bruh! He says it with a straight face
@@phantomKE I know right? He just leveled up with these jokes!
🤣🤣
jokes aside, he's not a bad looking dude at all, getting jacked wouldn't hurt tho, as long as he doesn't covert to a life coach and talk about it non-stop like that other coder youtuber John Sonmez.
He has a girlfriend in Canada.
Your metaphors are next level.
here's your big mac.
Aaaah I see what you did there...
🎈 🏠 🎈
"you are not a karen"
I avoid client-side rendering in order to save CPU cycles for cryptocurrency mining.
hahaha
hilarious comment! but crypto mining is an inefficient form of revenue on client's computer, see TPB case experiment.
@@TechdubberStudios It may pay less than ads, but it's many times better. I support websites that responsibly use cryptomining, and I block ads. Please, don't say that ads are better. They have never been any good to anybody's web browsing experience.
Oh, and you can use cryptomining along with Arc, another earning method that does not involve ads.
I'm done with Google's creepy trackers. Cryptocurrency mining is the future.
@@ezshroom I am genuinely 100% with you on the crypto movement. I hate ads. Always have hated them. But there are at least 2....3 big corporations that come to mind that were built on the ads business model, but with crypto mining... can't find one. And browser-crypto-mining is not exactly a new technology. I really want it to replace ads. I really do. Hate the pop-ups, spying, tracking, that's going on. And the first corpo that comes to mind would be Netflix, when considering whom should adopt the crypto model. Because the users stay on netflix and binge-watch hours and hours!
@@ezshroom also, do you happen to know any website/forum/subreddit focusing on browser-based mining? I would really like to join and dig in more into this subject.
the girlfriend problem might be solved if you stop walking around wearing asexual flag shirts
hahaha
lmao, good catch, respect
He's just playing hard to get. Karen gets it.
But with this if he ever gets one, she will be the right one. Lol
He do check a lot of aesthetic boxes from the virgin meme... Though I probably do too 😆
I love the tint on your glasses, it's serial killer-ish, where can i get a pair like those?
a package arrives at your door after the 3rd kill
@@bawad respect
They are the left-behinds after each kill. That's the way you get it.
@@bawad quick scope no scopes?
Those tints are wiped off blood from killing
For your sake and ours, I hope you DON'T get a girlfriend too soon.
I guess u never heard of prerender.io
been using it for years
Lol just killed the whole argument. Never heard of it before. Just goes to say that tech is exponential. Wonder if it will cause the cosmic crash eventually.
yep yep yep , you just commented before me
@@ayushkhanduri2384 same case, I just searched prerender before I add a comment about it to check if it's already mentioned and here it was.
🤯
solutions:
0) pre-rendering with parcel or webpack
1) server side rendering
your solutions are not client side rendering. he mentioned it.
Love the joke about girlfriend and client side rendering at the beginning
Fantastic job explaining this! As always, the hilarious dry humor and "next level" metaphors help drive home points and keep things entertaining. Really helped clear up a bunch of stuff and get me pointed in the right direction. Many thanks!
Very useful, thank you for pointing to react-snap. Happy Hacking Ben 🙌🏻
I was just watching one of your videos on react native animation earlier xD
Keep up the good job 🔥
Solution: NextJS, Angular Universal, Nuxt, etc.
Also check out the create-exact-app npm (that's exact not react). Like NextJS but Express-forward design, full control at the server side level of what's going on.
@@jpsimons Just FYI, next.js also gives you full server side control. You can just run next as a library within an express server. In my experience, it's super ergonomic while preserving the state-of-the-art benefits of next (code splitting, automatic static optimization, incremental static generation, etc.). Having said that, I have not yet checked out create-exact-app, and am not sure how it differs from nextjs.
Why do I not like the sound of Angular Universal?
@@angshu7589 because you are not carrot farmer. Although color of your profile picture kinda resembles the carrot :D
@Adithya R Svelte Sapper is still in early development. I love Svelte, but Sapper is still far away from production-ready
I love how Ben roasts Angular devs. I thought of that carrot farmer line off and on all day and cracked up every time.
Sorry I'm new here, and I've noticed that Ben clearly hates Angular.
Can someone give a quick background please???
I avoided serverside rendering a meta tag by registering a sub-domain, doing the serverside-rendering there and making my app only compatible with a set number of user-agents. Brilliant!
This is not true.
You can have Client Side Rendering with Server-side routing.
You send a different HTML page with some meta tags to describe your page for a different route
This is my project following CSR and I don't have this problem
github.com/polavi-project/ecommerce/
this feels like server side rendering. If you click anything at the ecommerce its loading all again... I didn't study your code, but this feels like classic php rendered eshop :D
Greate video! I use Laravel on the server side to serve up everything. Static html pages and React apps or a combo of both. It's easy to embed a react app within a .blade template file. Meanwhile Laravel takes care of everything else, like API services, user registration and authentication, etc. Best of both worlds.
I just put my meta tags with variables like %PAGE_NAME% %PAGE_IMAGE% and replace these later while serving the page with express. doesn't work while client-side routing but It works for link previews.
Nice GatsbyJS colorway on that shirt 🤙
I think your points on pre-rendering are slightly off. Not mentioning tools like Gatsby, Scully, or Gridsome is a miss as they can be used to render those dynamic routes you say cannot be pre-rendered. It's worth mentioning that JAMStack options are becoming really incredible for developers and give you the same benefits as server side rendering out of the box.
"It's like I spent a bunch of time building a house and now I want that house to fly." LMAO
prerender.io if u din knw abt it. Works fine for most cases. Bt i hv been moving sites to nextJs recently. Better speed. Easier SEO.
Yep, that was a huge relief after realizing this problem when I was almost done with my first website. Easy to set up too. Never tried Next.JS, but I'm interested. Is it worth it for multi page apps?
7:20 would it be possible to use the same url but check the header value on the for example nginx server?
like is this user agent a bot (twitter, fb, etc.) => proxy to your slim API for only the meta data response
and if its a real user (mac, windows, chrome, firefox user agent etc.) => proxy to your real page / default response / SSR page.
maybe im forgetting something. i dont know if this could work.
I had to do that once, I used a Lambda function since it was hosted on AWS, and the function intercepts the CloudFront distribution request and updates the HTML if the request comes from a robot, adding the OpenGraph tags.
Why do you want another domain for that. The server that delivers the app index.html could also deliver the meta response instead, based on user agent. One problem in both situations you could get (separate service or combined), is that the bots make checks whether the content they see is different to a regular user. Whether any of them does it, I don't know, but I would check this possibility.
I'm about that carrot farmer lifestyle tbh
me too 😦, that's why in my free time will enjoy build something else
@@mohdhaziq9859 I like the carrot life tho, has worked well for me
Sapper + svelte gives you the best of both worlds
Our course -- I followed and understood all the way to the end. This is because I'm an unemployed ex-Tech Lead [who has never worked at a FANG company], and a thousandaire.
there is a workaround : just add conditional tag in the small server that builds your page. you can still use client side rendering except for meta tags
Hey Ben,
I'm starting a new react web app for something like delivery.com and doordash.com where SEO is a major thing.
Like rich structured snippets of stars etc in google results.
Is SSR the only option?
If your content could be pre-generated (for example, blog), one of the options would be to use Gatsby.js
yeah
In that case Gatsby or NextJS or any other you recommend?
I don't know if Gatsby will do but their web.dev score hits 💯 so was thinking that but in a dilemma
Next.js
@@bawad do you know an easy way to migrate from create react app to next.js? I've found problems with REDUX, svg images and react router active route (NavLink). Great video btw ;)))
Who needs a girlfriend when you can wear ace coloured shirt
A solution to your problem could be to build a single page application, with each end point for the app being pre rendered.
It's basically jamstack. Once a user loads one page, the others do not need to be loaded.
There are benefits to client side rendering?
Btw rather than checking the user agent, just include a client side redirect location="whatever.com"
Because the bots dont have javascript they wont be redirected, but users will
The problem with client side rendering is mostly that most of the time it's used for something that doesn't need it. Most websites are mostly static.
I made my "yarn deploy" script change react's build/index.html to build/index.php, then it throws in a little php script that grabs some info from the good ole' mongoDB and 'echo's the metatags into the , only when that url is directly accessed. (You can get the URL params in PHP) It's only a few lines of code. The hardest part was figuring out how to configure Nginx's reverse proxy to render PHP. O_O
If you manage the web server, you could use the web server's router to do the same exact hack you described without the need for a different subdomain, just a route that checks the user-agent of the client and returns different HTML based on it.
The preview still won't work when users copy paste the link directly from the browser url bar
I started out doing server side rendering using Java/PHP spitting out html. Now that I'm researching client side rendering the first limitation I found is I have no f*cking access to any server resources. All server resources a page needs have to be directly accessible from the client. BAD.
I'm not a fan of server side unless it's to generate a HTML based email. Just seems so cluttered when doing work. Even the emails I generate on the server side are dreadful to work with but barrable since there aren't many.
My company has a massive CRA project that could benefit a conversion to NextJS, but we rely so much on browser globals outside of effects
Subbed for the consistent Angular claps 💀
my reaction to this video is LOL. as someone already mentioned, you could've done a decent server side routing
In your example of the sub domain. I think that’s not needed / don’t need to redirect. If you are using a server side program it can decide to return the react app or the preview on the normal domain using the same user agent logic or an IP address list of known Facebook servers.
Cant this be solved by a reverse proxy like NGINX just to cache the SSR rendered page but only serving it to bots?
That's what I was thinking. Maybe something like this to handle the routing internally to an SSR-generated page: www.nginx.com/resources/wiki/modules/user_agent/
or this: serverfault.com/questions/775463/nginx-redirect-based-on-user-agent
I'm sure there are other reverse proxies that support this, but this doesn't seem to be the most portable solution.
Edit: added an example of a similar implementation from ServerFault
Don’t build a SPA if you are building a website. The A in SPA is for application. Use SSG or SSR. Pre-rendering the page for web crawlers is a hack especially if you have a page that needs to display dynamic data like product price. You need to constantly update you pre-rendered pages and it’s not everyday that your pages will be crawled by SEO engines.
Lastly, don’t ever mess with SEO and meta tags if you don’t know what you’re doing. Always check with an SEO expert or use a good SEO tool. SEO is not just some random buzz word that web developers need to learn quickly. SEO is a long and sometimes painful game.
lol dude just make a request to your server to update it for you. complicated sure, but who the hell cares people just want it to work. what's interesting that I also haven't considered in my application is that users might want to visit the site if someone shares a link, if user makes a request all i can do is spit json back at them, i have no template for them to visit, and only users can see if they are logged in, i'll need to make guest template for non logged in users
You can write normal HTML with some normal CSS just to make the preview template, imagine that this is the loading page, and only once the client-side rendering is ready, you can show the content of that on top of that base temporary HTML template.
I've never done client-side rendering before, and I've never used React, but I think this should work. Why are you saying that it's not possible?
Detecting the user agent and serving different content is called "dynamic rendering": developers.google.com/search/docs/guides/dynamic-rendering
I've actually been wondering about this lately... whether to use Next.js or set up dynamic rendering so that only bots get the SSR version.
But I don't get why you're talking about adding the "share." subdomain to serve different content? Why not just use dynamic rendering on the regular URLs?
There is another solution. Force those corporation without budget limits to evaluate js. Create petition. And lunch law suite attack
I done this in a similar way. Except the share link page didn't check any user agent headers. It just featured a JavaScript redirect to the normal URL.
Bots don't run JavaScript, don't redirect, they get the meta tags they need. User does run JS, redirects to the main SPA. And, for good measure, a fallback clickable link on the share page shows in case redirecting is blocked.
I'm sure someone has already pointed out, but if someone copies the url from the browser, they won't have the `share.` prefix. Probably need a global UA interceptor that results in pages being rendered server-side.
Year 2030: Browsers introduce a feature to request server-rendered pages by pretending to be a facebook/twitter bot.
It works, that's what I do too. But forget about the subdomain solution. Lots of people don't care about some button "sharable link". They grab the url from the browser and send it to their Friends via FB or Whatsapp. I detect the crawler and redirect to a subdomain. When it comes to SEO - according to google they are fine with that, as long as the content of the redirect is not that different from the non-redirected version.
But it's all hacky and not elegant. Facebook should recognize, how the web actually works nowadays and update their crawlers to execute javascript and allow to make xhr requests.
Use Angular-Universal ... Oh wait
Use Next.js ...
when you share a deep link, the deep link shouldn't point to the webapp, but to server-generated pages with proper meta tags. The generated pages are dummy pages that redirects to the webapp.
Isn't this what client side routing or search queries are for? On document load before the window. Try some JSON data from thread, clone or populate a document fragment with your html and finally render the fragment using the query parameter and its prefix/suffix..
The User-Agent hack will work, but you risk getting penalized by search engines - some of them test with a real browser string to see if you're trying to manipulate the search engine. You might be able to look more "honest" to a bot by serving a Vary: User-Agent header and being up front about it, but then your caching is in the toilet. I'm afraid SSR is pretty much the only way to go, though personally I wouldn't use a framework to render the whole page on the server side (which is only an option if your server side is JS anyhow) but just render the non-interactive layout and content, then inject the interactive parts on the client side. The content that's interesting in terms of SEO tends to be non-interactive, and interactive elements generally aren't SEO relevant, so this tends to work out quite well.
So everyone knows Javascript is slow, C++ is fast...
What if you want to create a browser game where performance is important? Simple, code your back end in C++ and have it render each frame and then send the result to the client. Problem solved!
Had this issue while using a MeteorJS website running with ReactJs as the client side is that we created a crawler ( i think there is an npm project for this) that would go to every page, render it and save it on our DB. When a non-human (Google) would access the site it would served this rendered HTML, making it SEO friendly. Basically our server would use the User Agent to define what type of content the user would get served. Hope this help
can you please explain what do you mean by 'render it and save it on db' , do you mean like render the dom elements and attach to html and store it in some link and add that link in db , or what exactly are you storing in db, if this is the case wouldn't it be too much for pages which are dynamic like /rob/photos, /sam/photos and likewise to be stored in db or am i missing something
Your solution is nice, but people copy the url from the browser, and paste it where they wanna share it, and that won't work (they link will not have the "share" subdomain..
Or am I missing something?
Anyway I use angular, and it has server side rendering so I think i will just use that.. You can also just switch to angular.. I know you like it
Don't look at the User-Agent: that way lies madness. Instead, generate share.careerexpectations.com/reactjs with all the appropriate meta tags - and a line like this: window.location = "careerexpectations.com/reactjs". If a bot or spider or search engine or whatnot visits that page, the script tag will be ignored; if a browser does, the meta tags will be (mostly) ignored but the script will be executed and the user will be redirected appropriately. Alternatively, of course, drop the share subdomain, and just generate every page. It might be a little slower, since you'll be generating all this meta-data for browsers that don't care about it, every time the user reloads from a page other than the root, but it's simpler and you don't have to worry about what happens if the user shares the URL from the location bar instead of pressing the Share button.
Ngl I'm pretty sure the implementations are for security reasons. It wouldn't be fun if someone can simply inject arbitrary javascripts through link previews.
Why not send the proper meta tags according to the route by default in html and leave it up to scripts to overwrite them for the web client SPA. This way they web crawlers get their static tags because they never render with the scripts and SPAs get their correct content. Maybe I need to elaborate more but I hope you get the gist. It's possible this is how some of the SPA framework static rendering frameworks (Vue>NUXT, etc) operate as this doesn't seem like that crazy of an idea
Ok sure, you wont get the meta thing when you share a link. But what about the hard refresh on all pages and the form validations you need a server to be active on all your calls. Submit a simple form, wait for the server to respond. Nope, did not work, show the errors to the client. And repeat. (please do not say that you will send validation function alongside the page). How would you load a huge data with a spinner? Serve a page with a loader, and when the data is loaded, respond back again with the data?! (WebSocket) Or would you rather serve a page, that does another API call to get the data. How would you manage all that?!
Nice post. However I am curious as to why these FB bots can't read a client rendered app? I am sure that these bots are ultimately rendering the app on a web engine/browser, because they need to generate the preview of the rendered content. The static html will also most likely load some resources like CSS and JS over the network (unless the bot expects an inlined css/js), so it's not going to be an instant preview. So maybe the limitation is not technical but rather functional, in that the bot is probably not willing to wait too long for the page to load (which is the case with most client rendered apps)?
McLovin teaching javascript
yo ben, how about we make an AI Girlfriend, could be a fun project right?
Hey I saw Wes Bos in one of his videos, he used cloud functions to generate the preview and puppeteer i guess to take a screenshot of the url
This isnt a problem with client side rendering. It's a problem with the places building these link previews. They could just as easily run a bit of js to get an accurate preview. Google already does this afaik. But facebook tends to be a bit behind on this sort of thing.
You ever tried NextJS? It does SSR for the initial request (b/c it could be a bot), but CSR when you click links.
For some wierd cases like mine where only /particularRoute need to work like SSR what i actually tried was hosted gatsby project in particular route of CRA project and it worked just need to handle few re-routing cases
Look into Rendertron, and do your user-agent stuff in your web server, if it's a bot send it to Rendertron that generates a static html from the current route by visiting your site first and evaluating the Javascript.
Or if you are going todo a special service for this, skip the subdomain business? Just proxy to you service if it's a bot.
Prerender.io has a nginx setup you can't use for this.
Is this really a hack? This makes a lot of sense. Also, you could add some redirect rules for that bot request, that way none of your external links have to be to a different domain. Pretty sure something like S3 supports this for your static files.
"I spent my time building a house, and then decided I wanted this house to fly" This is exactly what I did and why I'm here lol. Goodbye React, hello Nextjs.
This might impact Google rankings down the line for you, as in you would not benefit from external signals generated by your users. If you don't care about that then I think it's a solid way to go!
This guy tries to sound like he knows what he's talking about but he either overcomplicates things or over simplifies things and I can't follow his train of thought. The mark of an expert is to be able to explain really complicated ideas to a toddler.
Also, the fact that he thinks Javascript is the best language there is exposes his level of knowledge in this field even more.
The solution to the preview links would be to have a back-end endpoint that fetches that links metadata and pass in a URL we want to preview. This isn't something that is unsolvable in CSR.
I would suggest using prerender.io and then you don't have to mess around the SSR. You can run it yourself using this docker image github.com/tvanro/prerender-alpine. I have it running on a site that has over 100k pages.
I was experimenting with client side rendering in 2012, it would generate the page based on json
Looking for exactly this solution. Was thinking of using aws api gateway to check the routes and then redirect based on headers. Social og links are super important for shareability..
These are some very irritating drawbacks this guy just mentioned. The same I faced with seo related tags like keyword that don't appear in view source code
JAM Stack is a solution , Gatsby or next.js is very powerful and mature
been struggling for two days to understand what are SSR and SSG and CSR and i literally just got every thing on a 10 min video about big mac and ice creem
There's some services that can help, prerender.io got popular for it but now there's other alternatives www.prerender.cloud/docs/alternatives, including the one built into Netlify docs.netlify.com/site-deploys/post-processing/prerendering/
This video is 2 years old, I don't know if it existed when you made it but today there is a tool called puppeteer that uses a headless Chrome browser to actually pre-render your HTML that you can feed to bots and still feed the client side rendered to humans
Ben awad, you are on your grin so much that when you get to be a millionaire-you sound like your on your purpose. Also can I get you you coach me in programming? Do you have a discord?
my solution for this is:
1. set a lambda as an index
2. on the index lambda, it should run a headless tool (like a headless chrome) to render the page if the request is a bot, else just serve the raw js
Lol, all serious developers have a common problem, that they don't have a girl friend. Well that's curable.
I was secretly hoping to see the carrot farmer when I went to careerexpectations.com/angularjs
o-o anything wrong with just using twig/blade and just use vue compiled version when you want shiz to update in real time?
The problem I see with the magic link is that when a user just copies the url instead of pressing a share button to get the special share.* link that there will not be meta tags and when you do use the special share.* then there are meta tags. And I think most users will probably just copy the url they are on now instead of looking for a share button
Nice solution. Only downside guess would be that Google wants you to show the same content to their bot as the user. But probably doesn't matter if you don't want to index those pages.
I think the problem of not having a girlfriend is worse than having one that is religious angular developer.
You just need to convert to angular.
The drawbacks of not understanding the title (and watching anyways)
Metaphors killing me 😂
No one roasts angular like Ben
why are your eyebrows so thick and dark🥺🥺
3:17 thanks for explaning this here lol, i was like why wouldn't you be able to just stick the meta tags in the requested html
I will learn WebD so that I can enjoy these digs by Ben😂
It's not a limitation of client side rendering, it's a limitation of these link preview generators
Please make a video on react 18 update, especially covering the SSR enhancements in react.