I love this video. It is exactly what I was looking for. Thanks a lot. Instead of defining guessing 100 pages for clearing cache. You can use something like : In the below command, 4 is the number of items displayed on the page. $NoOfItemsOnPage = 4; $pagesCount = ceil(count(User::all())/$NoOfItemsOnPage); for($i=1; $i
You don't need to use loop for cleaning all cache. Just use cache tags. Cache::tags(['user-pagination'])->put('user-page-1', $data); // Adding Cache::tags(['user-pagination'])->flush(); // Removing
nice video, you can use tags to group cache keys, i.e. you can set key for each page and give them all one tag 'users', and you can use Cache::flush('users') to forget all users tag keys at one time.
Having worked a lot with Magento before coming to Laravel, cache tags was was first thought. Magento's solution to cache back ends that don't support tags was to nuke the entire cache on those back ends. Brute force, but effective (if not very efficient). Another approach, depending on the size of the data set, might be to cache the entire record set, then apply the pagination after fetching from cache. Then you can use the same cache object for all requests, and paginate differently in the controller. Depending on the complexity of the presentation of the data, it might be more effective to query the data, and render the output, and cache the rendered output. If a significant amount of the request time comes from rendering the view (e.g. a heavy server side view), then just query caching might not yield significant improvement. A profiler or a tool like New Relic will help you dig into the execution time and determine whether that's the case.
Did you know that in ttl parameter in remember static function you could parse a carbon instance like Carbon::parse('20 seconds') instead of using calculation it would be more readable for humans :)
Hi Great Content , I think it will be a problem in your clearCache Observer ,if the user go from 1 to 3 and 4.... page not passing by 2 for example, it will not clear the cache of other pages only the first page because of the Break but its helps me anyway thank you
Model observer is indeed a good example for updating cached data. But unfortunately caching won't work at all if you use datatables with search and sorting functions (I believe you use datatables as well in this example). There is an option for remote search and remote data processing is required if you have >30 entries for a table, that way you would only load 1 page at a time from the DB
Think you're the smartest in laravel 🌹 ... Can you explain further about the "observer" ? .. and why not a real life example about implementing the redis cache .. thanks in advance..
measure, then cache what's slow and important. Premature optimization is the root of all evil. If you implement a caching strategy for paginated records, I think the strong implication is that you're not too concerned about record ordering on pages. I would advise against caching this particular type of data. There's a vanishingly small - if any - performance gain to be had, at the expense of record page ordering inconsistencies.
How to cache pagination if you have a complex table with 10+ filters and 5+ columns for sorting and 4-6 page-size options. In addition, data in table may have loads of foreign keys. I think it is bad solution to observe each of related models to only clear the cache.
I sometimes prefer to tweak the DB cache/buffer configurations since those are typically configured by default to very small systems so once you tell the DB it can use more ram the performance sometimes might improve. I know that this still makes the DB call, but still I can see how the 2nd call for the same table is always faster. Also the DB cache implementation is better that whatever I can implement on Laravel since updates on DB cache are also reflected on the cache. IMHO Laravel cache is only better on stuff that need to be calculated (not just read).
i never like using an arbitrary number like 100 to set the page limits. you could make a function to query for count and divide by your pages and use that number for your upper page limit.
Hallo. Great content. How about if we have front and back end in Laravel, in front user can Adding something (a CRUD for auth User's) but I have and a Admin. How to use the same Controller, or how to make something like this?
Great video! Thank you a lot. Could you do a video about practice example when using queue with "WithoutOverlapping", "ShouldBeUnique" feature in Lavavel? Therefore, we can understand the benefit when using these features in real cases.
Unless you would use Collection pagination instead of DB pagination, you would lose the benefit of caching once you do any sorting/filtering... or make it unnecessarily tough.
I think it's good idea if Laravel had something like Cache::startsWith('user-page-')->forget(); Otherwise you won't be able to clear the cache properly
@@LaravelDaily Thanks for your answer! How could I make site loading faster in this case? Passing Model::all() to the view where the Datatable creates pagination can slow down loading time a little bit when dealing with lots of records. Is there any way to deal with this without any serious hacking/reworking?
@@MrKataklysm if you have big data with a relationship, using an eloquent eagerload maybe is the problem you get slower. i'd prefer you may use query builder of join to handle this, in my experience, it will run more faster. and dont forget to Index your column to make it more faster im running with datatables, which load at least minimun 100K data, and its cost me 1-2sec to load the datatables serverside
Sir, Have you ever tried spatie/laravel-backup ? I faced an issue when taking db backup ... something related to dump. Terminal: ( Command not found : sh: mysqldump: command not found )
It's all in Laravel Scout docs, then 90% of the video would be installing the Elasticsearch or Meilisearch or Algolia. Which is in their docs, again. Not worth the video, in my opinion.
Caching is for large datasets, not for queries, with limits. for example, if you paginate 100k per 50 lines, why u wish cache that output? 50 rows is still minimum consumption of ram, but, on other side 2000 files with cached data (size is depended on data, for example simple 5 fields like name, email etc. is smaller that blogposts with full html contents etc.) ... then, using cache with some good mindset is sill something what u need to focus on
how to use other modal relation with cache ? just like that Cache::remember('chirps', now()->addMinutes(1), function () { return Chirp::with('user')->latest()->get(); });
Caching the pagination usually means something is wrong with the way we load the data like unoptimised queries, lack of indeces etc. Also caching pagination is kinda pointless since 90% times the data will change relatively often so clear the cache all the time beats the initial purpose of caching.
how about you help with a video to paginate but in a random order , cause anytime I query in a random order , and paginate it , I get duplicates from page to page
@@LaravelDaily I'm building a startup yeah? and I intend to infuse machine learning as time goes on , but everytime the user opens the app, I don't want them to see the same content they saw on their last session especially when new content hasn't been added and while I still try to make machine learning do this I wanted to give the impression that it already exist , so everytime you open the app , the content is shuffled , and the user keeps staying interested and seeing something new everytime
Hmm, reminds me of how Facebook/Twitter show us the personalized data every time we load their homepage, so maybe it is a realistic scenario after all. Well, in this case, there's no other way I would think of than caching that randomized data specifically for one user for one session.
Yap I'm agree with you, it's a lot of risk if we're using cache in pagination. But maybe the person who ask this, have an example of using pagination in a content that have low of possibility to change. For example, a list of some "big client" in your company profile. The list have 10 data, and you want to paginate it 3 data each page
@@LaravelDaily Why we use cache? We use cache to ease DB workload and speed-up the system response, don't we? So in my case I want to show people products in random order so every product has it's chance to be seen. The question is is it worth the effort.
@@motomono You could choose to call Products::inRandomOrder()->... and then cache the results of that for, let's say, 5 mins. Pro: You will only call the database once every 5 minutes. Con: Every user will see the same results for those 5 minutes. Don't know how much of a problem that is for you. You can reduce TTL, or cache more records and then display those randomly.
@@Remls Hi mate, tank you for the response. Unfortunately this approach doesn't work well with pagination - the problem is that on every page the user will see freshly randomized products so everytime the user get's back to let's say page no. 2 the products sowed on this page will be different. We want to randomize the product set once, then paginate them so te client can walk through tha pages and go back to the page x and see the same what he left.
@@motomono Maybe store randomised list of all IDs in cache, and then call find() on a slice of that array based on what page the user wants? You can cache the pages as well if you wish (but discard the cached pages if the cached list of IDs has expired).
According your video pagination is not a good idea and we can solve slow pagination with other solutions. In the other hand for caching and clearing a lot of data we can use cache tags or redis array key or wildcard key and... So foreach is not a good way
When using redis or memcached, you can used tagged cache (they are like folders for cache). You can then just clear all the cache for the tag.
I love this video. It is exactly what I was looking for. Thanks a lot.
Instead of defining guessing 100 pages for clearing cache. You can use something like :
In the below command, 4 is the number of items displayed on the page.
$NoOfItemsOnPage = 4;
$pagesCount = ceil(count(User::all())/$NoOfItemsOnPage);
for($i=1; $i
thank you so much
depends which cache provider you’ll use but what I usually do with redis is use tags for this so I can clear the container.
You don't need to use loop for cleaning all cache. Just use cache tags.
Cache::tags(['user-pagination'])->put('user-page-1', $data); // Adding
Cache::tags(['user-pagination'])->flush(); // Removing
Yes, but: "Cache tags are not supported when using the file, dynamodb, or database cache drivers."
nice video, you can use tags to group cache keys, i.e. you can set key for each page and give them all one tag 'users', and you can use Cache::flush('users') to forget all users tag keys at one time.
Having worked a lot with Magento before coming to Laravel, cache tags was was first thought. Magento's solution to cache back ends that don't support tags was to nuke the entire cache on those back ends. Brute force, but effective (if not very efficient).
Another approach, depending on the size of the data set, might be to cache the entire record set, then apply the pagination after fetching from cache. Then you can use the same cache object for all requests, and paginate differently in the controller.
Depending on the complexity of the presentation of the data, it might be more effective to query the data, and render the output, and cache the rendered output. If a significant amount of the request time comes from rendering the view (e.g. a heavy server side view), then just query caching might not yield significant improvement. A profiler or a tool like New Relic will help you dig into the execution time and determine whether that's the case.
Did you know that in ttl parameter in remember static function you could parse a carbon instance like Carbon::parse('20 seconds') instead of using calculation it would be more readable for humans :)
or you can use now()->addDays(5)
Hi Great Content , I think it will be a problem in your clearCache Observer ,if the user go from 1 to 3 and 4.... page not passing by 2 for example, it will not clear the cache of other pages only the first page because of the Break
but its helps me anyway thank you
Yeah, it should be chaned to "continue" instead of "break"
Mmm, great catch, agree.
@@LaravelDaily or maybe remove the else statement?
Great tutorial Povilas, Thanks!
Model observer is indeed a good example for updating cached data. But unfortunately caching won't work at all if you use datatables with search and sorting functions (I believe you use datatables as well in this example). There is an option for remote search and remote data processing is required if you have >30 entries for a table, that way you would only load 1 page at a time from the DB
I think that the key like `"users-" . $queryString` might help here
Think you're the smartest in laravel 🌹 ...
Can you explain further about the "observer" ? .. and why not a real life example about implementing the redis cache .. thanks in advance..
you can use Illuminate\Support\Facades\Artisan;
in the UserController/ update function
Artisan::call('cache:clear');
all cache cleared
Redis has Redisearch module with FILTER/SORT/LIMIT/OFFSET abilities for more advanced useage.
measure, then cache what's slow and important. Premature optimization is the root of all evil. If you implement a caching strategy for paginated records, I think the strong implication is that you're not too concerned about record ordering on pages. I would advise against caching this particular type of data. There's a vanishingly small - if any - performance gain to be had, at the expense of record page ordering inconsistencies.
Hey nice video! But you can use Tags in Cache, so you don't have to do a foreach.
Great! Now I must check if it works with Sqlsrv 2008 R2 as altervative to DataTables.
How to cache pagination if you have a complex table with 10+ filters and 5+ columns for sorting and 4-6 page-size options.
In addition, data in table may have loads of foreign keys. I think it is bad solution to observe each of related models to only clear the cache.
I wouldn't use cache with so many variables of data
is caching specific to that single user sessionid? or the "key" in that cache still accessible to other users?
I sometimes prefer to tweak the DB cache/buffer configurations since those are typically configured by default to very small systems so once you tell the DB it can use more ram the performance sometimes might improve. I know that this still makes the DB call, but still I can see how the 2nd call for the same table is always faster. Also the DB cache implementation is better that whatever I can implement on Laravel since updates on DB cache are also reflected on the cache. IMHO Laravel cache is only better on stuff that need to be calculated (not just read).
To clean cache you can use wildcards "*" instead "for" loops atleast with redis
Yes, you can use Tags with Redis for this. But not everyone uses Redis for caching.
I would use a cache tag, like "user-pages". So, I would just delete the cache by the assigned tag, when it's needed.
i never like using an arbitrary number like 100 to set the page limits. you could make a function to query for count and divide by your pages and use that number for your upper page limit.
I'm thinking about using cache tags for clearing all users-related at once
Also, wouldn't caching be pointless if your table has sortable columns?
I use cache mainly for statistics and long queries instead of doing calculations for every request
Even though this video is old, what if we use scopeFilter and continue to use pagination?
Hallo. Great content. How about if we have front and back end in Laravel, in front user can Adding something (a CRUD for auth User's) but I have and a Admin. How to use the same Controller, or how to make something like this?
I would personally use separate controllers, because there's a big change that in the future functionality may be different.
@@LaravelDaily ok thanks for quick answer. Great job by the way.
Great video! Thank you a lot. Could you do a video about practice example when using queue with "WithoutOverlapping", "ShouldBeUnique" feature in Lavavel? Therefore, we can understand the benefit when using these features in real cases.
Those are pretty niche cases, not sure if it would be interested for masses on my channel. But will add to the extended to-do list under "maybe".
Sir . Can laravel make paginate with clear url ? 🙏
Sir how to connect Microsoft access database in my laravel app
Real life cases wow 🔥
Unless you would use Collection pagination instead of DB pagination, you would lose the benefit of caching once you do any sorting/filtering... or make it unnecessarily tough.
A quite general question: How do you handle caches during development?
I didn't found a good way yet to disable them entirely during development.
In .env file use CACHE_DRIVER=null
@@LaravelDaily I tried that in the past and that didn't worked. Maybe I did something wrong. I will give it another shot. Thanks a lot! :)
Null object patern works always with no matter what. In this case with Cache driver it will work. But I believe it comes straight from the box
I think it's good idea if Laravel had something like Cache::startsWith('user-page-')->forget(); Otherwise you won't be able to clear the cache properly
It has 'Cache::tags' if you use Redis or Memcached driver
What do you think about using cache in cases when you are using front-end pagination, like jQuery Datatable for instance?
I don't think caching makes sense in that case
@@LaravelDaily Thanks for your answer! How could I make site loading faster in this case? Passing Model::all() to the view where the Datatable creates pagination can slow down loading time a little bit when dealing with lots of records. Is there any way to deal with this without any serious hacking/reworking?
@@MrKataklysm if you have big data with a relationship, using an eloquent eagerload maybe is the problem you get slower. i'd prefer you may use query builder of join to handle this, in my experience, it will run more faster. and dont forget to Index your column to make it more faster
im running with datatables, which load at least minimun 100K data, and its cost me 1-2sec to load the datatables serverside
Use server side rendering of datatables, which would load only one page of data
@@LaravelDaily Thanks for the answers! I'll give it a try.
Sir,
Have you ever tried spatie/laravel-backup ?
I faced an issue when taking db backup ... something related to dump.
Terminal: ( Command not found : sh: mysqldump: command not found )
You don't have mysqldump installed on your server
There will be another issue if i change per page size.
Is laravel collection methods are slower than PHP array functions ?
Not really. At least not in a way that you would actually "feel" in reality.
Hi!
Thanks. My qiuestion would be, is there a solution to clear cache like files with asterix? like users-* eg. thanks
You can use Tags for this, but it's supported properly only by Redis and Memcached drivers, from what I remember.
Hi, can you please do a video on configuring vite js, laravel, inertia and vue 3 running on homestead? I would greatly appreciate it.
I don't use homestead, I use Laravel valet, so I can't do a video you're asking for
@@LaravelDaily I would greatly appreciate it 🙏🏼
@@LaravelDaily I can't get my head around loading scss and js files with vite. Also, their documentation is not that good.
Suggestion :: requirement
How to use redis and database together in controller
Must validate the page number
Povilas, you should do video about Elastic search integratikn to Laravel. Or basically any other fulltext search index engine
It's all in Laravel Scout docs, then 90% of the video would be installing the Elasticsearch or Meilisearch or Algolia. Which is in their docs, again. Not worth the video, in my opinion.
Okay, maybe you're right
Caching is for large datasets, not for queries, with limits. for example, if you paginate 100k per 50 lines, why u wish cache that output? 50 rows is still minimum consumption of ram, but, on other side 2000 files with cached data (size is depended on data, for example simple 5 fields like name, email etc. is smaller that blogposts with full html contents etc.) ... then, using cache with some good mindset is sill something what u need to focus on
how to use other modal relation with cache ?
just like that
Cache::remember('chirps', now()->addMinutes(1), function () {
return Chirp::with('user')->latest()->get();
});
here is error show
Undefined property: stdClass::$user
Caching the pagination usually means something is wrong with the way we load the data like unoptimised queries, lack of indeces etc. Also caching pagination is kinda pointless since 90% times the data will change relatively often so clear the cache all the time beats the initial purpose of caching.
how about you help with a video to paginate but in a random order , cause anytime I query in a random order , and paginate it , I get duplicates from page to page
To me, randomizing the data and then paginating it doesn't make much sense. In what real-life scenario you would actually need this?
@@LaravelDaily I'm building a startup yeah? and I intend to infuse machine learning as time goes on , but everytime the user opens the app, I don't want them to see the same content they saw on their last session especially when new content hasn't been added and while I still try to make machine learning do this I wanted to give the impression that it already exist ,
so everytime you open the app , the content is shuffled , and the user keeps staying interested and seeing something new everytime
@@ifeoluwaAlao You don't cache random things.
@@genechristiansomoza4931 it's not about cache ,it's just normal query from eloquent
Hmm, reminds me of how Facebook/Twitter show us the personalized data every time we load their homepage, so maybe it is a realistic scenario after all. Well, in this case, there's no other way I would think of than caching that randomized data specifically for one user for one session.
Wow sir
Yap I'm agree with you, it's a lot of risk if we're using cache in pagination. But maybe the person who ask this, have an example of using pagination in a content that have low of possibility to change. For example, a list of some "big client" in your company profile. The list have 10 data, and you want to paginate it 3 data each page
Even more interesting problem is - how to use cache with random()
A bigger question is WHY would you do that.
@@LaravelDaily Why we use cache? We use cache to ease DB workload and speed-up the system response, don't we? So in my case I want to show people products in random order so every product has it's chance to be seen. The question is is it worth the effort.
@@motomono
You could choose to call Products::inRandomOrder()->... and then cache the results of that for, let's say, 5 mins.
Pro: You will only call the database once every 5 minutes.
Con: Every user will see the same results for those 5 minutes. Don't know how much of a problem that is for you. You can reduce TTL, or cache more records and then display those randomly.
@@Remls Hi mate, tank you for the response. Unfortunately this approach doesn't work well with pagination - the problem is that on every page the user will see freshly randomized products so everytime the user get's back to let's say page no. 2 the products sowed on this page will be different. We want to randomize the product set once, then paginate them so te client can walk through tha pages and go back to the page x and see the same what he left.
@@motomono
Maybe store randomised list of all IDs in cache, and then call find() on a slice of that array based on what page the user wants?
You can cache the pages as well if you wish (but discard the cached pages if the cached list of IDs has expired).
Guys is it just me or does somebody else also see that paginating using OFFSET is a very bad habbit?
According your video pagination is not a good idea and we can solve slow pagination with other solutions.
In the other hand for caching and clearing a lot of data we can use cache tags or redis array key or wildcard key and...
So foreach is not a good way