Laravel Collections vs Arrays: Performance Test
HTML-код
- Опубликовано: 1 окт 2024
- People say that Laravel Collections operations are slower than arrays. But by how much? Is the difference significant? Let me show you an example and share my thoughts.
- - - - -
Support the channel by checking out my products:
- My Laravel courses membership: laraveldaily.t...
- Laravel QuickAdminPanel: bit.ly/quickad...
- Livewire Kit Components: livewirekit.com
- - - - -
Other places to follow:
- My weekly Laravel newsletter: bit.ly/laravel-...
- My personal Twitter: / povilaskorop
I think it was Steve McConnell that said that it is important to "make the code work, then make it fast". Until it works and you test and measure it you don't know which parts are making your application slow. This is an example of what he was saying.
The more experienced you are the more you know what code constructs are faster than others but the real test is to measure it.
But only when the application works.
Agree! I rephrase the same thought as:
1. Make it work
2. Make it maintainable (readable / reusable for others)
3. Make it scalable (performance)
IMO, developer time is more important than the run-time. Since we have very fast connections and very strong hardwares, slighty slower code won't effect user experiance negatively. If a table becomes huge and starts slowing down the whole system, it will always be an option to recreate its queries and codes to improve the performance. So my motto "use collections until you shouldn't." Spending your time today for a future that might never come is not a good idea.
It adds up.
I agree with this point a lot. A broader approach of this is not to perform "premature optimization". Of course, it depends on the situation, in some cases you need to take care of the scalable solution from the very beginning, because re-writing would be hugely expensive, but in most cases, you start with MVP and then refactor/improve different parts, like if you diagnose later that collections specifically make some feature slow.
@@LaravelDaily I agree with you.
I think this is also the reason for using eloquent over query builder...
I wish we analyzed time complexities of each solution before running the test. Anyway, if you want effectively use Laravel collections, it is a good idea to know time complexities of the methods.
I've not done benchmarks, but I do recognise the time saved debugging and understanding code that I've come back to after a significant time period as a good reason to use collections, as opposed to loops as in the first example.
I think the main power of Collection is the sugar they add. Yes, they could slow your code to operate. But coding is not only execute the code. It's also about writing and maintaining your code. So, the true power of Collection is that you code faster.
Great video. Could you use countBy collection method rather than groupBy and mapWithKeys ?
This is specific to this use case but here the groupBy is doing not necessary calculation.
Interesting, never tried countBy
Would be interesting to compare this with `lazy` collections
That would be interesting! For some situations lazy collections might make sence. I do not have enough experience with them to fully know when to use them. Excellent video idea I think.
Lazy collections are made to use less RAM, not to be faster. Even though I would be interested in a speed comparison, just for curiosity
AFAIK, lazy collections are based on lazy sequence evaluation principles (using generators) which mean that a sequence will (in most cases) be iterated over only once no matter the amount of operations done to it. So lazy collections should be faster than regular collection who AFAIK store the "intermediate" data in an internal buffer
collect($users)->countBy(function ($user) {
return Str::of($user[1])->before(' ')->toString(); //strstr($user[1],' ', true);
});
->except(['Ben', 'Vesta','Lisa'])
->sortDesc();
Thanks, great improvements! I've never used countBy() before, will try it out and probably tweet to others.
I like this video.
Once we hade perfommance issue with data-transformation, and my Lead Dev, ask me to write DB query; instead of using Eloquent Collection.
For 100.000 rows On DB, the differences was Huge.
It takes more time for devs, but for huge amount of data It is always better to write Optimized code on the begining, instant of rewriting when the dB get bigger.
Comparing native implementation against a wrapper of native implementation? Native would always be faster.
Great comparison thanks, I'd like to know, what javascript framework is more popular with Laravel, and what is more efficient ????
It's personal preference about efficiency. In terms of popularity, Vue is by far the most popular among Laravel developers.
I'd be interested in the time an SQL query would take to perform this operation.
In this particular case, I believe you could make better use of Lazy Collection (included in the framework since laravel 6). One of the examples from the original PR was precisely about dealing with these kinds of large CSV files.
Some methods are not available to LazyCollections however, like mapWithKeys, but I think you can get the exact same result by changing with `->map->count()`.
use Illuminate\Support\LazyCollection;
LazyCollection::make(
function () {
$handle = fopen(public_path('users.csv', 'r'));
while( ($line = fgets($handle)) !== false ) {
yield $line;
}
})
->map(...)
->filter(...)
->groupBy(...)
->map->count()
->sortDesc();
Try it out!
Even if you're not using a large csv as the source, LazyCollections are available in the query builder too via the cursor() method.
Yes, planning a video on exactly that, on Lazy Collections, after a few people pointed it as well. Thanks!
so the conclusion is array 2x fast than collection?
No, you probably haven't watched the full video with my thoughts. The conclusion is "fast but it depends".
Great explanation! Great comparison! Great experience! Thank you!
Do it with a raw SQL query and try again.
Because the groupby method is an aggregation that's why its is slow. You will notice it if you are working on TSQL with complex queries
I think you're comparing oranges to apples, please use other method like countby or some other method, why we need a group by here
I don't think it's a fair comparison.
In the first example, only one loop is made, that is: 100 000 lines.
In the second, there are 2 maps, a filter and a group, totaling 400 000 lines.
As stated in the video, to really draw a conclusion, it is necessary to compare collection methods similar to the ones you will use.
Nice video!
It is perfectly fair. Both methods start with a dataset and output a result. One method being faster than the other because of the multiple loops over the dataset. Is that difference significant? Nope, not really. Both results are sub one second. If the difference had been greater, say ten seconds, then the difference would be significant and you'd choose the faster one or find another way to use collections faster.
This is exactly the type of thing you will find in your code, anyone's code, and the trick is to find out where the significant bottlenecks are and why they occur and concentrate the development in those areas to speed up the process.
@@tetleyk The purpose of the video is to compare performance between two things: arrays and collections.
So for it to be a FAIR comparison, equal methods must be used.
So for that, you should have used, in the sequence, in the first example: array_map, array_filter and two other custom methods to do the groupBy and mapWithKeys.
There's no way to be a fair comparison of performance when each example was done in a different way.
At no time was it said in the video that it was for you to learn to identify performance problems, but a comparison between two things was shown: arrays and collections
@@1PeZiNhO1 No, that is not what the purpose was, and foreach is a valid array method. Keyword, rather, but it's still valid.
You are forced to chain map and filter to collections, but you're not forced to do that with arrays.
I haven't read the code of Collection but most of them should be wrappers for php native methods so you shouldn't see too much performance difference.
Will the use of Lazy Collections in this case save time or is it just reducing the memory the application uses??
Aren't those two things directly related?
It makes no sense to use something Lazy, IF you need to access all data to get the result you want. (Group by, Sort etc.).
Supplier for the best Laravel tips and tutorials!
I am also interested to know what is easier to write and to read. Maybe there are situations where the execution time is not that important, but the time to develop the code is. First make things work, later make it faster when needed. That being said, I really like this topic because I feel it is important to make informed decisions on stuff like this. Thanks for the video!
Is there any way to measure the memory usage of a collection? I have a killed job issue when operating huge data through collection...
Yes with Laravel telescope, see my video: ruclips.net/video/wDKTCN_cJrU/видео.html
Hello Sir,
can you make a video on web 3.0 and where does php and laravel stands in it and how to create projects using these. or any other info related web 3.0 and php/laravel.
Thanks
I don't work with web 3.0 so can't make a video about that. But a few resources I've seen about it:
- github.com/m1guelpf/laravel-web3-login
- medium.com/geekculture/laravel-authentication-using-web3-15d0fb030a48
- www.freecodecamp.org/news/add-a-metamask-login-to-your-laravel-app/
Great content 👌
If you want to have a single run through using collections then I think using lazy collection can achieve that.
An excellent example for collections and arrays 👍
I hope we once get a typed collection data structure
Should of ran it on millions of rows imo
sir kindly can you share the link to the video in which you teach about cursor() loading, I think right after this but unfortunately, I didn't find it in your list.
I don't remember any video about it
@@LaravelDaily Model::cursor(); something like this?
Yes so I don't remember shooting videos about it
There is almost never a need to use array_key_exists unless you are testing for existence of a key AND want to get a return value of true if the value itself is null.
If you know values won't be null OR you wish to ignore them, isset is MUCH faster, especially at this scale of number of records
thank you...
Collection readability is less intuitive.
Please can we get your laravel live wire video Here on RUclips