If you convert the text into an array of integers, where each integer is an index into an array (or tree) of unique words from the text, you could possibly speed up things by a lot, depending on how long it takes to set up the arrays/tree.
Yeah, I need to try Mojo too. I'm finding that Mojo isn't yet good at text processing. I hope and assume that it will be get better as it is developed more and more.
I can imagine 2x is just a function of the complexity x the sample size, which makes me wonder about the curve at scale. There would also be some unrelated but important measures, such as speed of development and the effect of higher level abstractions vs lower level optimisation too
If you convert the text into an array of integers, where each integer is an index into an array (or tree) of unique words from the text, you could possibly speed up things by a lot, depending on how long it takes to set up the arrays/tree.
Very good idea! I'll have to try this.
what about Go ?
I haven't yet ventured into Go for text processing.
Also try mojo.
Yeah, I need to try Mojo too. I'm finding that Mojo isn't yet good at text processing. I hope and assume that it will be get better as it is developed more and more.
So rust essentially takes half the time than python....nice, but I thought rust would be a lot faster.
Yeah. Nearly twice as fast.
could be a lot faster. it all depends on what's being done and how efficient the code is. not always tho
I can imagine 2x is just a function of the complexity x the sample size, which makes me wonder about the curve at scale. There would also be some unrelated but important measures, such as speed of development and the effect of higher level abstractions vs lower level optimisation too
nice video :)
Thanks!