There’s very little resources on python multiprocessing, and ive struggled hard with it. Watching this vid helped me a lot, thx! P.S. Underrated channel, can see ur videos blowing up!
That’s a nice way to do it ! I did it otherwise a few months ago. I basically had a variable set to None, and I started a Thread with as a Target a function that returns the loaded object. So I did a while variable is None: and displayed the loading screen, and if the variable was not None anymore it would mean the loading process had ended. I guess it’s “hackier“ than your way
That is a good way of doing it, but it uses threading. I also did the same initially only to find that the FPS dropped to about 40 if I had a lot of images to load. That is why I decided to use multiprocessing to do the same. Thanks for sharing your approach :) [Edit: Spelling Correction]
Wooow. This is very useful. I believe this can help to make good visuals for game intro and transitions. I wanna test this out really sooon. Very great video and editing btw. I only want to know that let's say I have a bunch of heavy calculations (100) and they take 5 seconds to complete. Using this multiprocessing, is it possible to boost performance?
Thank You! Indeed you can boost the performance, but it depends on your CPU. in most cases, with the example you gave you should be able to improve performance using a standard CPU. But lets say you have 10K calculations, then an i3 processor single/dual core may not be able to reduce the time taken much compared to what an i5 or an i7 processor can do :)
Hello, I am attempting to make a game using pygame, but am completely new to multiprocessing. The game currently runs reliably at 60+ fps, but I would like to try to get it to run at 120 fps by separating the rendering from the game logic. I dont see many other examples of people separating these processes, is this a waste of time?
Hi, it might not be a waste of time depending on the context, but have you already tried using other methods of optimization? Like using convert or convert_alpha on surfaces and minimizing draw / blit calls? I am pretty sure you can optimize your game enough to run at 120+ FPS. However, if you must use other ways to speed up, then you can do the following -> make 1 process for rendering make another process for game logic connect them with a pipe object and transfer all information via pipe Since you don't have the actual object, but only the JSON - serialized information of the object, ur rendering should be made a bit differently. So something like: if object['type'] == 'car': screen.blit(car_image, object['position']) else ... something like that u need to load all assets in rendering process, and all information processing in the rendering part Only game logic will be processed in the other process. The problem starts when there are about 5000 - 10000 particles to render on to the screen Sending too many objects via the pipe per frame might slow it down. You might need to optimize the amount of data sent too. Maybe define a reference of objects, and use small hashes or integers to represent data instead of strings. Because strings can easily take a good chunk of memory, and block the pipe from processing further. Even if using ints, it might be problematic to pass all objects simultaneously. You might need to implement a queue of objects, and you send them in chunks. However, as long as the chunks are being processed, the rendering frame should not refresh. It should refresh only when all the processing has been done. This method should give you a major performance boost, but it might be too much of work. Also there could be various inconsitensies in the rendering frame, and 'lag' created, so your program needs to account for that too. I have passed large objects of 1-2 MB and they seem to work fine so far. So as long as you don't have too many objects and only few important properties to pass into the objects, then you should be able to implement a quite good setup. If you implement that setup, please do share it here, I am interested in seeing it : )
@@codekingchannel Yes, I am converting images on import: self.SPACE = pygame.transform.scale(pygame.image.load(os.path.join('Assets', 'space2.png')), (width, height)).convert(self.HUD) # background image It seems to make a really big difference for large images such as the scrolling background, but after a bit of testing it didn't seem to improve the performance of ship images. I think this is most likely because they have to be rotated every frame, and unfortunately I don't see a good way to get around this. (clip of the game: ruclips.net/video/SCMTbBTGiRQ/видео.html) I see what you're saying- the pipe itself may cause more of a performance issue than it fixes. There is one other project I have found that implements this kind of multiprocessing: www.reddit.com/r/pygame/comments/4k4np5/using_python_multiprocessing_in_pygame_game/ Are there any other obvious performance improvements that I might be missing? Thanks for the help :)
@@ln9296 Idk what optimizations you have tried so far, maybe caching could help? Also how do you render font? Fonts take a lot of FPS sometimes if poorly implemented. Make 1 font object first and then render text using it. Don't initialize font objects every frame. Your game looks good so far, but it does not seem it should consume that much FPS. Also the FPS you are specifying, is it based on FULLSCREEN MODE or non-full-screen? Also depends on how you are rendering objects. Multiple loops or one major loop. Also are unnecessary objects garbage collected or they still hover around? One more thing, for the same object type, are you loading different images for each object? If so, you should try to load only once and re-use them multiple times. Also how big are your transforms? Small sprite transforms should not be taking up too much FPS. But if you rotate the entire screen or scale up the entire screen, it would be highly FPS consuming because it is CPU based code. In that case, you should use the SCALED flag to use the GPU for scaling up the screen.
@@codekingchannel 1) fonts are initialized outside of the main loop so that shouldn't be a problem 2) I actually didn't know pygame had a full-screen mode... could you elaborate on this point? 3) object rendering is all handled in one function, but there is a separate loop for every object type (ships, stations, asteroids etc) will this slow things down? 4) there is no garbage collection, objects are removed from their respective lists using 'list.pop(index)' as I found this was much faster and more reliable than 'list.remove(object)' I havent noticed performance worsening after letting the game run but maybe I havent let it run long enough... should I use 'del object' after removing the object from a list? 5) all images (with the exception of ship images) are stored in a dictionary which is referenced during the rendering process. ships each have their own unique images as they need to have the weapon load out blitted on. I have also taken care to ensure only objects on the screen are drawn. 6) Large images such as the background are scaled/rotated outside of the main loop. ship images have to be rotated every frame- again not sure if there is a way around that? 7) I am not too familiar with the scaled flag, from what I can find it seems to be used to scale the entire game window and not individual surfaces blitted onto it? in addition to all of this I am working to improve the performance of the collision detection and nearest neighbor searches- I plan on using something vaguely similar to an rtree
@@ln9296 [1] good [2] write `pygame.display.set_mode((WIDTH, HEIGHT), pygame.SCALED | pygame.FULLSCREEN)` [3] Separate loop might slow it down, but might not have noticeable difference (it should be fine for now i guess) [4] that is not a very recommended way of removing objects make a property for the object if it is to be removed or not once the attribute becomes true, then remove it via list comprehension all_objects = [ i for i in all_objects if not i.to_remove ] it is mainly because of this list comprehension, I advised you to have 1 big game object list because list comprehension is much faster in this case You may also use the filter built-in function to filter the required objects out of all the objects. Both filter and list comprehension have similar performance [5] Good good [6] Use functools.lru_cache to cache the rotated images to some extent, that might improve performance but take slightly more memory [7] SCALED flag is used for entire window, yes, and you can use it to effectively scale up the window in case you were blitting all into a small surface first and then scaling up to a bigger surface. Only if you were doing this, SCALED is recommended, otherwise no need for SCALED flag. However it is always recommended to use SCALED when using FULLSCREEN mode to preserve Aspect ratio, but the scaling is pixelated, which means if you wanted a smooth scale, then it won't be implemented properly. This is mostly a pixel art scaling function Yes collision detection could be consuming some FPS too, optimizations on that part might help.
![How to Make a 3D Game in Python from Scratch [ Pygame ]](http://i.ytimg.com/vi/SmKBsArp2dI/mqdefault.jpg)
man you are under rated, Keep up the good work
Thank You so much!
There’s very little resources on python multiprocessing, and ive struggled hard with it. Watching this vid helped me a lot, thx!
P.S. Underrated channel, can see ur videos blowing up!
I am glad that the video helped :)
Thanks a lot! ❤️❤️❤️
That’s a nice way to do it ! I did it otherwise a few months ago. I basically had a variable set to None, and I started a Thread with as a Target a function that returns the loaded object. So I did a while variable is None: and displayed the loading screen, and if the variable was not None anymore it would mean the loading process had ended. I guess it’s “hackier“ than your way
That is a good way of doing it, but it uses threading. I also did the same initially only to find that the FPS dropped to about 40 if I had a lot of images to load. That is why I decided to use multiprocessing to do the same. Thanks for sharing your approach :)
[Edit: Spelling Correction]
Very Nice.
Wooow. This is very useful. I believe this can help to make good visuals for game intro and transitions. I wanna test this out really sooon. Very great video and editing btw.
I only want to know that let's say I have a bunch of heavy calculations (100) and they take 5 seconds to complete. Using this multiprocessing, is it possible to boost performance?
Thank You!
Indeed you can boost the performance, but it depends on your CPU.
in most cases, with the example you gave you should be able to improve performance using a standard CPU.
But lets say you have 10K calculations, then an i3 processor single/dual core may not be able to reduce the time taken much compared to what an i5 or an i7 processor can do :)
Hello, I am attempting to make a game using pygame, but am completely new to multiprocessing. The game currently runs reliably at 60+ fps, but I would like to try to get it to run at 120 fps by separating the rendering from the game logic. I dont see many other examples of people separating these processes, is this a waste of time?
Hi, it might not be a waste of time depending on the context, but have you already tried using other methods of optimization? Like using convert or convert_alpha on surfaces and minimizing draw / blit calls? I am pretty sure you can optimize your game enough to run at 120+ FPS.
However, if you must use other ways to speed up, then you can do the following ->
make 1 process for rendering
make another process for game logic
connect them with a pipe object
and transfer all information via pipe
Since you don't have the actual object, but only the JSON - serialized information of the object, ur rendering should be made a bit differently.
So something like:
if object['type'] == 'car': screen.blit(car_image, object['position'])
else ...
something like that
u need to load all assets in rendering process, and all information processing in the rendering part
Only game logic will be processed in the other process.
The problem starts when there are about 5000 - 10000 particles to render on to the screen
Sending too many objects via the pipe per frame might slow it down. You might need to optimize the amount of data sent too.
Maybe define a reference of objects, and use small hashes or integers to represent data instead of strings. Because strings can easily take a good chunk of memory, and block the pipe from processing further. Even if using ints, it might be problematic to pass all objects simultaneously. You might need to implement a queue of objects, and you send them in chunks.
However, as long as the chunks are being processed, the rendering frame should not refresh. It should refresh only when all the processing has been done.
This method should give you a major performance boost, but it might be too much of work.
Also there could be various inconsitensies in the rendering frame, and 'lag' created, so your program needs to account for that too.
I have passed large objects of 1-2 MB and they seem to work fine so far. So as long as you don't have too many objects and only few important properties to pass into the objects, then you should be able to implement a quite good setup.
If you implement that setup, please do share it here, I am interested in seeing it : )
@@codekingchannel Yes, I am converting images on import:
self.SPACE = pygame.transform.scale(pygame.image.load(os.path.join('Assets', 'space2.png')),
(width, height)).convert(self.HUD) # background image
It seems to make a really big difference for large images such as the scrolling background, but after a bit of testing it didn't seem to improve the performance of ship images. I think this is most likely because they have to be rotated every frame, and unfortunately I don't see a good way to get around this.
(clip of the game: ruclips.net/video/SCMTbBTGiRQ/видео.html)
I see what you're saying- the pipe itself may cause more of a performance issue than it fixes. There is one other project I have found that implements this kind of multiprocessing: www.reddit.com/r/pygame/comments/4k4np5/using_python_multiprocessing_in_pygame_game/ Are there any other obvious performance improvements that I might be missing?
Thanks for the help :)
@@ln9296 Idk what optimizations you have tried so far, maybe caching could help?
Also how do you render font?
Fonts take a lot of FPS sometimes if poorly implemented. Make 1 font object first and then render text using it. Don't initialize font objects every frame. Your game looks good so far, but it does not seem it should consume that much FPS. Also the FPS you are specifying, is it based on FULLSCREEN MODE or non-full-screen?
Also depends on how you are rendering objects. Multiple loops or one major loop. Also are unnecessary objects garbage collected or they still hover around? One more thing, for the same object type, are you loading different images for each object? If so, you should try to load only once and re-use them multiple times. Also how big are your transforms? Small sprite transforms should not be taking up too much FPS. But if you rotate the entire screen or scale up the entire screen, it would be highly FPS consuming because it is CPU based code.
In that case, you should use the SCALED flag to use the GPU for scaling up the screen.
@@codekingchannel
1) fonts are initialized outside of the main loop so that shouldn't be a problem
2) I actually didn't know pygame had a full-screen mode... could you elaborate on this point?
3) object rendering is all handled in one function, but there is a separate loop for every object type (ships, stations, asteroids etc) will this slow things down?
4) there is no garbage collection, objects are removed from their respective lists using 'list.pop(index)' as I found this was much faster and more reliable than 'list.remove(object)' I havent noticed performance worsening after letting the game run but maybe I havent let it run long enough... should I use 'del object' after removing the object from a list?
5) all images (with the exception of ship images) are stored in a dictionary which is referenced during the rendering process. ships each have their own unique images as they need to have the weapon load out blitted on. I have also taken care to ensure only objects on the screen are drawn.
6) Large images such as the background are scaled/rotated outside of the main loop. ship images have to be rotated every frame- again not sure if there is a way around that?
7) I am not too familiar with the scaled flag, from what I can find it seems to be used to scale the entire game window and not individual surfaces blitted onto it?
in addition to all of this I am working to improve the performance of the collision detection and nearest neighbor searches- I plan on using something vaguely similar to an rtree
@@ln9296
[1] good
[2] write `pygame.display.set_mode((WIDTH, HEIGHT), pygame.SCALED | pygame.FULLSCREEN)`
[3] Separate loop might slow it down, but might not have noticeable difference (it should be fine for now i guess)
[4] that is not a very recommended way of removing objects
make a property for the object if it is to be removed or not
once the attribute becomes true, then remove it via list comprehension
all_objects = [ i for i in all_objects if not i.to_remove ]
it is mainly because of this list comprehension, I advised you to have 1 big game object list
because list comprehension is much faster in this case
You may also use the filter built-in function to filter the required objects out of all the objects. Both filter and list comprehension have similar performance
[5] Good good
[6] Use functools.lru_cache to cache the rotated images to some extent, that might improve performance but take slightly more memory
[7] SCALED flag is used for entire window, yes, and you can use it to effectively scale up the window in case you were blitting all into a small surface first and then
scaling up to a bigger surface. Only if you were doing this, SCALED is recommended, otherwise no need for SCALED flag. However it is always recommended to use SCALED
when using FULLSCREEN mode to preserve Aspect ratio, but the scaling is pixelated, which means if you wanted a smooth scale, then it won't be implemented properly. This is
mostly a pixel art scaling function
Yes collision detection could be consuming some FPS too, optimizations on that part might help.