blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

René Dudfield
Hello,

this blit_list, blits, blit_mult, blit_many call...

I feel we should call it "Surface.blits". To go with drawline/drawlines.
It would take a sequence of tuples which match up with the blit arguments.

The original Surface.blit API.
    http://www.pygame.org/docs/ref/surface.html#pygame.Surface.blit
  blit(source, dest, area=None, special_flags = 0) -> Rect

The new blits API.
    blits(args) -> [rects]
    args = [(source: Surface, dest: Rect, area: Rect = None, special_flags: int = 0), ...]
    Draws a sequence of Surfaces onto this Surface...

    :Example:
    >>> surf.blits([(source, dest),
                  
(source, dest),
                  
(source, dest, area),
                  
(source, dest, area, BLEND_ADD)]
    [Rect(), Rect(), Rect(), Rect()]



One potential option...
  • Have a return_rects=False argument, where if you pass it, then it can return None instead of a list of rects. This way you avoid allocating a list, and all the rects inside it. I'll benchmark this to see if it's worth it -- but I have a feeling all those allocations will be significant. But some people don't track updates, so allocating the rects is not worth it for them. eg. the implementation from Leif doesn't return rects.

It can handle these use cases:
  • blit many different surfaces to one surface (like the screen)
  • blit one surface many times to one surface.
  • when you don't care about rects, it doesn't allocate them.
  • when you do care about update tracking, it can track them.
It can *not* handle (but would anyone care?):
  • blit many surfaces, to many other surfaces.

Areas not included in the scope of this:
  • This could be used by two sprite groups quite easily (Group, RenderUpdates). But I think it's worth trying to compile the sprite groups with Cython instead, as a separate piece of work.
  • Multi processing. It should be possible to use this API to build a multi process blitter. However, this is not addressed in this work. The Surface we are blitting onto could be split up into numberOfCore tiles, and rendered that way. This is classic tile rendering, and nothing in this API stops an implementation of this later.


Enhancements, objections?



cheers,





On Mon, Mar 27, 2017 at 6:38 PM, René Dudfield <[hidden email]> wrote:
Cool about the blit function. Since the rect and surfaces are by reference, then you only need to make sprite_image_list once (if you are blitting the same thing every time). Additionally, I think it would likely be fast enough for another commonly requested case (blit the same image to many different spots). It would also work nicely for sprite groups. I think it's worth some more API discussion, before merging it in. We'd also need some tests and docs.



On Mon, Mar 27, 2017 at 4:59 PM, Leif Theden <[hidden email]> wrote:
I think a function that accepts a sequence of tuples in the form of (dest_surface, dest_position, source_area, blend_mode) would be enough.  It is only needed as a power-user optimization and I don't seem much value in watering it down or splitting it into multiple functions for beginners.  No need to overthink and complicate it.  Cython may help sprite rendering in Groups, but it would have to be implemented and tested.

Has anyone considered releasing the GIL during blit operations?  Is it possible the release the GIL during a blit, then block when needed, in the cases of subsequent draw operations?  The optimized case would be to allow math operations to execute while the display buffer is being modified, while you are iterating over sprites.

Here is my preliminary C code for a "blit_multi" function.  It is called "blit_list".  It only supports (dest_surface, dest_position) tuples, but it works as so far.
https://gist.github.com/bitcraft/1785face7c5684916cde

Use it like this:

sprite_image_list = [(s.image, s.rect) for s in self.sprites]
display.blit_list(sprite_image_list)

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

Leif Theden
My votes:
  • don't bother with return value.  If a person cared (probably doesn't), then the rects can be inferred from the list sent to Surface.blits()  (again, not a fan of dirty rects)
  • or, make the default behavior not return a list of rects (return=False).  default-on would be incorrect, imo.
  • many sources to many destinations would be complex, and there are no other pygame methods that operate like that (that i am aware of). so no for now, unless very easy to implement (probably not).

I would maybe consider enforcing a fixed tuple size that is sent to the function.  The cost of dynamically unpacking and checking the length of each might be detrimental (benchmark, maybe?)

I'm not so sure that multiprocessing is a way forward, because of the lock contention involved with the destination surface.  Complex schemes to make lists of non-overlapping rects and delegating to workers smells np-complete to me.  Threads/processes waiting around for a lock seems like it would ruin the benefits of parallelism.  Who knows, however?

This would be a great benefit for the built in sprite groups.  Looking forward to seeing it implemented.



On Tue, Mar 28, 2017 at 11:35 PM, René Dudfield <[hidden email]> wrote:
Hello,

this blit_list, blits, blit_mult, blit_many call...

I feel we should call it "Surface.blits". To go with drawline/drawlines.
It would take a sequence of tuples which match up with the blit arguments.

The original Surface.blit API.
    http://www.pygame.org/docs/ref/surface.html#pygame.Surface.blit
  blit(source, dest, area=None, special_flags = 0) -> Rect

The new blits API.
    blits(args) -> [rects]
    args = [(source: Surface, dest: Rect, area: Rect = None, special_flags: int = 0), ...]
    Draws a sequence of Surfaces onto this Surface...

    :Example:
    >>> surf.blits([(source, dest),
                  
(source, dest),
                  
(source, dest, area),
                  
(source, dest, area, BLEND_ADD)]
    [Rect(), Rect(), Rect(), Rect()]



One potential option...
  • Have a return_rects=False argument, where if you pass it, then it can return None instead of a list of rects. This way you avoid allocating a list, and all the rects inside it. I'll benchmark this to see if it's worth it -- but I have a feeling all those allocations will be significant. But some people don't track updates, so allocating the rects is not worth it for them. eg. the implementation from Leif doesn't return rects.

It can handle these use cases:
  • blit many different surfaces to one surface (like the screen)
  • blit one surface many times to one surface.
  • when you don't care about rects, it doesn't allocate them.
  • when you do care about update tracking, it can track them.
It can *not* handle (but would anyone care?):
  • blit many surfaces, to many other surfaces.

Areas not included in the scope of this:
  • This could be used by two sprite groups quite easily (Group, RenderUpdates). But I think it's worth trying to compile the sprite groups with Cython instead, as a separate piece of work.
  • Multi processing. It should be possible to use this API to build a multi process blitter. However, this is not addressed in this work. The Surface we are blitting onto could be split up into numberOfCore tiles, and rendered that way. This is classic tile rendering, and nothing in this API stops an implementation of this later.


Enhancements, objections?



cheers,





On Mon, Mar 27, 2017 at 6:38 PM, René Dudfield <[hidden email]> wrote:
Cool about the blit function. Since the rect and surfaces are by reference, then you only need to make sprite_image_list once (if you are blitting the same thing every time). Additionally, I think it would likely be fast enough for another commonly requested case (blit the same image to many different spots). It would also work nicely for sprite groups. I think it's worth some more API discussion, before merging it in. We'd also need some tests and docs.



On Mon, Mar 27, 2017 at 4:59 PM, Leif Theden <[hidden email]> wrote:
I think a function that accepts a sequence of tuples in the form of (dest_surface, dest_position, source_area, blend_mode) would be enough.  It is only needed as a power-user optimization and I don't seem much value in watering it down or splitting it into multiple functions for beginners.  No need to overthink and complicate it.  Cython may help sprite rendering in Groups, but it would have to be implemented and tested.

Has anyone considered releasing the GIL during blit operations?  Is it possible the release the GIL during a blit, then block when needed, in the cases of subsequent draw operations?  The optimized case would be to allow math operations to execute while the display buffer is being modified, while you are iterating over sprites.

Here is my preliminary C code for a "blit_multi" function.  It is called "blit_list".  It only supports (dest_surface, dest_position) tuples, but it works as so far.
https://gist.github.com/bitcraft/1785face7c5684916cde

Use it like this:

sprite_image_list = [(s.image, s.rect) for s in self.sprites]
display.blit_list(sprite_image_list)


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

Radomir Dopieralski
I think that special_flags could be shared for all the blits in the
list. I can't think of a case where I would want to use different flags
for each.

On Tue, 28 Mar 2017 23:54:32 -0500
Leif Theden <[hidden email]> wrote:

> My votes:
>
>    - don't bother with return value.  If a person cared (probably
> doesn't), then the rects can be inferred from the list sent to
> Surface.blits() (again, not a fan of dirty rects)
>    - or, make the default behavior not return a list of rects
>    (return=False).  default-on would be incorrect, imo.
>    - many sources to many destinations would be complex, and there
> are no other pygame methods that operate like that (that i am aware
> of). so no for now, unless very easy to implement (probably not).
>
> I would maybe consider enforcing a fixed tuple size that is sent to
> the function.  The cost of dynamically unpacking and checking the
> length of each might be detrimental (benchmark, maybe?)
> I'm not so sure that multiprocessing is a way forward, because of the
> lock contention involved with the destination surface.  Complex
> schemes to make lists of non-overlapping rects and delegating to
> workers smells np-complete to me.  Threads/processes waiting around
> for a lock seems like it would ruin the benefits of parallelism.  Who
> knows, however?
>
> This would be a great benefit for the built in sprite groups.  Looking
> forward to seeing it implemented.
>
>
>
> On Tue, Mar 28, 2017 at 11:35 PM, René Dudfield <[hidden email]>
> wrote:
>
> > Hello,
> >
> > this blit_list, blits, blit_mult, blit_many call...
> >
> > I feel we should call it "Surface.blits". To go with
> > drawline/drawlines. It would take a sequence of tuples which match
> > up with the blit arguments.
> >
> > The original Surface.blit API.
> >     http://www.pygame.org/docs/ref/surface.html#pygame.Surface.blit
> >   blit(source, dest, area=None, special_flags = 0) -> Rect
> >
> > The new blits API.
> >     blits(args) -> [rects]
> >     args = [(source: Surface, dest: Rect, area: Rect = None,
> > special_flags: int = 0), ...]
> >     Draws a sequence of Surfaces onto this Surface...
> >
> >     :Example:  
> >     >>> surf.blits([(source, dest),  
> >                    (source, dest),
> >                    (source, dest, area),
> >                    (source, dest, area, BLEND_ADD)]
> >     [Rect(), Rect(), Rect(), Rect()]
> >
> >
> >
> > One potential option...
> >
> >    - Have a return_rects=False argument, where if you pass it, then
> > it can return None instead of a list of rects. This way you avoid
> > allocating a list, and all the rects inside it. I'll benchmark this
> > to see if it's worth it -- but I have a feeling all those
> > allocations will be significant. But some people don't track
> > updates, so allocating the rects is not worth it for them. eg. the
> > implementation from Leif doesn't return rects.
> >
> >
> > It can handle these use cases:
> >
> >    - blit many different surfaces to one surface (like the screen)
> >    - blit one surface many times to one surface.
> >    - when you don't care about rects, it doesn't allocate them.
> >    - when you do care about update tracking, it can track them.
> >
> > It can *not* handle (but would anyone care?):
> >
> >    - blit many surfaces, to many other surfaces.
> >
> >
> > Areas not included in the scope of this:
> >
> >    - This could be used by two sprite groups quite easily (Group,
> >    RenderUpdates). But I think it's worth trying to compile the
> > sprite groups with Cython instead, as a separate piece of work.
> >    - Multi processing. It should be possible to use this API to
> > build a multi process blitter. However, this is not addressed in
> > this work. The Surface we are blitting onto could be split up into
> > numberOfCore tiles, and rendered that way. This is classic tile
> > rendering, and nothing in this API stops an implementation of this
> > later.
> >
> >
> >
> > Enhancements, objections?
> >
> >
> >
> > cheers,
> >
> >
> >
> >
> >
> > On Mon, Mar 27, 2017 at 6:38 PM, René Dudfield <[hidden email]>
> > wrote:
> >> Cool about the blit function. Since the rect and surfaces are by
> >> reference, then you only need to make sprite_image_list once (if
> >> you are blitting the same thing every time). Additionally, I think
> >> it would likely be fast enough for another commonly requested case
> >> (blit the same image to many different spots). It would also work
> >> nicely for sprite groups. I think it's worth some more API
> >> discussion, before merging it in. We'd also need some tests and
> >> docs.
> >>
> >>
> >>
> >> On Mon, Mar 27, 2017 at 4:59 PM, Leif Theden
> >> <[hidden email]> wrote:
> >>  
> >>> I think a function that accepts a sequence of tuples in the form
> >>> of (dest_surface, dest_position, source_area, blend_mode) would
> >>> be enough.  It is only needed as a power-user optimization and I
> >>> don't seem much value in watering it down or splitting it into
> >>> multiple functions for beginners.  No need to overthink and
> >>> complicate it.  Cython may help sprite rendering in Groups, but
> >>> it would have to be implemented and tested.
> >>>
> >>> Has anyone considered releasing the GIL during blit operations?
> >>> Is it possible the release the GIL during a blit, then block when
> >>> needed, in the cases of subsequent draw operations?  The
> >>> optimized case would be to allow math operations to execute while
> >>> the display buffer is being modified, while you are iterating
> >>> over sprites.
> >>>
> >>> Here is my preliminary C code for a "blit_multi" function.  It is
> >>> called "blit_list".  It only supports (dest_surface,
> >>> dest_position) tuples, but it works as so far.
> >>> https://gist.github.com/bitcraft/1785face7c5684916cde
> >>>
> >>> Use it like this:
> >>>
> >>> sprite_image_list = [(s.image, s.rect) for s in self.sprites]
> >>> display.blit_list(sprite_image_list)
> >>>
> >>>  



--
Radomir Dopieralski

--
Radomir Dopieralski
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

René Dudfield
On Wed, Mar 29, 2017 at 8:33 AM, Radomir Dopieralski <[hidden email]> wrote:
I think that special_flags could be shared for all the blits in the
list. I can't think of a case where I would want to use different flags
for each.


Yeah, I think you may be right. When I've used it (for particles as an example) it was for everything in a Group. I searched through some code online and found a few uses where it was used in Sprites. But then all of those Sprites were rendered the same. There's some sort of nice symmetry by using the same arguments with blit, and blits however.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

DiliupG
When will this fabulous upgrade be available? :)

On 29 March 2017 at 12:58, René Dudfield <[hidden email]> wrote:
On Wed, Mar 29, 2017 at 8:33 AM, Radomir Dopieralski <[hidden email]> wrote:
I think that special_flags could be shared for all the blits in the
list. I can't think of a case where I would want to use different flags
for each.


Yeah, I think you may be right. When I've used it (for particles as an example) it was for everything in a Group. I searched through some code online and found a few uses where it was used in Sprites. But then all of those Sprites were rendered the same. There's some sort of nice symmetry by using the same arguments with blit, and blits however.



--
http://www.diliupg.com

**********************************************************************************************
This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
**********************************************************************************************

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

René Dudfield
Not sure. I need to do some benchmarking, and then more research into if the API can cover everything we want it to do. (which of course means figuring it out). So far I think the API I proposed will help with the Groups we have, and will also help with people who don't care about dirty rects.

It's not really much code... as you can see Leif has already done something like it... it's just a for loop around blit basically.
If someone who just knows python wants to help, they could begin writing unit tests for it in test/surface_test.py

My pygame things are basically these at the moment. If anyone wants to do the blits() API, that's fine too!

* keep improving https://pygame.org/wiki/GettingStarted , gather and file usability bugs with pip, python, pygame...
* help finalise github move decision/move.
* blits
* cython sprites.*Group
* SDL2 stuff


cheers,




On Thu, Mar 30, 2017 at 8:02 AM, DiliupG <[hidden email]> wrote:
When will this fabulous upgrade be available? :)

On 29 March 2017 at 12:58, René Dudfield <[hidden email]> wrote:
On Wed, Mar 29, 2017 at 8:33 AM, Radomir Dopieralski <[hidden email]> wrote:
I think that special_flags could be shared for all the blits in the
list. I can't think of a case where I would want to use different flags
for each.


Yeah, I think you may be right. When I've used it (for particles as an example) it was for everything in a Group. I searched through some code online and found a few uses where it was used in Sprites. But then all of those Sprites were rendered the same. There's some sort of nice symmetry by using the same arguments with blit, and blits however.



--
http://www.diliupg.com

**********************************************************************************************
This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
**********************************************************************************************


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: blits proposal, was Re: [pygame] Dirty rect overlapping optimizations

Leif Theden
Pull request sent with basic functionality of Surface.blits.

https://bitbucket.org/pygame/pygame/pull-requests/79

On Thu, Mar 30, 2017 at 2:44 AM, René Dudfield <[hidden email]> wrote:
Not sure. I need to do some benchmarking, and then more research into if the API can cover everything we want it to do. (which of course means figuring it out). So far I think the API I proposed will help with the Groups we have, and will also help with people who don't care about dirty rects.

It's not really much code... as you can see Leif has already done something like it... it's just a for loop around blit basically.
If someone who just knows python wants to help, they could begin writing unit tests for it in test/surface_test.py

My pygame things are basically these at the moment. If anyone wants to do the blits() API, that's fine too!

* keep improving https://pygame.org/wiki/GettingStarted , gather and file usability bugs with pip, python, pygame...
* help finalise github move decision/move.
* blits
* cython sprites.*Group
* SDL2 stuff


cheers,




On Thu, Mar 30, 2017 at 8:02 AM, DiliupG <[hidden email]> wrote:
When will this fabulous upgrade be available? :)

On 29 March 2017 at 12:58, René Dudfield <[hidden email]> wrote:
On Wed, Mar 29, 2017 at 8:33 AM, Radomir Dopieralski <[hidden email]> wrote:
I think that special_flags could be shared for all the blits in the
list. I can't think of a case where I would want to use different flags
for each.


Yeah, I think you may be right. When I've used it (for particles as an example) it was for everything in a Group. I searched through some code online and found a few uses where it was used in Sprites. But then all of those Sprites were rendered the same. There's some sort of nice symmetry by using the same arguments with blit, and blits however.



--
http://www.diliupg.com

**********************************************************************************************
This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
**********************************************************************************************



Loading...