pygame with SDL2 proposal

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
61 messages Options
1234
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

pygame with SDL2 proposal

René Dudfield
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

lhughes41
This post has NOT been accepted by the mailing list yet.
This is extremely wise (practical).
bw
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

bw
In reply to this post by René Dudfield
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import
pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy,
CFFI, plus all the SDL2 libs), I found that I would really like to see
in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes
all of the SDL2 API to programmers, not just what the C developers
choose to expose. This would make pygame superbly extensible. And it
would allow easy ad hoc inclusion of third-party prebuilt libs,
something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is
sorely needed in my opinion. One can use optimization tricks to
compensate for CPython's speed limit, but one cannot do many things that
are computationally intensive without farming out to a third-party
opaque library. pypy could let coders do a lot more in Python, when a
full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because
pygame has a pyd and I'm not spiffy enough to port it to anything but
Python. If CFFI and pypy are an option worthy of consideration, Rect
would be best done in a shared library. Rect is truly a workhorse, and
pypy Rects can't compete with pygame Rects under heavy usage of Rect()
and inflate and move, etc. In pygame most of my time is spent in blit().
In pypy most of my time is spent in Rect, because the class is 100%
Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import
gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard
for software rendering compatibility, I was shooting for the new texture
features. As expected, I discovered that all of the software-blit
optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and
discussion into the next gen, and I understand this is something
completely different. I certainly don't propose gsdl2 as a new code
base. But it would be grand if choose to use CFFI. I have no stake in
seeing my code transform into a butterfly, but if parts of it are useful
then good.

Also, an acceptable alternative might be to provide CFFI-compatible
shared libs so that people who want to use pypy have that choice, even
if they do not have the nice pygame API available. If so, anybody could
write a pygame-like API just like I did, and it doesn't need to be
maintained by the pygame core developers.


On 3/18/2017 5:02 AM, René Dudfield wrote:

> Hello,
>
> so, I've spent some more time reading source code, and investigating
> SDL2 and the options a lot.
>
> I *think* this plan below could be the best way forward for us.
>
> tldr;
>  * have a single source SDL1, and SDL2 code base with a compile time
> option. (like we have single source py2 and py3).
>  * move bitbucket/pygame repo to github/pygame.
>
>
> Rather than reimplementing everything, and introducing lots of
> compatibility bugs, Instead I think a gradual approach would work
> better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not
> such a great way forward (for the pygame project, I think it was
> necessary and good for Ren'Py).
>
> A big breaking change is kind of silly for how little resources we
> have. Python 3 managed to pull it off... after a decade, and with
> massive resources having poured into it. And it only took off once
> they put compatibility code back in. Here's the thread where some
> discussion has been happening.
> https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition
>
>
> What we do have is some patches to get the current code base working
> with SDL2.
> https://bitbucket.org/llindstrom/pygame-1.10-patch
>
> I think it should be possible with some work to improve an SDL2
> compatibility layer, so that pygame code base can work with either (as
> a compile time option). Then newer APIs can be introduced in time, in
> a non break-every-program kind of way. Also, it's been proven that
> 'hardware' blitting does not need to break SDL1 API compatibility to
> use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other
> projects], ...).
>
> Having a pygame2, or whatever separate repo would also make
> maintenance harder. Since for the foreseeable future, we will likely
> need to do maintenance releases for this anyway (at least I want to!,
> and I know other users of pygame will).
>
> ---
>
> For pip uploads, they would continue to be for SDL1, until we can
> finish the SDL2 changes, and it works better. There would be no new
> additions until compatibility is mostly there.
>
> The work would progress by slowly adding in compatibility changes
> whilst keeping pygame working. By keeping the SDL2 patch set as is,
> and slowly reducing the patch set until it is size zero. So we always
> have a pygame with sdl2, and a pygame with sdl1 that is producible.
> Eventually the patch set will disappear.
>
> ---
>
> A pygame2 module would just cause confusion amongst users, and that
> really boring 'pygame 1 or pygame 2' type debate would go on, and on
> like the "python 2, verses python 3" one did. For me, just avoiding
> that discussion for the next decade would be worth it.
>
> Then there would need to be two sets of documentation on the website.
> And two sets of tutorials... and... we'd have 2 of everything to *do*,
> and 2 of everything for people to look at.
>
> Finally, "import pygame2" looks kind of weird. I've grown used to
> "import pygame".
>
> ...
>
>
>
> I'm keen to get moving on this. I've been trying to get feedback on
> the 'pygame sdl2 addition' issue for the last month, and the issue has
> been open since 2013. So I'd like to put a time limit on this decision
> for one more week.
>
> I'd really like to hear back from contributors, and users (everyone).
>
>
>
>
> ps. Interestingly SDL_gfx has an SDL2 release now.
> http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

René Dudfield
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.


CPython has had some really nice speed improvements recently (since 2015). Python 3.6 is good, and 3.7 is looking significantly faster for some benchmarks.
    https://speed.python.org/comparison/?exe=5%2BL%2Bdefault%2C5%2BL%2B2.7&ben=223%2C224%2C225%2C226%2C227%2C228%2C229%2C230%2C231%2C232%2C233%2C234%2C235%2C236%2C237%2C238%2C239%2C287%2C240%2C241%2C242%2C243%2C244%2C245%2C246%2C247%2C248%2C249%2C250%2C251%2C252%2C253%2C254%2C255%2C288%2C256%2C257%2C258%2C259%2C260%2C261%2C262%2C263%2C264%2C265%2C266%2C267%2C268%2C289%2C269%2C270%2C271%2C272%2C273%2C274%2C276%2C275%2C277%2C278%2C279%2C280%2C281%2C282%2C285%2C284%2C283%2C286&env=1&hor=true&bas=none&chart=normal+bars

Especially with all the FASTCALL work.
    https://haypo.github.io/fastcall-issues.html
    http://faster-cpython.readthedocs.io/fat_python.html
Accepted CPython jit API pep.
    https://www.python.org/dev/peps/pep-0523/


I like this trend where people have decided to work together to get things into the CPython project, rather than do forks.


cheers,




On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

René Dudfield

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.


CPython has had some really nice speed improvements recently (since 2015). Python 3.6 is good, and 3.7 is looking significantly faster for some benchmarks.
    https://speed.python.org/comparison/?exe=5%2BL%2Bdefault%2C5%2BL%2B2.7&ben=223%2C224%2C225%2C226%2C227%2C228%2C229%2C230%2C231%2C232%2C233%2C234%2C235%2C236%2C237%2C238%2C239%2C287%2C240%2C241%2C242%2C243%2C244%2C245%2C246%2C247%2C248%2C249%2C250%2C251%2C252%2C253%2C254%2C255%2C288%2C256%2C257%2C258%2C259%2C260%2C261%2C262%2C263%2C264%2C265%2C266%2C267%2C268%2C289%2C269%2C270%2C271%2C272%2C273%2C274%2C276%2C275%2C277%2C278%2C279%2C280%2C281%2C282%2C285%2C284%2C283%2C286&env=1&hor=true&bas=none&chart=normal+bars

Especially with all the FASTCALL work.
    https://haypo.github.io/fastcall-issues.html
    http://faster-cpython.readthedocs.io/fat_python.html
Accepted CPython jit API pep.
    https://www.python.org/dev/peps/pep-0523/


I like this trend where people have decided to work together to get things into the CPython project, rather than do forks.


cheers,




On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

ThomasMNS
This post has NOT been accepted by the mailing list yet.
In reply to this post by René Dudfield
Hi René,

This sounds like a reasonable plan, allowing for Pygame to move forward to sdl2 while recognising the current status of the project, namely
- The currently small number of contributors making large changes unrealistic
- Pygame's status as a good beginners tool, helped by the large amount of excellent books, courses, code examples etc available. If these no longer worked with a new Pygame, new users are going to find it far more difficult to get involved
- Pygame's usage in teaching environments, tutors are not going to want to change their curriculum to use a new Pygame if the current version is working fine for them
- As you've pointed out the issues with maintaining 2 sets of projects, docs etc, when just maintaining the current ones has proved difficult

Kindest regards,
Tom
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Leif Theden
In reply to this post by René Dudfield
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Daniel Pope

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Leif Theden
I don't follow your idea about mistakes in Kivy. It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi. I love kivy and have been using it for a long time. It's not extremely stable (API changes), but it is light years ahead of pygame in terms of usability and functionality.

It may be a bit more cumbersome in Kivy for use with games or extreme beginners, but I don't think it would take much to have a games API if the community put effort in to it.

I'm all for a simple pygame too, and I don't think the comparison to Kivy is appropriate. Kivy is a framework for touch applications primarily.
On Sun, Mar 19, 2017 at 9:28 AM Daniel Pope <[hidden email]> wrote:

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Daniel Pope

> It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi

That's not the part that I am claiming is a tight coupling. The tight coupling is between the OpenGL/input bindings for mobile - which could be great, but is incomplete and undocumented - and the UI framework, which I don't like.

> I don't think it would take much to have a games API if the community put effort in to it.

Exactly - if it has to be an API of Kivy, rather than a separate project built upon it, then that's a tight coupling.


On Sun, 19 Mar 2017, 15:47 Leif Theden, <[hidden email]> wrote:
I don't follow your idea about mistakes in Kivy. It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi. I love kivy and have been using it for a long time. It's not extremely stable (API changes), but it is light years ahead of pygame in terms of usability and functionality.

It may be a bit more cumbersome in Kivy for use with games or extreme beginners, but I don't think it would take much to have a games API if the community put effort in to it.

I'm all for a simple pygame too, and I don't think the comparison to Kivy is appropriate. Kivy is a framework for touch applications primarily.
On Sun, Mar 19, 2017 at 9:28 AM Daniel Pope <[hidden email]> wrote:

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Leif Theden
Idk. You could argue that pygame had a "tight coupling" to pushing pixels with a CPU. Or the display for that matter (ever try to use the sound module before initializing the display?) In any case, I don't see this comparison to Kivy going anywhere.

On Sun, Mar 19, 2017 at 11:49 AM Daniel Pope <[hidden email]> wrote:

> It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi

That's not the part that I am claiming is a tight coupling. The tight coupling is between the OpenGL/input bindings for mobile - which could be great, but is incomplete and undocumented - and the UI framework, which I don't like.

> I don't think it would take much to have a games API if the community put effort in to it.

Exactly - if it has to be an API of Kivy, rather than a separate project built upon it, then that's a tight coupling.


On Sun, 19 Mar 2017, 15:47 Leif Theden, <[hidden email]> wrote:
I don't follow your idea about mistakes in Kivy. It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi. I love kivy and have been using it for a long time. It's not extremely stable (API changes), but it is light years ahead of pygame in terms of usability and functionality.

It may be a bit more cumbersome in Kivy for use with games or extreme beginners, but I don't think it would take much to have a games API if the community put effort in to it.

I'm all for a simple pygame too, and I don't think the comparison to Kivy is appropriate. Kivy is a framework for touch applications primarily.
On Sun, Mar 19, 2017 at 9:28 AM Daniel Pope <[hidden email]> wrote:

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Thomas Kluyver
Going back to basics for a moment: what is the advantage for game developers of SDL 2 over 1.2? Is it more performant? Does it offer easier to understand abstractions? Can you make games do things that would be impossible with 1.2? And are there any downsides to the new version? I think we need an idea of the differences to have a useful discussion on supporting SDL 2.

Thomas

On 19 Mar 2017 6:04 p.m., "Leif Theden" <[hidden email]> wrote:
Idk. You could argue that pygame had a "tight coupling" to pushing pixels with a CPU. Or the display for that matter (ever try to use the sound module before initializing the display?) In any case, I don't see this comparison to Kivy going anywhere.

On Sun, Mar 19, 2017 at 11:49 AM Daniel Pope <[hidden email]> wrote:

> It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi

That's not the part that I am claiming is a tight coupling. The tight coupling is between the OpenGL/input bindings for mobile - which could be great, but is incomplete and undocumented - and the UI framework, which I don't like.

> I don't think it would take much to have a games API if the community put effort in to it.

Exactly - if it has to be an API of Kivy, rather than a separate project built upon it, then that's a tight coupling.


On Sun, 19 Mar 2017, 15:47 Leif Theden, <[hidden email]> wrote:
I don't follow your idea about mistakes in Kivy. It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi. I love kivy and have been using it for a long time. It's not extremely stable (API changes), but it is light years ahead of pygame in terms of usability and functionality.

It may be a bit more cumbersome in Kivy for use with games or extreme beginners, but I don't think it would take much to have a games API if the community put effort in to it.

I'm all for a simple pygame too, and I don't think the comparison to Kivy is appropriate. Kivy is a framework for touch applications primarily.
On Sun, Mar 19, 2017 at 9:28 AM Daniel Pope <[hidden email]> wrote:

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Leif Theden
One huge advantage is that it is supported by the main developers. SDL 1.2 won't be getting much development (any?) to combat bitrot and depreciations of the interfaces it relies on.

After that, there are new interfaces in SDL 2 to make mobile games and apps fit into the platform better. There is easy hardware accelerated sprites. I would say those are the highlights.
On Sun, Mar 19, 2017 at 1:43 PM Thomas Kluyver <[hidden email]> wrote:
Going back to basics for a moment: what is the advantage for game developers of SDL 2 over 1.2? Is it more performant? Does it offer easier to understand abstractions? Can you make games do things that would be impossible with 1.2? And are there any downsides to the new version? I think we need an idea of the differences to have a useful discussion on supporting SDL 2.

Thomas

On 19 Mar 2017 6:04 p.m., "Leif Theden" <[hidden email]> wrote:
Idk. You could argue that pygame had a "tight coupling" to pushing pixels with a CPU. Or the display for that matter (ever try to use the sound module before initializing the display?) In any case, I don't see this comparison to Kivy going anywhere.

On Sun, Mar 19, 2017 at 11:49 AM Daniel Pope <[hidden email]> wrote:

> It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi

That's not the part that I am claiming is a tight coupling. The tight coupling is between the OpenGL/input bindings for mobile - which could be great, but is incomplete and undocumented - and the UI framework, which I don't like.

> I don't think it would take much to have a games API if the community put effort in to it.

Exactly - if it has to be an API of Kivy, rather than a separate project built upon it, then that's a tight coupling.


On Sun, 19 Mar 2017, 15:47 Leif Theden, <[hidden email]> wrote:
I don't follow your idea about mistakes in Kivy. It's not a tight coupling, it supports several backends and runs on many platforms, from PC to Rpi. I love kivy and have been using it for a long time. It's not extremely stable (API changes), but it is light years ahead of pygame in terms of usability and functionality.

It may be a bit more cumbersome in Kivy for use with games or extreme beginners, but I don't think it would take much to have a games API if the community put effort in to it.

I'm all for a simple pygame too, and I don't think the comparison to Kivy is appropriate. Kivy is a framework for touch applications primarily.
On Sun, Mar 19, 2017 at 9:28 AM Daniel Pope <[hidden email]> wrote:

Something that I think is important is that we should aim to avoid putting experimental stuff into Pygame.

I think it's a mistake Kivy made: tightly coupling excellent work in OpenGL ES bindings and deployment on mobile, with a crude UI framework and runtime.

I guess all I'm saying is that we should keep Pygame simple, and let other projects build the ecosystem on top of it.


On Sun, 19 Mar 2017, 13:27 Leif Theden, <[hidden email]> wrote:
My comment from the Reddit:

It has always been my feeling that pygame2 break compatibility with pygame1 and embrace modern computer features, namely the GPU. Shoehorning the clunky surface & blit API over sdl2 is a major regression. As is, no modern game library uses a system like this.

If it is moved to sdl2, then I would like to see it completely move to a modern rendering backend, without legacy features. Allow new programmers to try out shaders, experiment with opencl, maybe include a full featured SDL2 backend that you can drop into.

Also:

* first class gamepad support
* easier to understand sound API
* remove redundant modules like key, mouse, and joystick
* more focus on using the event queue
* more scaffolding, like a drop in event engine, easier setup for the screen, and game loop with less boilerplate cruft

At this point pygame1 is useful for quick prototyping and embedded systems because it can use a framebuffer. Beyond that? I'm not so sure that pygame can be relevant if it trys to support the old software API.

Even SDL2 dropped the software API, for the most part.

pygame_sdl2, guess what, it is trying to be source compatible, and despite being written in cython and compiled into native code, it is slower than pygame because of the legacy software renderer sdl2 doesn't optimize like the original did. To get pygame1 level performance with the legacy API, you will need to patch sdl2 and maintain a fork that contains optimized blitting code.

If you want to avoid a long transition and constant bickering about breaking changes, then you need to introduce a great feature to get people to switch. Implement a solid hardware-accelerated Sprite & Groups API and a "click to distribute" script in the std. lib and you will have people moving very quickly.

Also, drop python 2 support. Just my thoughts.
On Sun, Mar 19, 2017 at 8:04 AM René Dudfield <[hidden email]> wrote:

On Sun, Mar 19, 2017 at 8:57 AM, René Dudfield <[hidden email]> wrote:
Haha. "import pygumm" definitely does sound better. Seriously.

For *me* pypy is a non goal, and CFFI has some drawbacks. I know pypy also has plenty of upsides in certain cases... which has been mentioned a lot, so I won't repeat the good parts about it here.


This was my reasoning...
  • CFFI is slower on CPython. CPython is what 99% of people use.
  • People don't really contribute if something is in python which interfaces in C - and they only know python. This has been proven by other cffi and ctypes projects. Ctypes, and CFFI is a much rarer skill than standard platform tools. C still happens to be very widely known.
  • CFFI has issues on weird platforms... like iOS.
  • Cython, numba, numexpr are available. Numpy is not hard to install any more. These are faster than pypy in throughput, and latency.
  • Shaders in GLSL are pretty damn amazing if you need cross platform CPU/GPU acceleration.
  • pypy has jit warm up, jankiness because of jit and GC, and also doesn't work on weird platforms like iOS.
  • Converting everything to shared libraries, and supporting pypy is a whole lot more effort. However, in general I support sending things upstream and separating the C/asm code from the python code. So that it's easier for other people to use them, and there is less maintenance on pygame itself. However, packaging up shared libraries for multiple platforms is a lot of work. If those shared libraries are not already packaged for the 24 platforms, that actually creates maintenance burden as well.
  • pygame_cffi is slower on pypy than on CPython for many apps where the bottlenecks are already in the C/asm/platform code. Before the pypy project took funding, forked pygame, removed credits and copyright info, they were warned that blit was the bottleneck. So, like in a lot of things, it looks good in their benchmarks, but for real world things it doesn't perform so well - in many cases.
  • pypy uses significantly larger amounts of memory. Which is important for some older, or low memory systems.
  • pypy is slower than CPython where it matters for real time apps. Where pypy is faster, there are faster extensions available for CPython, and more appropriate for game JITs. Pypy has many performance cliffs, where it is really fast for some things after lots of warmup, but then for some magic un-obvious reason you fall into the valley of slow. The baseline pure python code performance is always faster in CPython when the jit can't work. Lower bound performance, and consistent performance turns out to be really important to real time apps. This means you can write a lot more code in python, without falling into the valley of slow.
Happy to take patches for pygame to help make it easier for anyone who wants to share code. Rect is mainly an SDL thing, with a whole bunch of niceties mostly at the python integration level. I'm not sure how to separate any of that really. There are pure python implementations of Rect, and perhaps you could use that to implement something in RPython for integration with pypy? But I think perhaps this runs into the case where integrating with C is slower in CFFI when there are lots of calls which take very little time. I'm sure the pypy CFFI people could help you there.

Again, there are definitely some good use cases for pypy, and CFFI. Even in games. Like in your game. But for me, and for the reasons above, I don't think it's a good idea for pygame. At the moment.

On Sun, Mar 19, 2017 at 6:32 AM, bw <[hidden email]> wrote:
Hi, thanks for inviting me to opine. :)

I like it. But "import pygumm" looks a lot more handsome than "import pygame2". Seriously.

Having had much recent experience with a SDL2 implementation (pypy, CFFI, plus all the SDL2 libs), I found that I would really like to see in the next generation:

1. CFFI wrapper. It is Python, which any of us can write, and it exposes all of the SDL2 API to programmers, not just what the C developers choose to expose. This would make pygame superbly extensible. And it would allow easy ad hoc inclusion of third-party prebuilt libs, something that is a fairly high bar to hurdle in pygame gen 1.

2. CFFI would also make pygame compatible with pypy, something which is sorely needed in my opinion. One can use optimization tricks to compensate for CPython's speed limit, but one cannot do many things that are computationally intensive without farming out to a third-party opaque library. pypy could let coders do a lot more in Python, when a full blown physics/geometry/numeric/scientific opaque lib is overkill.

3. I had acquired the problem of a slower Rect in pypy, however, because pygame has a pyd and I'm not spiffy enough to port it to anything but Python. If CFFI and pypy are an option worthy of consideration, Rect would be best done in a shared library. Rect is truly a workhorse, and pypy Rects can't compete with pygame Rects under heavy usage of Rect() and inflate and move, etc. In pygame most of my time is spent in blit(). In pypy most of my time is spent in Rect, because the class is 100% Python except for the four-slot CFFI data structure.

See what can be done with CFFI and pypy here:
http://pygame.org/project/2957
https://bitbucket.org/gummbum/gsdl2

I made a number of releases and games And a few drop-in tests "import gsdl2 as pygame", but it ain't perfect. However, I wasn't trying hard for software rendering compatibility, I was shooting for the new texture features. As expected, I discovered that all of the software-blit optimization tricks that I solved in gummworld2 were no longer needed.

Sounds like a few of you have already put a lot of thought and discussion into the next gen, and I understand this is something completely different. I certainly don't propose gsdl2 as a new code base. But it would be grand if choose to use CFFI. I have no stake in seeing my code transform into a butterfly, but if parts of it are useful then good.

Also, an acceptable alternative might be to provide CFFI-compatible shared libs so that people who want to use pypy have that choice, even if they do not have the nice pygame API available. If so, anybody could write a pygame-like API just like I did, and it doesn't need to be maintained by the pygame core developers.



On 3/18/2017 5:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Lenard Lindstrom
In reply to this post by René Dudfield
Hello,

Pygame relies heavily on the new buffer protocol to interact with
external modules like NumPy. The new buffer protocol is a CPython C api
level protocol for exposing array data. There is nothing comparable at
the Python level. In particular, the new buffer protocol allows for a
callback to call when a client releases a buffer. For a Pygame surface,
the callback unlocks the surface. The alternative to a callback is to
unlock the surface when the client object is garbage collected. This
will not work in PyPy, which lacks reference counting. So much of Pygame
needs to remain as extension modules to continue to use the new buffer
protocol, making CFFI less useful.

Lenard Lindstrom

On 17-03-18 10:32 PM, bw wrote:
> Hi, thanks for inviting me to opine. :)
[snip]
>
> Having had much recent experience with a SDL2 implementation (pypy,
> CFFI, plus all the SDL2 libs), I found that I would really like to see
> in the next generation:
>
>
[snip]
>
> 2. CFFI would also make pygame compatible with pypy, something which
> is sorely needed in my opinion. One can use optimization tricks to
> compensate for CPython's speed limit, but one cannot do many things
> that are computationally intensive without farming out to a
> third-party opaque library. pypy could let coders do a lot more in
> Python, when a full blown physics/geometry/numeric/scientific opaque
> lib is overkill.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Lenard Lindstrom
In reply to this post by René Dudfield
Hi René and everyone else,

I like this approach. My patches achieve the first step, using SDL2. The
next step is to introduce new SDL2 features as new modules and classes.

I suppose the SDL2 patches can be incorporated as conditionally compiled
code. They can be added one at a time, but must all be in place to work
properly. Then any new Pygame modules will expose SDL2 features only, so
not be built for SDL1.

As a long term goal SDL1 can be removed and SDL1 legacy modules
reimplemented on top of the SDL2 specific code.

Is there any reason new modules should not be written in Cython? If not
then we could try to use parts of pygame_sdl2 to quickly introduce SDL2
specific features.

Lenard Lindstrom


On 17-03-18 05:02 AM, René Dudfield wrote:

> Hello,
>
> so, I've spent some more time reading source code, and investigating
> SDL2 and the options a lot.
>
> I *think* this plan below could be the best way forward for us.
>
> tldr;
>  * have a single source SDL1, and SDL2 code base with a compile time
> option. (like we have single source py2 and py3).
>  * move bitbucket/pygame repo to github/pygame.
>
>
> Rather than reimplementing everything, and introducing lots of
> compatibility bugs, Instead I think a gradual approach would work
> better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not
> such a great way forward (for the pygame project, I think it was
> necessary and good for Ren'Py).
>
> A big breaking change is kind of silly for how little resources we
> have. Python 3 managed to pull it off... after a decade, and with
> massive resources having poured into it. And it only took off once
> they put compatibility code back in. Here's the thread where some
> discussion has been happening.
> https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition
>
>
> What we do have is some patches to get the current code base working
> with SDL2.
> https://bitbucket.org/llindstrom/pygame-1.10-patch
>
> I think it should be possible with some work to improve an SDL2
> compatibility layer, so that pygame code base can work with either (as
> a compile time option). Then newer APIs can be introduced in time, in
> a non break-every-program kind of way. Also, it's been proven that
> 'hardware' blitting does not need to break SDL1 API compatibility to
> use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other
> projects], ...).
>
> Having a pygame2, or whatever separate repo would also make
> maintenance harder. Since for the foreseeable future, we will likely
> need to do maintenance releases for this anyway (at least I want to!,
> and I know other users of pygame will).
>
> ---
>
> For pip uploads, they would continue to be for SDL1, until we can
> finish the SDL2 changes, and it works better. There would be no new
> additions until compatibility is mostly there.
>
> The work would progress by slowly adding in compatibility changes
> whilst keeping pygame working. By keeping the SDL2 patch set as is,
> and slowly reducing the patch set until it is size zero. So we always
> have a pygame with sdl2, and a pygame with sdl1 that is producible.
> Eventually the patch set will disappear.
>
> ---
>
> A pygame2 module would just cause confusion amongst users, and that
> really boring 'pygame 1 or pygame 2' type debate would go on, and on
> like the "python 2, verses python 3" one did. For me, just avoiding
> that discussion for the next decade would be worth it.
>
> Then there would need to be two sets of documentation on the website.
> And two sets of tutorials... and... we'd have 2 of everything to *do*,
> and 2 of everything for people to look at.
>
> Finally, "import pygame2" looks kind of weird. I've grown used to
> "import pygame".
>
> ...
>
>
>
> I'm keen to get moving on this. I've been trying to get feedback on
> the 'pygame sdl2 addition' issue for the last month, and the issue has
> been open since 2013. So I'd like to put a time limit on this decision
> for one more week.
>
> I'd really like to hear back from contributors, and users (everyone).
>
>
>
>
> ps. Interestingly SDL_gfx has an SDL2 release now.
> http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Leif Theden
I think that rewriting the extension modules in cython would be a great way to make the project accessible to new developers.

Writing extensions in C, as it is now, requires that developers have 1. Good C skills, 2. Understanding of the Python C library, 3. Knowledge of the SDL library, 4. Understanding of low-level details about the platforms it supports, and finally, 5. a good sense of how pygame works.

If you can imagine those skills charted as a Venn diagram, it's not difficult to understand why it is hard to maintain pygame as is, because only a handful of people have those skills and actually care.

Cython is not difficult to learn and looks like Python. It would be less of a barrier of entry for new developers to contribute.

Also, let's not forget that C in general is losing mindshare, as new developers tend to learn JS, java, and Python for their classes.

At that point however, Toms python_sdl2 is already doing that so, where does that leave pygame2?

If cython is chosen, better to let pygame 1.9 die gracefully and move on to a better platform and not duplicate effort (by moving to python_sdl2).

Good night, sweet prince, in that case.


On Sun, Mar 19, 2017 at 5:02 PM Lenard Lindstrom <[hidden email]> wrote:
Hi René and everyone else,

I like this approach. My patches achieve the first step, using SDL2. The
next step is to introduce new SDL2 features as new modules and classes.

I suppose the SDL2 patches can be incorporated as conditionally compiled
code. They can be added one at a time, but must all be in place to work
properly. Then any new Pygame modules will expose SDL2 features only, so
not be built for SDL1.

As a long term goal SDL1 can be removed and SDL1 legacy modules
reimplemented on top of the SDL2 specific code.

Is there any reason new modules should not be written in Cython? If not
then we could try to use parts of pygame_sdl2 to quickly introduce SDL2
specific features.

Lenard Lindstrom


On 17-03-18 05:02 AM, René Dudfield wrote:
> Hello,
>
> so, I've spent some more time reading source code, and investigating
> SDL2 and the options a lot.
>
> I *think* this plan below could be the best way forward for us.
>
> tldr;
>  * have a single source SDL1, and SDL2 code base with a compile time
> option. (like we have single source py2 and py3).
>  * move bitbucket/pygame repo to github/pygame.
>
>
> Rather than reimplementing everything, and introducing lots of
> compatibility bugs, Instead I think a gradual approach would work
> better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not
> such a great way forward (for the pygame project, I think it was
> necessary and good for Ren'Py).
>
> A big breaking change is kind of silly for how little resources we
> have. Python 3 managed to pull it off... after a decade, and with
> massive resources having poured into it. And it only took off once
> they put compatibility code back in. Here's the thread where some
> discussion has been happening.
> https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition
>
>
> What we do have is some patches to get the current code base working
> with SDL2.
> https://bitbucket.org/llindstrom/pygame-1.10-patch
>
> I think it should be possible with some work to improve an SDL2
> compatibility layer, so that pygame code base can work with either (as
> a compile time option). Then newer APIs can be introduced in time, in
> a non break-every-program kind of way. Also, it's been proven that
> 'hardware' blitting does not need to break SDL1 API compatibility to
> use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other
> projects], ...).
>
> Having a pygame2, or whatever separate repo would also make
> maintenance harder. Since for the foreseeable future, we will likely
> need to do maintenance releases for this anyway (at least I want to!,
> and I know other users of pygame will).
>
> ---
>
> For pip uploads, they would continue to be for SDL1, until we can
> finish the SDL2 changes, and it works better. There would be no new
> additions until compatibility is mostly there.
>
> The work would progress by slowly adding in compatibility changes
> whilst keeping pygame working. By keeping the SDL2 patch set as is,
> and slowly reducing the patch set until it is size zero. So we always
> have a pygame with sdl2, and a pygame with sdl1 that is producible.
> Eventually the patch set will disappear.
>
> ---
>
> A pygame2 module would just cause confusion amongst users, and that
> really boring 'pygame 1 or pygame 2' type debate would go on, and on
> like the "python 2, verses python 3" one did. For me, just avoiding
> that discussion for the next decade would be worth it.
>
> Then there would need to be two sets of documentation on the website.
> And two sets of tutorials... and... we'd have 2 of everything to *do*,
> and 2 of everything for people to look at.
>
> Finally, "import pygame2" looks kind of weird. I've grown used to
> "import pygame".
>
> ...
>
>
>
> I'm keen to get moving on this. I've been trying to get feedback on
> the 'pygame sdl2 addition' issue for the last month, and the issue has
> been open since 2013. So I'd like to put a time limit on this decision
> for one more week.
>
> I'd really like to hear back from contributors, and users (everyone).
>
>
>
>
> ps. Interestingly SDL_gfx has an SDL2 release now.
> http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

Tom Rothamel
I'll be honest and say I really haven't had any time to improve the features of pygame_sdl2 that aren't used by Ren'Py. It's hard enough to support one large project, and I just don't think I have another one in me at this time. I'd love to work with anyone who wants to bring pygame_sdl2 to parity with pygame, or to share/relicense any code that people want to move to other projects. (This could also include stuff distributed with Ren'Py at the moment, like Steam or IAP support.) 

Assuming that nobody wants to take this up as a project, I'll probably rename pygame_sdl2 to something else, so as not to overly confuse people on what it's about. 

I'd suggest there are three things a future pygame needs to focus on:

1) Mobile support - I think any game development platform that doesn't support mobile devices is just going to get increasingly marginalized. 

2) Providing access to modern graphics APIs, like OpenGL/OpenGL ES/Vulkan/DirectX - perhaps via something like ANGLE, and then supporting the rest of the stuff required to make a game around them - opening windows, loading images, audio, haptics, etc.

3) I would suspect that a non-backwards-compatible pygame would be poorly received if it keeps the same name and breaks the existing code. It would require any book mentioning pygame to be rewritten - and is it likely anyone would do that rather than try to teach people some other API/language?

But this is all backseat driving, since I really am kind of full up, schedule-wise. That being said, when evaluating SDL2 migration requirements, it's probably a good idea to first decide what the actual needs are. 




On Sun, Mar 19, 2017 at 6:46 PM Leif Theden <[hidden email]> wrote:
I think that rewriting the extension modules in cython would be a great way to make the project accessible to new developers.

Writing extensions in C, as it is now, requires that developers have 1. Good C skills, 2. Understanding of the Python C library, 3. Knowledge of the SDL library, 4. Understanding of low-level details about the platforms it supports, and finally, 5. a good sense of how pygame works.

If you can imagine those skills charted as a Venn diagram, it's not difficult to understand why it is hard to maintain pygame as is, because only a handful of people have those skills and actually care.

Cython is not difficult to learn and looks like Python. It would be less of a barrier of entry for new developers to contribute.

Also, let's not forget that C in general is losing mindshare, as new developers tend to learn JS, java, and Python for their classes.

At that point however, Toms python_sdl2 is already doing that so, where does that leave pygame2?

If cython is chosen, better to let pygame 1.9 die gracefully and move on to a better platform and not duplicate effort (by moving to python_sdl2).

Good night, sweet prince, in that case.


On Sun, Mar 19, 2017 at 5:02 PM Lenard Lindstrom <[hidden email]> wrote:
Hi René and everyone else,

I like this approach. My patches achieve the first step, using SDL2. The
next step is to introduce new SDL2 features as new modules and classes.

I suppose the SDL2 patches can be incorporated as conditionally compiled
code. They can be added one at a time, but must all be in place to work
properly. Then any new Pygame modules will expose SDL2 features only, so
not be built for SDL1.

As a long term goal SDL1 can be removed and SDL1 legacy modules
reimplemented on top of the SDL2 specific code.

Is there any reason new modules should not be written in Cython? If not
then we could try to use parts of pygame_sdl2 to quickly introduce SDL2
specific features.

Lenard Lindstrom


On 17-03-18 05:02 AM, René Dudfield wrote:
> Hello,
>
> so, I've spent some more time reading source code, and investigating
> SDL2 and the options a lot.
>
> I *think* this plan below could be the best way forward for us.
>
> tldr;
>  * have a single source SDL1, and SDL2 code base with a compile time
> option. (like we have single source py2 and py3).
>  * move bitbucket/pygame repo to github/pygame.
>
>
> Rather than reimplementing everything, and introducing lots of
> compatibility bugs, Instead I think a gradual approach would work
> better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not
> such a great way forward (for the pygame project, I think it was
> necessary and good for Ren'Py).
>
> A big breaking change is kind of silly for how little resources we
> have. Python 3 managed to pull it off... after a decade, and with
> massive resources having poured into it. And it only took off once
> they put compatibility code back in. Here's the thread where some
> discussion has been happening.
> https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition
>
>
> What we do have is some patches to get the current code base working
> with SDL2.
> https://bitbucket.org/llindstrom/pygame-1.10-patch
>
> I think it should be possible with some work to improve an SDL2
> compatibility layer, so that pygame code base can work with either (as
> a compile time option). Then newer APIs can be introduced in time, in
> a non break-every-program kind of way. Also, it's been proven that
> 'hardware' blitting does not need to break SDL1 API compatibility to
> use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other
> projects], ...).
>
> Having a pygame2, or whatever separate repo would also make
> maintenance harder. Since for the foreseeable future, we will likely
> need to do maintenance releases for this anyway (at least I want to!,
> and I know other users of pygame will).
>
> ---
>
> For pip uploads, they would continue to be for SDL1, until we can
> finish the SDL2 changes, and it works better. There would be no new
> additions until compatibility is mostly there.
>
> The work would progress by slowly adding in compatibility changes
> whilst keeping pygame working. By keeping the SDL2 patch set as is,
> and slowly reducing the patch set until it is size zero. So we always
> have a pygame with sdl2, and a pygame with sdl1 that is producible.
> Eventually the patch set will disappear.
>
> ---
>
> A pygame2 module would just cause confusion amongst users, and that
> really boring 'pygame 1 or pygame 2' type debate would go on, and on
> like the "python 2, verses python 3" one did. For me, just avoiding
> that discussion for the next decade would be worth it.
>
> Then there would need to be two sets of documentation on the website.
> And two sets of tutorials... and... we'd have 2 of everything to *do*,
> and 2 of everything for people to look at.
>
> Finally, "import pygame2" looks kind of weird. I've grown used to
> "import pygame".
>
> ...
>
>
>
> I'm keen to get moving on this. I've been trying to get feedback on
> the 'pygame sdl2 addition' issue for the last month, and the issue has
> been open since 2013. So I'd like to put a time limit on this decision
> for one more week.
>
> I'd really like to hear back from contributors, and users (everyone).
>
>
>
>
> ps. Interestingly SDL_gfx has an SDL2 release now.
> http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

René Dudfield
In reply to this post by Lenard Lindstrom
Yeah, conditional compilation, compatible functions, macros and opaque types. Which ever is nicer for each part.

Cython is fine for new things, IMHO.

cheerio,



On Sunday, March 19, 2017, Lenard Lindstrom <[hidden email]> wrote:
Hi René and everyone else,

I like this approach. My patches achieve the first step, using SDL2. The next step is to introduce new SDL2 features as new modules and classes.

I suppose the SDL2 patches can be incorporated as conditionally compiled code. They can be added one at a time, but must all be in place to work properly. Then any new Pygame modules will expose SDL2 features only, so not be built for SDL1.

As a long term goal SDL1 can be removed and SDL1 legacy modules reimplemented on top of the SDL2 specific code.

Is there any reason new modules should not be written in Cython? If not then we could try to use parts of pygame_sdl2 to quickly introduce SDL2 specific features.

Lenard Lindstrom


On 17-03-18 05:02 AM, René Dudfield wrote:
Hello,

so, I've spent some more time reading source code, and investigating SDL2 and the options a lot.

I *think* this plan below could be the best way forward for us.

tldr;
 * have a single source SDL1, and SDL2 code base with a compile time option. (like we have single source py2 and py3).
 * move bitbucket/pygame repo to github/pygame.


Rather than reimplementing everything, and introducing lots of compatibility bugs, Instead I think a gradual approach would work better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not such a great way forward (for the pygame project, I think it was necessary and good for Ren'Py).

A big breaking change is kind of silly for how little resources we have. Python 3 managed to pull it off... after a decade, and with massive resources having poured into it. And it only took off once they put compatibility code back in. Here's the thread where some discussion has been happening.
https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition


What we do have is some patches to get the current code base working with SDL2.
https://bitbucket.org/llindstrom/pygame-1.10-patch

I think it should be possible with some work to improve an SDL2 compatibility layer, so that pygame code base can work with either (as a compile time option). Then newer APIs can be introduced in time, in a non break-every-program kind of way. Also, it's been proven that 'hardware' blitting does not need to break SDL1 API compatibility to use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other projects], ...).

Having a pygame2, or whatever separate repo would also make maintenance harder. Since for the foreseeable future, we will likely need to do maintenance releases for this anyway (at least I want to!, and I know other users of pygame will).

---

For pip uploads, they would continue to be for SDL1, until we can finish the SDL2 changes, and it works better. There would be no new additions until compatibility is mostly there.

The work would progress by slowly adding in compatibility changes whilst keeping pygame working. By keeping the SDL2 patch set as is, and slowly reducing the patch set until it is size zero. So we always have a pygame with sdl2, and a pygame with sdl1 that is producible. Eventually the patch set will disappear.

---

A pygame2 module would just cause confusion amongst users, and that really boring 'pygame 1 or pygame 2' type debate would go on, and on like the "python 2, verses python 3" one did. For me, just avoiding that discussion for the next decade would be worth it.

Then there would need to be two sets of documentation on the website. And two sets of tutorials... and... we'd have 2 of everything to *do*, and 2 of everything for people to look at.

Finally, "import pygame2" looks kind of weird. I've grown used to "import pygame".

...



I'm keen to get moving on this. I've been trying to get feedback on the 'pygame sdl2 addition' issue for the last month, and the issue has been open since 2013. So I'd like to put a time limit on this decision for one more week.

I'd really like to hear back from contributors, and users (everyone).




ps. Interestingly SDL_gfx has an SDL2 release now. http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

René Dudfield
In reply to this post by Tom Rothamel
Hi Tom,

Is there any reason you couldn't use an SDL2 using pygame?

I assumed you would be able to.

best,

On Monday, March 20, 2017, Tom Rothamel <[hidden email]> wrote:
I'll be honest and say I really haven't had any time to improve the features of pygame_sdl2 that aren't used by Ren'Py. It's hard enough to support one large project, and I just don't think I have another one in me at this time. I'd love to work with anyone who wants to bring pygame_sdl2 to parity with pygame, or to share/relicense any code that people want to move to other projects. (This could also include stuff distributed with Ren'Py at the moment, like Steam or IAP support.) 

Assuming that nobody wants to take this up as a project, I'll probably rename pygame_sdl2 to something else, so as not to overly confuse people on what it's about. 

I'd suggest there are three things a future pygame needs to focus on:

1) Mobile support - I think any game development platform that doesn't support mobile devices is just going to get increasingly marginalized. 

2) Providing access to modern graphics APIs, like OpenGL/OpenGL ES/Vulkan/DirectX - perhaps via something like ANGLE, and then supporting the rest of the stuff required to make a game around them - opening windows, loading images, audio, haptics, etc.

3) I would suspect that a non-backwards-compatible pygame would be poorly received if it keeps the same name and breaks the existing code. It would require any book mentioning pygame to be rewritten - and is it likely anyone would do that rather than try to teach people some other API/language?

But this is all backseat driving, since I really am kind of full up, schedule-wise. That being said, when evaluating SDL2 migration requirements, it's probably a good idea to first decide what the actual needs are. 




On Sun, Mar 19, 2017 at 6:46 PM Leif Theden <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;leif.theden@gmail.com&#39;);" target="_blank">leif.theden@...> wrote:
I think that rewriting the extension modules in cython would be a great way to make the project accessible to new developers.

Writing extensions in C, as it is now, requires that developers have 1. Good C skills, 2. Understanding of the Python C library, 3. Knowledge of the SDL library, 4. Understanding of low-level details about the platforms it supports, and finally, 5. a good sense of how pygame works.

If you can imagine those skills charted as a Venn diagram, it's not difficult to understand why it is hard to maintain pygame as is, because only a handful of people have those skills and actually care.

Cython is not difficult to learn and looks like Python. It would be less of a barrier of entry for new developers to contribute.

Also, let's not forget that C in general is losing mindshare, as new developers tend to learn JS, java, and Python for their classes.

At that point however, Toms python_sdl2 is already doing that so, where does that leave pygame2?

If cython is chosen, better to let pygame 1.9 die gracefully and move on to a better platform and not duplicate effort (by moving to python_sdl2).

Good night, sweet prince, in that case.


On Sun, Mar 19, 2017 at 5:02 PM Lenard Lindstrom <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;len-l@telus.net&#39;);" target="_blank">len-l@...> wrote:
Hi René and everyone else,

I like this approach. My patches achieve the first step, using SDL2. The
next step is to introduce new SDL2 features as new modules and classes.

I suppose the SDL2 patches can be incorporated as conditionally compiled
code. They can be added one at a time, but must all be in place to work
properly. Then any new Pygame modules will expose SDL2 features only, so
not be built for SDL1.

As a long term goal SDL1 can be removed and SDL1 legacy modules
reimplemented on top of the SDL2 specific code.

Is there any reason new modules should not be written in Cython? If not
then we could try to use parts of pygame_sdl2 to quickly introduce SDL2
specific features.

Lenard Lindstrom


On 17-03-18 05:02 AM, René Dudfield wrote:
> Hello,
>
> so, I've spent some more time reading source code, and investigating
> SDL2 and the options a lot.
>
> I *think* this plan below could be the best way forward for us.
>
> tldr;
>  * have a single source SDL1, and SDL2 code base with a compile time
> option. (like we have single source py2 and py3).
>  * move bitbucket/pygame repo to github/pygame.
>
>
> Rather than reimplementing everything, and introducing lots of
> compatibility bugs, Instead I think a gradual approach would work
> better. I'm starting to think that perhaps pygame_sdl2 by Tom, is not
> such a great way forward (for the pygame project, I think it was
> necessary and good for Ren'Py).
>
> A big breaking change is kind of silly for how little resources we
> have. Python 3 managed to pull it off... after a decade, and with
> massive resources having poured into it. And it only took off once
> they put compatibility code back in. Here's the thread where some
> discussion has been happening.
> https://bitbucket.org/pygame/pygame/issues/174/pygame-20-the-sdl2-edition
>
>
> What we do have is some patches to get the current code base working
> with SDL2.
> https://bitbucket.org/llindstrom/pygame-1.10-patch
>
> I think it should be possible with some work to improve an SDL2
> compatibility layer, so that pygame code base can work with either (as
> a compile time option). Then newer APIs can be introduced in time, in
> a non break-every-program kind of way. Also, it's been proven that
> 'hardware' blitting does not need to break SDL1 API compatibility to
> use hardware accelerated APIs (Angle, SDL_gpu, [insert 4 or 5 other
> projects], ...).
>
> Having a pygame2, or whatever separate repo would also make
> maintenance harder. Since for the foreseeable future, we will likely
> need to do maintenance releases for this anyway (at least I want to!,
> and I know other users of pygame will).
>
> ---
>
> For pip uploads, they would continue to be for SDL1, until we can
> finish the SDL2 changes, and it works better. There would be no new
> additions until compatibility is mostly there.
>
> The work would progress by slowly adding in compatibility changes
> whilst keeping pygame working. By keeping the SDL2 patch set as is,
> and slowly reducing the patch set until it is size zero. So we always
> have a pygame with sdl2, and a pygame with sdl1 that is producible.
> Eventually the patch set will disappear.
>
> ---
>
> A pygame2 module would just cause confusion amongst users, and that
> really boring 'pygame 1 or pygame 2' type debate would go on, and on
> like the "python 2, verses python 3" one did. For me, just avoiding
> that discussion for the next decade would be worth it.
>
> Then there would need to be two sets of documentation on the website.
> And two sets of tutorials... and... we'd have 2 of everything to *do*,
> and 2 of everything for people to look at.
>
> Finally, "import pygame2" looks kind of weird. I've grown used to
> "import pygame".
>
> ...
>
>
>
> I'm keen to get moving on this. I've been trying to get feedback on
> the 'pygame sdl2 addition' issue for the last month, and the issue has
> been open since 2013. So I'd like to put a time limit on this decision
> for one more week.
>
> I'd really like to hear back from contributors, and users (everyone).
>
>
>
>
> ps. Interestingly SDL_gfx has an SDL2 release now.
> http://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx/
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pygame with SDL2 proposal

René Dudfield
In reply to this post by Tom Rothamel
"it's probably a good idea to first decide what the actual needs are"​

Some ideas on that...
  • SDL2 provides better platform support in some places, and some nicer features.
  • SDL1 provides better platform support in some places.
  • pygame apps would like to continue working.
  • ...
Things like mobile support, I agree are important, but I guess getting the basics done first is more important. Everything else can flow from there.

Currently we don't even have a good story for distribution on pypi, windows, linux, or mac (The platforms where people make things on). I think we should keep our goals humble, and achievable. Complete them, and move on to the next steps.

Daniel Pope had a good point about not putting in any experimental features, and to keep it simple. We've learned that lesson, and have removed things already that were not maintainable. It's the approach that CPython now takes as well - things need to gain popularity and prove stability before they are moved in. Additionally now it's much easier to use external packages via pip, so bundling all together is not as useful as it once was (binary distribution is still challenging however).

The app distribution work(so people can distribute their own games easily), I think should continue with the wider python community. A tool I'm working on has had massive contributions from someone who doesn't use pygame, or game tools at all. I think that Thomas Kluver probably has the same experiences with his packaging work. Trying to get things fixed in the python distribution tools themselves has proved challenging, but has also proved to sometimes work.

With this proposal, we should be able to work on other things like mobile, and extended graphics support afterwards. I agree that they are important. There's also nothing stopping people from working on them separately(or raising resources to fund that work).
1234
Loading...