Rendered at 20:50:25 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gomoboo 16 hours ago [-]
It always bugs me to think about the Python community not rallying around PyPy. Faster Python is right there and has been there for years. Sure there can be C-extension issues but projects like HPy showed you can remedy that. Why doesn’t everyone just use and support PyPy?
famouswaffles 13 hours ago [-]
"C-extension issues" are not something you can just gloss over. They're a huge chunk of python usage. Users won't rally around that. And Hpy is dead. If CPython embraced PuPy, worked with it to smooth those issues then maybe things would be different.
pjmlp 12 hours ago [-]
To the point that in Python culture those C extensions are seen as Python libraries, not bindings.
theandrewbailey 6 hours ago [-]
Maybe PyPy needs a Steve Jobs-like figure to write a "no C extension" manifesto, declaring that all Python apps must be 100% Python.
bastawhiz 6 hours ago [-]
Is there a single example of a language+runtime where this approach has ever been successful
pjmlp 4 hours ago [-]
Well, see popularity of CGO among Go devs, or JNI among Java devs, or the ongoing work to make all the learnings from System C# (from Midori) available in regular C#, while C++/CLI is slowly left in maintenance mode (last updated for partial C++20 support, and not part of FOSS .NET)
vova_hn2 2 hours ago [-]
A lot of Python code is just a relatively thin layer of "glue" between C libraries.
An ML pipeline will probably rely on numpy to do all the heavy lifting.
A web backend will probably use something like psycopg, which is a wrapper around libpq (official postgres client lib, written in C).
etc
bastawhiz 6 hours ago [-]
PyPy makes tradeoffs that might not be acceptable. For anything that isn't a long running server, PyPy can be much, much slower since the JIT doesn't have a chance to warm up. PyPy also implements things differently: repeated string concatenation in a loop is quadratic instead of linear like in CPython. There's also memory tradeoffs, depending on what you're doing.
The list of differences is quite long, and even though most of them are edge cases, it's not hard to get bitten by one on a project of any substantial size:
It's not that people are writing weird code, it's that you have a package that does something clever but safe, and then another package using it that calls it in a way that's unexpected but not documented as unsafe, and layers and layers of that compound until you have things like exceptions raised because non-string keys are getting set on type objects.
At the end of the day, Python isn't a terribly well-specified language. CPython behavior is functionally the specification. "Make it match CPython but faster" isn't really an option because it kills your ability to do a lot of the things that make PyPy fast. And if that was all it took, we wouldn't still be arguing over a JIT inside CPython. Pyston was an honest attempt at doing that, but it has been dead for years partially because it hit many of these walls.
vova_hn2 2 hours ago [-]
> Pyston was an honest attempt at doing that
I'm still kinda sad that Pyston didn't happen
> but it has been dead for years partially because it hit many of these walls
Maybe this is one of the reasons. But I've also heard that Dropbox (which was making Pyston [0]), that used to heavily rely on Python, just decided that gradual migration to Go [1] is a more feasible approach than creating a faster Python implementation.
An ML pipeline will probably rely on numpy to do all the heavy lifting.
A web backend will probably use something like psycopg, which is a wrapper around libpq (official postgres client lib, written in C).
etc
The list of differences is quite long, and even though most of them are edge cases, it's not hard to get bitten by one on a project of any substantial size:
https://doc.pypy.org/cpython_differences.html
It's not that people are writing weird code, it's that you have a package that does something clever but safe, and then another package using it that calls it in a way that's unexpected but not documented as unsafe, and layers and layers of that compound until you have things like exceptions raised because non-string keys are getting set on type objects.
At the end of the day, Python isn't a terribly well-specified language. CPython behavior is functionally the specification. "Make it match CPython but faster" isn't really an option because it kills your ability to do a lot of the things that make PyPy fast. And if that was all it took, we wouldn't still be arguing over a JIT inside CPython. Pyston was an honest attempt at doing that, but it has been dead for years partially because it hit many of these walls.
I'm still kinda sad that Pyston didn't happen
> but it has been dead for years partially because it hit many of these walls
Maybe this is one of the reasons. But I've also heard that Dropbox (which was making Pyston [0]), that used to heavily rely on Python, just decided that gradual migration to Go [1] is a more feasible approach than creating a faster Python implementation.
[0] https://dropbox.tech/infrastructure/introducing-pyston-an-up...
[1] "About a year ago, we decided to migrate our performance-critical backends from Python to Go to leverage better concurrency support and faster execution speed." - https://dropbox.tech/infrastructure/open-sourcing-our-go-lib...