No-GIL Python is a mistake
In a quest to improve performance and bring cross-core threading to the CPython runtime, the Python community has approved Python Enhancement Proposal (PEP) 703, which will remove the infamous Global Interpreter Lock. However, GIL removal presents some serious challenges for Python users.
Why no-GIL Python is a mistake
Members of the community who question the wisdom of GIL removal often cite the following concerns:
- Slower single-threaded execution.
- The need to support both GIL and non-GIL distributions.
- Incompatibility with non-Python functions.
- Program-freezing stop-the-world events.
- An exodus of users to a different language, particularly Mojo.
Slower Python apps
Compiler architects don't have to worry about deadlocks when they only allow one thread to run at a time. Data contention isn't an issue.
However, in a multi-threaded, multi-core environment, many key language features must include clock-cycle consuming locks. By adding such locks, a no-GIL Python might degrade performance for single-threaded applications.
Even the PEP 703 proposal itself admits the problem: "The changes proposed in the PEP will increase execution overhead for --disable-gil builds compared to Python builds with the GIL. In other words, it will have slower single-threaded performance."
Multi-version support
As early-adopters move to the no-GIL Python, maintainers of the most popular Python libraries will be expected to do the same. However, established users of those libraries will need continued support for GIL-based runtimes. That puts library providers and open-source maintainers in the unenviable position of supporting two distributions at the same time.
Incompatible external functions
Python's true strength is its ability to call functions and procedures in other languages, such as Java and C++, that are faster, more efficient and fully capable of threading across cores. However, all external functions currently in use expect to be invoked in a safe, single-threaded manner. If a no-GIL Python is to work, all those external systems must be updated as well.
Stop-the-world events
Memory management isn't easy when it involves multiple threads. To support garbage collection (GC), the no-GIL Python requires two stop-the-world (STW) events to reclaim memory.
During STW events, a program completely freezes. Are Python users ready for 15-second STW pauses as the GC algorithm cleans things up?
Mass user exodus
The transition to the no-GIL Python is fraught with uncertainty. Platform instability issues are expected, and users will need to rewrite working code to deal with no-GIL changes.
Meanwhile, Python-compatible runtimes, such as Mojo, already provide the benefits of a no-GIL Python along with additional enhancements that make them vastly superior languages to use. Expect a mass exodus to Mojo as the Python community tries to remove the GIL.
It won't work
The other big problem with no-GIL Python? It probably won't work.
Take the word of Python creator Guido van Rossum himself, who said in 2007, "This has been tried before, with disappointing results, which is why I'm reluctant to put much effort into it myself." He echoed those 17-year-old sentiments again in a 2023 Lex Fridman podcast episode. Even the experts are unsure if removing Python's GIL will work.
While I wish them luck in the endeavor, the pursuit of a no-GIL Python runtime seems to me like a giant mistake.
Darcy DeClute is a technical trainer and Agile coach who helps organizations apply Scrum-based principles to adopt a modern DevOps stack. She is a certified Professional Scrum Master, Professional Scrum Developer and Professional Scrum Product Owner as well as author of Scrum Master Certification Guide.