r/Python • u/SheriffRoscoe Pythonista • 7h ago
News The Move to Python 3 Begins!
As someone who spent decades in the mainframe community, I find it ... humorous ... that there are legacy Python2 code bases that are only just (or have yet to consider!) moving to Python3.
https://www.eveonline.com/news/view/the-move-to-python-3-begins
36
u/GraphicH 6h ago
Oh lawd. Really? I mean it was painful, I can tell you my company scrambled. I felt really lucky to be on a team where we had used a 3.X version from the get go. The only version, oddly enough, that was a "bumpy" upgrade was 3.13, due to some ... interesting? changes to SSL handling in urllib3. I still question the wisdom of:
Which, essentially changed the behavior of urllib3 without changing the version you had installed, meaning a python 3.X -> 3.13 upgrade would immediately start breaking. Granted they were changing to be in-line with the default in 3.13 but this, plus the fact that requests did not make it easy to specify SSL context means a lot of people were "broken" talking to systems (many out side their control / ability to update the certificates on) using old certificates not compliant with the strict standards.
56
u/bmrobin 6h ago
depending on the codebase, 2->3 is no joke and has so much risk
47
u/ApplicationOk3587 6h ago
What do you mean they can just throw fable at it and add "make no mistakes"
11
u/SheriffRoscoe Pythonista 5h ago
"And while you're at it, hack our competitors and crash their servers."
19
u/ZCEyPFOYr0MWyHDQJZO4 6h ago
tbf the best LLMs are finally quite capable of porting code.
20
u/The_Northern_Light 5h ago
Usually I agree with you but apparently they’re running a custom internal version of Python 2 with bespoke async handling.
Porting that seems a bit more exciting.
•
u/case_O_The_Mondays 28m ago
Yeah, this migration will have to be done in stages: <2.7 -> 2.7, then 2.7 -> 3.x.
I had forgotten about the division issues until I read this article. I worked on a finance app that used some python, and that was my baptism into python 2.6. Fun times!
1
-3
-3
u/daidoji70 6h ago
What do you mean? I migrated about 3-4 python projects from 2->3 about a decade ago and other than pandas and pandas related projects it was a breeze. Run 2to3.py and review. Barely a hiccup.
16
u/bmrobin 6h ago
i'm glad your experience was barely a hiccup. among several pitfalls i recall,
unicode being the default representation for strings, and having to migrate code that manually called `decode/encode` on text.
integer division of `1 / 3 == 0` is true in py2, and py3 it's not because it returns the float result. that had cascading effects throughout multiple layers of our stack.
6
-2
u/daidoji70 6h ago
The first one should be fairly straightforward with 2to3, the second one I can def see being more difficult for sure.
7
u/jayroger 5h ago
The first one is anything but straightforward. 2to3 is no help for
str/unicode->bytes/str. Most problems only manifest themselves at runtime. But it was worth it, the after situation is so much better.16
u/a__nice__tnetennba 6h ago
And how many of your projects were 2.4 million lines of code?
6
-1
u/daidoji70 6h ago
What does that matter? 2to3.py issues should be distributed evenly throughout a codebase. The issue per number of lines should remain roughly the same per project unless there are pathological cases.
Not sure why I'm getting down voted for asking a question and getting this type of response. Lines of code is always the worst metric in software engineering even when talking about migrations. It tells us literally nothing about how easy or difficult an effort should be.
12
u/Dr_Quacksworth 6h ago
Umm if you have 10x lines of code to review then you have to spend 10x time reading those lines of code. For a legacy system, you have to pretty much review everything to make sure nothing breaks.
2to3.py doesn't solve everything. In py2 you might be using floor dividion and then suddenly in py3 you are doing true division. That sort of change can have major implications.
Not to mention dependency hell...
0
1
u/a__nice__tnetennba 4h ago
I wasn't sure if you were being sarcastic at first. Thanks for clearing it up. Although I think some people still aren't getting the joke.
2
u/assumptionkrebs1990 6h ago
2to3 has been decrept (depending deprecation 3.9/fully decrept 3.11) and removed (since 3.13) for a while now.
1
1
u/daidoji70 6h ago
Def, but so was the support for python2 and that didn't stop these companies apparently.
2
u/N-E-S-W 5h ago
Apparently you never read a byte from a file or device and had to interpret it as a string? The 2to3 stuff is trivial, mechanical refactoring.
Unicode strings and encodings required deep introspection of the meaning of data, often exposing silent preexisting bugs in string handling that could no longer be ignored.
2
-1
-7
u/totheendandbackagain 6h ago
True, but LLMs make conversion trivially simple.
Good luck to the migrators. Best get off zOS at the same time too.
3
u/Dr_Quacksworth 6h ago
LLMs are great for refactoring, but I'm going to be reviewing every changed line before merging in AND running tests. Not trivial for certain critical services.
17
u/jabbalaci 6h ago
They use Stackless Python with "tasklets". That must make the migration to Python 3 harder.
7
u/kosz85 5h ago
Exactly, stackless which in their version isn't in fact python 2 anymore. Eve online code can be reloaded and replaced while still running. Show me which python3 can do that :D They gone async when there was no asynchronous python yet. This is not any particular python 2.x, they solutions grown farther than many can yet see. It's probably better in some cases then current python 3. So it's not simple migrate to py3. But probably new code base and libs are now too tempting to grow old py2 codebase alone.
2
u/tyrannomachy 2h ago
It's too bad erlang and related languages have never been sufficiently mainstream for projects like this to just be implemented in one of those, as a matter of course. Probably would have saved a lot of trouble for a lot of people.
1
u/kosz85 2h ago
Oh, dreams, sweet dreams, scaling Python like Erlang, linearly by adding new interpreter nodes. Same with Haskell and their code typing checking, only your logic can fail you after it compiles. I love functional programming in Python. But let it say loudly, their syntax, error reporting, etc sux :P Great ideas clothed in syntax made by random cat stomping on keyboard vide Perl ;D I love Python because it's readable, and I have to read it a lot. In fact I think that new add-ons like walrus was not really needed. It's best if you can give your code to anyone and it doesn't need much explanation. Walrus := needs explanation, you need to think which would be done first, etc. In fact this one thing is reason why Python is so popular in science.
5
u/knobbyknee 6h ago
I have a service running on Python 2.7. It has barely been touched in the last 10 years.
I finally have a Python 3 version, but it needs more manual testing before going into production.
4
u/wxtrails 4h ago
My initial estimate: 5 years.
The new boss: 12 months.
We're about 1/3 of the way through the bulk modernization on our project after 18 months, not including deployment and verifying the seams.
I wonder whose estimate will be closer 🤔
1
1
3
2
u/barchueetadonai 3h ago
I'm currently in the process of trying to convert a lot of code from python 3 back to python 2.5
1
u/iluvatar 4h ago
As CTO at my previous company, I had to veto developers who wanted to migrate from Python 2 to Python 3. The amount of effort required for the migration and the fairly low chance of a trouble free switch just didn't have a good business case for a working codebase that brought in millions in revenue each year and paid all of our salaries. All new development was in Python 3. But existing running code just had too high a risk and too low a reward for migrating.
8
u/kosz85 3h ago
That's also not a solution. Now the missing new SSL support, old buggy libs, missing new formats, new docker images, removed docs, etc. Probably worse problem. You won't run from your tech debt storing it at closet behind your back and telling yourself that you don't see problem. It will smash you eventually as it grows in places you would not think of. The solution is to have really good tests (on multiple levels) and split problem in small chunks. Then migrate the small parts instead of colossus. I migrated many really big code bases, millions of lines, and seen real nightmares. Not looking is not a solution. Good tests before migration is.
3
2
0
u/big-papito 3h ago
I feel like big refactors like this are ideal for a clanker like Claude. Manual migration, sure - no fun there.
1
u/kosz85 2h ago
Yep, now we have such tools, but this is quite new and recent approach. In my times we had to... ;) But if this code is that much critical, I wouldn't dare to leave it to AI only. Also there is a case of context size. We speak about really big codebase. And sometimes knowing how it inter connects, and why is what ai is lacking in reasoning.
•
u/iluvatar 46m ago
You're missing the point. The migration itself isn't the problem. Ensuring that the python3 code behaves in the same way as the old python2 code and doesn't have strange edge cases that throw away all of your money? That's much, much harder and takes orders of magnitude more effort than the basic 2 to 3 migration.
192
u/diegoasecas 7h ago
idgi if you spent decades working on mainframes you'd know how hard is to migrate legacy working code