| Summary: | Low performance with python3 compare to python2 on the same platform | ||
|---|---|---|---|
| Product: | buildroot | Reporter: | fabien.nogue |
| Component: | Other | Assignee: | unassigned |
| Status: | RESOLVED MOVED | ||
| Severity: | normal | CC: | buildroot, fabien.nogue, yann.morin.1998 |
| Priority: | P5 | ||
| Version: | 2021.11.3 | ||
| Target Milestone: | --- | ||
| Hardware: | Other | ||
| OS: | Linux | ||
| Host: | Target: | ||
| Build: | |||
| Attachments: | configure cmd of python2 and python3 | ||
What's the output of `python -m cProfile test_python.py 10000000` on each? I do see some variances even in Docker containers but not to that degree. On an rpi CM3 it doesn't take anywhere near the numbers you're posting below (only 8s). One option would be to profile the python interpreter itself. 3.12 has hooks for `perf` but they may not be representative of the situation you're seeing as recent interpreters try to perform opcode specialization. Otherwise you may have to build the interpreter with profiling enabled and use another external tool to collect that data. (In reply to Vincent Fazio from comment #1) Thank you for your reply. Here are the outputs of our test with the `cProfile` module: ``` # python2 -m cProfile test_python.py 10000000 2 function calls in 18.871 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 18.871 18.871 18.871 18.871 test_python.py:3(<module>) 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} # python3 -m cProfile test_python.py 10000000 3 function calls in 53.225 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 53.224 53.224 53.224 53.224 test_python.py:3(<module>) 1 0.000 0.000 53.225 53.225 {built-in method builtins.exec} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} ``` Is the line `built-in method builtins.exec` relevant? On the other hand, I am attempting to profile the `cpython` interpreter. I recompiled `python3` with options `--enable-optimizations` and `--enable-profiling`, run my test and got the `gmon.out`. I have `no time accumulated` when I run `gprof` on the output. So far I have no idea why the profiling output has no result. I need to investigate further. Thank you for your report.
The issue tracker for the Buildroot project has been moved to
the Gitlab.com issue tracker:
https://gitlab.com/buildroot.org/buildroot/-/issues
We are taking this opportunity to close old issues in this old
tracker. If you believe your issue is still relevant, please
open one in the new issue tracker.
Thank you!
|
Created attachment 9649 [details] configure cmd of python2 and python3 Hello, We are passing from python 2.7.18 to python 3.9.7 on buildroot 2021.11 and we have a big drop on the performance. On our platform (arm-cortexA8), python3 is 2.5 times slower than python2 Our searches on the internet never spoke about a loss of performance as big as this one. So we are thinking we have a problem about our python and the way it is compiled, this is why we post here. We create an image with both, python2 and python3 on it, no other software is running on background, and run the following test: ``` # cat test_python.py #!/usr/bin/python import sys n = int(sys.argv[1]) while n: n -= 1 # python2 --version Python 2.7.18 # time python2 ./test_python.py 10000000 real 0m 21.23s user 0m 20.73s sys 0m 0.48s # python3 --version Python 3.9.7 # time python3 ./test_python.py 10000000 real 0m 51.39s user 0m 51.34s sys 0m 0.02s ``` Our `./configure` are almost identic. And flags used during compile time as well. We tried python 3.11.5 on buildroot 2023.08.3 and performance are as bad as 3.9.7. Do you have any ideas about where we can lead our research? Thank you.