Benchmark Cinder vs. Cpython performance; improve pipeline and performance of the Cinder build #72
Comments
rtindru
changed the title
|
thanks for working on this @rtindru ! another potentially useful resource for building python might be the the "official docker build" - check out https://github.com/docker-library/python/blob/master/3.8/bullseye/Dockerfile#L44-L52 for example |
|
@kmod could you share the details of the benchmarks you ran? |
|
Steps to run pyperformance benchmark SetupI ran Install
|
Comparison using pyperf table |
|
@itamaro I seem to be running into issues when I run [root@4b3304b8eaa9 vol]# make
Running code to generate profile data (this can take a while):
# First, we need to create a clean build with profile generation
# enabled.
make profile-gen-stamp
make[1]: Entering directory '/vol'
make clean
make[2]: Entering directory '/vol'
find . -depth -name '__pycache__' -exec rm -rf {} ';'
/bin/sh: find: command not found
make[2]: [Makefile:2256: pycremoval] Error 127 (ignored)
find . -name '*.py[co]' -exec rm -f {} ';'
/bin/sh: find: command not found
make[2]: [Makefile:2257: pycremoval] Error 127 (ignored)
find . -name '*.[oa]' -exec rm -f {} ';'
/bin/sh: find: command not found
make[2]: *** [Makefile:2271: clean] Error 127
make[2]: Leaving directory '/vol'
make[1]: *** [Makefile:692: profile-clean-stamp] Error 2
make[1]: Leaving directory '/vol'
make: *** [Makefile:712: profile-run-stamp] Error 2 |
|
@rtindru the make command is trying to use |
|
That worked great, thank you! |
|
I got a bunch of these errors when I enabled lto: So I ran the benchmarks with no lto or optimizations, just The flask benchmark is here: https://github.com/pyston/python-macrobenchmarks/blob/main/benchmarks/bm_flaskblogging/run_benchmark.py This writes the timestamp per iteration to flaskblogging.json, and I used a not-yet-open-sourced script that computes the median response time and uses that as the benchmark result. |
|
I ran into an issue while building with optimizations Steps to reproduceErrorThis error is occurs when you do ./configure --enable-optimizations this version of python complies it either way. What I triedI ran However, the make build_all_merge_profile
make[1]: Entering directory '/vol'
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
"Parser/listnode.o" [New] 0,0-1 All
true
make[1]: Leaving directory '/vol'
# Remove profile generation binary since we are done with it.
make clean
make[1]: Entering directory '/vol'
find . -depth -name '__pycache__' -exec rm -rf {} ';'
find . -name '*.py[co]' -exec rm -f {} ';'
find . -name '*.[oa]' -exec rm -f {} ';'
find . -name '*.s[ol]' -exec rm -f {} ';'
find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';'
find build -name 'fficonfig.h' -exec rm -f {} ';' || true
/* List a node on a file */
#include "Python.h"
#include "token.h"
#include "node.h"
/* Forward */
static void list1node(FILE *, node *);
static void listnode(FILE *, node *);
void
PyNode_ListTree(node *n)
{
listnode(stdout, n);
}
static int level, atbol;
static void
listnode(FILE *fp, node *n)
{
level = 0;
atbol = 1;
list1node(fp, n);
}
static void
list1node(FILE *fp, node *n)
{
if (n == NULL)
return;
if (ISNONTERMINAL(TYPE(n))) {
int i;
for (i = 0; i < NCH(n); i++)
list1node(fp, CHILD(n, i));
}
else if (ISTERMINAL(TYPE(n))) {
switch (TYPE(n)) {
case INDENT:
++level;
break;
case DEDENT:
--level;
break;
default:
"Parser/listnode.c" 66L, 1278B 1,0-1 Top
find build -name '*.py' -exec rm -f {} ';' || true
find build -name '*.py[co]' -exec rm -f {} ';' || true
rm -f pybuilddir.txt
rm -f Lib/lib2to3/*Grammar*.pickle
rm -f Programs/_testembed Programs/_freeze_importlib
/* Grammar subroutines needed by parser */
#include "Python.h"
#include "grammar.h"
#include "token.h"
/* Return the DFA for the given type */
const dfa *
PyGrammar_FindDFA(grammar *g, int type)
{
/* Massive speed-up */
const dfa *d = &g->g_dfa[type - NT_OFFSET];
assert(d->d_type == type);
return d;
}
const char *
PyGrammar_LabelRepr(label *lb)
{
static char buf[100];
if (lb->lb_type == ENDMARKER)
return "EMPTY";
else if (ISNONTERMINAL(lb->lb_type)) {
if (lb->lb_str == NULL) {
PyOS_snprintf(buf, sizeof(buf), "NT%d", lb->lb_type);
return buf;
}
else
return lb->lb_str;
}
else if (lb->lb_type < N_TOKENS) {
if (lb->lb_str == NULL)
return _PyParser_TokenNames[lb->lb_type];
else {
PyOS_snprintf(buf, sizeof(buf), "%.32s(%.32s)",
_PyParser_TokenNames[lb->lb_type], lb->lb_str);
return buf;
}
}
else {
Py_FatalError("invalid label");
return NULL;
}
"Parser/grammar1.c" 47L, 1066B 1,0-1 Top
find build -type f -a ! -name '*.gc??' -exec rm -f {} ';'
rm -f Include/pydtrace_probes_ceval.h
rm -f Include/pydtrace_probes_gcmodule.h
rm -f Include/pydtrace_probes_import.h
rm -f profile-gen-stamp
make[1]: Leaving directory '/vol'
# This is an expensive target to build and it does not have proper
# makefile dependency information. So, we create a "stamp" file
# to record its completion and avoid re-running it.
touch profile-run-stamp
Rebuilding with profile guided optimizations:
rm -f profile-clean-stamp
make build_all CFLAGS_NODIST=" -fprofile-use -fprofile-correction" LDFLAGS_NODIST=""
make[1]: Entering directory '/vol'
>> Programs/python.o
>> Parser/acceler.o
>> Parser/grammar1.o
>> Parser/listnode.o
Parser/listnode.c: In function ‘list1node’:
Parser/listnode.c:66:1: error: ‘/vol/Parser/listnode.gcda’ profile count data file not found [-Werror=missing-profile]
66 | }
| ^
cc1: all warnings being treated as errors
make[1]: *** [Makefile:2207: Parser/listnode.o] Error 1
make[1]: Leaving directory '/vol'
make: *** [Makefile:738: profile-opt] Error 2 |
|
Going to try the build on a cloud linux machine as I am wondering if this is a Mac issue |
any luck on Linux? |
I'm seeing this error too on a stock Though I notice it only came up after removing Opened #74 |
|
I tried building the most recent >> libpython3.8_static.a
In function ‘assemble_lnotab’,
inlined from ‘assemble_emit’ at Python/compile.c:5961:25,
inlined from ‘assemble’ at Python/compile.c:6308:18:
Python/compile.c:5914:19: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
5914 | *lnotab++ = k;
| ^
>> python.exe
./Modules/_static.c:20: warning: type of ‘_PyCheckedDict_Type’ does not match original declaration [-Wlto-type-mismatch]
20 | extern PyTypeObject _PyCheckedDict_Type;
|
Objects/dictobject.c:6493:19: note: type ‘struct _PyGenericTypeDef’ should match type ‘struct PyTypeObject’
6493 | _PyGenericTypeDef _PyCheckedDict_Type = {
| ^
Objects/dictobject.c:6493:19: note: ‘_PyCheckedDict_Type’ was previously declared here
Objects/dictobject.c:6493:19: note: code may be misoptimized unless ‘-fno-strict-aliasing’ is used
./Modules/_static.c:21: warning: type of ‘_PyCheckedList_Type’ does not match original declaration [-Wlto-type-mismatch]
21 | extern PyTypeObject _PyCheckedList_Type;
|
Objects/listobject.c:3883:19: note: type ‘struct _PyGenericTypeDef’ should match type ‘struct PyTypeObject’
3883 | _PyGenericTypeDef _PyCheckedList_Type = {
| ^
Objects/listobject.c:3883:19: note: ‘_PyCheckedList_Type’ was previously declared here
Objects/listobject.c:3883:19: note: code may be misoptimized unless ‘-fno-strict-aliasing’ is used
In function ‘assemble_lnotab’,
inlined from ‘assemble_emit’ at Python/compile.c:5961:0,
inlined from ‘assemble’ at Python/compile.c:6308:0:
Python/compile.c:5914: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
5914 | *lnotab++ = k;
|
/usr/bin/ld: python.exe.lto.o: in function `std::__shared_ptr<strictmod::objects::StrictExceptionObject, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<strictmod::objects::StrictExceptionObject>, std::shared_ptr<strictmod::objects::StrictType>, std::weak_ptr<strictmod::objects::StrictModuleObject> const&, std::shared_ptr<sequence_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > >(std::_Sp_alloc_shared_tag<std::allocator<strictmod::objects::StrictExceptionObject> >, std::shared_ptr<strictmod::objects::StrictType>&&, std::weak_ptr<strictmod::objects::StrictModuleObject> const&, std::shared_ptr<sequence_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&&) [clone .isra.0]':
/vol/./StrictModules/Objects/exception_object.h:10: undefined reference to `vtable for strictmod::objects::StrictExceptionObject'
/usr/bin/ld: python.exe.lto.o: in function `void strictmod::objects::StrictType::addGetSetDescriptor<std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), decltype(nullptr)>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), decltype(nullptr)) [clone .constprop.0]':
/usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: python.exe.lto.o: in function `void strictmod::objects::StrictType::addGetSetDescriptor<std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), decltype(nullptr), decltype(nullptr)>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), decltype(nullptr), decltype(nullptr)) [clone .constprop.0]':
/usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::StrictFuncType::addMethods()':
/usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: /usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: /usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: python.exe.lto.o:/usr/include/c++/10/ext/new_allocator.h:150: more undefined references to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))' follow
/usr/bin/ld: python.exe.lto.o: in function `std::shared_ptr<strictmod::objects::StrictType> strictmod::objects::makeType<strictmod::objects::StrictExceptionType, char const (&) [10], std::shared_ptr<strictmod::objects::StrictModuleObject>&, std::vector<std::shared_ptr<strictmod::objects::BaseStrictObject>, std::allocator<std::shared_ptr<strictmod::objects::BaseStrictObject> > >, std::shared_ptr<strictmod::objects::StrictType> >(char const (&) [10], std::shared_ptr<strictmod::objects::StrictModuleObject>&, std::vector<std::shared_ptr<strictmod::objects::BaseStrictObject>, std::allocator<std::shared_ptr<strictmod::objects::BaseStrictObject> > >&&, std::shared_ptr<strictmod::objects::StrictType>&&) [clone .constprop.0]':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::PropertyType()':
/vol/./StrictModules/Objects/property.h:69: undefined reference to `vtable for strictmod::objects::StrictPropertyType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::AttributeErrorType()':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::AssertionErrorType()':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::ValueErrorType()':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::StopIterationType()':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::KeyErrorType()':
/vol/./StrictModules/Objects/exception_object.h:26: undefined reference to `vtable for strictmod::objects::StrictExceptionType'
/usr/bin/ld: python.exe.lto.o:/vol/./StrictModules/Objects/exception_object.h:26: more undefined references to `vtable for strictmod::objects::StrictExceptionType' follow
/usr/bin/ld: python.exe.lto.o: in function `strictmod::objects::StrictObjectType::addMethods()':
/usr/include/c++/10/ext/new_allocator.h:150: undefined reference to `strictmod::objects::StrictGetSetDescriptor::StrictGetSetDescriptor(std::weak_ptr<strictmod::objects::StrictModuleObject>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<strictmod::objects::BaseStrictObject> (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::StrictType>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&), void (*)(std::shared_ptr<strictmod::objects::BaseStrictObject>, strictmod::CallerContext const&))'
/usr/bin/ld: python.exe.lto.o: in function `__static_initialization_and_destruction_0(int, int) [clone .lto_priv.77] [clone .constprop.0]':
/vol/./StrictModules/Objects/type_type.h:9: undefined reference to `vtable for strictmod::objects::StrictTypeType'
/usr/bin/ld: python.exe.lto.o: in function `__static_initialization_and_destruction_0(int, int) [clone .lto_priv.77] [clone .constprop.0]':
/vol/./StrictModules/Objects/module_type.h:9: undefined reference to `vtable for strictmod::objects::StrictModuleType'
/usr/bin/ld: python.exe.lto.o: in function `__static_initialization_and_destruction_0(int, int) [clone .lto_priv.77] [clone .constprop.0]':
/vol/./StrictModules/Objects/property.h:172: undefined reference to `vtable for strictmod::objects::StrictGetSetDescriptorType'
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:796: python.exe] Error 1
make[3]: Leaving directory '/vol'
make[2]: *** [Makefile:724: build_all_generate_profile] Error 2
make[2]: Leaving directory '/vol'
make[1]: *** [Makefile:700: profile-gen-stamp] Error 2
make[1]: Leaving directory '/vol'
make: *** [Makefile:712: profile-run-stamp] Error 2Steps to reproduce: git clone https://github.com/facebookincubator/cinder
docker run -v "$PWD/cinder:/vol" -w /vol -it --rm ghcr.io/facebookincubator/cinder/python-build-env:latest bash
./configure --enable-optimizations --with-lto
make |
|
Got the same error on a RHEL box Steps to reproduce Error log |
|
The build failure is a known issue and tracked in #74. More cases don't need to be cited. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The current open source build of Cinder could be improved in a couple of ways
[ ] Benchmark the performance of Cinder open source build vs. Cpython
[ ] Incorporate best practices in the open source build pipeline
[ ] Improve the performance of the final build
CC: @itamaro
The text was updated successfully, but these errors were encountered: