genautomata: out of memory #10

New Issue

claunia · 2026-01-29T22:22:56Z

claunia commented

2026-01-29 22:22:56 +00:00

Originally created by @dmik on GitHub (Jan 9, 2020).

There is a bunch of build tools in gcc/build used to generate various stuff for GCC. One of these tools is genautomata.exe. It generates a source file from a machine definition (.md) file.

It is run as follows during make process in the gcc subdirectory of the output tree:

build/genautomata.exe ../../master/gcc/common.md ../../master/gcc/config/i386/i386.md \
  insn-conditions.md > tmp-automata.c

If it is built using GCC 4.9.2 and its libstdc++, all is fine. The resulting file is quite big though — 11 MB (as designed). However, if it is built using GCC 9.2.0 and its libstdc++, it exits with the following error message:

out of memory allocating 4064 bytes after a total of 4262222764 bytes

Sometimes a popup message box about SYS0147 is also shown (see https://github.com/bitwiseworks/gcc-os2/issues/6#issuecomment-572660490).

Note that I can't try out the binary built with GCC 9.2.0 with a libstdc++ DLL from GCC 4.9.2 because operator delete(void*, unsigned int) (__ZdlPvj) was never exported back then (which is also strange and needs some checking). So I'm not sure yet if it's the new GCC itself or just the new libstdc++ DLL which is guilty here.

Originally created by @dmik on GitHub (Jan 9, 2020). There is a bunch of build tools in `gcc/build` used to generate various stuff for GCC. One of these tools is `genautomata.exe`. It generates a source file from a machine definition (.md) file. It is run as follows during make process in the `gcc` subdirectory of the output tree: ``` build/genautomata.exe ../../master/gcc/common.md ../../master/gcc/config/i386/i386.md \ insn-conditions.md > tmp-automata.c ``` If it is built using GCC 4.9.2 and its libstdc++, all is fine. The resulting file is quite big though — 11 MB (as designed). However, if it is built using GCC 9.2.0 and its libstdc++, it exits with the following error message: ``` out of memory allocating 4064 bytes after a total of 4262222764 bytes ``` Sometimes a popup message box about SYS0147 is also shown (see https://github.com/bitwiseworks/gcc-os2/issues/6#issuecomment-572660490). Note that I can't try out the binary built with GCC 9.2.0 with a libstdc++ DLL from GCC 4.9.2 because `operator delete(void*, unsigned int)` (`__ZdlPvj`) was never exported back then (which is also strange and needs some checking). So I'm not sure yet if it's the new GCC itself or just the new libstdc++ DLL which is guilty here.

claunia closed this issue

2026-01-29 22:22:57 +00:00

claunia commented

2026-01-29 22:22:57 +00:00

@dmik commented on GitHub (Jan 9, 2020):

Re operator delete, it's a special form, see (5) in https://en.cppreference.com/w/cpp/memory/new/operator_delete. it seems that this form just didn't exist in the C++ standard prior C++14 (which GCC 4.x supports only partially). According to https://gcc.gnu.org/projects/cxx-status.html, this operator (called "sized deallocation") was only added in GCC 5.x. So its absense in the GCC 4.x DLL is fully expected.

@dmik commented on GitHub (Jan 9, 2020): Re operator delete, it's a special form, see (5) in https://en.cppreference.com/w/cpp/memory/new/operator_delete. it seems that this form just didn't exist in the C++ standard prior C++14 (which GCC 4.x supports only partially). According to https://gcc.gnu.org/projects/cxx-status.html, this operator (called "sized deallocation") was only added in GCC 5.x. So its absense in the GCC 4.x DLL is fully expected.

claunia commented

2026-01-29 22:22:58 +00:00

@dmik commented on GitHub (Jan 10, 2020):

I tried to just rebuild the CPP files that comprise genautomata.exe using GCC 9.2.0 (so that they are linked against its libstdc++) with the rest (e.g. libiberty.a) still built by GCC 4.9.2 and this is enough to break it. Which means that libiberty (that actually does the failed memory allocation in its xmalloc implementation) is not involved here. Most likely, libstdc++ is.

@dmik commented on GitHub (Jan 10, 2020): I tried to just rebuild the CPP files that comprise `genautomata.exe` using GCC 9.2.0 (so that they are linked against its libstdc++) with the rest (e.g. libiberty.a) still built by GCC 4.9.2 and this is enough to break it. Which means that libiberty (that actually does the failed memory allocation in its `xmalloc` implementation) is not involved here. Most likely, libstdc++ is.

claunia commented

2026-01-29 22:22:59 +00:00

@dmik commented on GitHub (Jan 11, 2020):

I did some more tests and found the following. The executable works well if linked against the static version of libstdc++. It also works well if worked normally against the DLL version but only if it is also linked against LIBCx (-lcx). And no, it's not mmap as one cold think. It's the default C heap placement. A program linked against -lcx not only gets mmap support and FPU control word change protection. It also makes the application use high memory for the default C heap regardless of presence of -Zhigh-mem among the link options. We did that specifically to solve the low heap problem with applications that link to DLLs which we can't rebild with -Zhigh-mem. And any such DLL will "vote" for low memory when the application starts so it will block usage of high memory for the C heap. And this is completely unwanted behavior.

Given that, the rest was quite easy to figure out. We build stdcpp6.dll via libtool now and it seems to ignore LDFLAGS. As a result, this DLL was built w/o -Zhigh-mem. Consequently, all apps linked against it (like genautomata.exe) would switch to low memory for the C heap. And this heap is indeed very small — typically something around 300 MB. This amount is certainly not enough for getautomata to do its job hence it bails. When linked against LIBCx, it gets high memory for the C heap regardless of any heap voting and this makes it pretty happy in terms of available memory (should be around 2.5 GB if there is enough physical RAM).

Now I need to figure out how to pass -Zhigh-mem to libtool fro all DLLs we build with it.

@dmik commented on GitHub (Jan 11, 2020): I did some more tests and found the following. The executable works well if linked against the static version of `libstdc++`. It also works well if worked normally against the DLL version but only if it is also linked against LIBCx (`-lcx`). And no, it's not `mmap` as one cold think. It's the default C heap placement. A program linked against `-lcx` not only gets `mmap` support and FPU control word change protection. It also makes the application use high memory for the default C heap **regardless** of presence of `-Zhigh-mem` among the link options. We did that specifically to solve the low heap problem with applications that link to DLLs which we can't rebild with `-Zhigh-mem`. And any such DLL will "vote" for low memory when the application starts so it will block usage of high memory for the C heap. And this is completely unwanted behavior. Given that, the rest was quite easy to figure out. We build `stdcpp6.dll` via libtool now and it seems to ignore LDFLAGS. As a result, this DLL was built w/o `-Zhigh-mem`. Consequently, all apps linked against it (like `genautomata.exe`) would switch to low memory for the C heap. And this heap is indeed very small — typically something around 300 MB. This amount is certainly not enough for `getautomata` to do its job hence it bails. When linked against LIBCx, it gets high memory for the C heap regardless of any heap voting and this makes it pretty happy in terms of available memory (should be around 2.5 GB if there is enough physical RAM). Now I need to figure out how to pass -Zhigh-mem to libtool fro all DLLs we build with it.

claunia commented

2026-01-29 22:23:01 +00:00

@dryeo commented on GitHub (Jan 12, 2020):

On 01/11/20 01:26 PM, Dmitriy Kuminov wrote:

Now I need to figure out how to pass -Zhigh-mem to libtool fro all DLLs
we build with it.

Other DLFLAGS such as -Zbin-files also don't seem to get passed

@dryeo commented on GitHub (Jan 12, 2020): On 01/11/20 01:26 PM, Dmitriy Kuminov wrote: > Now I need to figure out how to pass -Zhigh-mem to libtool fro all DLLs > we build with it. Other DLFLAGS such as -Zbin-files also don't seem to get passed

claunia commented

2026-01-29 22:23:02 +00:00

@dmik commented on GitHub (Jan 12, 2020):

Investigation shows that libtool normally respects LDFLAGS (as well as CFLAGS and CXXFLAGS). However, in case of GCC things are way more complex. There are three definitions there: build platform (where gcc is being built), host platform (where it will be then run) and target platform (for which it will generate binaries). LDFLAGS is only used for the first two. The libstdc++ library is built as for target and hence configure requires a special version of LDFLAGS for it which is called LDFLAGS_FOR_TARGET. So the solution is as easy as adding:

export LDFLAGS_FOR_TARGET="$LDFLAGS"

after the LDFLAGS export before running the root configure. With that, all DLLs we build get flags from LDFLAGS, including -Zhigh-mem. I did a full GCC build which proves that. Problem solved.

@dmik commented on GitHub (Jan 12, 2020): Investigation shows that libtool normally respects LDFLAGS (as well as CFLAGS and CXXFLAGS). However, in case of GCC things are way more complex. There are three definitions there: build platform (where gcc is being built), host platform (where it will be then run) and target platform (for which it will generate binaries). LDFLAGS is only used for the first two. The libstdc++ library is built as for target and hence configure requires a special version of LDFLAGS for it which is called LDFLAGS_FOR_TARGET. So the solution is as easy as adding: ``` export LDFLAGS_FOR_TARGET="$LDFLAGS" ``` after the LDFLAGS export before running the root configure. With that, all DLLs we build get flags from LDFLAGS, including `-Zhigh-mem`. I did a full GCC build which proves that. Problem solved.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bitwiseworks/gcc-os2#10