Manual:$wgShellCgroup

Under Linux: a cgroup directory used to constrain memory usage of shell commands. The directory must be writable by the user which runs MediaWiki.

If specified, this is used instead of ulimit, which is inaccurate, and causes malloc to return NULL, which exposes bugs in C applications, making them segfault or deadlock.

A wrapper script will create a cgroup for each shell command that runs, as a subgroup of the specified cgroup. If the memory limit is exceeded, the kernel will send a SIGKILL signal to a process in the subgroup. (MediaWiki will often report this error as error code 137, after the bash return status of a command receiving SIGKILL)

Set up
If cgroups are not set up, you may have to mount the cgroup file system. As root:

mount -t tmpfs cgroup_root /sys/fs/cgroup mkdir /sys/fs/cgroup/memory mount -t cgroup cgroup_memory -omemory /sys/fs/cgroup/memory

Next you need to do MediaWiki specific configuration:

mkdir -p /sys/fs/cgroup/memory/mediawiki mkdir -m 0777 /sys/fs/cgroup/memory/mediawiki/job

The reliability of cgroup cleanup can be improved by installing a notify_on_release script in the root cgroup [This step is optional] echo "/usr/local/bin/cgroup-mediawiki-clean" > /sys/fs/cgroup/memory/release_agent
 * Copy to /usr/local/bin/cgroup-mediawiki-clean
 * Run the command

Last of all, enable cgroups in your LocalSettings.php echo '$wgShellCgroup = "/sys/fs/cgroup/memory/mediawiki/job";' >> LocalSettings.php

The reliability of cgroup cleanup can be improved by installing a notify_on_release script in the root cgroup, see e.g. https://gerrit.wikimedia.org/r/#/c/40784

Troubleshooting
If cgroups don't work, things to try:
 * Install cgroup-bin package
 * Run the command . If the memory subsystem has a 0 under enabled, then memory cgroups may be disabled in your kernel. Try rebooting your computer with   on the kernel command line.
 * Older bash doesn't like the readarray call in limit.sh. You should use at least 4.1