Unable to install apps after upgrading to 7.5.0 UP8+
NOTE: However, not every app install will fail, usually memory allocation failure happens for larger apps. Sometimes it will also happen when trying to start/stop an app.
In /var/log/messages:
Sep 18 06:47:25 runc:[1:CHILD]: page allocation failure: order:8, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=runc-buildah-buildah2238631207.scope,mems_allowed=0
We get following error in /var/log/qradar/app/podman_build and /var/log/qradar.log or /var/log/qradar.error:
Sep 18 06:47:25 ::ffff:127.0.0.1 [hostcontext.hostcontext] [pool-2-thread-5] com.ibm.si.application.podman.registry.build.ImageBuildException: com.ibm.si.application.podman.commands.PodmanCommandException: Podman command [/usr/bin/podman build -t console.localdeployment:5000/qapp/1502:3.10.0-20240918064722 --build-arg APP_USER_ID=99 --build-arg APP_GROUP_ID=99 --build-arg BUILD_TIMESTAMP="2024-09-18 10:47:19 UTC" /storetmp/AppFW_1502] failed with exit code [1] and error message [error running container: from /usr/bin/runc creating container for [/bin/sh -c groupadd -o -g $APP_GROUP_ID $APP_GROUP_NAME && useradd -l -u $APP_USER_ID -g $APP_GROUP_ID $APP_USER_NAME && echo -n $BUILD_TIMESTAMP | sed 's/"//g' > $APP_ROOT/group_date]: time="2024-09-18T06:47:24-04:00" level=fatal msg="nsexec-1[4062222]: failed to unshare remaining namespaces (except cgroupns): Cannot allocate memory"
Unable to install apps after upgrading to 7.5.0 UP8+ or apps are in error state
This workaround does not need services to be restarted and should take effect when changes are made. Please check cat /proc/pagetypeinfo before and after a change to confirm if higher-order pages are available. Running this, compaction will be applied only once.
cat /proc/pagetypeinfo
# cat /proc/pagetypeinfo
# echo 1 > /proc/sys/vm/compact_memory
As this workaround works in most cases, it can be set to be applied automatically and periodically either by Adding it as the boot parameter or setting a crontab job, especially for these systems that experience severe memory fragmentation.
If enabling compact_memory alone does not work, try this, it's a time action to flush the slab objects and page cache:
# sysctl -w vm.drop_caches=3
NOTE: Keep in mind, any changes to the sysctl.conf file is not persistent and has to be re-applied after patches/upgrades.