meta-phosphor/obmc-init.sh: fix panic when rwfs is full
mkdir -p $work will fail if rwfs partition is full, and then it will
cause overlayfs mount fail as well. Finally it cause kernel panic and
bmc fails to boot:
rofs = mtd4 squashfs rwfs = mtd5 jffs2
[ 3.841846] jffs2: notice: (103) jffs2_build_xattr_subsystem: complete building xattr subsystem, 21 of xdatum (19 unchecked, 2 orphan) and 45 of xref (1 dead, 1 orphan) found.
mkdir: can't create directory 'run/initramfs/rw/work': No space left on device
[ 9.022304] overlayfs: failed to resolve 'run/initramfs/rw/work': -2
mount: mounting cow on /root failed: No such file or directory
chroot: can't execute '/bin/sh': No such file or directory
Unable to confirm /sbin/init is an executable non-empty file
in merged file system mounted at /root.
Change Root test failed!
Fatal error, triggering kernel panic!
[ 9.141777] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
[ 9.142489] CPU: 0 PID: 1 Comm: init Not tainted 5.10.36-60b3c9d #1
[ 9.142667] Hardware name: Generic DT based system
[ 9.142949] Backtrace:
[ 9.144308] [<80915100>] (dump_backtrace) from [<809153a4>] (show_stack+0x20/0x24)
[ 9.144623] r7:80b08ed4 r6:60000093 r5:00000000 r4:80d71c9c
[ 9.144816] [<80915384>] (show_stack) from [<80919b54>] (dump_stack+0x9c/0xb0)
[ 9.144982] [<80919ab8>] (dump_stack) from [<80915824>] (panic+0x114/0x33c)
[ 9.145136] r7:80b08ed4 r6:80d0a970 r5:00000000 r4:80dbe378
[ 9.145271] [<80915710>] (panic) from [<80126c14>] (do_exit+0x99c/0xa0c)
[ 9.145426] r3:00000001 r2:00000000 r1:00000100 r0:80b08ed4
[ 9.145552] r7:ffffe000
[ 9.145632] [<80126278>] (do_exit) from [<80127cf0>] (do_group_exit+0x50/0xc8)
[ 9.145787] r7:000000f8
[ 9.145863] [<80127ca0>] (do_group_exit) from [<80127d88>] (__wake_up_parent+0x0/0x30)
[ 9.146034] r7:000000f8 r6:00000004 r5:00000004 r4:010fd190
[ 9.146185] [<80127d68>] (sys_exit_group) from [<80100060>] (ret_fast_syscall+0x0/0x54)
[ 9.146425] Exception stack(0x810c5fa8 to 0x810c5ff0)
[ 9.146745] 5fa0: 010fd190 00000004 00000001 00000000 00000001 010fd190
[ 9.147049] 5fc0: 010fd190 00000004 00000004 000000f8 00000000 00000000 004e4a84 00000000
[ 9.147295] 5fe0: 004e4b40 7ec93a04 00474048 76df4a54
[ 9.147959] CPU1: stopping
[ 9.148650] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.10.36-60b3c9d #1
[ 9.148957] Hardware name: Generic DT based system
[ 9.149264] Backtrace:
[ 9.150182] [<80915100>] (dump_backtrace) from [<809153a4>] (show_stack+0x20/0x24)
[ 9.150598] r7:00000001 r6:60070193 r5:00000000 r4:80d71c9c
[ 9.150878] [<80915384>] (show_stack) from [<80919b54>] (dump_stack+0x9c/0xb0)
[ 9.151149] [<80919ab8>] (dump_stack) from [<8010f1f4>] (do_handle_IPI+0x2f8/0x32c)
[ 9.151430] r7:00000001 r6:0d15a000 r5:00000001 r4:80dbe278
[ 9.151649] [<8010eefc>] (do_handle_IPI) from [<8010f250>] (ipi_handler+0x28/0x30)
[ 9.151925] r9:810f6000 r8:81080800 r7:00000001 r6:0d15a000 r5:81084b80 r4:00000014
[ 9.152205] [<8010f228>] (ipi_handler) from [<80184754>] (handle_percpu_devid_fasteoi_ipi+0x80/0x154)
[ 9.152536] [<801846d4>] (handle_percpu_devid_fasteoi_ipi) from [<8017dce4>] (__handle_domain_irq+0x8c/0xe0)
[ 9.152911] r7:00000001 r6:00000000 r5:00000000 r4:80c6fbdc
[ 9.153147] [<8017dc58>] (__handle_domain_irq) from [<80101348>] (gic_handle_irq+0x7c/0x90)
[ 9.153508] r9:810f6000 r8:8f80200c r7:80c6fbe8 r6:8f802000 r5:810f7f38 r4:80d0541c
[ 9.153819] [<801012cc>] (gic_handle_irq) from [<80100b0c>] (__irq_svc+0x6c/0x90)
[ 9.154168] Exception stack(0x810f7f38 to 0x810f7f80)
[ 9.154541] 7f20: 00000000 000195e0
[ 9.154993] 7f40: 8ddcb144 8011ba00 810f6000 00000001 80d04d10 80d04d4c 80d9dfdc 80b0b4f0
[ 9.155402] 7f60: 00000000 810f7f94 810f7f98 810f7f88 80108fd4 80108fd8 60070013 ffffffff
[ 9.155762] r9:810f6000 r8:80d9dfdc r7:810f7f6c r6:ffffffff r5:60070013 r4:80108fd8
[ 9.156070] [<80108f90>] (arch_cpu_idle) from [<80926c68>] (default_idle_call+0x38/0x108)
[ 9.156382] [<80926c30>] (default_idle_call) from [<80159894>] (do_idle+0xdc/0x148)
[ 9.156676] [<801597b8>] (do_idle) from [<80159bd0>] (cpu_startup_entry+0x28/0x2c)
[ 9.156974] r9:410fc075 r8:8000406a r7:80dbe280 r6:10c0387d r5:00000001 r4:00000091
[ 9.157275] [<80159ba8>] (cpu_startup_entry) from [<8010f7c0>] (secondary_start_kernel+0x15c/0x180)
[ 9.157614] [<8010f664>] (secondary_start_kernel) from [<8010182c>] (__enable_mmu+0x0/0x14)
[ 9.157919] r5:00000051 r4:810e006a
Fix this by not removing workdir directly, but keeping the directory
and removing all entires under it. We will still get below warning,
but it can boot to shell and provide ssh access to give a chance of
recovery.
[ 9.519053] overlayfs: failed to create directory run/initramfs/rw/work/work (errno: 28); mounting read-only
Also update related patch file in meta-google to avoid build failure.
Signed-off-by: Heyi Guo <guoheyi@linux.alibaba.com>
Change-Id: I69c640c70ee3e9d1133b7d61d391501616c229d6
diff --git a/meta-google/recipes-phosphor/initrdscripts/obmc-phosphor-initfs/rwfs-clean-dev.patch b/meta-google/recipes-phosphor/initrdscripts/obmc-phosphor-initfs/rwfs-clean-dev.patch
index 7631c85..8760ccf 100644
--- a/meta-google/recipes-phosphor/initrdscripts/obmc-phosphor-initfs/rwfs-clean-dev.patch
+++ b/meta-google/recipes-phosphor/initrdscripts/obmc-phosphor-initfs/rwfs-clean-dev.patch
@@ -1,8 +1,8 @@
diff --git a/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh b/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh
-index 62be89f039..689fbea5bf 100644
+index 2a32d26ffc..a8cc76d440 100644
--- a/obmc-init.sh
+++ b/obmc-init.sh
-@@ -275,6 +275,14 @@ then
+@@ -287,6 +287,14 @@ then
fi
fi
@@ -17,13 +17,13 @@
if grep -w clean-rwfs-filesystem $optfile
then
echo "Cleaning of read-write overlay filesystem requested."
-@@ -399,6 +407,9 @@ HERE
+@@ -411,6 +419,9 @@ HERE
debug_takeover "$msg"
fi
+# Keep track of the last version to use this rwfs
+cp /etc/os-release "$rwdir"/os-release
+
- rm -rf $work
- mkdir -p $upper $work
-
+ # Empty workdir; do not remove workdir itself for it will fail to recreate it if
+ # RWFS is full
+ if [ -d $work ]
diff --git a/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh b/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh
index e61ede9..2a32d26 100644
--- a/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh
+++ b/meta-phosphor/recipes-phosphor/initrdscripts/files/obmc-init.sh
@@ -411,7 +411,13 @@
debug_takeover "$msg"
fi
-rm -rf $work
+# Empty workdir; do not remove workdir itself for it will fail to recreate it if
+# RWFS is full
+if [ -d $work ]
+then
+ find $work -maxdepth 1 -mindepth 1 -exec rm -rf '{}' +
+fi
+
mkdir -p $upper $work
mount -t overlay -o lowerdir=$rodir,upperdir=$upper,workdir=$work cow /root