Rootfs filesize analysis tool
This commit checks in a tool to be used for analysing the space taken up
in the rootfs, as well as an overnight run from my system.
Most other tools give estimates of the space savings given when things
are deleted by either xz compressing the individual file, and measuring
the cost, or doing some other things.
This script actually attempts to remove the file or folder in question,
and rebuild the squashfs without it, then measure the size savings. It
loops this over the entire filesystem (excluding symlinks) and sorts the
result by size savings.
It takes a _very_ long time to run (several hours on a very fast
machine) but should give the most accurate answer to "what can we delete
to reduce our flash usage".
Some things that surprised me:
1. Deleting python would save us ~2.7MB in the rootfs size after the
/usr/lib/python2.7 delete (2.2MB), and the libpython removal (500KB)
2. Redfish static data is taking up more space than originally thought
with the latest revisions of the spec. Filtering the schema files down
to the ones we actually use would probably save us 100KB.
3. keymaps take up 430KB! ! ! !. In comparison, this is almost exactly
the same size as libc takes on the bmc. A quick look at the things
shows that they're being pre-gzipped in the filsystem, which means that
the xz compressor can't compress similarities between files. Might be
something to look at.
4. bmcweb binary weighs in at 425KB. This can be optimised.
5. /usr/share/www weighs in at 678KB. We need to get this down.
6. cracklib takes up 217KB. This seems kinda lame.
Change-Id: I5636ff22009f341a1454afd480f54701c6cf7954
Signed-off-by: Ed Tanous <ed.tanous@intel.com>
diff --git a/edtanous/rootfs_size.py b/edtanous/rootfs_size.py
new file mode 100644
index 0000000..3751fcf
--- /dev/null
+++ b/edtanous/rootfs_size.py
@@ -0,0 +1,64 @@
+import subprocess
+import tempfile
+import os
+from os.path import join, getsize
+
+# TODO
+# - Make build directory an input parameter
+# - Make squashfs file an input parameter
+# - Fix 80 char violations and run through pep8
+
+BUILD_DIR = "/home/ed/openbmc-openbmc"
+# files below this size wont be attempted
+FILE_SIZE_LIMIT = 0
+
+SQUASHFS = BUILD_DIR + \
+ "/build/tmp/deploy/images/wolfpass/intel-platforms-wolfpass.squashfs-xz"
+
+original_size = getsize(SQUASHFS)
+print("squashfs size: ".format(original_size))
+
+results = []
+
+with tempfile.TemporaryDirectory() as tempremovedfile:
+ with tempfile.TemporaryDirectory() as tempsquashfsdir:
+ print("writing to " + tempsquashfsdir)
+ command = ["unsquashfs", "-d",
+ os.path.join(tempsquashfsdir, "squashfs-root"), SQUASHFS]
+ print(" ".join(command))
+ subprocess.check_call(command)
+ squashfsdir = tempsquashfsdir + "/squashfs-root"
+
+ files_to_test = []
+ for root, dirs, files in os.walk(squashfsdir):
+ for name in files + dirs:
+ filepath = os.path.join(root, name)
+ if not os.path.islink(filepath):
+ # ensure files/dirs can be renamed
+ os.chmod(filepath,0o711)
+ if getsize(filepath) > FILE_SIZE_LIMIT:
+ files_to_test.append(filepath)
+
+ print("{} files to attempt removing".format(len(files_to_test)))
+
+ for filepath in files_to_test:
+ name = os.path.basename(filepath)
+ newname = os.path.join(tempremovedfile, name)
+ os.rename(filepath, newname)
+ with tempfile.TemporaryDirectory() as newsquashfsroot:
+ subprocess.check_output(
+ ["mksquashfs", tempsquashfsdir, newsquashfsroot + "/test", "-comp", "xz"])
+
+ results.append((filepath.replace(squashfsdir, ""),
+ original_size - getsize(newsquashfsroot + "/test")))
+ os.rename(newname, filepath)
+
+ print("{:>6} of {}".format(len(results), len(files_to_test)))
+
+results.sort(key=lambda x: x[1], reverse=True)
+
+with open("results.txt", 'w') as result_file:
+ for filepath, size in results:
+ result = "{:>10}: {}".format(size, filepath)
+ print(result)
+ result_file.write(result + "\n")