So I’ve had issues with AppImage (and Snapcraft, and Flatpak, and whatever new flavor of the month versions exist) for a while. The arguments in favor are always the same, as are the rebuttals, and so on and so forth.
Snapcraft User: It makes life simpler!
Gray Beard: You end up with duplicate copies of libraries though.
SU: Who cares, disk is cheap!
GB: I still have to deploy to a server or whatever. And what about deploying security fixes if the bundled libraries are out of date?
SU: The important libraries like libc live outside the application, so your package manager deals with those. Besides, it’s not like code reuse is a thing anymore! Every application already uses a different {image,audio,windowing} library.
So I decided it was time to investigate for myself. I grabbed the AppImage of KDevelop because:
- I knew it existed
- I use KDevelop, so it’s something I’d possibly use as an AppImage (at the time I’m writing this, both the AppImage and the version of KDevelop installed via my package manager are 5.2.1)
The first thing I did was extract the contents of the AppImage file. This gave me a folder (/mnt/ai/kdevelop) with the squashfs expanded and available for poking around. I wanted to see how many libraries included in the image were already installed on my system:
[/mnt/ai/kdevelop] find usr/ -type f -name '*.so' | wc -l
179
[/mnt/ai/kdevelop] find usr/ -type f -name '*.so' |
while read f; do
if [ -e /${f} ]; then
echo "${f} exists";
fi
done | wc -l
141
At first glance that’s pretty bad, but let’s see how many of those are just for KDevelop. Since paludis (my package manager) can tell me what package owns a file, I’ll feed the output there and see how many of my system packages are bundled in:
[/mnt/ai/kdevelop] find usr/ -type f -name '*.so' |
while read f; do
if [ -e /${f} ]; then
echo "/${f}";
fi
done |
sed "s|^/usr/lib/|/usr/lib64/|" |
while read f; do
cave owner ${f};
done |
sed 's|::installed$||' |
sort |
uniq -c >/home/stephen/appimage/kdevelop.package-list
This is close to the previous script, but there are a few minor changes:
- I’m changing the file path in the AppImage from usr/lib to usr/lib64. This matches the path on my system where the file actually would live, since /usr/lib symlinks to /usr/lib64.
- I’m only getting the unique packages that provide files. If something provides 20 libraries that are bundled into the AppImage, I don’t need to see it 20 times (but I’ll get the count).
The results of this command:
[~/appimage] cat kdevelop.package-list 3 dev-libs/nspr-4.17 5 dev-libs/nss-3.34.1 9 dev-qt/qtgui-5.7.1-r1 5 dev-qt/qtimageformats-5.7.1 1 dev-qt/qtnetwork-5.7.1 2 dev-qt/qtsvg-5.7.1 1 dev-qt/qtvirtualkeyboard-5.7.1 58 dev-util/kdevelop-5.2.1 5 dev-util/kdevelop-python-5.2.1 20 kde-apps/kate-17.08.3 2 kde-apps/konsole-17.08.3 1 kde-frameworks/kauth-5.40.0 1 kde-frameworks/kglobalaccel-5.40.0 1 kde-frameworks/ki18n-5.40.0 1 kde-frameworks/kinit-5.40.0 19 kde-frameworks/kio-5.40.0-r3 1 kde-frameworks/ktexteditor-5.40.0 2 kde-frameworks/kwindowsystem-5.40.0 1 kde-frameworks/sonnet-5.40.0 3 kde-plasma/breeze-5.11.5
KDevelop owns most of them (expected), but there are some interesting things that got included. I was surprised to see so much of Qt bundled (I run a KDE Plasma desktop, so there could be bias at play), but some of these libraries are important. I use at least three things that leverage ktexteditor (KWrite, Kate, KDevelop), and nspr and nss are part of Firefox (nss is related to security). I was surprised not to see LLVM/Clang libraries, but they don’t show up in ldd for my system-installed KDevelop either.
Next step was to try and uninstall all these libraries and see what happened.
[~/appimage] cave uninstall $(awk '{print "="$2}' |
grep '^X' -A 2 >kdevelop.package-list.broken
[~/appimage] grep '^X' kdevelop.package-list.broken | wc -l
161
161 broken packages if I uninstall everything?! Not all of these are related to KDevelop either:
X www-client/firefox 58.0.1:0::installed Will be broken by uninstalls: Reasons: dependent upon dev-libs/nspr-4.17:0::installed (DEPEND), dependent upon dev-libs/nspr-4.17:0::installed (RDEPEND), dependent upon dev-libs/nss-3.34.1:0::installed (DEPEND), dependent upon dev-libs/nss-3.34.1:0::installed (RDEPEND) X media-video/vlc 2.2.8-r1:0::installed Will be broken by uninstalls: Reasons: dependent upon dev-qt/qtgui-5.7.1-r1:5::installed (DEPEND), dependent upon dev-qt/qtgui-5.7.1-r1:5::installed (RDEPEND) X app-office/libreoffice 5.4.5.1:0::installed Will be broken by uninstalls: Reasons: dependent upon dev-libs/nspr-4.17:0::installed (DEPEND), dependent upon dev-libs/nspr-4.17:0::installed (RDEPEND), dependent upon dev-libs/nss-3.34.1:0::installed (DEPEND), dependent upon dev-libs/nss-3.34.1:0::installed (RDEPEND) X net-wireless/wpa_supplicant 2.6-r3:0::installed Will be broken by uninstalls: Reasons: dependent upon dev-qt/qtgui-5.7.1-r1:5::installed (DEPEND), dependent upon dev-qt/qtgui-5.7.1-r1:5::installed (RDEPEND), dependent upon dev-qt/qtsvg-5.7.1:5::installed (DEPEND), dependent upon dev-qt/qtsvg-5.7.1:5::installed (RDEPEND)
So my browser, office suite, wireless network manager, and media player already depend on these libraries, and that’s ignoring all the KDE and Plasma packages I already have.
I get the desire for universal binaries, but this is insane. I don’t need multiple copies of Qt and KDE/Plasma libraries. If I wanted AppImages for my terminal emulator (Konsole) and GUI editors (Kate/KWrite), would that mean bringing in even more copies of all these same libraries? Yeah, disk is cheap, but the AppImage has 96M of libraries, the majority of which I already have installed. And this says nothing of the fact that if any of these packages need to be updated (nss and nspr stand out) I’m either dependent on the KDevelop team to release a new AppImage or modify it myself. If my distro is providing AppImages they might port security fixes to old software, but I don’t think it’s optimistic to assume the upstream developers will only support the latest versions of their software (hope you like bleeding edge for everything).
On top of that, where exactly is the line drawn between libraries that are assumed to exist and libraries that need to be bundled? Obviously Qt and KDE/Plasma got bundled, but libc didn’t. Last time I checked lots of packages release ABI-breaking updates (I recall soname changes between versions of libpng and ncurses), so those would have be to bundled to avoid breaking an AppImage with the system-wide package manager, but that also means even more duplicate code.
Like many things, to me this is a scaling problem. The idea is great on paper, and probably works well for small isolated packages, but once you’re dealing with dependencies stuff gets messy.
Note: I did the scripts above in one-liners, but added white space to make them more readable. I haven’t run the white space enhanced versions of my system, so apologies in advance if I accidentally messed something up.