Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle. The amount of dynamic linking issues I encountered on GNU/Linux was insane (fuck libstdc++). Not even glibc manages to keep forward compatibility working (breaking memcpy, breaking DT_HASH, ...)!
It's much better to just statically link your binaries (unfortunately many programs on GNU/Linux do not support this).
Before the security guys come out the woods: No you don't need to rebuilt all programs linked against your dynamic library of choice. Just keep the object files around and relink them against the updated version of the static libraries. If now package manager just allowed differential binary updates (actually why is this trivial optimization not a common thing?) the overhead in terms of download size is not that of a big deal too.
The value you gain is immense: binaries just work (modulo linux ABI issues but they are doing quite a good job at keeping this interface stable)
Apple uses dynamic linking to enable its OSes to evolve. Apple's libraries talk to daemons using IPC protocols such as XPC or MIG; if those libraries were statically linked, the daemons would have to support old protocols forever. And when Apple changes the appearance of its UI, they want all apps to get the changes immediately, and not wait for the apps to be recompiled. For example, way back I implemented window resizing from all sides and every app just got it on day 1; no recompilation necessary!
It’s the main reason why dynamic linking is popular. It’s just that the people on Hacker News often write client software for Linux rather than having to deal with ABI compatibility :)
> Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle
But space in fast cache IS worth the hassle. It's still extremely limited, and since we're hitting the limits of moore's law we're not getting much more of it.
The article touches on this. Statically linking can be a case of "tragedy of the commons" if you statically link a single program, you only see the benefits. But if you statically link all programs in a system, vs dynamic linking of all programs, you're giving up on the chance of having shared caching of commonly used pieces of code between programs, and you're giving up on minimizing code size so you don't have to prematurely evict useful code from the cache.
The article touches on this subject:
> It’s worth noting that the Swift devs [...] care much more about code sizes (as in the amount of executable code produced). More specifically, they care a lot more about making efficient usage of the cpu’s instruction cache, because they believe it’s better for system-wide power usage. Apple championing this concern makes a lot of sense, given their suite of battery-powered devices.
It's a common argument of computers getting powerful so we don't need to care that much for performance and/or efficiency regarding to cpu/memory/storage etc. In some limited cases the argument is valid but most of the time it's not.
First of all while maybe desktops and mobile phones are more powerful now, but we're getting more and more lower spec devices, like smart watches. Even when smart watches will be powerful enough one day, there will be eventually smaller computers, like smart contact lens, nano robots that run in blood vessels etc.
Secondly efficiency is still favourable for powerful computers. A small percentage of cost reduction can be big money save for large corps. A small percentage of energy save of all (or just a portion of) computers in the world can be a big win for the environment.
Lastly, we also run VMs and containers everywhere. Notice how we've already come up with all kind of ways to minimise VMs and containers size and footprint, in order to run more of them, to start them faster, and to transfer images quicker.
Yep. For hot call sites, these optimizations & inlining opportunities make a massive difference to performance. Static linking also allows for faster application startup time. (Though I don't have an intuition for exactly how slow dynamic linking is).
The only argument for dynamic linking being more efficient is that each dynamic library can be shared between all programs that use it. But not a net win in all cases. When you dynamically link a library, the entire library is loaded into RAM. When you static link, dead code elimination means that only the code you actually run in the library needs to be loaded.
But honestly, none of these arguments are strong. Dyld is fast enough on modern computers that we don't notice it. And RAM is cheap enough these days that sharing libraries between applications for efficiency feels a bit pointless.
The real arguments are these:
- Dynamic linking puts more power in the hands of the distribution (eg Debian) to change the library that a program depends on. This is used for security updates (eg OpenSSL) or for UI changes on Apple platforms. Dynamic linking is also faster than static linking, so compile times are faster for large programs.
- Static libraries put power in the hands of the application developer to control exactly how our software runs. VMs and Docker are essentially wildly complicated ways to force static linking, and the fact that they're so popular is evidence of how much this sort of control is important to software engineers. (And of course, statically linked binaries are usually simpler to deploy because static binaries make fewer assumptions about their runtime environment.)
> When you dynamically link a library, the entire library is loaded into RAM.
It doesn't. When you dynamic link a library, no part of it is loaded into RAM. It is page-faulted in, as it is used. In the end, only parts that were really used were loaded into RAM.
A page will be loaded in if any part of it is useful. Given that functions will be laid out more or less randomly throughout a shared library, and programs use a randomly scattered subset of the functions, I think its safe to say that you'll get a lot of bytes read in to ram that are never used.
Especially when we take the filesystem's read-ahead cache into account - which will optimistically load a lot of bytes near any executed function.
If your program makes use of some arbitrary 10% of the functions in a shared library, how much of the library will be read from disk? How much will end up in RAM? Its going to be much more than 10%. I'd guess that you'll end up with closer to 50% of the library loaded in memory, in one way or another. (Though I could be way off. I suspect most of the time the filesystem cache will end up loading the whole thing.)
If its 50% loaded, a shared library thats used once will waste 90% of its download size & disk space and 50% of its ram usage compared to the equivalent static library. And make the application slower to start because it needs to link at runtime. And make the program slower to run because of missed inlining opportunities.
> A page will be loaded in if any part of it is useful. Given that functions will be laid out more or less randomly throughout a shared library, and programs use a randomly scattered subset of the functions, I think its safe to say that you'll get a lot of bytes read in to ram that are never used.
Dynamic linking also allows for extensible applications that have to otherwise be implemented with slower OS IPC calls, and higher resource costs in process and CPU cores management.
Which also prevents good privilege separation/sandboxing. See PAM vs BSD Auth, where the former cannot be secured with anything like pledge/unveil or Capsicum, but the latter can.
So are you saying use Rust, more efficient at the expense of maybe using more memory? Or swift which may save some memory by being dynamically linked but perhaps is a little slower.
Honestly a ton of code is utility code, not run that often and nobody wants to take the time or expense to rewrite that Perl/python/bash script.
Some code efficiency is much more important than others.
> First of all while maybe desktops and mobile phones are more powerful now, but we're getting more and more lower spec devices, like smart watches. Even when smart watches will be powerful enough one day, there will be eventually smaller computers, like smart contact lens, nano robots that run in blood vessels etc.
My desktop/laptop distro dynamically link everything by default just so it can be slightly less work to port to a smartwatch? That sounds like a reasonable argument for having dynamically linking be possible, but it doesn't seem compelling for it being the default for the entire world.
You can't ship reasonable plugin for most of the popular content creation tools - be it audio, video effects, 3d modelling, etc. without support for dynamic/shared libs.
Okay, maybe you can - at the risk of performance cost - e.g. your out-of process plugin that need to move data in/out with a cost, and then it over complicates - because the plugin process might've died - how do you restart it, how do you get it back to state, etc. etc. You just don't have to solve this with .dll or .so
Apart from most perf critical cases, it's a good idea to run plugins in a separate process anyway. This way a bad plugin won't crash or corrupt your own process. You will be able to stop and unload it without risk of leaving locks locked or thread-locals leaking.
And also there's WASM now which has even better sandboxing and OS/architecture independence.
Note that this is how things were done before dynamic linking was invented, and dynamic linking was seen as a big step forward. Processes+shared memory is a really painful and slow paradigm compared to just invoking function calls on the same stack and in the same process.
"a bad plugin won't crash or corrupt your own process."
It still can. The plugin just has to return bad data you're not prepared to handle, or crash in a way you didn't anticipate and code around.
And note that with higher level languages the whole thing is moot. A plugin crashing just means it throws an exception which can be caught, reported and quite likely the host process will survive just fine even though it's all in-process and dynamically loaded.
Something existing and succeeding is not automatically a claim that it's the only one possible thing, or even a novel idea, so dismissal "it's just recycling old ideas" is non-sequitur.
I'm not even sure what you're trying to say with this comment. Is only 1958 LISP or maybe SmallTalk worthy and truly novel, and every other technology must come with a disclaimer that it's just a pale imitation and everything has been done before?
Maybe there was some underappreciated pioneer PL/BYTECODE-1968, but in 2023 there's WASM, CIL, Java bytecode (with marketing as large as the dotcom bubble), maybe Lua or eBPF. For practical application it absolutely doesn't matter which one was first or the most humble. What matters is that WASM is common popular standard now. It has easy to embed runtimes and decent language support. The marketing around it is a huge plus too, because I can tell people "use WASM" and it'll be cool and they'll use it, instead of e.g. instinctively reacting "ewwww, Java".
And good luck debugging / make it performant and still not clear how well the memory allocation is handled (in the dll-host situation, often the plugin may use the alloc/free functionality from the host, and give hints, in wasm case - this would be a bit of sandbox)
I'm not saying you need to never use dynamic libraries. Some tools (especially handling/supporting puligins) obviously should make use of dlopen and co.
It's just doesn't make sense in my head why e.g. nano dynamically links against 10 libraries. Lots of pain for little gain.
Latest dynamic linking bug I just encountered is that big picture mode of steam crashes somewhere in libstdc++ when said library is too new but works fine on older versions.
Cross process shared memory is a thing on Unix so there is no need for the performance cost of copying across a pipe. Sure you have to manage if it dies, but you have to manage it crashing you in the dl case.
But you don't have only Linux, you have (and probably first) OSX and/or Windows and then Linux, and you need an API that works well in that case. Being an in-process dll/so plugin, while fraught with perils gets you to avoid other issues (state, health, restart, identity, etc.)
Also sometimes you don't have a choice, but have to make a dll, for example:
(I wish most have used grpc/flatbuffers/whatever to communicate, but then every RPC call have to be checked/retried/handled, and or shared memory well handled, with (?) locks, etc. - not a trivial thing for someone who is deeply specialized in making a very good effect/renderer/etc instead of dealing with this extra complexity on top).
I use a Python script to read the shared memory and generate JSON with this information, then a web page using this JSON is overlaid using OBS, and this way we transmit races on YouTube.
I don't know C#, but it should work similarly. This also means you definitely don't need to use the same language to create and to consume the information.
Ah, so I (the programmer) build object files, send them to the distro maintainers, they link them with their specific versions of the libraries, ands send the resulting executable binary to me/other users.
...that's remote dynamic linking, only with (potentially) human intervention in the loop. I kinda see how it would have worked better if the distro maintainers cared about package quality and speedy updates but they don't (bacause they physically can't, it's way too much effort for way to few people).
Look, I don't argue that dynamic linking has zero problems — heavens know how much I've struggled with glibc but that's mostly because glibc is bloody insane. The whole slew of GNU dynamic-linking-related extensions to ELF is insane, now that I think of it.
The distro maintainers take your source code and package it. If they need to update the program due to an outdated dependency they don't need to recompile the whole program but rather just relink their cached object files with the newer version of the static library (if applicable).
In GNU/Linux, static libraries are precompile object files (.a are nearly indistinguishable from .o files)
Obviously it is not the same at all (LTO, inlining etc.
Indeed in some cases API and ABI breakage requires you to recompile your program from scratch. This is not differnt from dynamic libraries where ABI breakage is only discovered during runtime.
The use case Im talking about tough is your classical 3 bugfix in a library.
Now you have to convince all your software providers to ship those object files. I'm sure it won't be a problem, it's only a really huge change to the entire way in which software packaging and distribution works today...
It's fine as a thought exercise, and might work really well in a restrictive environment (e.g. Google) where you control all the source going into your system. For what most of us call the Real World, it's highly problematic.
Dynamic linking is the shit! It's so undersold. Static linking has been convenient & simple, but it gives up so much awesomeness.
On consumer systems, memory pressure is real, and being able to have quarter your program be effectively free is enormously helpful. Cache hits also go way up, which can be even more vital.
Server side, it depends. If you just run a db server and a monolith, it indeed doesnt matter. But if I look at the hosts we run at my work, if you look at many clusters, they're big nodes and they're running hundreds of processes. If we could get back decent double digit percent of memory that would be great. If caches would get more effective that'd be super. We're leaving a ton on the table because nothing is shared.
There's similar works to win back basic obvious performance happening in a number of places. Composefs has shown up on HN a couple times, trying to let containers effectively share files & page-cache.
Will I deny that dynamic linming has brought pain? No. But I think it's important to recognize that this is a fallback position. If we were better about not breaking conpatibility, not introducing changes, this would be less of an issue. If we had better software testing that could help us detect & see incompatibility faults as they arise, if we could canary test a new software Bill-of-Materials & have some early indicators, we might have, a decade ago, not collectively decided to give up & give in to "binaries just work".
> Just keep the object files around and relink them against the updated version of the static libraries.
This wouldn't work, as extra care needs to be taken to not break the ABI of a shared library. We have various conventions to aid with this including but not limited to "so versions", but these must be kept in mind by the library developer. Object file ABI stability isn't even guaranteed by the compiler.
Also, these days with LTO, the linking step takes a tremenduous amount of time anyway. Doing this wouldn't help much.
Dynamic linking is not the only way to deliver security fixes, just one particular, limited, C-oriented solution. Note that dynamic linking is also not sufficient to change C++ templates or C header-only libraries. It shifts all of the code replacement complexity from the OS (just replace a file) onto the language and library authors (ABI compat, fragile base classes, no inlining or generics).
Rust/Cargo gives you Cargo.lock (and more precise build plans if you want) that can be used to build a database of every binary that needs to be rebuilt to update a certain dependency. Distros could use it identify and update all affected packages. There are also caching solutions to avoid rebuilding everything. It's just something they didn't have to implement before, so naturally distros prefer all new languages to fit the old unbundled-C approach instead.
You ask the vendor, since they need to look after their software anyway. Don't count on closed-source abandonware to conveniently have all vulnerabilities placed in external libraries.
I am talking about apps on the iPhone where Apple does not have access to developers' source code, right? We might be talking about different situations.
Updates of system libraries there are definitely a plus.
However, besides the OS itself Apple doesn't do anything about app security. Not only they don't require unbundling, they don't even support it. Even if you use dynamic libraries in your app, they will be bundled with your app, and never updated separately.
From App Store perspective there's absolutely no difference between a Swift app diligently split into frameworks and a Rust monolith library. They treat both as an opaque bundle.
That's what you would expect from the App Store as well, since why change a dynamic library when the code-controlling developer can just build a new version with his own code when the developer wants to update the app.
Injecting new (the nature of not being available on app-submission) framework versions on the developers' behalf would be scary, since these versions weren't tested with the app because the versions did not exist before. As you said, ABI compat is impossible when no one cares about that much.
I like SDL2's approach to enable both: https://old.reddit.com/r/linux_gaming/comments/1upn39/sdl2_a... A game can statically link the library, but the user can with an environmental variable override the functions to point at their dynamic library instead. Very useful especially with games as you can then keep them running on new platforms or fix certain bugs even though the developers have long since disappeared or moved on.
But then if it works with shared libs, you kind of lose the advantage of the static linking which was mostly (if I understand correctly) that dynamic library was not working well...
It's up to the user if they want to see whether the dynamic library is "working well". They work well for many people! But the statically linked version will be used unless their custom env variable is declared, everyone can be happy.
But they do patch the known security exploits that are likely to be actively used. I'm happier with a security exploit (almost) nobody knows than with a published one that appears in hacking tutorials from 10 years ago.
There are two degrees of separation here though: The software vendors and then the linux distros.
If you sell software that requires your clients to upgrade their system-wide security stack, so they might not. If it is statically linked, no need for them to.
Not convinced at all. Say a big exploit gets published for a very popular library: you probably want to patch it as fast as possible. Whereas Rust will help you have fewer security issues in your own code, which is likely not as popular as some of your critical deps.
I too wonder if it's worth the effort to ship around a trillion versions of different libs (with their trillions of different versions of dependency libs)^n.
One neat thing about it is that we get to override functionality easily, either by replacing entire binaries (such as we do with glide-wrappers to get our 3Dfx fix on modern windows machines) or by using LD_PRELOAD to load hooking SOs to fixup data in old binaries as they are used.
With a "single vendor" providing your stdlib (Apple is a good case in point) dynamic linking is actually better. It is also absolutely no fun whatsoever to recompile your entire hard drive when doing development (which is what Rust work is mostly like, but also Crystal and other "static link everything" approaches). Static linking for a "fast deployable binary for minimum common denominator OS" like the one Go supports is great, but it is great for a usecase (downloadable binaries with narrow functionality) and when your dependency stack is small (no UI, no accessibility, no graphics, no native code).
IMO dynamic linking is rad if you have a known system compatibility table and known vendor (distro) selection for the OS components/key deps. The issue rather is that it is a pain in the butt to configure and understand.
That pretty much sums up what everyone does on Windows. Lots of things are linked dynamically, but apart from the C/C++ runtime library and the OS libraries you just ship all those DLLs with your software.
But this works because in Windows each software is installed in it's own folder, and the search path for dynamic linking starts in the binary's folder. That way you can just dump everything in your installation folder without worrying about comparability with other software. In a Unix or Linux this is much harder to achieve. Sure, you can install into your own folder in /opt and add a wrapper script to load libraries from there, but it's hardly idiomatic.
I did it for 10+ years at my last job, you need a build system that hammers on everything really hard to set the rpath on everything, you shouldn't need wrappers. It definitely isn't idiomatic though.
You assume there’s a philosophy or coherent reasoning behind it, rather than “This is the way we did it with static libraries, so when we adopted shared/dynamic libraries we didn’t change anything else.” Because near as I can tell that’s exactly what happened when BSD and Linux implemented Sun-style .so support in the early 1990s, and there hasn’t been any attempt to rethink anything since then.
Probably because the purpose of the dynamic linker serves the typical O/S layout where there's only one copy of different dynamic libs and everything is linked against those, and packages installed by package managers are authoritative for the things they ship. Distro maintainers want this and lots of system admins expected packages to behave like this.
There's an alternative universe somewhere in which containerization took a different path and Unix distros supported installing blobbier things into /opt, but without (or optionally) the hard container around it. Then fat apps could ship their own deps.
The problem is that there's a lot of pushback from people who want e.g. only one openssl package on the system to manage and it legitimately opens up a security tracking issue where the fat apps have their own security vulns and updates need to get pushed through those channels. It was more important to us though to be able to push a modern ruby language out to e.g. CentOS5, so that work was more than an acceptable tradeoff.
Containerization of course has exactly the same problem, and static compilation probably just hides the problem unless security scanners these days can identify statically compiled vulnerable versions of libraries.
I need to look at NixOS and see if it supports stuff like multiple different versions of interpreted languages like ruby/python/etc linking aginst multiple different installed versions of e.g. openssl 1.x/3.x properly. That would be even better than just fat apps shoved in /opt, but requires a complete rethink of package management to properly support N different versions of packages being installed into properly versioned paths (where `alternatives` is a hugely insufficient hack).
> and static compilation probably just hides the problem unless security scanners these days can identify statically compiled vulnerable versions of libraries
Some scanners like trivy [1] can scan statically compiled binaries, provided they include dependency version information (I think go does this on its own, for rust there's [2], not sure about other languages).
It also looks into your containers.
The problem is what to do when it finds a vulnerability. In a fat app with dynamic linking you could exchange the offending library, check that this doesn't break anything for your use case, and be on your way. But with static linking you need to compile a new version, or get whoever can build it to compile a new version. Which seems to be a major drawback of discouraging fat apps.
Indeed, containers and static linking are just hiding the problem.
> each software is installed in it's own folder, and the search path for dynamic linking starts in the binary's folder.
I think the benefit of this is that an app can be fat but doesn't have to. And an app can be made fat afterwards if need be. The app folder is just the starting point for searching. If the library is not there it is probably shared.
Wrappers and a build system that tweaks everything feel like a hack, not a system wide solution.
There is also Gobo Linux. I wonder if they solved this.
It is done on Linux, for most large popular software. Blender, Firefox, VSCode all are distributed this way. The reason it's not done more is probably some combination of culture and tooling.
Interesting suggestion - basically treating a statically linked binary as a cache which is invalidated when any of the dependencies is updated, triggering either proactive or reactive relinking.
The disadvantage vs. dynamic linking is that you can't easily replace libraries on the fly. It is less flexible but it might actually be better for security. Dynamic linking could be emulated, albeit slowly, by relinking each time a program is launched. This would be slower but potentially useful for certain purposes such as debugging.
The vast majority of libraries on your system are used by only one program. I'd imagine dynamic linking also freezes progress and improvements on libraries because it's extremely difficult to roll out changes without breaking packages that haven't been tested and updated for them.
That post keeps being linked in these discussions but it's highly misleading.
> Do your installed programs share dynamic libraries?
> Findings: not really
The posted findings actually show that common libraries ranging from libc to libX11 are used by high-hundreds to thousands of binaries. The fact that there are also a lot of not-widely-shared dynamic libraries doesn't seem particularly significant.
> Wouldn't statically linked executables be huge?
> Findings: not really
> On average, dynamically linked executables use only 4.6% of the symbols on offer from their dependencies. A good linker will remove unused symbols.
This analysis is completely wrong because it ignores the effect of transitive dependencies. If all exported symbols are independent and don't depend on anything else, then using 4.6% of the symbols means the linker can strip the library down to 4.6% of the size. Simple libraries like libc or libm are pretty close to that ideal. But for more complex libraries, the exported symbols are just the entry points for a deep nest of interlinked functions, and often a single symbol will transitively depend on a large fraction of the library.
> This analysis is completely wrong because it ignores the effect of transitive dependencies.
I hear your criticism, and I'd love to see the corrected results for this. My instinct is that most fully statically linked binaries wouldn't be that much bigger than their dynamically linked counterparts even with transitive inclusion. But I don't know for sure.
Honestly I'm surprised its not easier to run these experiments. If dyld can traverse the call graph through dynamic libraries, why can't it just output the result as a single statically linked binary? I don't know how useful this would be - and it would miss out on inlining & optimization opportunities. But it'd be great for these sort of questions.
At least with ELF, shared libraries are organized into segments and you really only have a handful of them. The information about where individual function boundaries are has been thrown away and in principle this can matter, e.g. if cold code was moved away from hot code. Therefore, the linker can't just split the segments apart.
As for your instinct, it'd be wildly wrong depending on the use case. Maybe if you only link with libc, sure. But now consider linking against something like a GUI framework...
We run hundreds of processes on big nodes at work & we absolutely would save hella hella memory & have way better cache utilization if we could share the massive massive massive reams of code each process duplicates for itself.
Static linking is basically like running Electron. You take a wonderful multi-machine capable environment then use it to run a single thing, exacerbating the size of what should be minor overhead.
As a Linux user, there's like 30 dependencies used by basically half of what I run. If you click in to the article & look at the first chart, it shows this clearly: it tapers down quickly yes but so what? What actually matters? There are a good solid chunk of core libraries used en volume. You dont save much disk space, yes. But many libraries are used by many-hundreds of consumers. Thats where the pain is, that's the loss, that's where you are burning memory space again and again and again with every process you load, and worse, is the cache you are contesting for no reason. You could share that. You should. Giving it up is only because someone has decided it's too hard & too difficult to try,.. and sometimes they are right. But most Linux distros have been working through & managing it & doing ok for years.
>We run hundreds of processes on big nodes at work & we absolutely would save hella hella memory & have way better cache utilization if we could share the massive massive massive reams of code each process duplicates for itself.
Assuming those hundreds are mostly duplicates why can't they all share their entire code section?
There's ~60 different things we run. It would be tempting to try to make some kind of a group scheduler. But on the other hand, we get some nice load levelling/load-averaging from having mixed processes. Ultimately there's not really any reason for us to have static linking, but it's just "how it's done" now and it's unquestioned and there aren't as many well established alternatives as there ought to be.
> The vast majority of libraries on your system are used by only one program.
I find this is a Linux-ism rather than the common case on Windows or macOS. With Windows and macOS a given version of the OS is a fixed set of libraries and services available for third party software. Even third party libraries tend to call into the system libraries for some things. On Linux the only fixed facility of the OS is going to be kernel syscalls since even the libc can differ between distros.
I think dynamic libraries which are only used by a single program are even more common on windows and macos.
Look at how many DLLs ship with applications, and get installed inside the Program Files/MyApp directories! (Seriously - do a search for .dll files there. They're everywhere). Practically all of those DLLs are only be used by a single program. In each case, the price is being paid for dynamic linking (in startup time and the memory cost from not having dead code elimination). But with none of the benefits.
MacOS and iOS apps have the same problem - but I think the modern apple security policies make it even worse. As far as I know there's no way for a mac application bundle to install a shared library on macos or ios which is made available to other applications. The "Drag the .app to the Applications folder" requires that every application ships with its own set of "shared libraries". Outside of the shared libraries provided by the operating system, why would applications ever use shared libraries when they won't ever be shared anyway?
(If its not clear, I'm using the terms "shared library" and "dynamic library" interchangeably here because they mean the same thing.)
On Apple’s platforms the primary motivator for sharing code within an application is that an application isn’t just one component. You may have an application that has any number of app extensions in it that provide access to its features in various settings; if you build that functionality into a framework and use it from the app itself and any app extensions, it’ll still only be loaded into memory once even though there might be several processes mapping it.
Creating frameworks and libraries also helps the developer break down software into more testable components, which doesn’t necessarily require them to be dynamic but if you do go that route you remove an axis of variance that can result in different behavior.
For example, a static library on UNIX is just an indexed archive of object files, so it doesn’t have a single overall set of imports and exports or a single set of initializers to run at load, whereas a dynamic library is a fully-linked module with its own set of dependencies, imports, exports, and initializers. If you assume equivalence between them you’re eventually going to be wrong, often in hard-to-diagnose ways.
iOS developers have the choice of bundling shared code into a static library, or a dynamic framework, and the tradeoffs between them has historically been a little hard to determine, and have also been a moving target.
Some reasons to choose a dynamic library, even though it’s only ever consumed by a single app bundle:
* Shared code between apps embedded in the same bundle (see sibling comment)
* Some apps may want to load portions of code on-demand with dlopen()
* Dependencies / duplicate symbols. Common example is an analytics static library that contains sqlite, and either the main app or another library also linking sqlite, and getting link warnings about “duplicate symbols, which one is used is undefined”. I think this isn’t necessarily a static / dynamic library issue, but 3rd party static libs were often a “single file with all dependencies” and didn’t always understand why that was a problem.
* for a long time, it was the cleanest way to provide & consume headers & object file when distributing closed-source 3rd party libraries.
There may be more, I’m fuzzy on some of the details. If I remember correctly, some of the reasons could be considered tooling issues: creating a Framework was easier for some cases, even if you didn’t want/need the dynamic linking inherent in that choice.
This is a somewhat myopic take on the value of the dynamic linking.
One important aspect is that a dynamic library provides a host of implementations that have been tailored or optimised for specific subsets of CPU's or specific generational CPU features. Let's take «memcpy» as an example that is taken for granted. «memcpy» is defined as weak symbol in the libc.so or similar, and the dynamic linker will replace it with a version that has been optimised for the CPU flavour and CPU flavour's features that have been detected at the runtime or replace it with a less performant, default generic implementation if the dynamic linker does not recognise or support a particular CPU variety. Remember how «memcpy» got an instant manyfold speedup when the AVX-512 optimised version appeared? Yep, all dynamically linked binaries got an instant speedup without even noticing the change. The same will happen (likely, it has already happened) for aarch64 SVE2 vector instructions, and it will happen for RISC-V 64 bit once the vector extension is ratified etc. Doing away with the dynamic linking will slow the adoption of new CPU features down substantially.
If you do away with the dynamic linking, you will not be able to troubleshoot or debug a failing app by substituting the default «malloc» with a debug (or a more performant) implementation via LD_PRELOAD. The Boehm garbage collector was widely used this way (whilst it was being maintained) to overload «malloc» and «free» to successfully run leaky apps that would run out of memory without the GC.
Then there are linking times. Linking Chrome and KDE binaries has been reported to take over an hour and require vast amounts of disk space. We now have «sold» and «mold» so it might be less of an issue but they are not a mainstream linkers yet. Updating a single dynamic library is undisputably faster than relinking 100k binaries that use it. 99.999% of end users will never bother with the relinking anyway.
Naturally, you could claim that the same can be accomplished by fiddling with link maps supplied alongside of object files, but… the link maps are hostile to developers and require a large body of the low-level fringe knowledge that libc authors have taken the burden of. The link maps would have to be very detailed thus very large and also require every developer to have an advanced degree in witchcraft. That would hamper software development efforts for nearly everyone.
Which is not to mention that distributing object files has been tried before, and it has never caught on especially amongst commercial vendors for a number of mostly non-technical reasons. The amount of IP that can be extracted from an object file and the ease of extraction has made the vendors unwilling to distribute anything that is not a final binary product and balk at the idea of it.
You could make a stronger case with suggestion with the OS/400 style of the static binary translation (or AOT) which is functionally combines benefits of object files and some dynamic library features (with the exception of LD_PRELOAD that would have to be replaced with a separate AOT run to «re-link» the final binary product with whatever version of «malloc» / «memcpy» / etc is required), and LLVM has tried that with Bitcode, but for a reason unclear to me, Bitcode seems to have been fading away.
In (open)SUSE we also use delta RPMs. They work in a slightly odd way: The CPIO archive of the installed RPM is is rebuilt based on the installed files, then a binary diff is applied to create the new RPM. That is then installed. (ref: https://github.com/rpm-software-management/deltarpm)
It saves a lot of bandwidth, but takes a huge amount of local resources to install them so I just disable deltas everywhere.
It's much better to just statically link your binaries (unfortunately many programs on GNU/Linux do not support this).
Before the security guys come out the woods: No you don't need to rebuilt all programs linked against your dynamic library of choice. Just keep the object files around and relink them against the updated version of the static libraries. If now package manager just allowed differential binary updates (actually why is this trivial optimization not a common thing?) the overhead in terms of download size is not that of a big deal too.
The value you gain is immense: binaries just work (modulo linux ABI issues but they are doing quite a good job at keeping this interface stable)