Debian User Forums

Why is the root mounted on an overlay file system?

This is the content of the overlay folder from a live USB:
A live Debian USB, with or without persistence, is run via an overlay filesystem by overlaying an "rw" sub-folder on the filesystem.squashfs. Without specifying the persistence Linux boot parameter (and the corresponding persistence partition or persistence file), the rw sub-folder will be created in RAM and will go puffed when the system is turned off, Of course you don't have to use overlayfs but you will have to recompile the default Debian Linux kernel.

Recently there have been a number of so-called "immutable" Linux distros that came out, including: Fedora Silverblue, Vanilla OS, NixOS and perhaps a number of others that I am not aware of. Now the Arch-based Manjaro Linux is also developing its own immutable variant.

An immutable OS has its obvious advantages. However, it also has its own inconveniences, i.e., the changes including upgrades made to your system won't get saved. For Bookworm, since it is very stable, this is usually not a problem. For Sid, on the other hand, since it's being upgraded almost on a daily basis, this can be a pita without persistence. There is a way to create an immutable Sid that can be conveniently upgraded without resorting to a persistence partition. I will try to discuss that when time permits. For many of us who are not code contributors, working on Sid (at least on a part-time basis) and identifying and reporting bugs is probably one of the best avenues to contribute back to the great Debian ecosystem.

pwzhangzz wrote: 2024-08-10 10:33There is a way to create an immutable Sid that can be conveniently upgraded without resorting to a persistence partition.

To create an immutable Sid, you must first create a Testing (now corresponding to Trixie) Live USB with persistence, upgrading it to Sid (plus other customizations to suit individual needs), then merging the persistence with the existing filesystem.squashfs file to create a new filesystem.squashfs file. This becomes a new immutable Sid.

To create an immutable Sid with persistence but without the requisite persistence partition(s), all you need to do is to run the following command right before you power off the live session:
Here /extra0 is an external partition that I have created to store things that I don't want to be in the live iso. It can be encrypted or in another hd/ssd, etc. When you restart a new live Sid session. simply copy the rw folder back to RAM to recover persistence:
Of course you don't have to (& you shouldn't) run this (especially the first step) more frequently than you have to. This is the beauty. If anything goes wrong, then it never happens. "Never happens" is much better than "going back in time" where something unexpected can (& will) always happen.

Adding a little humor here . Sid is called "unstable" because it's being constantly upgraded sometimes more than on a daily basis (thus "unstable in existence not necessarily in operation). But when you make it "immutable", it is actually more stable than "stable". This is not a joke. I love Sid.

Recently I had a whirlwind tour of Europe visiting about a dozen client offices. I am no firewall expert but I know enough to know that I cannot blindly trust any firewall. An immutable system comes in very handy, I don't have to wake up during the mid of the night worrying that someone might have planted a virus into my hd. For frequent business travelers, an immutable OS is almost tantamount to a utopia.

Another advantage for business travelers is that the immutable Debian can be created in a portable USB stick. If in the unthinkable and most unfortunate moment that you lose your laptop, you can always go to a computer store, grub a computer (make sure that it does not have a Nvidia GPU), plug in your immutable Debian equipped USB, and you are right back in business. No one will notice any difference. Caveat. According to the US laws, a computer is a capital budget item, it cannot be expensed but must be depreciated over 5 years. Thus, the computer spec preferably should be fairly modern. Bookworm may have problem with relatively new hardware, but Sid usually works great, again, as long as it does not have Nvidia GPU. But business machines seldomly, if ever, need discrete GPUs. You can also try to borrow a computer from your client & and proceed as if nothing had happened (which will make your presentation very impressive). Sid also works great with old hardware.

pwzhangzz wrote: 2024-08-21 15:45

Code: Select all

sudo rsync -av --ignore-errors --delete /run/live/overlay/rw/ /extra0/rw

Code: Select all

sudo rsync -av /extra0/rw/  / 

In practice, we often exclude google chrome cache and config files from persistence:
The second exclude=google-chrome/ parameter will exclude both ~/.config/google-chrome and ~/.cache/google-chrome folders from retained persistence. This is one of the advantages that a "portable" (for lack of a better word) persistence is better than using a fixed persistence partition. If not managed properly, these folders can grow to multiple gigabytes. But more important, some of our older male students only trust using their immutable system to watch internet porns lest it be found out by their kids. This is the main reason that keeps them from using Windows or Apple. Keeping google caches in persistence would defeat this purpose*.

Using "portable" persistences (again for lack of a better term) has other advantages. For example, as a GNOME user, I always wanted to try KDE/Plasma. After booting from the immutable Sid (or Bookworm), I can easily do the requisite apt install and switch the DE from GNOME to KDE (or any other DE), then save the /run/live/overlay/rw folder as /extra0/rw.KDE or something like that. Next time when I want to see how much KDE has improved, I can simply rsync the rw.KDE folder to the RAM or to the fixed persistence partition. But, as our older but wiser students manifested, the biggest advantage is the peace of mind. Especially when using Sid.

* Of course, with a "fixed" persistence partition, you can always erase (or re-shuffle) the /rw folder so that the cached stuff won't be retained during the next boot. True to the Debian philosophy, the immutable Debian discussed in this thread is extremely flexible. Also unlike the Fedora Silverwhatever, the immutable option of Debian can be run exactly like a conventionally installed system. You don't have to learn new things. Thus less chances of possible messed ups.

After running Silverblue as a retiree in a home environment for the past 6 months, I thought the snapshots was the holy grail to revert from bad updates . The problems I see with Silverblue which would apply to all immutable cores, is that containerized packages are always going going to be restricted, and in my case downloading Evolution (flatpak) did not pull in the gspell dependencies which depnds on enchant, which depends on hunspell, Flatpak use it's own private local storage for things like the message database and cached messages, so my only real option was to layer Evolution and all its dependencies on the lib-ostree, something developers frown upon as it makes the system unstable when there are updates. If they address this containerized sandboxing issue and allow those packages root access to the degree it is needed to function properly, then I guess we could welcome immutable distros, as it's what SteamOS, Android and macOS are to some degree already, and many more testing such as openSUSE's Aeon. I can understand immutable catching on in mass-deployment corporate situations and information kiosks, but I also see a place for a virtually unbreakable system with home users, since if the new image encounters a problem, it automatically reverts to the previously working snapshot.

Maybe this is not an aspect of immutable that you were considering, and it is relatively easy to resolve if the will is there to engage the containerized packagers in the conversation, but it is also the reason I am now a happy Debian user with Evolution flathub with separate BTRFS subvolumes and snapshot submenu on the grub menu, so I have all of those benefits without the immutability. So I have no plans to return to Silverblue as I have everything I want from Debian.

distro-nix wrote: 2024-09-16 00:32 After running Silverblue as a retiree in a home environment for the past 6 months

As I mentioned in this thread, we have a pro bono project to help retired professionals to more productively use their computers. Unlike Your Honorable ( ), our students are only interested in using their computers as an appliance and have zero appetite in doing--or allowing us to do--anything below the surface. Immutability is probably the only reason they are still willing to stick with Linux (Debian). Of course, the unfortunate fact that our live usb is nowhere near good (usable) enough bears the entire blame*. Debian is a great system.

Fedora Bluewhatever or other immutability-minded distros actually do not provide the same level of immutability as I mentioned here. Plus they add an extra layer of complexity than Linux already has. We need simplicity, not additional complexity.

The following is an example of the grub.cfg file that we are using for booting an immutable Sid* live from a usb stick:
The mult-boot config allows the live-usb creator to iteratively adjust the content of the immutable Sid to suit the target user(s). End users only need the first menuentry thus making this live usb extremely simple. Similar configuration (with slight modifications) can be appended to the grub multiple-boot structure of an hd to emulate an immutable but also iteratively adjustable live Sid "usb" without needing an actual usb.

* One of the reasons that I used Sid in this example is that our students typically purchased a new computer when they started the class--or are already owners of a new Windows PC but don't know how to use it. Of course my personal preference has nothing to do with this decision. For most of the Debian experts on this forum, Bookworm, because of its maturity, probably would make a better candidate for use as the starting point of an immutable OS.

The size of the remasterized Sid iso is actually smaller than the original (Debian Testing) iso:
In preparing the remasterized Sid iso, we have added more than 20 packages. But the biggest "offenders" were the bunches of config and cache files for google chrome (totaling about 800 MB, uncompressed). The size of the remasterized iso can be further reduced. But because of our very limited resources, we can only be interested in a workable/tolerable immutable Debian-based OS with bare minimum functionalities, definitely not an optimized one.

As shown above, the Sid live iso has been updated twice. Typically we upgrade the Sid live iso about once a month after it was created from the Testing live iso. We are doing this for free, but Debian does not prohibit potential entrepreneurs from charging service fees.

You can also add VirtualBox or Virt Manager in the live usb and install Windows 11 VM on the same disk but a different partition. This is probably the easiest way to create a portable Windows usb. The Debian system is immutable, the Windows is not, but is customized and can be put in hibernation when not in use (which is most of the time ) thus minimizing potential troubles and releasing resources.

During the Pandemic, many schools used Fed COVID funds to purchase Chromebooks for their students to do on-line classes at home. The main reason was that, unlike Windows, Chromebooks are basically immutable making them much easier to administer. I hired a couple of high school kids to try our immutable Sid (at that time corresponding to the future Bullseye), and they actually liked Debian better than Chromebooks. Most of Chromebook functions (at least those that are typically used by the students) can be duplicated with the Google Chrome browser, which was made part of the Debian live iso, but a full blown Debian OS, even in the form of an immutable live usb stick, is much much more powerful and flexible--and more fun to use--than a Chromebook. Later on, it was discovered that those Chromebooks were not upgradable (I have no idea what that means), thus creating a huge e-waste problem. Don't know whether that problem was resolved. But with the immutable Debian usb, it can be easily upgraded or even discarded (the usb stick can be re-used for other purposes). Totally friendly to the environment. It is also not tied to any particular hardware.

Walmart has an ASUS Vivobook Go 15.6 inch laptop on sale for $250 (the price has since gone up to $300 but it can fluctuate) with the following specs:
I am not making any comment one way or another but just wanted to point out that this Windows-pre-installed laptop has specs that are better than most chromebooks at actually a lower price (& bigger screen!). There are two major "problems" with this low-priced laptop. First, it is (from my experience) always loaded with tons of adware. BestBuy typically charges $79 to remove these adware & may not do a complete job. But these adware only work in Windows, if you are running Debian, they won't bother you at all. Second, it only has 256GB ssd. But again if you are running an immutable Debian usb (with or without persistence), hd size really doesn't matter. A 256GB Samsung compact flash drive (at 400MB/s) costs less than $25 & the price keeps falling.

pwzhangzz wrote: 2024-10-05 16:33 I hired a couple of high school kids to try our immutable Sid (at that time corresponding to the future Bullseye), and they actually liked Debian better than Chromebooks.

Inkscape v 1.4 was released today with appimage being the "preferred" format for Linux users:

https://inkscape.org/release/inkscape-1.4/

I have been trying (though in vain) to learn Inkscape since its inception. But because of the relatively steep learning curve and its cluttered interface, never went anywhere. However, kids don't seem to have any problem mastering it, and they actually love the convoluted (to me anyway) interface. As I mentioned previously, our immutable usb extensively uses appimages for major apps. Appimages such as those for LibreOffice, GIMP, Google Chrome, Krita, Inkscape, VLC, and various other video/audio editing apps are stored in an external partition which is mounted to the live iso during boot. This keeps the live iso small. but more importantly, they can be easily added and upgraded without going through the remasterization steps. Inkscape 1.4 is no exception:

pwzhangzz wrote: 2024-10-14 18:51As I mentioned previously, our immutable usb extensively uses appimages for major apps. Appimages such as those for LibreOffice, GIMP, Google Chrome, Krita, Inkscape, VLC, and various other video/audio editing apps are stored in an external partition which is mounted to the live iso during boot. This keeps the live iso small. but more importantly, they can be easily added and upgraded without going through the remasterization steps. Inkscape 1.4 is no exception:

#screenshot showing Inkscape-1.4#

nowadays, no kids will allow you to install Linux in their PC. No one is interested. The only way to get them to pay some attention is through a bootable usb that is fully self-explanatory and which can be put into productive use immediately (like 5 minutes). The live usb must also include a handful of exciting apps.

Many years ago, one of our students brought his daughter to the class. We gave her a usb stick which has Krita installed as part of the persistence (at that time running Ubuntu). Just like when we became self-taught and self-proclaimed Linux experts ( ), she sort of became an expert in Krita entirely on her own. Eventually she was charging $100 a pop to do digital portraits from photos (she was too young to work at the McDonald's).

As our knowledge improved, we have done away with the persistent usbs--because they are not immutable. Relatively wide availability of appimages made this possible. The following is a screenshot of Krita 5.2.6:

This.Is.Awesome.

@Best_Threads

One of the advantages of running an immutable Debian is that there is no fear to try new apps (the only way to move Debian desktops forward, imnsho). The following is a screenshot of running GIMP 3.0 RC1 in Sid:


Time to start getting acquainted with the new GIMP!

The application where I could best visualize an immutable flavor od Debian or any other distribution is in either an education or cooperate setting where you need the ability to do mass deployments of the same image of said flavor of Linux in a timely fashion.

Another case specific application in which an immutable configuration would be of great advantage would be in a distribution targeted towards a pacific subset oi users for instance Bazite or SteaMos which are both distributions aimed at gamers.

One of the critical elements missing from a professionally competent Linux desktop/workstation is a workable CAD program. Last month, FreeCAD finally released its 1.0 version (I think the previous version was 0.19):
Hope this means something. The 1.0 version of FreeCAD can be run, externally via its appimage, from an immutable/portable Sid/Bookworm usb stick. This allows students to be more willing to casually give it a try. (Although FreeCAD comes with a Windows version, most school kids nowadays are savvy enough not to install any unknown program into their computer. Anything goes wrong, even though totally unrelated, YOU will be held responsible!)

This appimage is put together by FreeCAD developers and it provides a convenient way for users to make contributions, including monetary donations*:
* As I mentioned above, both Krita and Inkscape also provide official appimages (as their preferred download format) which came out the same time as their new version release announcement, and both appimages have prominent spaces for donations. I don't know if this is the new trend for doing business in FOSS. But if it is, I am all for it! Tears in my eyes! Way to go!

I use librecad, the version in stable's repo is 2.2.0-alpha

arzgi wrote: 2024-12-09 14:39 I use librecad, the version in stable's repo is 2.2.0-alpha

Thanks for the plug. Will be great if you can teach us some of the tricks? How about (the great) Blender?

Also, there was a thread about backing up cell phone data to Debian (can't find it). Anyone kind enough to share their experience about using GSConnect? Ditto scrcpy?

pwzhangzz wrote: 2024-12-09 16:44

arzgi wrote: 2024-12-09 14:39 I use librecad, the version in stable's repo is 2.2.0-alpha

Thanks for the plug. Will be great if you can teach us some of the tricks? How about (the great) Blender?

Also, there was a thread about backing up cell phone data to Debian (can't find it). Anyone kind enough to share their experience about using GSConnect? Ditto scrcpy?

https://dokuwiki.librecad.org/doku.php/usage:manuals There is libreacad forum, where I grabbed that link. I can recmommend Johny Heikell's LibreCAD for Real Dummies.

I use syncing Plain app from F-droid repo. It just creates a web page, where you can sync every file, also directories, or folders in commercial OSes parlance. Does a lot of other things too, but that would outside the topic.

Blender is great, if you need to do 3d morphing, I used it only once, did the trick.

pwzhangzz wrote: 2024-12-09 01:13most school kids nowadays are savvy enough not to install any unknown program into their computer. Anything goes wrong, even though totally unrelated, YOU will be held responsible!

Twenty-some years ago, one of our interns started a Linux club in his school. The school administrators at that time were progressive enough to allow them to establish a Linux-based server for the school. His work was so impressive that he got a full scholarship from one of the elite east-coast universities. Many of us on this forum probably also started our Linux path when we were students. However, things have mutated. An immutable Debian as I described in this thread can be delivered as an "app" (by conventional definition an "app" should be portable and manageable in size). This is probably the only way to get (average) students willing to try Debian.

CwF wrote: 2024-06-26 04:30

friendlysalmon88 wrote: 2024-06-26 02:52 we need to embrace solutions like Silver Blue and its competitors.

No, we don't, not really

@Cwf I understand your point of contention with my opinion and readily acknowledge that I'm way out of my league to have even opened up my big dumb mouth.