Re-master your flash installation

[Forester]

In any case, the steps above involved will be somewhat tricky to the Linux newbies. I wish there
is a cheatcode called "upgrade=/dev/sda4:/master/KNOPPIX/KNOPPIX" which will automatically
perform the upgrade. I will leave this as an exercise for those interested to do so by modifying minirt.gz.

If you organise thing right, you can use cheat codes that are already there (assuming Knoppix 6.4.3 or equivalent).

Suppose the Knoppix you have just remastered is on filesystem /dev/sdb1 under the directory KNOPPIX. Suppose further that you arranged for the new remastered Knoppix to be on device /dev/sda1 under the directory /home/yourself/remaster. This directory contains both the new KNOPPIX file and the cleaned-up knoppix-data.img (or aes) file. I don't think it has to contain all the other little files normally found in the KNOPPIX directory but it wouldn't hurt.

You can now reboot to the remastered knoppix for test purposes with the cheat codes:

fromhd=/dev/sda1 knoppix_dir=home/yourself/remaster

The directory /home/yourself/remaster has to be a real directory (avoid symbolic links and mount points). A nice feature of this approach is that /dev/sda1 doesn't have to be a vfat file system so long as it is a) visible at boot time and b) of a type the init script understands. This means it could (you didn't get this from me) be your ntfs Windows 7 main disk.

If the new remastered Knoppix seems to be working OK you can use it to copy the new KNOPPIX file over the old one:

mount /dev/sdb1 /media/sdb1
cp -pf /mnt-system/home/yourself/remaster/KNOPPIX /media/sdb1/KNOPPIX/

Then clean up the old knoppix-data.img:

mkdir knoppix_tmp
sudo mount -o loop=/dev/loop1 /media/sdb1/KNOPPIX/knoppix-data.img knoppix_tmp/
cd knoppix_tmp
sudo rm -fr $(ls --ignore=home --ignore=var);

Finally, reboot Knoppix from your USB stick 'as normal'.

[Capricorny]

If you organise thing right, you can use cheat codes that are already there (assuming Knoppix 6.4.3 or equivalent).
....

fromhd=/dev/sda1 knoppix_dir=home/yourself/remaster

The directory /home/yourself/remaster has to be a real directory (avoid symbolic links and mount points). A nice feature of this approach is that /dev/sda1 doesn't have to be a vfat file system so long as it is a) visible at boot time and b) of a type the init script understands. This means it could (you didn't get this from me) be your ntfs Windows 7 main disk.

I wouldn't recommend newbies experimenting with remastering on Win7 NTFS partitions. I think it is much safer to shrink the Win partition on a disk, set up an ext3 partition there, and do everything on "native Linux".

Apart from that, I think the method looks very promising. I will try out something like this myself now, as I feel forced to do it: Upgrading all packages on Knoppix 6.4.4 DVD today, filled up about 2.6 of total 4 GB persistent store. Therefore, people should not attempt it without really in need of it. OR, as a first step in remastering.

I'll start out using internal hard disk, then I may try with USB-sticks - but I think a better external storage for remastering would be an external SSD drive with USB3 connection.

[Capricorny]

To follow up, I think it may be an advantage to do as much as possible before starting the remastering process. That implies doing package installs, upgrades etc the "ordinary" way as much as possible. But how to, within 4GB space? No way there, but we don't have to confine ourselves to FAT32 and DVD limitations.

I just tried out expanding knoppix-data.img to 8.5 GB, and that works well on ext3. I place the Knoppix package on an ext3 partition and run from there, starting from USB with the fromhd= cheatcode. I'm not sure if there will be some kind of performance problems with large loop-mounted images, but I haven't seen any so far. So I will try with an even larger persistent store, allowing me to do more comprehensive installs and compile large programs.

If we, after cleaning up and remastering, get outside the 4GB limits, there are several ways to still use USB sticks as basis. The simplest is to format the stick to ext2/ext3, install (for example) GRUB on it and just dump the Knoppix files to it. Somewhat more elaborate would be to split the compressed packages on two volumes, or work with several uncompressed volumes - that has worked just fine for me.

I think using Linux file systems on sticks is very practical. For exchange of data with other platforms, we can set up a smaller FAT (or even NTFS) partition on the stick, and have that partition automatically mounted on boot-up. Typically, a 16 GB stick could be partitioned into 15 GB ext2 and 1 GB FAT32, the ext2 volume: 4 GB compressed, 10 persistent and 1 spare.

[kl522]

I just tried out expanding knoppix-data.img to 8.5 GB, and that works well on ext3. I place the Knoppix package on an ext3 partition and run from there, starting from USB with the fromhd= cheatcode. I'm not sure if there will be some kind of performance problems with large loop-mounted images, but I haven't seen any so far. So I will try with an even larger persistent store, allowing me to do more comprehensive installs and compile large programs.

I have used ext3 persistent store for quite sometime now. The only performance problem with large ext3 and ext2 file system ( not specifically to loop-mounted ones ) is when you have to do file system check. The file system check time is really slow when compared to ext4.

[Capricorny]

... The file system check time is really slow when compared to ext4.

I use a 16.5 GB ext2 persistent store, for testing now, seems like checking takes some time.. But, it seems to work. Full system upgrade went well. Which is not obvious, as, for example, I found that dpkg halts in updates on one missing newline in one file list - regardless of dependencies. Now is the time for purging/pruning. Is there any available hinting for this? The packages list, like in http://ftp.knoppix.nl/os/Linux/distr...ckages-dvd.txt doesn't give too much clue, it seems. Is there any way to pick out the games here? Here are some deletion candidates among the 40+ MB packages. Are any of these strictly necessary or very useful?

dpkg-query -W --showformat='${Installed-Size} ${Package}\n' | sort -n ...... 
....
42884  context 
44452  wireshark-common   
45644  lmodern 
46912 kdewallpapers  
49972 gnome-games-data  
51448 scribus-ng  
52616 kde-l10n-es  
59692 evolution-common  
60760 pinyin-database  
62844 openvas-plugins-dfsg  
65260 tuxpaint-stamps-default  
68852 kdebase-workspace-data  
85476 mingw32  
85972 inkscape  
86044 neverball-data  
98064 gcompris-data  
101889 linux-image-2.6.37-64  
120892 wesnoth-1.8-data  
136684 wesnoth-1.8-music  
203404 crossfire-maps

In particular, identifying game packages in a simple way would be nice.

[kl522]

I use a 16.5 GB ext2 persistent store, for testing now, seems like checking takes some time.. But, it seems to work. Full system upgrade went well.

16.5 GB persistent store is considered a huge persistent store but it is still probably not near to a point where file system check is unbearably slow. I used to have a 130 G ext3 file system which file system check takes unbearably long time to complete. I have since migrated to ext4. I am glad that I did. 130 GB is already slow, I can't imagine having a 1 TB ext3 file system !

As I am writing, I did a small mod to my minirt.gz to support ext4. I think it is about time it should do that.

Cheers.

[Capricorny]

130 GB is already slow, I can't imagine having a 1 TB ext3 file system !

As I am writing, I did a small mod to my minirt.gz to support ext4. I think it is about time it should do that.

Very interesting if we could use ext4 - but I found that time-wise, 8GB persistent is quite OK. And I really don't think we need more when remastering. We definitely need more than 4 for upgrades and new installs, but using 8GB after remastering, we can do an awful lot of things.

Very much to my surprise, DVD remastering, mostly following Forester's procedure, succeeded at the first try. Of course not perfectly, as I tried out alternatives and got a few things somewhat wrong. And the compressed KNOPPIX image was only 3.2 GB, with Oracle XE 10g and a lot of R packages installed. Removed a few games etc - there seems to be ample room for things like vmware workstation and a couple of servers - in fact, the USB VFAT stick concept could be enough for most uses.

I had 4GB RAM and 10 GB swap - clearly enough. And, according to top, the I5-430M ran at 370% CPU during compression, indicating that create_compressed_fs parallelized nicely.