Re-master your flash installation

[kl522]

I just tried out expanding knoppix-data.img to 8.5 GB, and that works well on ext3. I place the Knoppix package on an ext3 partition and run from there, starting from USB with the fromhd= cheatcode. I'm not sure if there will be some kind of performance problems with large loop-mounted images, but I haven't seen any so far. So I will try with an even larger persistent store, allowing me to do more comprehensive installs and compile large programs.

I have used ext3 persistent store for quite sometime now. The only performance problem with large ext3 and ext2 file system ( not specifically to loop-mounted ones ) is when you have to do file system check. The file system check time is really slow when compared to ext4.

[Capricorny]

... The file system check time is really slow when compared to ext4.

I use a 16.5 GB ext2 persistent store, for testing now, seems like checking takes some time.. But, it seems to work. Full system upgrade went well. Which is not obvious, as, for example, I found that dpkg halts in updates on one missing newline in one file list - regardless of dependencies. Now is the time for purging/pruning. Is there any available hinting for this? The packages list, like in http://ftp.knoppix.nl/os/Linux/distr...ckages-dvd.txt doesn't give too much clue, it seems. Is there any way to pick out the games here? Here are some deletion candidates among the 40+ MB packages. Are any of these strictly necessary or very useful?

dpkg-query -W --showformat='${Installed-Size} ${Package}\n' | sort -n ...... 
....
42884  context 
44452  wireshark-common   
45644  lmodern 
46912 kdewallpapers  
49972 gnome-games-data  
51448 scribus-ng  
52616 kde-l10n-es  
59692 evolution-common  
60760 pinyin-database  
62844 openvas-plugins-dfsg  
65260 tuxpaint-stamps-default  
68852 kdebase-workspace-data  
85476 mingw32  
85972 inkscape  
86044 neverball-data  
98064 gcompris-data  
101889 linux-image-2.6.37-64  
120892 wesnoth-1.8-data  
136684 wesnoth-1.8-music  
203404 crossfire-maps

In particular, identifying game packages in a simple way would be nice.

[kl522]

I use a 16.5 GB ext2 persistent store, for testing now, seems like checking takes some time.. But, it seems to work. Full system upgrade went well.

16.5 GB persistent store is considered a huge persistent store but it is still probably not near to a point where file system check is unbearably slow. I used to have a 130 G ext3 file system which file system check takes unbearably long time to complete. I have since migrated to ext4. I am glad that I did. 130 GB is already slow, I can't imagine having a 1 TB ext3 file system !

As I am writing, I did a small mod to my minirt.gz to support ext4. I think it is about time it should do that.

Cheers.

[Capricorny]

130 GB is already slow, I can't imagine having a 1 TB ext3 file system !

As I am writing, I did a small mod to my minirt.gz to support ext4. I think it is about time it should do that.

Very interesting if we could use ext4 - but I found that time-wise, 8GB persistent is quite OK. And I really don't think we need more when remastering. We definitely need more than 4 for upgrades and new installs, but using 8GB after remastering, we can do an awful lot of things.

Very much to my surprise, DVD remastering, mostly following Forester's procedure, succeeded at the first try. Of course not perfectly, as I tried out alternatives and got a few things somewhat wrong. And the compressed KNOPPIX image was only 3.2 GB, with Oracle XE 10g and a lot of R packages installed. Removed a few games etc - there seems to be ample room for things like vmware workstation and a couple of servers - in fact, the USB VFAT stick concept could be enough for most uses.

I had 4GB RAM and 10 GB swap - clearly enough. And, according to top, the I5-430M ran at 370% CPU during compression, indicating that create_compressed_fs parallelized nicely.

[kl522]

Very interesting if we could use ext4 - but I found that time-wise, 8GB persistent is quite OK. And I really don't think we need more when remastering.

True and fully agree.

However there is another subtle need to have minirt.gz supporting ext4, ie when the persistent store itself is a file sitting on an ext4 filesystem. Right now this configuration is not supported, one cannot have a poor-man install on an ext4 file system.

[Capricorny]

However there is another subtle need to have minirt.gz supporting ext4, ie when the persistent store itself is a file sitting on an ext4 filesystem. Right now this configuration is not supported, one cannot have a poor-man install on an ext4 file system.

That is a rather serious limitation for me. The hardware and file system agnosticism of Knoppix is one of the great advantages, making many system details mostly an efficiency issue.

And the need for efficient 100+ GB volumes is clear - for many, if not most, users.

Would it really be enough to tweak minirt.gz for Knoppix - because support is integrated otherwise?

I have a couple more questions. First, I noticed that the ISO9660 standard was frequently violated when creating the iso file system. Some file names got changed too - what is the expected effect of this? Anything to do about it?

Second, I come back to the cloop issue. Is there anything new on this front? Seems to me that for our special purpose, we don't need the particular features of cloop (being file system agnostic maybe the most important contrast to squashfs, which is a file system?). I am going to make some comments on this in the squashfs thread.

[kl522]

I am going to make some comments on this in the squashfs thread.

By all means please go ahead to do it. However I will not be joining you there. I better keep my mouth shut on this topic. Everything I want to say about squashfs is already said. I have no rights whatsoever to comment on the choice of it, only the author can decide on the choice. Wow, this is indeed like trying to meddle into people's choice of religion. No way a third person can make any comments on your choice of religion, don't you think so ?

[kl522]

That is a rather serious limitation for me. The hardware and file system agnosticism of Knoppix is one of the great advantages, making many system details mostly an efficiency issue.

And the need for efficient 100+ GB volumes is clear - for many, if not most, users.

Would it really be enough to tweak minirt.gz for Knoppix - because support is integrated otherwise?

Since I am not the author I could only ***SPECULATE*** why this is not supported. For the author, it is trial to change minirt.gz to support ext4 and yet it is not supported probably :-

1. The author does not believe in ext4, thinking it might be unsafe.
( Well, this is unlikely to be true because even the much more dangerous configuration of poor-man install on NTFS is at the least "installable" ).

2. The code for support for various file system has long been written and nobody has provided the feedback for the need for ext4 or rather, the author is distant enough to be able to hear about the need for ext4 from anyone yet.
( This is highly likely because of the way KNOPPIX development has been taking it's course all the years - a solo project ).