The title is kind of confusing. If you have already installed knoppix on flash, and you
have enabled a persistent store, meaning you could go ahead with adding, upgrading
and customizing packages, why do you still need to re-master knoppix ?
Reasons as follows :-
1. When you upgrade your files/packages, the original copy resided on the
read-only compressed image is still there. The system exposes you to the ones on
the persistent store, and hides away the ones on the compressed image.
This is a waste of space, if you have upgraded many files/packages, for
every file/package in the persistant store, there is a counterpart in the
compressed image. Re-master will delete the hidden copies.
Even when there is no compressed image counterpart, due to you added some
new interesting packages, by moving your added packages to the compressed
image, you also free up the (uncompressed) space on the persistent store.
2. Another reason to re-master is that you want to maintain a clean separation between
experimental things verses the "stable" portion of your customization, ie you want to
move the stable and proven things to the compressed image, so that you can later
proceed to add your experimental things to the persistent store. In case you
are not happy with the new experimental things you added, you can always erase it
from your persistent store, or even you can wipe out the entire persistence store.
If you have previously re-mastered your flash installation, deleting things from the
persistent store is an easy, clear-cut and safe operation; because even if you delete
everything, you will still have the last snapshot of a fully working, stable and proven,
and yet already fully customized version of knoppix to start with. There is no need
to falling back to original uncustomized knoppix.
The end result of a flash re-master is not a new copy of CD or DVD, as you have already
booted off from flash, CD or DVD is irrelevant. The end result of flash re-master is just a
new copy of compressed image KNOPPIX.
This is how I re-master my flash installation, in brief. Note there is no "chroot" needed :-
Find a directory or a partition where you have a few GB of space :-# mkdir -p /lots_of_free_space/source/KNOPPIX( Whatever you want it to appear in the new compressed image, you copy it there )
# mkdir -p /lots_of_free_space/master/KNOPPIX
# cd /lots_of_free_space/source/KNOPPIX
# cp -a /dev .
# cp -a /UNIONFS/bin .
# cp -a /UNIONFS/etc .
# cp -a /UNIONFS/lib .
# cp -a /UNIONFS/sbin .
# cp -a /UNIONFS/usr .
# cp -a /UNIONFS/var .
# cp -a /UNIONFS/opt .
# cp -a /UNIONFS/home .
# cp -a /UNIONFS/root .
.....
# cd ..
# mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" \
-hide-rr-moved -cache-inodes -pad /lots_of_free_space/source/KNOPPIX \
| nice -5 /usr/bin/create_compressed_fs - 262144 > /lots_of_free_space/master/KNOPPIX/KNOPPIX
After a long time, you will get a compressed image at /lots_of_free_space/master/KNOPPIX/KNOPPIX,
which you can use it to replace your /mnt-system/KNOPPIX/KNOPPIX.
You can see that the steps above are quite simple. This last step of replacing existing KNOPPIX
compressed image must be perform "offline", ie you should perform it when you are not
using /mnt-system/KNOPPIX/KNOPPIX.
There is a challenge, because once you have booted in knoppix, you are already using
/mnt-system/KNOPPIX/KNOPPIX. So either you boot from a CD with your flash detached
and then put the flash back after you have booted so that you can overwrite it, OR you
could configure /mnt-system/boot/syslinux/syslinux.cfg to add the cheatcode "debug"
in it. At the first shell prompt, the /mnt-system/KNOPPIX/KNOPPIX is not mounted yet,
this is when you could manually perform this somewhat "risky" task.
Basically the steps will be roughly like this :-
If your free space which contains the compressed image resides on a separate partition :-# mount /dev/sdaX /lot_of_free_spaceIf using the cheatcode "debug", once done, you type "exit" to proceed with the rest of booting
If you have enough space, you can rename KNOPPIX for backup purposes :-
# mv /mnt-system/KNOPPIX/KNOPPIX /mnt-system/KNOPPIX/KNOPPIX.bak
Now overwrite it with the re-mastered copy :-
# cp /lots_of_free_space/master/KNOPPIX/KNOPPIX /mnt-system/KNOPPIX
You can skip mounting knoppix-data.img by renaming it,
# mv /mnt-system/KNOPPIX/knoppix-data.img /mnt-system/KNOPPI/knoppix-data.img.bak
to verify that you new image is working. You will have to exit 6 times before it will be booted
completely.
If you are happy with it, you can shutdown and boot another time into "debug" shell, this time
you can "restore" your knoppix-data.img and free up the space in there :-
# mv /mnt-system/KNOPPIX/knoppix-data.img.bak /mnt-syste/KNOPPIXknoppix-data.img
# mkdir /tmp/tmp
# mount -o loop /mnt-systsem/KNOPPIX/knoppix-data.img /tmp/tmp
( Now you can delete whichever files you want to delete to free up the space, as
they have already been transferred to the compressed image ).
# cd /tmp/tmp
# rm -rf bin sbin lib etc var usr .......
Once you have finished :-
# cd /
# umount -d /tmp/tmp
Again, if you are using "debug" cheatcode, you have to type 'exit' 6 times before you
will be booted completely with the freed-up persistant store.
In any case, the steps above involved will be somewhat tricky to the Linux newbies. I wish there
is a cheatcode called "upgrade=/dev/sda4:/master/KNOPPIX/KNOPPIX" which will automatically
perform the upgrade. I will leave this as an exercise for those interested to do so by modifying minirt.gz.
Last note: the "debug" cheatcode is a very useful thing. As you can use it to rename/move the
knoppix-data.img ( or knoppix-data.aes ) for the purpose of resizing.
I really appreciate that you share this!
The first rule about remastering is, don't do it. But it's not the last rule...
I would like to postulate a few more good (IMHO) reasons for remastering, I fully agree with yours.
3. Bloat. The DVD contains gigabytes (uncompressed) of programs we will probably never run. While some essentials (every man his own!) are missing.
4. Bugfixes made permanent, moved from "user space".
5. Extra package install/updating made more systematic.
6. Smaller total system footprint. Will always be relevant, if not so important economically as storage gets cheaper.
7. Possible to make several cloop images. Could be useful for things like heavy development tools.
8. Could even try other approaches than through cloop.
I imagine that a good way of doing the full KNOPPIX upgrade, would be to copy the newly made compressed image to, for example, an external USB HD, and do the "field testing" on that. Speaking for myself, I'm pretty sure I would want to make quite a few adjustments.
I created an account just for this tutorial.
I tried your method of remastering as I like the way it works.
But did however made the mistake of trying to do the source/master directory on a ntfs filesystem. this caused all persmissions to be lost while copying and after the remaster nearly everything was broken.
Of course I was going to rage on this tutorial when I realized it maybe couldve been because it was ntfs and took another memorystick and formatted it to an ext2 filesystem. this did the job
is there a way to not break anything while still copying to a ntfs filesystem?
dinosoep wonders ...
Well, yes there is. Have you noticed how Knoppix uses (compressed) loop devices a lot ? You can think of these as filesystems-in-a-file instead of filesystems-on-a-partition. You can use the same trick to create an Linux file system on an ntfs partition.is there a way to not break anything while still copying to a ntfs filesystem?
There are four steps:
1. Mount your ntfs partition under Knoppix (if not already mounted). That will probably be something like:
2. Create a large file on it. How big depends but for remastering the Knoppix CD let's guess 4 Gb is enough. You need something like:Code:mount /media/sda1
3. Create a file system in the big file:Code:dd bs=4K count=1M if=/dev/zero of=/media/sda1/big.one
4. Mount the file as a loop back device:Code:mkfs -t ext3 -F /media/sda1/temp/big.one
You may need to sudo one or more of the commands and, of course, adapt the parameters to your particular situation. The mkfs command takes many parameters and some may be more useful in this situation than others.Code:mount -o loop /media/sda1/temp/big.one /lots_of_free_space
As a moderator, I passed the above post, as it is not spam and on topic.
However, as a user and someone who has been on this site for nearly 8 years and seen over and over again the reports of disaster after writing to NTFS, I couldn't just pass it and not warn others that it is not a universal opinion. Yea, you can create a file system within another file system under Linux. But to do so you still need to write to the physical file system. Not just to create it but during all of the writing processes. I know of no reason to think that this is safer than just writing to NTFS (which is unfortunately still not safe).
If you want another file system on your hard disk consider shrinking the main partition, or use that "back up" partition that the vendor wasted disk space to create (after making a backup disc), and create a second Linux or FAT partition there, or even just add another inexpensive hard disk. An extra partition can be a Linux partition or a FAT partition (A FAT partition will be subjected to the 4GB-1 file size but lets you transfer files easily between Windows and Linux). But if you write to a NTFS partition, have very good backups of everything and keep making backups, the effects of corruption are often not noticed for some time after the damage is done.
---
Verifying of md5 checksum and burning a CD at slow speed are important.
This subject has been mentioned quite many times. 8 years of observation is quite a long time, especially in the world of IT. I am not sure how extensive your observation is substantiated with facts, as software version changes, some bad experience in the past with bugged software may not be applicable to newer versions of software.
The fact is over the recent years, the use of NTFS (for read and write access ) in Linux has got wide spread usage and penetration. If you look at embedded devices, quite many of them are Linux based, they read/write usb flashes and SATA harddisk. I came to know quite many of the media players are Linux based. I also know that Samsung TV is Linux-based too. These devices read/write on NTFS file system (and FAT32) !
I cannot help but to think that read/write of NTFS on Linux has gotten to a usable degree of safety - if one does not use those fancy features of NTFS. But if indeed your observation - substantiated with facts - does indicate that NTFS writing is still unsafe, perhaps it is a problem unique to Knoppix ? I am wondering ......
I think the creation of an image file, organizing a file system on it and loop-mounting that file system, may be relatively safe on NTFS. But, as NTFS is a rather swiftly and erratically moving object, I can't really understand how anyone can be quite sure everything will always go well. Even if everything works perfectly today, there is a chance that the next service pack, or even bugfix patch, may introduce problems. Therefore, I like to make a distinction between simple tasks like creating, updating and reading an image file, and more general file system use. In particular, if all reading/writing to the image file is to be done under Linux, one may mostly avoid inconsistencies etc between Windows and Linux NTFS implementations.
I agree with you that creation of an image file, organizing a file system on it and loop-mounting that file system is considered relatively safe - with a catch I shall explain below.
I was reading the ntfs-3g write up, they mentioned this in the Wiki :-
From what I read, if you want to use NTFS-3g to fix access control list and permission on a Windows system, you might not succeed because of the catchy support of ACL and permission. However, if you want to implement what Knoppix needed, ie compressed loop mount a file on NTFS, it would appear to be safe.NTFS-3G supports all operations for writing files: files of any size can be created, modified, renamed, moved, or deleted on NTFS partitions. Transparent compression is supported, but there is no support for encryption. Support to modify access control lists and permissions is available.
On the other hand ......
I did mention before for Knoppix 6.2.1 ( which I did not check if it is still valid after that ), it is in my humble opinion that knoppix does not shutdown the persistent file system properly, more so for FUSE. If you look at the shutdown script, basically it did not manage to unmount the UNIONFS/aufs2. The FUSE user space driver is terminated before the persistent file system got unmounted. If FUSE keeps buffer, the buffer content is it flushed ? I am not sure. Specifically I noticed when using vanilla Knoppix 6.2.1 with a ext2 persistent store loop mounted on NTFS, I noticed the system had to fsck the persistent store, and each time there is inconsistencies and it requires manual intervention. That was Knoppix 6.2.1 anyway .....
sorry for replying this late, I hoped that knoppix forums would sent me a mail when someone reacted.
this was exactly what I was looking for
I have been writing from linux mint to a ntfs partition a lot and I got no problems whatsoever so I think writing is getting more and more stable to ntfs
Vintage Apple Macintosh 6100/66 PowerPC 601 66MHz 8MB Retro Desktop Computer PC
$189.99
Vintage Apple Macintosh IIci Computer No Power For Parts and Repair No OS No HDD
$159.95
Vintage Apple IIe Computer A2S2128 (825-1351-A)
$174.99
Apple III Monitor A3M0039 - Rare Vintage Apple
$130.00
Vintage Macintosh SE/30 Mac Apple Computer M5119 -
$349.95
Vintage 1992 Apple Macintosh PowerBook Duo 210 FOR PARTS/REPAIR
$50.00
Vintage Apple Logo Calculator. New in original box. LAST ONE...................
$32.00
Vintage Apple PowerBook 170 w/ Adapter and Battery Charger - Tested Working
$199.99
Vintage Rare Apple 20th Anniversary Macintosh Computer (TAM) FATBack Cover
$390.00
Vintage Apple Imagewriter I Printer 1985 SEALED RARE COLLECTOR'S ITEM
$199.95