-
Senior Member
registered user
Re-master your flash installation
The title is kind of confusing. If you have already installed knoppix on flash, and you
have enabled a persistent store, meaning you could go ahead with adding, upgrading
and customizing packages, why do you still need to re-master knoppix ?
Reasons as follows :-
1. When you upgrade your files/packages, the original copy resided on the
read-only compressed image is still there. The system exposes you to the ones on
the persistent store, and hides away the ones on the compressed image.
This is a waste of space, if you have upgraded many files/packages, for
every file/package in the persistant store, there is a counterpart in the
compressed image. Re-master will delete the hidden copies.
Even when there is no compressed image counterpart, due to you added some
new interesting packages, by moving your added packages to the compressed
image, you also free up the (uncompressed) space on the persistent store.
2. Another reason to re-master is that you want to maintain a clean separation between
experimental things verses the "stable" portion of your customization, ie you want to
move the stable and proven things to the compressed image, so that you can later
proceed to add your experimental things to the persistent store. In case you
are not happy with the new experimental things you added, you can always erase it
from your persistent store, or even you can wipe out the entire persistence store.
If you have previously re-mastered your flash installation, deleting things from the
persistent store is an easy, clear-cut and safe operation; because even if you delete
everything, you will still have the last snapshot of a fully working, stable and proven,
and yet already fully customized version of knoppix to start with. There is no need
to falling back to original uncustomized knoppix.
The end result of a flash re-master is not a new copy of CD or DVD, as you have already
booted off from flash, CD or DVD is irrelevant. The end result of flash re-master is just a
new copy of compressed image KNOPPIX.
This is how I re-master my flash installation, in brief. Note there is no "chroot" needed :-
Find a directory or a partition where you have a few GB of space :- # mkdir -p /lots_of_free_space/source/KNOPPIX
# mkdir -p /lots_of_free_space/master/KNOPPIX
# cd /lots_of_free_space/source/KNOPPIX
# cp -a /dev .
# cp -a /UNIONFS/bin .
# cp -a /UNIONFS/etc .
# cp -a /UNIONFS/lib .
# cp -a /UNIONFS/sbin .
# cp -a /UNIONFS/usr .
# cp -a /UNIONFS/var .
# cp -a /UNIONFS/opt .
# cp -a /UNIONFS/home .
# cp -a /UNIONFS/root .
( Whatever you want it to appear in the new compressed image, you copy it there )
.....
# cd ..
# mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" \
-hide-rr-moved -cache-inodes -pad /lots_of_free_space/source/KNOPPIX \
| nice -5 /usr/bin/create_compressed_fs - 262144 > /lots_of_free_space/master/KNOPPIX/KNOPPIX
After a long time, you will get a compressed image at /lots_of_free_space/master/KNOPPIX/KNOPPIX,
which you can use it to replace your /mnt-system/KNOPPIX/KNOPPIX.
You can see that the steps above are quite simple. This last step of replacing existing KNOPPIX
compressed image must be perform "offline", ie you should perform it when you are not
using /mnt-system/KNOPPIX/KNOPPIX.
There is a challenge, because once you have booted in knoppix, you are already using
/mnt-system/KNOPPIX/KNOPPIX. So either you boot from a CD with your flash detached
and then put the flash back after you have booted so that you can overwrite it, OR you
could configure /mnt-system/boot/syslinux/syslinux.cfg to add the cheatcode "debug"
in it. At the first shell prompt, the /mnt-system/KNOPPIX/KNOPPIX is not mounted yet,
this is when you could manually perform this somewhat "risky" task.
Basically the steps will be roughly like this :-
If your free space which contains the compressed image resides on a separate partition :-# mount /dev/sdaX /lot_of_free_space
If you have enough space, you can rename KNOPPIX for backup purposes :-
# mv /mnt-system/KNOPPIX/KNOPPIX /mnt-system/KNOPPIX/KNOPPIX.bak
Now overwrite it with the re-mastered copy :-
# cp /lots_of_free_space/master/KNOPPIX/KNOPPIX /mnt-system/KNOPPIX
You can skip mounting knoppix-data.img by renaming it,
# mv /mnt-system/KNOPPIX/knoppix-data.img /mnt-system/KNOPPI/knoppix-data.img.bak
If using the cheatcode "debug", once done, you type "exit" to proceed with the rest of booting
to verify that you new image is working. You will have to exit 6 times before it will be booted
completely.
If you are happy with it, you can shutdown and boot another time into "debug" shell, this time
you can "restore" your knoppix-data.img and free up the space in there :-
# mv /mnt-system/KNOPPIX/knoppix-data.img.bak /mnt-syste/KNOPPIXknoppix-data.img
# mkdir /tmp/tmp
# mount -o loop /mnt-systsem/KNOPPIX/knoppix-data.img /tmp/tmp
( Now you can delete whichever files you want to delete to free up the space, as
they have already been transferred to the compressed image ).
# cd /tmp/tmp
# rm -rf bin sbin lib etc var usr .......
Once you have finished :-
# cd /
# umount -d /tmp/tmp
Again, if you are using "debug" cheatcode, you have to type 'exit' 6 times before you
will be booted completely with the freed-up persistant store.
In any case, the steps above involved will be somewhat tricky to the Linux newbies. I wish there
is a cheatcode called "upgrade=/dev/sda4:/master/KNOPPIX/KNOPPIX" which will automatically
perform the upgrade. I will leave this as an exercise for those interested to do so by modifying minirt.gz.
Last note: the "debug" cheatcode is a very useful thing. As you can use it to rename/move the
knoppix-data.img ( or knoppix-data.aes ) for the purpose of resizing.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
kl522
# cd ..
# mkisofs -R -U -V "KNOPPIX.net filesystem" -publisher "KNOPPIX www.knoppix.net" \
-hide-rr-moved -cache-inodes -pad /lots_of_free_space/source/KNOPPIX \
| nice -5 /usr/bin/create_compressed_fs - 262144 > /lots_of_free_space/master/KNOPPIX/KNOPPIX
After a long time, you will get a compressed image at /lots_of_free_space/master/KNOPPIX/KNOPPIX,
which you can use it to replace your /mnt-system/KNOPPIX/KNOPPIX.
Maybe I was too agressive with the blocksize 262144 ? Most web site only recommends 65536.
I suspect blocksize 262144 might be related to some intermittent problem which I am facing.
-
Senior Member
registered user
I really appreciate that you share this! ![Smile](images/smilies/icon_smile.gif)
The first rule about remastering is, don't do it. But it's not the last rule...
I would like to postulate a few more good (IMHO) reasons for remastering, I fully agree with yours.
3. Bloat. The DVD contains gigabytes (uncompressed) of programs we will probably never run. While some essentials (every man his own!) are missing.
4. Bugfixes made permanent, moved from "user space".
5. Extra package install/updating made more systematic.
6. Smaller total system footprint. Will always be relevant, if not so important economically as storage gets cheaper.
7. Possible to make several cloop images. Could be useful for things like heavy development tools.
8. Could even try other approaches than through cloop.
I imagine that a good way of doing the full KNOPPIX upgrade, would be to copy the newly made compressed image to, for example, an external USB HD, and do the "field testing" on that. Speaking for myself, I'm pretty sure I would want to make quite a few adjustments.
-
I created an account just for this tutorial.
I tried your method of remastering as I like the way it works.
But did however made the mistake of trying to do the source/master directory on a ntfs filesystem. this caused all persmissions to be lost while copying and after the remaster nearly everything was broken.
Of course I was going to rage on this tutorial
when I realized it maybe couldve been because it was ntfs and took another memorystick and formatted it to an ext2 filesystem. this did the job ![Smile](images/smilies/icon_smile.gif)
is there a way to not break anything while still copying to a ntfs filesystem?
-
dinosoep wonders ...
is there a way to not break anything while still copying to a ntfs filesystem?
Well, yes there is. Have you noticed how Knoppix uses (compressed) loop devices a lot ? You can think of these as filesystems-in-a-file instead of filesystems-on-a-partition. You can use the same trick to create an Linux file system on an ntfs partition.
There are four steps:
1. Mount your ntfs partition under Knoppix (if not already mounted). That will probably be something like:
2. Create a large file on it. How big depends but for remastering the Knoppix CD let's guess 4 Gb is enough. You need something like:
Code:
dd bs=4K count=1M if=/dev/zero of=/media/sda1/big.one
3. Create a file system in the big file:
Code:
mkfs -t ext3 -F /media/sda1/temp/big.one
4. Mount the file as a loop back device:
Code:
mount -o loop /media/sda1/temp/big.one /lots_of_free_space
You may need to sudo one or more of the commands and, of course, adapt the parameters to your particular situation. The mkfs command takes many parameters and some may be more useful in this situation than others.
-
Administrator
Site Admin-
![Quote](images/misc/quote_icon.png)
Originally Posted by
Forester
Well, yes there is. Have you noticed how Knoppix uses (compressed) loop devices a lot ? You can think of these as filesystems-in-a-file instead of filesystems-on-a-partition....
As a moderator, I passed the above post, as it is not spam and on topic.
However, as a user and someone who has been on this site for nearly 8 years and seen over and over again the reports of disaster after writing to NTFS, I couldn't just pass it and not warn others that it is not a universal opinion. Yea, you can create a file system within another file system under Linux. But to do so you still need to write to the physical file system. Not just to create it but during all of the writing processes. I know of no reason to think that this is safer than just writing to NTFS (which is unfortunately still not safe).
If you want another file system on your hard disk consider shrinking the main partition, or use that "back up" partition that the vendor wasted disk space to create (after making a backup disc), and create a second Linux or FAT partition there, or even just add another inexpensive hard disk. An extra partition can be a Linux partition or a FAT partition (A FAT partition will be subjected to the 4GB-1 file size but lets you transfer files easily between Windows and Linux). But if you write to a NTFS partition, have very good backups of everything and keep making backups, the effects of corruption are often not noticed for some time after the damage is done.
---
Verifying of md5 checksum and burning a CD at slow speed are important.
-
Senior Member
registered user
![Quote](images/misc/quote_icon.png)
Originally Posted by
Harry Kuhman
However, as a user and someone who has been on this site for nearly 8 years and seen over and over again the reports of disaster after writing to NTFS, I couldn't just pass it and not warn others that it is not a universal opinion. Yea, you can create a file system within another file system under Linux. But to do so you still need to write to the physical file system. Not just to create it but during all of the writing processes. I know of no reason to think that this is safer than just writing to NTFS (which is unfortunately still not safe).
This subject has been mentioned quite many times. 8 years of observation is quite a long time, especially in the world of IT. I am not sure how extensive your observation is substantiated with facts, as software version changes, some bad experience in the past with bugged software may not be applicable to newer versions of software.
The fact is over the recent years, the use of NTFS (for read and write access ) in Linux has got wide spread usage and penetration. If you look at embedded devices, quite many of them are Linux based, they read/write usb flashes and SATA harddisk. I came to know quite many of the media players are Linux based. I also know that Samsung TV is Linux-based too. These devices read/write on NTFS file system (and FAT32) !
I cannot help but to think that read/write of NTFS on Linux has gotten to a usable degree of safety - if one does not use those fancy features of NTFS. But if indeed your observation - substantiated with facts - does indicate that NTFS writing is still unsafe, perhaps it is a problem unique to Knoppix ? I am wondering ......
-
Senior Member
registered user
I think the creation of an image file, organizing a file system on it and loop-mounting that file system, may be relatively safe on NTFS. But, as NTFS is a rather swiftly and erratically moving object, I can't really understand how anyone can be quite sure everything will always go well. Even if everything works perfectly today, there is a chance that the next service pack, or even bugfix patch, may introduce problems. Therefore, I like to make a distinction between simple tasks like creating, updating and reading an image file, and more general file system use. In particular, if all reading/writing to the image file is to be done under Linux, one may mostly avoid inconsistencies etc between Windows and Linux NTFS implementations.
-
Senior Member
registered user
I agree with you that creation of an image file, organizing a file system on it and loop-mounting that file system is considered relatively safe - with a catch I shall explain below.
I was reading the ntfs-3g write up, they mentioned this in the Wiki :-
From what I read, if you want to use NTFS-3g to fix access control list and permission on a Windows system, you might not succeed because of the catchy support of ACL and permission. However, if you want to implement what Knoppix needed, ie compressed loop mount a file on NTFS, it would appear to be safe.
On the other hand ......
I did mention before for Knoppix 6.2.1 ( which I did not check if it is still valid after that ), it is in my humble opinion that knoppix does not shutdown the persistent file system properly, more so for FUSE. If you look at the shutdown script, basically it did not manage to unmount the UNIONFS/aufs2. The FUSE user space driver is terminated before the persistent file system got unmounted. If FUSE keeps buffer, the buffer content is it flushed ? I am not sure. Specifically I noticed when using vanilla Knoppix 6.2.1 with a ext2 persistent store loop mounted on NTFS, I noticed the system had to fsck the persistent store, and each time there is inconsistencies and it requires manual intervention. That was Knoppix 6.2.1 anyway .....
-
sorry for replying this late, I hoped that knoppix forums would sent me a mail when someone reacted.
this was exactly what I was looking for ![Very Happy](images/smilies/icon_biggrin.gif)
I have been writing from linux mint to a ntfs partition a lot and I got no problems whatsoever so I think writing is getting more and more stable to ntfs
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
![Compaq Mechanical AT/XT Keyboard Mainframe Collection Gray (Missing Keys) RARE picture](/store/img/g/jagAAOSwlu1lecVO/s-l225/Compaq-Mechanical-AT-XT-Keyboard-Mainframe-Collect.jpg)
Compaq Mechanical AT/XT Keyboard Mainframe Collection Gray (Missing Keys) RARE
$384.99
![IBM mainframe server cpu IBM93 vintage cpu collector's item picture](/store/img/g/EYQAAOSwzWxmThYD/s-l225/IBM-mainframe-server-cpu-IBM93-vintage-cpu-collect.jpg)
IBM mainframe server cpu IBM93 vintage cpu collector's item
$125.99
![Vintage Hewlett Packard HP 2100S Microprogrammable Computer System Mainframe #2 picture](/store/img/g/eBcAAOSwRstk0rkC/s-l225/Vintage-Hewlett-Packard-HP-2100S-Microprogrammable.jpg)
Vintage Hewlett Packard HP 2100S Microprogrammable Computer System Mainframe #2
$1499.99
![Vintage IBM 4341 / 4361 / 4381 MCM Processor Mainframe CPU picture](/store/img/g/vw0AAOSwTFRmRYkE/s-l225/Vintage-IBM-4341-4361-4381-MCM-Processor-Mainframe.jpg)
Vintage IBM 4341 / 4361 / 4381 MCM Processor Mainframe CPU
$120.00
![AST Bravo MS P/60 Vintage Mainframe Computer Old IBM Style NO HDD picture](/store/img/g/l58AAOSwOIpl~fXd/s-l225/AST-Bravo-MS-P-60-Vintage-Mainframe-Computer-Old-I.jpg)
AST Bravo MS P/60 Vintage Mainframe Computer Old IBM Style NO HDD
$129.95
![Digital DEC LK201 Terminal Keyboard RJ11 Connection Mainframe picture](/store/img/g/DsgAAOSwK51mPTos/s-l225/Digital-DEC-LK201-Terminal-Keyboard-RJ11-Connectio.jpg)
Digital DEC LK201 Terminal Keyboard RJ11 Connection Mainframe
$49.00
![*PAIR* 1960's Vintage GE-600 Series Mainframe Computer PCBs Bitslice Accumulator picture](/store/img/g/-qYAAOSwP21mTl9Q/s-l225/-PAIR-1960-s-Vintage-GE-600-Series-Mainframe-Compu.jpg)
*PAIR* 1960's Vintage GE-600 Series Mainframe Computer PCBs Bitslice Accumulator
$14.95
![IBM Mainframe Core Memory, 1959 picture](/store/img/g/fAkAAOSwRRJlP-cX/s-l225/IBM-Mainframe-Core-Memory-1959.jpg)
IBM Mainframe Core Memory, 1959
$2000.00
![Lotus Works Vintage Software 3.5in Disks Original Seal 1990 Mainframe Collection picture](/store/img/g/sAIAAOSwl1VlRTR9/s-l225/Lotus-Works-Vintage-Software-3-5in-Disks-Original-.jpg)
Lotus Works Vintage Software 3.5in Disks Original Seal 1990 Mainframe Collection
$383.99
![Unisys Mechanical Clicky Keyboard AT/XT PN:F4208-00 Mainframe Collection picture](/store/img/g/0OIAAOSwI8BlU4TF/s-l225/Unisys-Mechanical-Clicky-Keyboard-AT-XT-PN-F4208-0.jpg)
Unisys Mechanical Clicky Keyboard AT/XT PN:F4208-00 Mainframe Collection
$319.99