Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: Adding more persistent disk images

  1. #1
    Senior Member registered user
    Join Date
    Sep 2006
    Posts
    802

    Adding more persistent disk images

    In Knoppix 6.X, the persistent image is a file, knoppix-data.img, which at boot gets loop-mounted on KNOPPIX-DATA and merged with the cloop-mounted Knoppix image in unionfs.
    If something has gone wrong with this image, you may just mount the media (for instance fram another running version of Knoppix), and delete the file. (Don't try to delete a mounted image). On next booting from this device, you will be asked to create a new persistent image.

    This image is very tightly integrated with the Knoppix version you are running, and on FAT32 it is limited to 4GB. I find I need more space, I'd like not to have it integrated with the current Knoppix version, and I want to store it on USB-sticks etc.
    Typically, I want to install Eclipse, Groovy, Grails and several other Java packages on the stick independent of the running Knoppix. I also want to use USB sticks without all the limitations of the FAT32 file system.
    One way to accomplish this, is adding more persistent disk images. I use a 16GB stick, with ca 1 GB for Knoppix 6.2 from CD, 4GB for standard persistent disk image, and two more persistent images, which will be mounted at boot time, but not merged with KNOPPIX in unionfs.

    I choose to put the images together with the standard image inside the /knoppix folder.
    First step is to create the file, which is done with dd.
    Second is to setup a loop device on the file, with losetup
    Third is creating an ext2 file system on the image.
    Next, create a mounting point and test-mount the loop device. After testing, unset the loop connection with losetup -d
    Now, the new image is ready for mounting with mount -o loop <image> <mount point>.

    Code:
    root@Microknoppix:/media/sdd1/knoppix# dd if=/dev/zero of=knoppix-data2.img bs=512 count=8000000
    8000000+0 records in
    8000000+0 records out
    4096000000 bytes (4.1 GB) copied, 336.48 s, 12.2 MB/s
    root@Microknoppix:/media/sdd1/knoppix# losetup /dev/loop6 knoppix-data2.img 
    root@Microknoppix:/media/sdd1/knoppix# mkfs -t ext2 /dev/loop6
    mke2fs 1.41.3 (12-Oct-2008)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    250480 inodes, 1000000 blocks
    50000 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=1027604480
    31 block groups
    32768 blocks per group, 32768 fragments per group
    8080 inodes per group
    Superblock backups stored on blocks: 
    	32768, 98304, 163840, 229376, 294912, 819200, 884736
    
    root@Microknoppix:/media/sdd1/knoppix# mkdir /mnt/knxdata2 && mount /dev/loop6 /mnt/knxdata2
    root@Microknoppix:/media/sdd1/knoppix# umount /mnt/knxdata2 && losetup -d /dev/loop6
    root@Microknoppix:/media/sdd1/knoppix# mount -o loop knoppix-data3.img /mnt/knxdata2
    After this setup is performed, necessary mounting code may be added to /etc/rc.local. I prefer to stay entirely outside the hard-coded directory system, which eliminates a lot of otherwise natural choices. I use store as a common mounting point for extra "partitions", and mount the development tools volume as /store/share and the user projects as /store/varl.

    Edit 20100220:
    It seems to be possible to make a backup of the standard persistent image, compress it, and have it copied back at boot time if something has gone wrong with the running image. From Klaus' comments to his list of 6.2.0 cheat codes:
    If you place an update*.zip or update*tar.gz file on the medium holding
    the KNOPPIX data, it will be unpacked onto the overlayed filesystem
    before starting "init", thus allowing quick reconfiguration of the
    system.
    One consequence of this, is that it may be good practice to back up the persistent image now and then. Accessing it when it is not mounted. For instance, put a KNOPPIX copy somewhere else, and either use that with the fromhd= cheat code, or (g)zipping the knoppix-data.img file from it.

  2. #2
    Senior Member registered user
    Join Date
    Feb 2010
    Posts
    512
    Edit 20100220:
    It seems to be possible to make a backup of the standard persistent image, compress it, and have it copied back at boot time if something has gone wrong with the running image. ...
    I looked inside /init (Knoppix 6.2.1) and found the code for updating the system:
    Code:
    # Check for updates on-disk, install them if necessary.
    ls /mnt-system/KNOPPIX/update*.zip /mnt-system/KNOPPIX/update*.tar.gz /mnt-system/KNOPPIX/update*.taz 2>/dev/null | while read update; do
     if [ -r "$update" ]; then
      message -e "\r${CRE}${GREEN}${UPDATING} ${YELLOW}$update${NORMAL}"
      case "$update" in
       *.zip) ( cd / ; unzip -o "$update" >/dev/null 2>&1 ) ;;
       *.tar.gz|*.taz) ( cd / ; tar -zxf "$update" >/dev/null 2>&1 ) ;;
      esac
     fi
    done
    As far as I understand the code, one has to create an ordinary zip or tar.gz archive inside the running system and place this archive in the KNOPPIX directory. The archive will be extracted into /.

    For example, I could run KNOPPIX from the live cd and make all the necessary changes. Then I would create the update-special.tar.gz archive with my configuration files which is copied to a flash device with an installed KNOPPIX on it. I would delete an existing persistent image and boot the system from that flash device. After the creation of a new persistent image I would get a KNOPPIX with my special configuration. (I tried this with a backup of may /home/knoppix and it worked fine.)

  3. #3
    Senior Member registered user
    Join Date
    Sep 2006
    Posts
    802
    Quote Originally Posted by klaus2008
    As far as I understand the code, one has to create an ordinary zip or tar.gz archive inside the running system and place this archive in the KNOPPIX directory. The archive will be extracted into /.

    For example, I could run KNOPPIX from the live cd and make all the necessary changes. Then I would create the update-special.tar.gz archive with my configuration files which is copied to a flash device with an installed KNOPPIX on it. I would delete an existing persistent image and boot the system from that flash device. After the creation of a new persistent image I would get a KNOPPIX with my special configuration. (I tried this with a backup of may /home/knoppix and it worked fine.)
    Sure. But if you run it on the living system, my understanding is that you will necesarily copy the whole UNIONFS structure, wherever you pick directories that are "mixed". (But of course I may be wrong.) How will that work out on updating? If you mount the image, non-running, and do the zipping you will at least get a somewhat smaller thing.. But will that blank out the UNIONFS part from the compressed image when it is unzipped? Haven't tried.

  4. #4
    Junior Member registered user
    Join Date
    Feb 2010
    Posts
    12
    I think the best thing to do is to copy the KNOPPIX-DATA directory, since that neatly isolates all the personal files. Just be careful: there are a lot of sneaky dot-files in there that probably shouldn't be tarred. I haven't had a chance to explore that further.

    All I know is I tried to use Klaus's method and it only partially, I suspect when it hit one of the weird-looking aufs dot-files. So I tried a different tar without them (I did this by using two computers, one with the old image and one with the new one, making the old one an NFS server with /KNOPPIX-DATA exported, then copying all of it to a mount point on the computer running the new image, then copying that into my home directory as "knoppix" which prevented me from copying files that shouldn't be copied. In retrospect it probably would have worked just as well if I'd just moved the data on the old image to the old image's home directory [which would skip the problematic files], tarred it up and transferred it to the new image). But instead of using Klaus's method I used the "-b debug" cheatcodes and executed the command myself to see what would happen. There were no errors and the files were moved properly. I'll try it again with some of the problem files in the .tar and without them using Klaus's method when I have some time to spend testing.

  5. #5
    Junior Member
    Join Date
    Oct 2010
    Posts
    9
    i also have been working on getting KNOPPIX-DATA onto a dvd. i have tested garaden's solution above and can verify that it works properly once all the "dot-wh-dot" (.wh.*) files have been removed from the tarred KNOPPIX-DATA directory.

    i also created a hacked version of the knoppix initialization script in the initramfs file minirt.gz which i will document a bit further at my original post http://www.knoppix.net/forum/threads...t-image-on-dvd

    sincerely,
    proctor

  6. #6
    Senior Member registered user
    Join Date
    Dec 2009
    Posts
    423
    Quote Originally Posted by proctor View Post
    i also have been working on getting KNOPPIX-DATA onto a dvd. i have tested garaden's solution above and can verify that it works properly once all the "dot-wh-dot" (.wh.*) files have been removed from the tarred KNOPPIX-DATA directory.
    Oh no, I wonder how people come to this conclusion about those dot-wh-dot files. In my view, if you want to copy a consistent copy of "file sets" ( ie collection of files ), you should rather copy the /UNIONFS counterpart, and not copy /KNOPPIX-DATA and omitting the dot-wh-dot.

    A package is basically a set of files. Very often when you update a package (using synaptics for example), that might result in removing some older files on the read-only file system which does not have it's counterpart in the updated package. To mark that these read-only files as deleted, thereby there will be dot-wh-dot files created on the read-write file system.

    The mechanism maybe too complicated to be explained here. But all in all, if you copy files on the read-write file system ( and omitting the dot-wh-dot files ), you will likely end up with having inconsistent packages.

  7. #7
    Junior Member
    Join Date
    Oct 2010
    Posts
    9
    by "copy the /UNIONFS counterpart" do you mean the entire system as opposed to "only the changes?" on my test system /UNIONFS consists of around 2GB data, whereas /KNOPPIX-DATA is only 140MB....

    is there a better way to do what i am trying to do (use the persistent overlay image knoppix-data.img on a read-only media)?

    sincerely,
    proctor

  8. #8
    Senior Member registered user
    Join Date
    Dec 2009
    Posts
    423
    Quote Originally Posted by proctor View Post
    by "copy the /UNIONFS counterpart" do you mean the entire system as opposed to "only the changes?" on my test system /UNIONFS consists of around 2GB data, whereas /KNOPPIX-DATA is only 140MB....

    is there a better way to do what i am trying to do (use the persistent overlay image knoppix-data.img on a read-only media)?

    sincerely,
    proctor
    Sorry it wasn't clear to me what you are trying to accomplish. But I am only saying that if you copy /KNOPPIX-DATA but omitting the dot-wh-dot files, you will have inconsistent packages.

  9. #9
    Junior Member
    Join Date
    Oct 2010
    Posts
    9
    well this is very valuable information to have!

    what happens when the dot-wh-dot files are left intact is that when the init system tries to mount the tarred structure into the existing overlay, the dot-wh-dot files conflict/interfere with their already existing counterparts in the UNIONFS system.

    do you know of a workaround for this issue?

    proctor

  10. #10
    Junior Member
    Join Date
    Oct 2010
    Posts
    9
    hi,

    after looking in more detail at aufs it seems to me that using the tarred /KNOPPIX-DATA (really a tarred mounted /KNOPPIX/knoppix-data.img) with removed dot-wh-dot files should not disrupt an aufs file system because the tarred system is not overlaid into the UNION and so therefore not merged in the same fashion that aufs uses: they are simply copied into the UNION, into the aufs.

    it seems to me that this is actually the preferred method, and not my patch (at http://www.knoppix.net/forum/threads...t-image-on-dvd).

    would anyone disagree with this?

    proctor

Page 1 of 2 12 LastLast

Similar Threads

  1. Replies: 2
    Last Post: 01-10-2009, 12:31 PM
  2. Persistent images: how to make diffs of it?
    By hotplainrice in forum General Support
    Replies: 0
    Last Post: 11-02-2006, 11:02 PM
  3. writing floppy disk images to the floppy
    By linuxman in forum General Support
    Replies: 2
    Last Post: 12-10-2005, 11:47 PM
  4. switching persistent disk images on the fly?
    By rwcitek in forum Hardware & Booting
    Replies: 0
    Last Post: 05-29-2005, 08:37 PM
  5. Adding another hard disk.
    By captain in forum The Lounge
    Replies: 1
    Last Post: 05-20-2004, 06:19 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Lot of 50 4GB DDR4 Memory PC4-3200 SODIMM Laptop RAM 3200MHz picture

Lot of 50 4GB DDR4 Memory PC4-3200 SODIMM Laptop RAM 3200MHz

$330.00



Gigastone DDR3 Desktop RAM 32GB (4x8GB) 1600MHz PC3-12800 CL11 1.5V 240 Pin picture

Gigastone DDR3 Desktop RAM 32GB (4x8GB) 1600MHz PC3-12800 CL11 1.5V 240 Pin

$59.99



A-Tech 16GB 2 x 8GB PC3-12800 Laptop SODIMM DDR3 1600 Memory RAM PC3L 16G DDR3L picture

A-Tech 16GB 2 x 8GB PC3-12800 Laptop SODIMM DDR3 1600 Memory RAM PC3L 16G DDR3L

$33.99



16GB 2 x 8GB DDR3 1333 REG Memory RAM for DELL PRECISION T5500 T5600 T7500 T7600 picture

16GB 2 x 8GB DDR3 1333 REG Memory RAM for DELL PRECISION T5500 T5600 T7500 T7600

$17.99



HyperX FURY DDR4 8GB 16GB 4GB 32GB 2666MHz PC4-21300 Desktop RAM Memory DIMM 288 picture

HyperX FURY DDR4 8GB 16GB 4GB 32GB 2666MHz PC4-21300 Desktop RAM Memory DIMM 288

$26.95



Samsung 16GB 2Rx4 PC3L-12800R DDR3-1600 1.35V ECC REG RDIMM Server Memory RAM 1x picture

Samsung 16GB 2Rx4 PC3L-12800R DDR3-1600 1.35V ECC REG RDIMM Server Memory RAM 1x

$10.99



HyperX FURY DDR4 4GB 8GB 16GB 3200 2400 2666 MHz Desktop RAM Memory DIMM 288pin picture

HyperX FURY DDR4 4GB 8GB 16GB 3200 2400 2666 MHz Desktop RAM Memory DIMM 288pin

$26.95



[ BULK LOT OF 10 ] Desktop RAM 4GB DDR3 PC3 Micron, SAMSUNG, HYNIX picture

[ BULK LOT OF 10 ] Desktop RAM 4GB DDR3 PC3 Micron, SAMSUNG, HYNIX

$39.99



HyperX FURY DDR3 16GB 2x 8GB 1600 MHz PC3-12800 Desktop RAM Memory DIMM 240pins  picture

HyperX FURY DDR3 16GB 2x 8GB 1600 MHz PC3-12800 Desktop RAM Memory DIMM 240pins

$34.95



A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G picture

A-Tech 8GB DDR3 1600 PC3-12800 Laptop SODIMM 204-Pin Memory RAM PC3L DDR3L 1x 8G

$17.99