Intesting. I just did this too. Remastered KNOPPIX to shrink it a bit and added chntpw and captive-ntfs and my own KDE menus, boot splashes, and artwork and put it on a 1G fob booted w/grub. Pretty slick.

I haven't tried it on a system w/scsi drives, but I'm a little surprised it wouldn't boot for you. Did you still have the device.map file in /boot/grub? If so I wonder if that's what is confusing things.

I didn't use the wiki procedure to put mine together. I just copied my remastered CD to the fob, copied the grub files over, and used grub -> setup to write the mbr. No device.map file.

This is just a total WAG, but I wonder how it would have worked w/out device.map.

Later,
Z

I didn't use GRUB. For something standalone like a USB key, SYSLINUX seems like a better choice, since it doesn't require a device map at all. SYSLINUX is entirely self-contained and just "sees" the media it's booting off of, and no others. This is bad for a multibooting desktop machine, but very good for removable/rearrangeable things like USB keys, since there's no more problems with devices appearing in different places on various computers.

As far as remastering goes, I'm waiting until the official release of 5.0. Even though I have the DVD image already, I'm hoping they have a CD-sized image, so I can easily fit it into the USB key.

Grub doesn't need device map either, unless there is some sort of confusion between BIOS drive mapping and OS drive mapping, IIRC.

My questions was whether grub would have worked for you if you hadn't included device.map when you installed it /booted from it. W/out device.map, the drive booted from (the flash fob) should just be drive 0 in menu.lst entries regardless of other drives on the box.

I don't mean to argue the merits of SYSLSINUX vs GRUB, just wondering why grub didn't work well for you. It has certainly worked OK for me so far.

Z.[/quote]

[/quote]

That's cool, I didn't know that grub did not require the device map. It had failed for me in the past when I tried it. SYSLINUX also is handy for Knoppix since Knoppix uses ISOLINUX already. Maybe I'll give GRUB another chance someday.

If/when you do, please let us know what you find. I'm also looking forward to your full SYSLINUX write-up.

My neighbor does a lot of winXX rescue/recovery - now _there's_ a growth industry - and I'm trying to put together a set of flash drives that boot on as many machines as possible.

Along with the grub/syslinux thing, it also seems to matter (to some boxen) just how the fob gets formatted in the first place. I've put up a page describing my struggles:

http://www.beezmo.com/FloobyDustDir/FDKnoppixUsb.htm

No doubt there are errors/omissions. I welcome feedback. Be gentle

Thanks again for your SYSLINUX info.

Z.

Thanks. I've gotten busier recently, so that writeup might have to wait a while.

However, I checked that link on your page, and what I did is very, very similar to what is here:

http://rz-obrian.rz.uni-karlsruhe.de/knoppix-usb/

It looks like he used an older version of Knoppix. The USB scripting has changed since then. I wonder if the sleep is still needed. Sometimes I get failed boots, where it just hangs after Knoppix has found the USB device. It starts to boot, Knoppix loads up, it loads the USB drivers, and then it hangs. Lots of SCSI errors in the log. Most of the time, though, it works. Maybe my USB key is flaky: I keep it on a keychain (like a real key and it's been bumped around a few times.

I also didn't know about the HP utility, and just used Linux cfdisk and mkdosfs tools to format the USB key. What is the "magic" that is done by the HP utility? It might be useful to zero out the entire USB key, and then use the HP tool on it, to see just what changes it makes to the partition table and such. I wonder how it would be different from the generic Linux tools?

I do plan to revisit this project once the Knoppix 5.0 CD (not DVD) comes out! Hopefully they'll have fixed the USB ehci.ko bug by then

I'm pretty sure it's the (fake) geometry: sectors/cylinder, etc. I have looked at the partition table w/fdisk in expert mode after a partiton/format with the HP tool and it looks pretty strange. I'll dig back into this when I get a minute. It should be possible to maximize "bootability" entirely with Linux tools once this is puzzled out (he said optimisticly).

Z.

Just had to add this to the links in this thread:

http://www.sysresccd.org/Howto_install-usb-stick

Some more good information on making USB thumbdrives bootable under both Linux and Windows.

Finally got around to analyzing what that HP formatter utility does!

I ran it on Windows XP.

Tried it with FAT and FAT32 filesystems. Tried it with bootable DOS system files, and without. So, total of 4 trials. The DOS system files came from the generic DOS boot disk you get when you use Windows XP to format a floppy. The HP formatter also offers the choice of NTFS, but I have no interest in keeping my data locked up inside Microsoft's proprietary silo

Here's what we get in the MBR.

Boot sector code:

http://mirror.href.com/thestarman/asm/mbr/Win2kmbr.htm

The HP formatter tool's boot sector compared identical to the standard Microsoft 2000/XP boot sector, but with one exception. There was a 3-byte patch at offset 0xCA: Microsoft's bytes of B8 01 02 were changed to some new bytes of EB 1A 90.

What this does is overwrite the start of the old-fashioned CHS INT 13 commands, with a jump to the newer LBA commands (INT 13 BIOS Extensions), which are located at offset 0xE6. This effectively FORCES the use of LBA, no matter what the size of the disk is. (Without this patch, a very small disk that fit into the CHS limits would still have been eligible to use the old CHS commands.)

Since every BIOS that is even remotely capable of seeing bootable USB thumbdrives, would have to already have supported INT 13 BIOS Extensions for years, this isn't a problem. It's probably a good thing, as it means the bogus CHS values, in the table below, will be completely ignored.

Other things between the boot sector code and the partition table:

Windows 2000/XP disk serial number at 0x1B8: The HP formatter writes a random value here, as it should, but strangely, the 4th byte (at 0x1BB) is always 00! So, there's only 3 bytes of randomness here, not 4. Not sure at all why this was done. A bug?

Error message pointers at 0x1B5: My Microsoft boot sector has the standard values of 2C 44 63 for US English. The HP formatter sets these to 00 00 00, though, so if the MBR were ever to print an error message, it would show up blank or garbage! Another bug?

Partition table:

It uses partition 1, which will be marked active if the bootable DOS system files were installed. CHS size is */255/63.

So, it's definitely not USB-ZIP compatible. (USB-ZIP compatibility would have required the use of partition 4 and CHS size of */64/32.)

Partition types are always 0x0E for FAT16 LBA, or 0x0C for FAT32 LBA. As with the patch above, HP forces LBA to always be used, no matter what the size of the partition is. (I don't have any very small USB keys, so can't fully prove this.)

CHS start is standard 0/1/1. This means the "hidden track" is still intact, and so the remaining 62 sectors of cylinder 0 head 0 are empty wasted space. So, feel free to install your favorite boot managers, or viruses, here.

CHS end is messed up, though. It does not match the LBA end! Instead, the CHS end is truncated to the highest possible full cylinder boundary. On my USB key, I have a total of 2001888 sectors. This should give me a CHS end of 124/155/63, but it doesn't. The HP formatter instead writes a CHS end of 123/254/63. It ends at the boundary of the last full cylinder (123) instead of continuing to include the final partial cylinder (124).

LBA start is 63 (0x3F), which is correct.

LBA size is the entire size of the USB key, minus those 63 sectors. Mine works out to 2001825 sectors, which is correct.

So, there you have it. That's pretty much a complete breakdown of what the HP formatter utility does, at least in the boot sector.

I didn't analyze past the boot sector, because once the MBR is running and LBA has been forced on, the BIOS hopefully won't be able to cause the rest of the boot process to fail. Assuming the BIOS returns valid data when reading the disk with the INT 13 BIOS Extensions, things should just work from there, as it's all generic filesystem and kernel loader stuff from there. It shouldn't be BIOS-dependent, from that point forward.

Still, I'd be curious to know if any other failures happen, when trying to boot from a USB thumbdrive that has been correctly recognized by the BIOS. What BIOS versions are in the computers that worked to boot drives formatted with the HP formatter but didn't work to boot drives formatted with other tools?

That, to me, is by far the hardest problem: getting the BIOS to see the drive, in the first place! If the BIOS can see the USB thumbdrive well enough to set it up as the bootable "hard drive" (the 0x80 device), and run the MBR properly, things just work from there, I have found. This is assuming the BIOS can provide valid INT 13 support when loading data, until the operating system gets fully up and running.

Testing with my SYSLINUX MBR and Knoppix distro on USB, that I made earlier, versus the bootable DOS disks created by this HP formatter utility, unfortunately didn't help me boot up from any more computers. I still had the same successes and the same failures. As before, the hard part is getting the BIOS to just see the USB thumbdrive in the first place. Cleverly patching the MBR won't help, if the BIOS isn't willing to load the MBR at all....

Josh