Working with ntfs

[Forester]

@ Forester

My scenario is very simple, and I can probably refine my question to be simpler.

Thanks utu for your description. I can't see anything dangerous in what you are doing.

I'm a bit surprised you say you copy the entire contents of the USB onto the hard drive and all you get is a file. I'd expect a folder with a few files and a couple of sub folders.

Anyway, the important thing is you are writing to the ntfs file system using Windows so you aren't running any (more) risks (than the next guy).

If you not remastering then I think you will find the only file that ever changes on the USB's FAT file system is knoppix-data.img so you could probably simplify your procedure further and even save yourself a little time and disk space.

[Forester]

NTFS

So, on balance, while I am not thrilled with being forced, as it were, to use a substandard filesystem, ... I am content for the time being with using ntfs3g.

Hmm .... I fear that you may be being a tad libellous in branding (Windows) NTFS a sub-standard file system for I suspect much of what you complain about is a result of (the way you are) using (the Linux) ntfs-3g driver. Tut, tut. Shame on you.

This brings us to the most serious drawback of the use of NTFS - it requires defragging, in some form or the other, relatively often.

From the one time I read up on Windows NTFS, I remember something along the lines of "A whole generation grew up defragging FAT file systems under Windows 98 and continue religiously defragging NFTS file systems when there really is no need to." I've defragged NTFS file systems but only as a prelude to cuttings the b******ds down to size so I could use the disk space for something useful.

It is not that fragmentation does not occur on NTFS: it is more that fragmentation has no adverse affect on performance. If you do really have performance problems due to fragmentation, it may be an ntfs-3g issue.

I often (daily) have several torrents running simultaneously (sometimes more).

Think about what you're doing. You are down loading a large file in small chunks in random order and, you admit, sometimes you are down loading more than several at once. I think fragmentation is inevitable.

debian@aspen:~$ sudo filefrag /media/iso/KNOPPIX_V6.2.1DVD-2010-01-31-EN/KNOPPIX_V6.2.1DVD-2010-01-31-EN.iso
/media/iso/KNOPPIX_V6.2.1DVD-2010-01-31-EN/KNOPPIX_V6.2.1DVD-2010-01-31-EN.iso: 106113 extents found

That's a torrent download I did recently under Linux to an XFS file system. I quote Wikipedia: "XFS is particularly proficient at handling large files and at offering smooth data transfers." 106 133 extents: looks pretty fragmented to me. My bitTorrent client claims the download had less that 4 500 chunks !

I keep an eye on the processor load ... to avoid making the system unresponsive by overloading it, it works fine (sort of, anyway.)

The processor load surprised me at first.

For me, just one torrent download can max out my broad band. Not only does my browser become unresponsive, but so do the browsers anyone else might be using elsewhere in the house. So I don't: I limit the bandwidth of individual torrents but I can also limit the grand total.

If I had fast enough broadband, I'd expect the disk access to max out before the processor load (same effect - system goes treacle). Remember you're using ntfs-3g. That's a File System In User Space (aka FUSE) driver. No way is it going to be as efficient as a lean, mean Linux Kernel driver or the genuine Windows kernel driver from M$soft. Furthermore, since NTFS is a complex file system, I expect the design goals for ntfs-3g were correct operation rather than efficient use of processor or memory. Now I'm not so surprised.

... in the near future I expect to eliminate my daily NTFS-access needs ...

Sounds like you've a serious habit to kick.

If it were me, I'd free up space on the NTFS partitions, defrag 'em and shrink 'em and use the space free for a new partition with a file system better suited to the job. I'd do some research starting with jfs and xfs.

I suspect that even using the space freed for another NTFS partition dedicated to torrent downloads would give better performance. Also, if Harry's nightmare comes true, all you lose is a few TV recordings, not them and your OS, personal correspondence, bank records and the e-mail address of that cute little ....

Cheers!