Working with ntfs

[krishna.murphy]

There are several problems with NTFS, and IMHO it should be avoided where possible. That said, M$oft being as ubiquitous as it is, it is frequently necessary to deal with it even if one chooses not to operate on/under/within their OS-s on a daily basis. Thus I am grateful that ntfs-3g is available, and included in Knoppix. My usage/reasoning is detailed below, but I am not a member of the "don't ever write to NTFS in Linux" camp - it's not SO bad, these days, and sometimes necessary (or at least convenient.)

I am using Knoppix version 6.2.0-DVD from the FAT32-formatted first partition of my hard drive, and there are no doubts that this imposes limitations. In addition to the 4GB of KNOPPIX itself, I have a persistent store of 2 GB, wherein all my system additons (e.g. Google Chrome) and some files kept in home, etc. (all under ext3) reside, leaving about 2GB of FAT32 storage for certain downloads, Windows-readable. Since this is the system my wife and I use to watch downloaded TV and movies (mostly), it follows that more storage for media is required. I fill this need by using slack space on TWO other NTFS partitions with large amounts of relatively dead storage. I often (daily) have several torrents running simultaneously (sometimes more), which can definitely ratchet up the processor load noticeably, especially as those partitions fill up and the files get more fragmented.

This brings us to the most serious drawback of the use of NTFS - it requires defragging, in some form or the other, relatively often. The Linux ext2/3/4 filesystems are relatively immune to fragmentation, and what does exist therein (always some, usually not much) is not a serious handicap and NEVER requires user (or admin) intervention, being automatic.

I have also used Knoppix copied to an NTFS (Windows) partition, and thereunder running, causing auto-mounting when booted. While this is an anecdotal report, and certainly NOT an endorsement of such a practice, I can report that the most serious trouble I had using that system on a daily basis was also processor load, perhaps due to allowing the automatic use of the Windows swap file. Experientially, I found that ntfs3g was quite capable, and robustly handled the access Knoppix required of it to fulfill the role of providing underlying filesystem support, and shares files with Windows with no trouble (unless I tried to hibernate Windows and then access files it thought were locked while it was asleep.)

So, on balance, while I am not thrilled with being forced, as it were, to use a substandard filesystem, and in the near future I expect to eliminate my daily NTFS-access needs, I am content for the time being with using ntfs3g. I don't lose ANY data, and if I keep an eye on the processor load (via the cute little graphic available on the taskbar called "CPU usage monitor") to avoid making the system unresponsive by overloading it, it works fine (sort of, anyway.)

Cheers!
Krishna

[Forester]

@ Kishna

Thanks for telling us a little of how you use Knoppix. I see posts by Junior Members that don't say how they are trying to use Knoppix with replies from Senior Members saying you can do this and that and the other. I read these replies and I think "that's all very well but ...".

I am using a USB install of the Knoppix 6.4.3 EN DVD at work because I've been tasked to write some scripts for Linux servers and I don't agree that 'cross-development' from Windows is the better option. I dare not repartition the hard drive or anything: it isn't mine. I cannot boot Knoppix: my boss requires I use corporate e-mail and messaging services (so he can yell at me without embarrassment). I want a portable solution because next week I may be using a different PC and next month I may be working for a different client. My compromise was to install VirtualBox on Windows 7 and run Knoppix in a virtual machine. There are advantages as well as disadvantages.

At work I don't use ntfs-3g even though I use the ntfs file system. I have virtual disks, which are large ntfs files on the outside but Linux file systems on the inside. When Knoppix accesses these file systems, the (VirtualBox) driver calls Windows and Windows accesses the nfts file system. That's about as safe as it gets.

One nice thing about this arrangement is that I can keep all the work stuff that probably should not leave the building at work but still take Knoppix and any personal stuff home. I do that because I don't have the time at work to learn as much as I want to. I certainly can't be messing around remastering Knoppix on company time.

At home, you may have guessed, I run Linux, not Windows and I have no real use for ntfs-3g. It was not always that way. At one time I had a home network with several PC and a laptop running Windows 2000. I used Windows shares to make one large network file system. When I first installed Linux at home I naturally used a dual-boot arrangement. In order to 'share' files between Linux and Windows I learnt how to set up a samba network. When a PC was running Linux, I used ntfs-3g to mount the old Windows shares so that other PCs could access data over the samba network just as if the PC were still running Windows. Since I didn't really expect to use this all that often, I used the auto mounter to mount the ntfs file systems only when they were actually in use.

The solution is 'technically' sound but, I suppose, how 'safe' it is depends entirely on how safe ntfs-3g is. I was using this several years ago when it was first included in an Ubuntu distributed it (I don't use Ubuntu any more but that's a different story). I never heard of ntfs-3g being unreliable. I never had a problem but I never used it all that much. The auto mounter and the samba network are still in place but they only ever get used when I do a 'fire drill' to make sure they are still working.

I cannot say whether ntfs-3g is safe or not but I have messed around with different file systems and operating systems and machines and stuff to have a fair idea of what the options are and their pros and cons.

[Forester]

NTFS

So, on balance, while I am not thrilled with being forced, as it were, to use a substandard filesystem, ... I am content for the time being with using ntfs3g.

Hmm .... I fear that you may be being a tad libellous in branding (Windows) NTFS a sub-standard file system for I suspect much of what you complain about is a result of (the way you are) using (the Linux) ntfs-3g driver. Tut, tut. Shame on you.

This brings us to the most serious drawback of the use of NTFS - it requires defragging, in some form or the other, relatively often.

From the one time I read up on Windows NTFS, I remember something along the lines of "A whole generation grew up defragging FAT file systems under Windows 98 and continue religiously defragging NFTS file systems when there really is no need to." I've defragged NTFS file systems but only as a prelude to cuttings the b******ds down to size so I could use the disk space for something useful.

It is not that fragmentation does not occur on NTFS: it is more that fragmentation has no adverse affect on performance. If you do really have performance problems due to fragmentation, it may be an ntfs-3g issue.

I often (daily) have several torrents running simultaneously (sometimes more).

Think about what you're doing. You are down loading a large file in small chunks in random order and, you admit, sometimes you are down loading more than several at once. I think fragmentation is inevitable.

debian@aspen:~$ sudo filefrag /media/iso/KNOPPIX_V6.2.1DVD-2010-01-31-EN/KNOPPIX_V6.2.1DVD-2010-01-31-EN.iso
/media/iso/KNOPPIX_V6.2.1DVD-2010-01-31-EN/KNOPPIX_V6.2.1DVD-2010-01-31-EN.iso: 106113 extents found

That's a torrent download I did recently under Linux to an XFS file system. I quote Wikipedia: "XFS is particularly proficient at handling large files and at offering smooth data transfers." 106 133 extents: looks pretty fragmented to me. My bitTorrent client claims the download had less that 4 500 chunks !

I keep an eye on the processor load ... to avoid making the system unresponsive by overloading it, it works fine (sort of, anyway.)

The processor load surprised me at first.

For me, just one torrent download can max out my broad band. Not only does my browser become unresponsive, but so do the browsers anyone else might be using elsewhere in the house. So I don't: I limit the bandwidth of individual torrents but I can also limit the grand total.

If I had fast enough broadband, I'd expect the disk access to max out before the processor load (same effect - system goes treacle). Remember you're using ntfs-3g. That's a File System In User Space (aka FUSE) driver. No way is it going to be as efficient as a lean, mean Linux Kernel driver or the genuine Windows kernel driver from M$soft. Furthermore, since NTFS is a complex file system, I expect the design goals for ntfs-3g were correct operation rather than efficient use of processor or memory. Now I'm not so surprised.

... in the near future I expect to eliminate my daily NTFS-access needs ...

Sounds like you've a serious habit to kick.

If it were me, I'd free up space on the NTFS partitions, defrag 'em and shrink 'em and use the space free for a new partition with a file system better suited to the job. I'd do some research starting with jfs and xfs.

I suspect that even using the space freed for another NTFS partition dedicated to torrent downloads would give better performance. Also, if Harry's nightmare comes true, all you lose is a few TV recordings, not them and your OS, personal correspondence, bank records and the e-mail address of that cute little ....

Cheers!