Archive for 'Windows'

One of the perennial problems I see no matter where I work or who I contract for is mysteriously slow network speeds to or from a Windows machine. I’ve amassed quite a list of tips and tricks for addressing this issue, and now I’m listing them all here.

I won’t go into a thorough treatment of exactly what’s going on with each command and feature that is being enabled or disabled. I’ll leave that as an exercise for the reader. This is a quick-n-dirty jumping off point for deeper problem resolution procedures. Also, these troubleshooting steps are not offered in any particular order, with the exception of the first four which try and scope the problem down to hardware versus software.

With no further rambling, here is my list of tricks when trying to solve a slow network connection on a Windows host:

Start With the Physical Layer

It’s almost always the easiest thing to check and is more often the cause of problems than most people would suspect. Thanks to Pauska in the comment below for reminding me of this. Switch cables out, switch NICs if possible (I like to keep a USB NIC around for this), try different switch ports, wall jacks — everything. It’s quick work and can reap a quick reward. Plus, with the physical layer out of the way, you can trust the observations that you make in the software layer.

Boot From a Live CD

Remove the OS from the equation and see if you can isolate the issue to hardware. Grab a Live CD that has an OS on it with support for your hardware. Once you boot from it, perform some tests on the bandwidth to see if the problem still exists. If so, then you may be safer in assuming that the problem exists somewhere other than the operating system (unless the same configuration that’s causing the problem exists in both operating systems).

Search for Network Related Errors

Perhaps there’s a lot of collisions on the network or the network card is having a large amount of CRC errors. A quick way to see current TCP/IP statistics is to run netstat -s. Look for any interesting numbers that speak to receive errors or re-transmissions.

Use Performance Monitor counters to analyze error data live. If errors and re-transmissions seem unusually high, you have a jumping off point for further exploration.

Inspect Traffic with Network Monitor

Launch Microsoft Network Monitor or Wireshark (or whatever packet sniffer you prefer) and inspect the packet stream. There will almost certainly be a trail of information that can lead you to the ultimate problem. The trouble is: can you persevere to the end? It’s no easy thing to digest TCP conversations en mass.

In reality, this is where the root cause analysis will begin and often where it will end. However, if you want to flail at some network related options to try and narrow down the culprit, read on.

Disable Windows Network Task Offloading

Add a DWORD registry key titled ‘DisableTaskOffload’ with value of 1 to the registry hive  HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\.

Check that it currently exists and what the value is with the following PowerShell cmdlet:

get-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\ -name "DisableTaskOffload"

Check the whole parent hive if you want:

get-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

Create the new registry entry:

New-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\" -Name "DisableTaskOffload" -Value 1 -PropertyType "DWord"

Disable TCP chimney offloading

You will need to disable TCP offloading in the Windows OS as well as the hardware’s drivers, however we’ll talk more about disabling hardware offloading in the next point. By the way, TCP offloading only works if it is enabled both in Windows and in the hardware’s driver.

First, let’s check to see if any connections are currently offloaded to hardware using netstat -t

InHost means that the TCP connection is being handled… well… in the host. If there are connections being offloaded to the hardware, know that disabling this will wreak some havoc with them.

To determine the state of offloading within the OS, run the following at a command prompt:

netsh int tcp show global

Look at the state of the “Chimney Offload State” setting. If it’s enabled, disable it with the following command:

netsh int tcp set global chimney=disabled

Disable All Hardware Network Offloading

Now you need to inspect your network card’s capabilities. Go to Device Manager, open up the properties of the NIC and select the Advanced tab. Search for any options that reference offloading. TCP, UDP, checksum, whatever. Disable it. “But! But! Offloading roxors!!” I know, this is just for troubleshooting purposes. Once you figure out where the bottleneck is, you can start determining the root cause. That’s for later though.

Each card has different features and terminology, so I can’t be more specific. For now, just disable anything to do with offloading.

Disable Receive Side Scaling

Check to see if it’s enabled with the following command:

netsh int tcp show global

Disable receive side scaling with:

netsh int tcp set global rss=disabled

Disable NetDMA

Once again, check to see if it’s enabled with the following command:

netsh int tcp show global.

See if the registry key for the setting exists using PowerShell:

get-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters -name EnableTCPA

To disable it, create its registry key and give it the proper value. Using PowerShell:

New-Item -Path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\EnableTCPA
New-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\" -Name "EnableTCPA" -Value 0 -PropertyType "DWord"

Disable Autotuning

Check to see if autotuning is enabled with:

netsh interface tcp show global

Disable it with:

netsh int tcp set global autotuning=disabled

Uninstall Remote Differential Compression

Go to Add/Remove Programs or Programs and Features (run >> appwiz.cpl). Choose the option to turn Windows features on or off. Uninstall Remote Differential Compression.

More information about RDC can be found at the Wikipedia page on Remote Differential Compression.

Alter NIC and Switch Port Speed and Duplex Settings

First, document your NICs current link speed and duplex settings. Then document the switch port’s settings.

In Windows, go into Device Manager, open the NIC in question and go to the advanced tab. The exact naming of the property for the card’s speed and duplex settings will vary, but you’ll know it when you see it.

Auto-negotation can be a pain. Set your NIC to 100 or 1000 Mbps Full Duplex if possible. Continue to frob with the possibilities. Personally, I wouldn’t bother with half-duplex settings, but – as they say - any port speed in a storm!

Update your NIC Drivers

Sounds simple. Sounds stupid. It works. Do it.

Not only should you use the latest drivers, but also look for discussions concerning your network card and its performance relative to the driver version. Perhaps it’s an older driver that you need. See if you can track down older versions and try those.

Check for Third Party Security Tools

If an antivirus utility is set to scan live traffic for malicious payloads, that can negatively impact throughput. Check to see what security tools are installed on the node that is having throughput problems and temporarily disable any features that affect live traffic.

Reset the TCP/IP Stack

You know that you’re flailing when you start resetting the TCP/IP stack. Read more about the procedure in Microsoft KB299357. At an elevated command prompt, run the following command:

netsh int ip reset resetlog.txt

Reset Winsock2

To read more about the practice of repairing winsock2 corruption read Microsoft KB811259. To reset winsock, use the following command:

netsh winsock reset

Reset only the catalog with the following command

netsh winsock reset catalog

Note that if you are using Windows XP SP1 or earlier, you will have to manually reset winsock using the instructions in Microsoft KB811259.

Do you have any tips or tricks for a slow Windows network connection? Let me know in the comments below and I’ll include them here!

My Problem:

Using Internet Explorer 9 on a brand new installation of Windows 7 Professional, a user could not open certain PDFs that were located on a website. Some PDF would appear to begin downloading and then after a few moments, a simple error message would pop up:

The file is damaged and could not be repaired.
Local\EWH$@et`08b

The document could be opened if it was first downloaded and then opened with Adobe Reader. It was only a problem if IE tried to open it in a browser tab.

Oddly, various other PDFs that were accessed with the browser could open as normal.

Possible Solutions:

There are two possible solutions to this issue that I am aware of.

First:

The problem might be due to be an overflowing temporary internet files folder. I noticed that other PDFs could be viewed in IE. The ones that could open were smaller than the PDFs that were giving the user problems.

A temporary fix is to delete all temporary internet files and restart IE. A more permanent fix is to empty the temporary files folder at each exit. You can also increase the disk space available to the temporary internet files folder.

To delete temporary internet files upon exiting IE, go to Tools Menu >> Internet Options >> Advanced Tab >> Security Section >> Check the box next to “Empty Temporary internet Files Folder when Browser is Closed”

To increase the amount of space on your hard drive that IE can use to store temporary files, go to Tools Menu >> Internet Options >> General Tab >> Browsing History section >> Click the “Settings” button >> Edit the number next to “Disk Space to Use”

Second:

A second solution that is possible is as simple as updating Adobe Reader. I know, I know – it’s too simple. However, check to make sure you have the latest version. If you do, uninstall and re-install it.

In the user’s case, it was an older version of Adobe Reader. I updated it to the latest version (Adobe Reader X point something-or-other as of the writing of this post).

Other Possibilities:

There remains two other major culprits. The first being IE itself. Some have said that using one particular version of internet explorer causes the problems. No one seems to agree which version solves the problem because it seems that any version of IE going back to version 6 has experience this issue. That leads me to believe that the problem is rooted in something fundamental to IE and/or the Windows OS in a way that IE relies upon. You might want to try uninstalling IE and re-installing it.

Lastly, make sure that you have the proper updates for your installation of Windows. Another one of the potential problems that existed in my scenario is that the client machine did not have the latest Windows updates.

While auditing and revising the backup policies for some servers, I came to an older file server that I hadn’t had significant contact with in a while. I knew I had made a volume mount point from one volume to another volume, but couldn’t remember where it was.

Like a good SysAdmin, I documented it in a private wiki so the information was a simple click away. However, it struck me that I should know how to list all volume mount points on a Windows Server 2003 or 2008 box. Who knows? Maybe I made a couple of extra VMPs and forgot about it.

If I had development skills, I could use some of the valuable MSDN content to create a small app to enumerate mounted folders that uses the calls “FindFirstVolumeMountPoint”, “FindNextVolumeMountPoint” and “FindVolumeMountPointClose”. However, as far as any kind of programming is concerned I couldn’t hack my way out of a wet paper bag.

The Windows 2000 Resource Kit has three tools that deal with volume mount points or “Junction Points”. These tools also work on Server 2008 and are included by default, at least on my Server 2008 R2 machine. Those tools are: linkd.exe, mountvol.exe and delrp.exe.

Mountvol when used by itself with no switches will first show the command’s help text but then list all volumes that are available on the system as well as a what junctions connect to that volume:

Notice that all my drive letters are listed, but underneath one of them, the E: drive, is a path to a folder that resides on the C: drive. This is the location index files for my backup program. Super, so now I can see all the junction points that are associated with a drive letter! As I suspected, there was only one junction point.

It should be noted that I also found a scripted way of enumerating volume mount points. The script is from The Scripting Guys and is located at this Microsoft TechNet link. Not that it does not work for Windows Server 2008, but supposedly does for Windows Server 2008 R2 I’ll reproduce the script here for you benefit:

strComputer = "."
Set objWMIService = GetObject("winmgmts:" _
    & "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set colItems = objWMIService.ExecQuery _
    ("SELECT * FROM Win32_MountPoint")

For Each objItem In colItems
  WScript.Echo "Directory: " & objItem.Directory
  WScript.Echo "Volume: " & objItem.Volume
  WScript.Echo
Next

When pasted into notepad, saved as a.vbs file and run the cscript, the output is similar, but not as well formatted as mountvol:

A while back I wrote an article for Simple Talk concerning one way to reset a Windows password on a machine that you have physical access to. I decided to make an accompanying screencast to show it in action. Below are two identical videos. One on YouTube, the other on Vimeo. Choose the video site that you prefer the best.

Note that the YouTube video is rendered in 720p so crank up the quality and watch in full screen. For some reason the Vimeo video isn’t in HD even though I have one HD upload per month.

Vimeo

YouTube

Etc. Notes

Let me know if you spot any glaring inconsistencies. The presentation portion of the screencast was made with Prezi. I used the Windows version of Camtasia Studio 7 to make the screencast.

Do you have any topics that you’d like to see explained in a screencast? Let me know in the comments below.

(P.S. Yes, I realize now that I start sentences with the words “so” and “now” far too much. I’ll work on fixing that in the next screencast I do.)

Earlier this week, I found out how to copy file and directory permissions using RoboCopy. To quote Jessica @UberGeekGirl DeVita after I tweeted about my adventures with RoboCopy:

Robocopy ftw

However, there are two main disadvantages to RoboCopy that I’ve come across so far.

RoboCopy Disadvantage #1

It needs the .NET Framework 2.0. Okay, that’s not a huge disadvantage because there’s not much that you can do with a Windows server without the .NET 2.0 framework. Remember, Server 2003 was initially branded as “Windows .NET Server.” However, if you’re still hobbling along with older versions of Windows or you’re really, really obsessed with removing possible attack vectors, you’ll need to keep this requirement in mind.

RoboCopy Disadvantage #2

It is a CLI tool. I know, I know. You’re probably howling “That’s not a disadvantage!” And really, it’s not a huge disadvantage. However, if you only interact with it occasionally then you might prefer to deal with a nice GUI that can help you navigate the myriad of options available. Of course, if you want a GUI to layer on top of RoboCopy there are several to choose from so you’re not completely left in the dark:

• Better RoboCopy GUI
• RoboCopy GUI
• SH-Soft RoboCopy GUI

I admit, the above two “disadvantages” are rather wispy. However, it’s possible that you could bump into them. And even if you never do bump into them, you might still want to consider an alternative to RoboCopy.

Enter RichCopy

RichCopy is a tool that was initially developed internally at Microsoft back in 1996. It was developed by Microsoft employee Ken Tamaru. If the above to disadvantages are of any significance to you, then RichCopy might be of special interest because it flies in the face of both.

From my preliminary investigation, one product is not a superset of the other. It appears that some features of one are not in the other and vice versa. For example, RichCopy doesn’t seem to have the ability to build a directory structure using empty folders and zero-length files like RoboCopy can with the /create switch. RichCopy does, however, support FTP transfer and the saving of commonly used preferences to different profiles for easy reuse (wow do I wish that was a RoboCopy feature). Much is made of RichCopy’s multithreaded nature, however many people seem to be unaware that RoboCopy is also multithreaded with the use of the /MT[:n] switch. In fact, the latest versions of RoboCopy default to seeding /MT: with the value 8.

Another feature that RichCopy has that I can’t seem to replicate in RoboCopy is the “Consolidate Multiple Sources” option. RoboCopy selects file on a [source] [destination] basis, whereas RichCopy can copy based on multiple sources, however the contents of the multiple sources are consolidated under a the single parent directory of the destination. That can snarl you up a bit if you’re not familiar with the consequences of concatenating multiple folders’ content.

Here’s a quick tip for anyone who decides to try RichCopy: Immediately turn on the display of advanced options by selecting View >> Advanced Options. This allows you to see many more of the options available on the File Copy Options window.

RichCopy’s executable can be ran at the command line and includes an extensive list of switches, however they are not at all similar to RoboCopy’s switches so be prepared to memorize a whole new set. An unexpected bonus in RichCopy is that there is a massive list of errors that you can choose to have cancel the entire copy operation:

Sure, that feature by itself is pretty cool, however I found it cooler that I now have a list of error codes and their human-readable messages. Of course, that also brought up this topic:

RichCopy’s GUI is no-nonsense, but not daunting for casual users (I think? Perhaps I’m a bad judge of non SysAdmin types). You can control just about every aspect of file copying that you could conceivably need in most situations (a notable exception being the creation of empty files and folders). The exclusion and inclusion lists are impressive:

If you’re looking for a RoboCopy GUI, perchance look at RichCopy instead. It has most of the functions and can even access FTP shares (not that any of us are still using FTP). The feature to save commonly used options as individual profiles could come in very hand for frequently performed file copying activities. Saved profiles are accessible through the command line tool as well!

Do you have a RoboCopy alternative that you prefer? Or is RoboCopy forever unparalleled? Let me know in the comments.

My Problem

I have sometimes had to copy the permissions on one directory over to another directory. Sometimes it’s simply due to the migration of files from one server to another. Other times it’s for the purpose of backing up ACL entries before an ACE edit. When frobbing around with permissions, it’s often nice to make a dummy folder or file and copy the pre-frobbed permissions over. That way any post-frobbing disasters can be rectified rather quickly.

At first, I thought copying permissions would be a simple matter of using icacls to perform some kind of permission dump. Sadly, and somewhat surprisingly, I was not able to find an easy way to do that. Certainly you can simply pipe the output of icacls to a text file, however I could not find an easy way to consume that text-based permission record. That’s where the wonder of RoboCopy comes in.

My Solution

Reading Microsoft KB323275 reveals yet another interesting use of RoboCopy.

robocopy [source] [destination] /secfix [include appropriate exclusion filters here]

Yes, once again RoboCopy comes to the rescue for things other than copying files and folders. If you’ve been an admin on Windows boxxen for terribly long, I hope you’re at least moderately familiar with RoboCopy.

In this usage, have a peek at the /secfix switch. The official TechNet help for the switch simply says:

Fixes file security on all files, even skipped ones.

However there is a larger note at the bottom of the help document that states:

When using the /SECFIX copy option, specify the type of security information you want to copy by also using one of these additional copy options:

• /COPYALL
• /COPY:O
• /COPY:S
• /COPY:U
• /SEC

If you’re simply performing a permissions copy, make sure that you use the proper file selection options (/XO, /XN, /XF, etc.) to get only the files and/or folders you want. No use in copying an entire directory structure if you only need the permissions on one folder to be copied. Also, if you’re only interested in copying permissions and not the files themselves, keep in mind the /CREATE switch:

/CREATE - Creates a directory tree and zero-length files only.

Perhaps I should start collecting various uses for RoboCopy and compile them into a series of “Stupid RoboCopy Tricks” posts. =)

(This is the sequel to “How Does a Windows Administrator Solve Every Problem?“

Apologies to Hyperbole and a Half)

Want to know how to find your Windows Product ID? I won’t tell you right away. Keep reading and I’ll clear up some common misconceptions that you might not know you have.

Recently I dove into the topic of how to discern what installation media was used to install Windows. It’s possible to find that information out using the Product ID number. The search engine results for anything Windows Product ID related were disconcerting. There is a lot of confusion around what the Product ID is. Before I show you how to find the Product ID, let me tell you what it is and what it isn’t.

What the Windows Product ID is Not

The Windows Product ID is not your Product Key (also known as your License Key). The Product ID is not the code that you type in to install Windows. A Product / License Key looks like the following:

XXXXX-XXXXX-XXXXX-XXXXX-XXXXX

That’s five sets of five numbers separated by dashes. To reiterate, the Product ID is not the above number. The above number is the Product Key (aka License Key).

What the Windows Product ID Is

The Windows Product ID is a 20-character number that follows this form:

12345-123-1234567-12345

That’s a five digit number, followed by a three digit number, followed by a seven digit number and finally a five digit number. The Product ID is a number that is generated based on the Product Key (the thing that you pay money for and can install Windows with). The Product ID is then combined with a “Hardware ID” that is generated based on the types of hardware that you have in your PC. Those two things combine to form the Installation ID. When you activate Windows, the Installation ID is associated with the Product ID.

What the Windows Product ID is Good For

Apart from being an internal number that Microsoft uses to make sure your copy of Windows is genuine, it does have a few surprising uses. You can determine what installation media was used to install Windows from it. You can also figure things out like the Microsoft Product Code (MPC) for the installation which tells you the locale and even if it was an upgrade or not.

How to Find Your Windows Product ID

Finally we come to what you probably wanted to see all along. How to find the Windows Product ID. Ultimately it’s located at the following registry key:

HKLMSOFTWAREMicrosoftWindows NTCurrentVersionProductId

And you can navigate there through RegEdit or via the command line using the “reg” command like this:

reg query "HKLMsoftwaremicrosoftwindows ntcurrentversion" /v ProductId

Notice that you must use quotes since there is a space in the key’s name and you have to run the command in an elevated command prompt.  When you have your Product ID, you can then do some interesting things with it like learn what media your installation came from and if it was an upgrade or not.

Aside from that, the Product ID will probably never be something that you have to write down or keep track of.

EDIT: As commentor Brian points out, the media type for all Vista and beyond installations is the same. Media type was only different in XP and prior versions of Windows. The license key that was used to install Windows is what will now determine the channel ID. I’ve had a hard time tracking down documentation on this subject, so it’s a bit fuzzy. However, I still remain skeptical that the media files between OEM, TechNet and MSDN are completely identical in Vista and beyond. I have no proof of this though, and it remains to be tested if my suspicions are true.

Far too many times, I’ve troubleshot a Windows PC and come to find out that the image was made from media that did not match the license that I was trying to work with. Unfortunately, I know many IT Professionals that use MSDN or TechNet images in a pinch for production machines, and rationalize that “It’s the same bits, and I really do have the license for it, I just don’t have the right media at this moment.” That’s true, to an extent, but it’s still completely illegal and seems to have a technical detriment at times as well.

While Vista and beyond theoretically use the same media regardless of TechNet, OEM, Retail and etc, I still have my doubts. Nonetheless, the license key used to install Windows is still very important. Many times I have suspected that a TechNet or MSDN license was used to activate Windows in a production environment, but had no knowledge of how to discern the truth of the matter.

Was this PC installed from the MSDN image or license? Maybe an OEM disc that someone had laying around? Perhaps a Volume License image? I suspected that there was a way to tell, because in many instances certain Windows features didn’t behave like I thought they should when the image was from TechNet or MSDN. There seemed to be a way that Microsoft “just knew” that the image wasn’t from the media type that it should have been.

While I don’t know about any tell-tale signs deep in the Windows bits, I now know that there is a high level way of discerning a Windows image’s origins. Thanks to this ServerFault question “Which media was used to install Windows 7“

I saw it and decided to launch into an investigation. I had had that very question running through my mind many times, but could never get to the bottom of it. In fact, after sifting through a mountain of search engine results to try and answer the ServerFault question, I still couldn’t find an answer. I favorited the question with the hopes that someone would answer it in the coming weeks or months. Fortunately, I didn’t have to wait that long. The question’s author found the answer just a little while later.

The crux of the matter is within the Windows Product ID and how one interprets the numbers. A Windows Product ID looks like this: 12345-123-1234567-12345. Notice that the Product ID is not the Product Key, the latter being what you are essentially paying for when you buy Windows. Searching for information on how to find the Product ID comes back with plenty of misguided articles that confuse the two. Here’s Microsoft way of finding the Product ID for some of the most popular iterations of Windows.

You can also find the Windows Product ID at the following registry key: HKLMSOFTWAREMicrosoftWindows NTCurrentVersionProductId

Oddly, I found the Windows Product ID at this seemingly unrelated key: HKLMSOFTWAREMicrosoftInternet ExplorerRegistrationProductId

The major source of information for how to interpret the Product ID number is from a free tech support community (that I had not heard of before this topic came up) called LunarSoft at their Windows Product IDs page. Searching around for other sources of Windows Product ID information finds that everyone seems to be gathering their information from them, even answers on Microsoft’s own support forums will link back to their Product ID page. If anyone knows where the official Microsoft information can be found, let me know.

The key part of the Product ID that is important for discovering what image was used to install Windows is the “Channel ID” – the three digit number that is the second number in the four number PID. In my case, my Channel ID is 292, however that number isn’t on the list at LunarSoft. Apparently, while LunarSoft’s list is great, it is a bit dated. You can see this forum thread that makes mention of the outdated nature of the list.

There is still some confusion, but apparently 292 stands for Windows Ultimate Retail, which stands to reason since my installation is Windows Ultimate installed from a disc I scored for free at an official Windows 7 launch party in Pittsburgh. I think the list of Channel IDs is in need of some confirmation, but I can’t find any official documentation on the subject. However, between LunarSoft’s Windows Product ID page and the forum thread over at MyDigitalLife I think you should be mostly taken care of.

Once you have your Channel ID, compare it to either LunarSoft’s list or the MyDigitalLife forum post and you’ll have a pretty good idea of what media was used to install Windows. I’ll be on the look out for any official and up-to-date documentation on the Channel ID in the mean time.

Do you know of a better way? Have any insights on official documentation? Let me know in the comments below.

As Preston Kutzner recently said to me, NTFS is a harsh mistress when accessing it via Linux. I am using Fedora 14 and have a LaCie 2big external hard drive connected via USB. The 2big is configured as a RAID 1 set using it’s own built-in RAID hardware. When I try to open the drive in Nautilus I receive the following error:

Unable to mount LaCie 2Big.

Error mounting: mount exited with exit code 12: Failed to read last sector (1953519615): Invalid argument

HINTS:

• Either the volume is a RAID/LDM but it wasn’t setup yet,
• or it was not setup correctly (e.g. by not using mdadm –build …),
• or a wrong device is tried to be mounted,
• or the partition table is corrupt (partition is smaller than NTFS),
• or the NTFS boot sector is corrupt (NTFS size is not valid).

Failed to mount ‘/dev/sdc1′: Invalid argument. The device ‘/dev/sdc1′ doesn’t seem to have a valid NTFS. Maybe the wrong device is used? Or the whole disk instead of a partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

Being new to the world of Happy Little Penguins I spent a full day Googling, learning plenty of new things at every step and button mashing in the form of running shell commands that I had only just learned about moments earlier.

I’m certain that through this whole ordeal a solution existed and that evidences of the problem were staring me in the face the whole time. I am further certain that a more elegant solution existed than the scorched-earth one that I chose (that you will find out about in just a moment), but I am too much of a neophyte to pick up on them much less be able to act on anything I might have noticed.

I could see the 2big in the /dev folder at /dev/sdc1. The 2Big is listed in /dev/disk/by-id as the following:

usb-LaCie_2_BigQuadra_00D04BA80A0443AE-0:0
usb-LaCie_2_BigQuadra_00D04BA80A0443AE-0:0-part1

It is listed in by-label as the following:

LaCiex202Big

It is listed in by-path as the following:

pci-0000:00:1d.7-usb-0:3.4:1.0-scsi-0:0:0:0
pci-0000:00:1d.7-usb-0:3.4:1.0-scsi-0:0:0:0-part1

Finally, it is listed by-uuid as the following:

lrwxrwxrwx. 1 root root  10 Mar  2 14:29 3E421CD2421C90AF -> ../../sdc1

fdisk -l /dev/sdc shows the following:

Disk /dev/sdc: 1000.2 GB, 1000153686016 bytes
255 heads, 63 sectors/track, 121595 cylinders, total 1953425168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xe7479c04
 
Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            2048  1953521663   976759808    7  HPFS/NTFS

I created a new folder /mnt/2big and ran mount /dev/sdc1 /mnt/2big and received the same error as I did in Nautilius (as if I wouldn’t have?). I decided to search the generic error, rather than focus on anything to do with the LaCie drive. “Error mounting: mount exited with exit code 12” brought back some interesting things. One of the “solutions” was to reformat the disk with gparted. That is not an option I wanted to exercise unless as a last resort.

I then tried: ntfsfix /dev/sdc1

Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
NTFS volume version is 3.1.
NTFS partition /dev/sdc1 was processed successfully.

I then rebooted into Windows expecting that a chkdsk would be automatically requested to be performed. It was not, so I booted back in Linux to try a few more things before attempting a manual chkdsk within Windows. I installed testdisk thinking that I could perform some kind of partition table rebuild with it. After analyzing the 2big I received this interesting error:

Disk /dev/sdb - 1000 GB / 931 GiB - CHS 121596 255 63
The harddisk (1000 GB / 931 GiB) seems too small! (< 1000 GB / 931 GiB)
Check the harddisk size: HD jumpers settings, BIOS detection...
The following partition cant be recovered:
 
Partition               Start        End    Size in sectors
HPFS - NTFS              0  32 33 121601  25 24 1953519616 [LaCie 2Big]
 
[ Continue ]
 
NTFS, 1000 GB / 931 GiB

At about that point, someone wondered what NTFS driver I was using so I made sure that I was using ntfs-3g. I then decided to use Cfdisk to do some probing. Cfdisk /dev/sdb1 got me this error:

FATAL ERROR: Bad primary partition 0: Partition ends after end-of-disk

So apparently the partition was sized larger than the disk. And Windows is okay with this and will perform without complaint? No attempt to fix it is made? Insert angry face here.

I then plugged the 2big into a Windows Vista machine intending to chkdsk it, but as the disk was being mounted I saw a dialog box warning “Do you want to scan and fix LaCie 2big? There might be a problem with some files on this device or disc. This can happen if you remove the device or disc before all files have been written to it.” Interesting, so suddenly there was some kind of file system error detected. There was an option to scan and fix, but since I was not 100% sure what command would be run, I continued without scanning. Instead I manually ran chkdsk using the /F /V and /X options (Fix errors, show messages and force volume dismount respectively). I plugged the drive into Fedora but still received the same errors as before when mounting it.

At that point, quite a lot of time had been spent researching and testing, so I decided on the easy fix. I had already synced the 2big to another drive before attempting this project so I gparted it (I was too lazy to even try parted), created a new partition and then copied everything back to the 2big. Problem “solved”.

I know that there was likely a less destructive way of ending that saga, but I haven’t had sufficient beatings with the Linux cluebat to know how. Is dealing with NTFS always this frustrating on Linux? What practices and standards has Microsoft been using with how Windows interacts with NTFS volumes that it can apparently have an unhealthy partition table and still work without warning or fixing the problem? I’d seriously consider using ext4 for all my drives and using a plugin like Ext2Read within my windows machines if I didn’t often physically share some drives with other people’s Windows machines.

What are your experiences with NTFS on *NIX machines? Exceedingly painful or am I doing it wrong?