Today I can go no farther (part 6)

This is part of the hard disk recovery documentation.

Part 6.

Today I can go no farther (so I stopped)

The last days of this project are spent on two tiring tasks that do not gain me very much, but must be done to carry this project to its logical conclusion. One of these is to decrypt a few very small, but important NTFS-encrypted files. The other is to wring the last readable bits out of the broken Seagate drive by splitting all the error regions to isolate the unreadable regions as much as possible. These can proceed in parallel.

I set Knoppix to slave away on the broken Seagate drive using ddrescue again. This ended up taking more than one day constantly running and made the disk really hot. I had to manually intervene from time to time just so this will get somewhere. This is where I wished ddrescue was smarter about skipping around unreadable regions.

The other task was more interesting from a learning perspective. When a file is to be encrypted by NTFS, it generates a file-specific symmetric “file encryption key” and stores the key with the file. Of course the key isn’t stored in plaintext but is encrypted using the user’s public key and is only decodeable by the user’s private key. Normally the user should export the private key into a .pfx certificate file and store it on a smart card or USB pen drive or something, and more importantly, remove it from the certificate store on Windows. I did make a copy of the private key off the computer. I could just re-import the certificate, and then decrypting the data would be trivial. But the external storage device that holds the private key isn’t with me during the break, unfortunately. On the other hand, I didn’t bother to remove the private key from the computer either, because I wasn’t really serious about this particular piece of data, just toying around with the NTFS encryption feature, mostly.

That made the encrypted file potentially recoverable without delay. Windows doesn’t store the user’s private key in plaintext either, of course. From what I understand, the user’s private key is encrypted somehow with user’s password hash, then maybe it is encrypted again with a master key belonging to the system. Somehow, the system’s master key is the weak link. I think it is stored in the SAM database of the system registry, which is in %SYSTEMROOT%\system32\config, though I may be mistaken on this point. The user’s various keys are stored in %USERPROFILE%\Application Data\Microsoft\Protect and %USERPROFILE%\Application Data\Microsoft\Crypto. Here is another explanation of this.

To make a long story short, if all of these files are available, and the user’s password is known, then it is possible to recover the encrypted files directly from the drive without an explicitly exported private key (hence encryption with these files around is ultimately pointless).

I ended up using the Elcomsoft Advanced EFS Data Recovery tool (AEFSDR). They have a trial version, too. Unfortunately it’s crippleware, unlike Mount Image Pro. All I can say is, if you really want to decrypt files, you find a way around the “cripple” in crippleware. Enough said.

For the record, AEFSDR is pretty good and fast. However, I am also not completely satisfied by it. I haven’t found any bugs, but the problem with it is its incompleteness. I don’t know what file system access method it uses, but it cannot see any removable drives, including removable USB drives or any mounted virtual disks (e.g. by Mount Image Pro). This is severely limiting, because encrypted files are not easily copied (Windows says Permission Denied), and I basically ran into a wall because of this. Please, AEFSDR, go work on this some more. You are selling a recovery tool, so your users are most likely in desperate straits with limited resources… they don’t want to deal with your programming kludges.

The way around it is to use NTBACKUP (part of Windows), which can archive encrypted files to a backup image, and then you can restore these files onto a permanent physical disk. After also copying the security related files from Windows to a permanent physical disk, AEFSDR decrypts all files successfully.


After I got as much as I am ever going to get out of the Seagate disk using ddrescue (I got another few tens of megabytes out of the disk), I retraced the steps on the new image. This time, the image loads as a valid NTFS partition and Windows will read it (apparently the last few bits recovered was pretty important). However, the Windows directory is still not readable, although almost all other directories are directly readable. NFI also works on the image — this is very good, as it means I can finally profile what I lost down to the last sector. I also ran chkdsk on it again — fewer errors to correct this time, but hopefully fewer zero-filled data issues to deal with at the end of the day. This chkdsk’d image is now the gold recovery image. I backed it up immediately.

That is the end of it. The story has a happy ending, but what a giant waste of time. Back up your data.

Lessons today:

  • AEFSDR is nice. But I wouldn’t buy it, either.
  • Remove your private keys from the computer.
  • Near total data recovery is very possible from an apparently dead drive, but a delicate procedure needs to be followed.
  • The right tools are very important, and a complete disk recovery toolchain simply does not exist — you are left to find your own pieces.

On to the Appendix.


  1. mike
    March 8th, 2007 | 15:04

    Sorry you had so much trouble.

    I need to use Elcoms ware unless you know of another better and or cheaper.

    I can not boot into drive, but efs is the problem.

    Using another computer with the lame disk attached as a slave what do I do to use NT Backup ?

    USB attachment seems the only way to attach the troublesome drive.

    XPPro is the OS.

    Got any suggestions or help–appreciated. !!!!

    Mike 3-8-07

  2. me
    March 8th, 2007 | 20:04

    Elcomsoft is probably the thing to use. So, if the file system is readable, then just start NT Backup, which comes with XP (it’s under Accessories -> System Tools -> Backup), go to the “Backup” tab and select the encrypted files from your USB attached drive and save them to a backup archival file. Then “restore” the backup to a new location (it will ask you where the files should go.) That’s it really. Once you have your files on a primary drive, you can try the decryptor. You really need the folders I mentioned that contain the crypto data, and a password to your user account, also. If you haven’t changed keys or passwords on your account, the recovery should be straightforward …

    I mentioned getting around the “cripple” in AEFSDR. Google will help you there, but see whether any files are reported to be recoverable first.

Leave a reply