Copy NTFS to a smaller partition with OSS tools

If you want to "upgrade" a computer with a HDD to using a smaller disk, usually a SSD, you will need to come up with a way to shrink your windows partition. There are some for-pay software packages which do this for you, but no OSS tool which will do it with a simple click. There's a slightly meandering sequence of steps necessary to get everything working correctly. Your best allies are the gparted livecd, and a USB to SATA (probably) adapter to let you hook both disks up to your system at once. This process should work for basically any version of Windows from 2000 on, but I used it specifically with Windows 7.

First, a warning. This process could easily eat your data if it goes wrong. Make a backup first. Read the entire article before attempting this process.

Briefly, the process works like this:

  1. Copy the partitions which can be copied
  2. Shrink the partitions too large to copy
  3. Create a new partition for the migration
  4. Copy the now-shrunken filesystem from the old partition
  5. Grow the copied filesystem to fill the new partition

I went through this process on a Dell Inspiron Mini 10V, which I am keeping around basically just to run VCDS so that I can use it for problem diagnosis on my Audi A8.

Copy Partitions

On many if not most systems, Windows creates two partitions. The first partition is a 'boot' partition which is used to keep all of the boot files near the beginning of the HDD. Some older computers, even some which support Windows 7, won't boot operating systems if their boot files are located after the 2GB mark on a disk. If you have more than one partition, you need to copy them all for Windows to boot correctly. If you have only one partition on your disk, skip this step entirely.

The easiest way to copy those partitions is with 'dd' or even 'cat'. You want to copy the data directly from one disk to the next. In my example, the source disk was /dev/sda and the target was /dev/sdc, while my gparted live usb was /dev/sdb. The easiest way to do this is to simply dd if=/dev/sda of=/dev/sdc bs=1M. For the uninitiated, this means copy data from /dev/sda to /dev/sdc, in blocks of 1MB. By default, dd writes a byte and not a block at a time. You want to write in blocks of at least 4 kB, which is a typical hard sector size for SSDs, or a multiple thereof. If you want the command to terminate when you actually don't need to do any more copying, you should also figure out how many MB those partitions are, add a couple dozen for good luck, and add "count=number" to the dd command line. If you don't, though, it will just waste time writing everything it can to the other disk, and then the command will terminate and you can proceed.

Shrink Partitions

Shrinking Windows partitions can be a bit of a pain, because of the MFT. The "master file table" is a backup of part of the information Windows needs to keep track of your files, and it is located in the middle of the partition. However, the free tool ntfsresize can handle this problem. This is located on the gparted livecd (or live usb, if you have created one.) Once you have removed enough content from the partition to shrink it, and defragmented the files and relocated them to the beginning of the partition, you use ntfsresize to do that job. I used Auslogics Disk Defrag Free to handle that part. It doesn't matter much which defragger you use, as long as it has free space consolidation. This will relocate everything but the MFT so that the filesystem can be shrunk.

Once you've defragmented in Windows, you can boot gparted live, get into the command line interface (you will be prompted during boot to start a shell) and actually shrink the partition. The smaller the better, but obviously the size needs to be between the smallest size that it can be, and the size of the new partition. Then you use the ntfsresize command to actually do the dirty work. First you figure out how small the partition can be, then you resize it. So first I did:

ntfsresize --info /dev/sda3

This gave me an amount of "space in use" in megabytes. I then added a couple of hundred onto this number for good luck (and so that if Windows has to create any files during boot, I don't cause a problem) and then resized the partition to that size.

ntfsresize --size numberM /dev/sda3

Where number is the new size in megabytes. Once you have shrunk the filesystem, it is marked as dirty so that it will be automatically checked at boot. If you are paranoid, go ahead and boot back into Windows and let it check the disk, then boot gparted live again. Otherwise, try ntfsfix -d /dev/sdc3 (or whatever your device is) to be able to proceed to the next step.

Keep in mind that you are shrinking your working partition here. You should have a backup before you try this! I did not make a backup and it still came out fine, but I was willing to live dangerously because there was no data on the system when I did this, only the operating system. If I had failed, I could have just done a fresh install on the SSD and lost only time.

Create a new Partition

Once you have copied the partitions you can copy, you need to make a partition to migrate your old partition's data into. If you just copied the entire disk, you've got a bogus partition now which not only cannot be used, but whose extents are actually outside the physical extents of the disk. This can only cause problems. You want to delete that partition. For me it was partition 3, because Dell hides a secret partition used for diagnostics or something on the disk. You can delete it with parted, fdisk, cfdisk... or if you booted gparted live into the graphical mode (you can open a shell from there, too) then you can even use parted. Then you want to create a new partition to replace it, and the new one should be the same kind as the old one, and start on the same sector.

How do we actually accomplish that? One way to do it is with parted. parted /dev/sda unit s print will print your partition table, with sizes and start points listed in sectors. We don't actually care about the end sector, because we want to fill up the disk. We only care about the starting sector, because we want our partition to start in the same exact place. Run parted /dev/sdc, delete the last partition from your new disk, then recreate it. print will print your partition table; rm 3 would remove partition 3, and mkpart starts making a new partition. You want the "type" to be ntfs, and the "start" to be the starting sector, plus an s on the end. So if you were starting on sector 232384512, you'd want to type 232384512s. The end of the partition should be the end of the disk.

If you wanted to copy your filesystem exactly, you'd want to jump through more hoops to figure out what the ending sector of your disk was, but we don't care about that at all. We're trying to resize the partition to fill the available free space, right?

Copy the Filesystem

Now that you have someplace for the data to go, you can actually copy it. We do this with ntfsclone:

ntfsclone \
    --force \
    --overwrite /dev/sdc3 \
    /dev/sda3

This example copies the filesystem directly from sda3 to sdc3. It may actually be faster to back up to an image and then restore, but this is simpler. Once you have done this, it is actually possible to boot the system, but you will find that the reported size of the filesystem is whatever you've shrunk it to. It won't automatically use the whole partition. There are some other steps to go through to get to that point.

Grow the filesystem

This was actually shockingly easier than I thought it would be, because NTFS is a modern filesystem that is happy to be repaired. The steps are performed with testdisk, another utility which is included in the gparted live distribution. First you run testdisk, then you:

  1. Choose disk (/dev/sdc in my case)
  2. Choose Intel
  3. Choose Advanced
  4. Choose with up/down arrows the correct partition and with left/right arrows the Option [Boot]
  5. Choose [Rebuild BS]
  6. Testdisk complains that "Extrapolated boot sector and current boot sector are different." - Choose [Write]
  7. Confirm with Y and Quit

Now, one of two things will happen. If you do have a boot partition, then Windows will fail to boot, then it should boot again and then offer you the chance to repair your system. Go ahead and do this, once it's done, you're done. If this does not happen (for any reason) then you will need a Windows 7 boot CD (or USB) to correct the problem. Use it to boot into the recovery console, then run FSCK /F C:. This is essentially what Windows does during system repair anyway. Reboot, and you should come up in windows with the filesystem using the complete partition.

Summary

You could pay fifty bucks for a piece of software which would just transfer your system over for you, or you can employ a little adaptability and do it for free with OSS tools with horrible interfaces. When I put a Samsung SSD in my desktop, their migration tool actually left my system unbootable. Meanwhile, this procedure leaves you in full control of the process; if you make a mistake, you'll probably know exactly where.

Add new comment

Default

  • Use [fn]...[/fn] (or <fn>...</fn>) to insert automatically numbered footnotes.
  • You may link to images on this site using a special syntax
  • Web page addresses and e-mail addresses turn into links automatically.
  • To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.
  • Internal paths in single or double quotes, written as "internal:node/99", for example, are replaced with the appropriate absolute URL or path. Paths to files in single or double quotes, written as "files:somefile.ext", for example, are replaced with the appropriate URL that can be used to download the file.
  • Filtered words will be replaced with the filtered version of the word.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote> <q>

Issue

  • Lines and paragraphs break automatically.
  • To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>

Drinking Game

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <img> <p> <br> <pre> <h2> <h3> <h4>
  • Images may be embedded like: [image:node_id align=alignment hspace=n vspace=n border=n size=label width=n height=n nolink=(0|1) class=name style=style-data node=id] Leave off any attributes you don't want.
  • [img_assist|...] tags will be displayed, maybe. Please don't make more of them.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Refresh Type the characters you see in this picture. Type the characters you see in the picture; if you can't read them, submit the form and a new image will be generated. Not case sensitive.  Switch to audio verification.