How do you limit the bandwidth for a file copy?

I've got an old windows 2000 box in a remote location with a T1 connection and a vpn to my location. I normally use smb mounts to transfer files but now it's time to decommission the server and copy it's backups to my location. I have about 40 gigabytes (compressed) to copy. I'm prepared for it to take a long time, but I have a few caveats.

• I need to limit the bandwidth so terminal service connections to the site are not affected
• I want to be able to resume a partial transfer

There are a few small files and several large files (10-20 gigabytes). I'm familiar with rsync on *nix platforms but have had bad luck with windows and I don't know that it will really keep partially transfered files. What do you use?

From serverfault wizard

• robocopy.exe has a switch called inter-packet gap, allowing you to insert a time window in between the packets of your copy, and thereby reduce the impact on the channel.

  It's not exactly "use no more than 30% of the available bandwidth", but you can acheive the same effect with a little math. You can always specify some number of milliseconds and let it run for a bit, then CTRL+C to interrupt, adjust your command as needed, then resume. I've done just this when I didn't want to overload the WAN during the business day with massive replications.

  robocopy has another switch /z allowing for "resumeable" transfers, so if the transfer is interrupted you can pick up where you left off, and don't need to shift the whole 40 GB again.

  I'm not sure how granular the resume bit is, because I've never really tested for example a single 40GB ZIP file. Test something smaller first ;-)

  There are some nice GUIs for robocopy which can assist with the syntax, but anyone with a Linux background will grok it easily. Grab the latest versions from a copy of Windows 2003 or later. Otherwise you'll find it as a Windows 2000 Resource Kit Tool download.

  In the Wikipedia entry for robocopy, someone noted that the penalty for restartable copying (the /z switch) is 6x slower performance (see Known Flaws).

  nray : You're doing it wrong - robocopy has been a tool of choice for years now, tested on any number of different networks. The "robo" is supposed to mean "robust".

  KPWINC : +! for Robocopy. Used it several times like this and never had an issue.

  Avery Payne : +1 for a well-known, well-used tool. Strange as it sounds, it's sometimes easier to plunk robocopy down into a (ick) batch file and have it do repeated copies this way via System Tasks.
  From nray

• You could try setting up FTP on the 2000 box, and using a FTP client at the other end, like filezilla to deal with partial copies.

  Oskar Duveborn : +1 To mention the bandwith limits available in Filezilla wouldn't hurt.
  From Dan

• 40 GB of files are going to take more than 60 hours to copy over a T-1. It might be easier to mail an external hard drive in both directions. You can have someone plug it into the server, copy/encrypt your files and then have them mail it back to you.

  davr : looks like about 60 hours according to my math: 40GB * 1024(MB/GB) * 8(bit/byte) / 1.5mbit / 3600(sec/hr) = ~60 hours. Of course there will probably be overhead and stuff, so will take a little longer, so maybe say 70 or 80 to be safe? But 1.5mbit is pretty slow, even a really old PC should have no trouble feeding data at that rate.

  nray : "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." —Tanenbaum, Andrew S. (1996)

  wizard : The utility of spare hard drives is lost on my office. I don't mind waiting.
  From Carl Campos

• You say you've used rsync on *nix, there are several ports of rsync to windows you might look into. For example:

  • RsyncWin32
  • DeltaCopy
  • cwrsync

  Personally I've found rsync pretty reliable, and it can handle interruptions in the middle of transfers and so on. Rsync also has a flag to limit transfers to an exact kbit/sec transfer speed, so that would seem to work for you as well.

  From davr

• I use Robocopy to back up all of my servers as well as a live Mailserver backup. Once it is set up right, its truly set it and forget it. Best thing that ever happened to my IT life. Just putting in my 32 cents (allowing for inflation)

  From RoadKingRick