SCP File Transfer Stalls

If you are trying to do a large data transfer to (or from) an ARSC system from outside of the ARSC network, and the SCP file transfer stalls, this could be due to network overload, firewall problems, or some other technical issue, on the network between your system and ARSC. These sorts of issues can be difficult for us to investigate because the offending network equipment is usually not under our direct control.

An easy possible solution is to use the "rsync" program instead of the "scp" program. For example:

rsync -P --append -e ssh local_file user@host:remote_file

This fundamentally does the same thing as the scp utility; however, when rsync is used with the -P and the --append flags, it preserves partially transferred files on the remote end and will append data onto the end of this file if the command is restarted. This means that, if the transfer fails for some reason, you can run the command again and it will continue where it left off, rather than starting from the beginning.

(We have included the -e ssh option to be safe, but on most systems this is the default.)

Please remember that there are differences between scp and rsync which will become significant if you attempt to craft more complicated rsync commands. In particular, rsync behaves differently depending on whether or not a directory name is suffixed with a trailing forward-slash, whereas scp does not make such a distinction. For example, these two commands do NOT do the same thing:

scp -r local_dir/ rsync -r local_dir/

To learn more about the rsync interface, please read through the RSYNC(1) manual page, especially the USAGE section:

Back to Top