|
As of February 25, 2003, the lab does not allow any file transfer method that utilizes clear-text password authentication with the exception of anonymous FTP. To facilitate off-site file transfers, the computer center provides several securely authenticated services via the ftp.jlab.org system.These are as follows:
- SRM - the Storage Resource Manager - for files known to the mass storage system (or importing files destined to be written into the silo), the SRM service provides transparent access, that is secure and attempts to fully utilize available network bandwidth.
- BBFTP - for arbitrary disk files on the /work or /cache file systems, bbftp is secure and does parallel transfers to maximize throughput
- SCP - for copying individual files via ssh protocol
- SafeTP - a secure replacement for traditional ftp services
- SFTP - a secure replacement which tunnels ftp over ssh.
When required, anonymous ftp service is available for read-only access.
scp
scp is an application that replaces rcp for remote file coping. Users can use the machine ftp.jlab.org for scp access to /home, /group, /work, and /site filesystems.It can also be used from login1 and login2 directly for transfers to and from /home, /group, /scratch and /site :
scp -p tmp/tabletest.pl wilma.widomaker.com:tabletest.pl
This example shows a single-line command to copy a file from the /home directory:
wilma> ssh -t login2.jlab.org scp -p tmp/table.pl wilma.widomaker.com
SafeTP Service
The machine ftp.jlab.org provides a SafeTP server which has access to /home, /group, /work, and the /site filesystems.SafeTP is a packaged developed at the University of California at Berkeley to provide a secure method for file transfer between Unix or Windows clients and secure FTP servers on Unix or Microsoft WindowsNT/Windows 2000. The Berkeley documentation is available at http://safetp.cs.berkeley.edu/ . The currently supported version is 1.46 which was released November 27, 2000. SafeTP benefits include:
- Multi-platform Secure authentication and secure data (client option)
- Transparent - works with existing terminal based and GUI based FTP clients
- Freely available
For an FTP client, a part of the SafeTP suite transparently interposes itself between outgoing ftp traffic from any standard FTP client to provide secure authentication when it detects a connection to a secure FTP server.
Further detailed information including software installation and configuration can be found on the JLAB SafeTP web pages.
sftp Service
The machine ftp.jlab.org provides a sftp server which has access to /home, /group, /work, and the /site filesystems.sftp is ftp tunneled over ssh. sftp clients are readily available from the Unix command line, or with applications like PUTTY. See the man pages for usage details.
NOTE: Because sftp requires an ssh server to be run on the machine, we have implemented a restricted shell which will prevent the machine from being used as another interactive login server. You will see an error message if you ssh to the machine ftp.jlab.org. Remember this machine is for file transfers only. Use login1 or login2 for interactive login sessions.
Anonymous FTP Service
The machine ftp.jlab.org supports anonymous ftp service for file retrieval only. It does not support write access. The file system served is available at /site/ftp. Users sharing files using anonymous ftp should ensure that the information is suitable for general public distribution as those files are readily available to anyone in world.
BBFTP Service
bbftp (http://ccweb.in2p3.fr/bbftp/) provides an alternative to ftp for wide-area file transfers, that:
- encrypts username and password using SSL
- optimizes file transfer speed by using large TCP window sizes and by copying each file as several parallel transfers
- clients are available for Linux, Solaris (2.6 and 7), HP-UX 10.20, AIX and IRIX.
- accepts a list of commands in a file.
This service is to be preferred over traditional ftp, since it is both more secure and tries to optimize network bandwidth usage. The facility can be used to transfer files already present on disk (/work or /cache). Since it is more secure than traditional ftp the filesystems are mounted read-write and the service can accept incoming file transfers. Users' home directories and group areas are also available for this service.
Method of Use:
The server for bbftp is bbftp.jlab.org.
The standard bbftp command takes an input command file containing a list of instructions (cd, put, get etc). Options are available to enable compression, specify the number of parallel transfer streams etc. This command is suitable for command line use - the user name and password are supplied when the command is invoked.
However, for use in a script bbftp would normally need the username and password to be stored in a file. This is obviously not a good solution. For this reason the bbftpcd and bbftpc commands should be used. The user should start the bbftpcd daemon to contact the remote host - supplying username and password. This daemon will just run in the background on the user's machine and keep the username and password in memory. Then the bbftpc (client) command should be used in a script to initiate the actual file transfers - it talks to the local daemon (bbftpcd) which in turn authenticates the user to the remote server and does the file transfer.
Full details of the use of the bbftp command is explained in the man pages - for bbftp, bbftpc and bbftpcd.
Observed Performance:
A few tests of bbftp were done between JLAB and ODU. All tests were for files being copied from JLAB to ODU.
Without Compression With Compression p value CPU Utilization mbit/sec CPU Utilization mbit/sec 1 5.6 5.8 2 10.9 10.9 3 14.5 100% 15.9 4 20.7 16.0 5 23.4 16.4 6 25.7 16.6 7 20% 27.7 16.6 8 25.8 16.6 9 24.6 16.4 10 22% 21.9 16.4SRM - Storage Resource Manager
The SRM utility is a grid-enabled middleware tool for transferring a large amount of data between JLab and a remote site.
This document is maintained by {helpdesk@jlab.org}
Copyright Jefferson Lab 2007