|
|
Issue 10 |
March 2002 |
Changes to JLab
Computing Access from OffsiteThe following
information is intended to clarify the changes that will be seen over the next
few weeks as interactive access points to the lab computing services are
tightened.
On the
Unix Openssh client: ssh
-2 login1.jlab.org
Unix Ssh.com client: ssh2
login1.jlab.org
If you successfully
login you will have no trouble after the maintenance day. If you do experience
problems, please contact your local system administrator and have them install
either a current version of openssh or a current version of ssh.com's
SSH client.
Offsite Windows users
will need to upgrade to a SSH client application which supports SSH Protocol 2.
The
Logins from off-site
will be possible only using ssh and only through
dedicated login servers - login1.jlab.org, login2.jlab.org (or via generic name
jlab.org). Access to jlabs1
from offsite will no longer be available. In addition we will
require that all remote logins to these machines be validated with a password;
we will not allow RSA/DSA keys since they can be setup to use a blank or null
passphrase. From these machines you may login to any other machine on site. No
other systems will be permitted to provide direct login access to the outside.
You may set up ssh to connect internally in a single
command as shown in the examples below, but you will be asked for your password
at each step.
Examples:
ssh
-t login1.jlab.org ssh jlabs1
ssh
-t jlab.org ssh jlabh1
ssh
-f -t login2.jlab.org linxone xterm
-name linxone
Refer to http://cc.jlab.org/docs/services/unix/login_server.html
for more details on the login systems.
scp:
Can be used from
login1 and login2 directly for transfers to and from /home, /group, /scratch
and /site :
scp
-p tmp/tabletest.pl wilma.widomaker.com:tabletest.pl
This example shows a
single-line command to copy a file from the /home directory:
wilma>
ssh -t login2.jlab.org scp -p tmp/table.pl wilma.widomaker.com
ftp:
We provide an ftp
service with encrypted login (SafeTP), as well as anonymous ftp on ftp.jlab.org.
Should the remote ftp client not be capable of using SafeTP the server
will fall back to standard ftp, but since this sends passwords in the clear
this fall back will be removed in the future.
bbftp:
Use this for bulk
data transfer (i.e. large files) as it is very efficient, and passwords are
encrypted. Refer to http://cc.jlab.org/docs/scicomp/how-to/off-site-data-transfers.html
for more details on secure ftp and bbftp. The /work and /cache filesystems can
be accessed via bbftp.
NOTE: In the future there will be a dedicated system which
will provide import and export services utilizing scp
and bbftp to the /work and /cache filesystems from offsite. These services will
be provided via the host xfer.jlab.org as soon as testing and configuration of
this service has been completed. Announcements regarding the availability of
this resource will be sent via the weekly
This service will be
provided on login1 and login2 and will make use of ssh.
The existing service on jlabs1 will be withdrawn as it uses a very insecure
method. The following is all that is needed for a remote CVS client:
setenv
CVS_RSH ssh
setenv
CVSROOT username@login.jlab.org:/group/xxx/CVS
Then proceed as
normal (e.g. cvs checkout mystuff
etc) - you will be prompted for a password, which is your usual jlab login
password. In the above example replace "username" with your user id,
and replace "/group/xxx/CVS" with the full pathname of your CVS
repository. The existing CVS remote service (using the pserver)
will be closed on March 19.
Please contact the
Computer Center Helpdesk at helpdesk@jlab.org for
assistance or more information.
On the April 16 Maintenance Day, TeraTerm (ttssh), the SSH client application used by most Windows users to connect to CUE Unix systems (db1, jlabs1, jlabh1…), will cease to function because SSH Protocol 1 support will be terminated. TeraTerm only supports the use of SSH Protocol 1 when making connections. SSH Protocol 1 is not supported under the latest security upgrades which mandate the use of SSH protocol 2 on site. The replacement for TeraTerm will be the SSH client application PuTTY, which does provide SSH protocol 2 support. PuTTY will be available for installation for CUE configured Windows PCs from the standard supported software installation area via Start ->JLab-CUE -> Client Installed Programs. We will be announcing and providing further configuration instructions for PuTTY once this information has been finalized. We have found PuTTY to be compatible with the Ingres (db1) and Xwindows applications currently in use at JLab.
We are currently
evaluating the openssh and ssh.com clients and have made them both available
for local users. Both of these clients provide SSH protocol 2 support, which as
stated previously, will be mandated on the April 16 maintenance day. These
clients can be evaluated on our internal systems as follows:
Openssh client: jlabs1>use openssh2
Ssh.com client: jlabs1>use ssh2
Users will need to
setup the necessary protocol 2 dsa keys in order to
use these clients. This can be performed automatically for the ssh.com client
by executing the following script:
jlabs1>site/bin/update_2_ssh2
This script will
require that a passphrase be entered during this upgrade, and you can either
press return (for a NULL passphrase) or type in a valid passphrase at this
time.
Macintosh users will
not be required to perform any additional reconfiguration of the currently
supported Macintosh SSH client application, Nifty SSH. Nifty SSH supports both
SSH protocol 1 and 2. Please see the JLab Nifty SSH documentation located at http://cc.jlab.org/docs/services/macintosh/nifty_ssh.html.
CUE Password Expiration As part of the general
password requirements set forth by DOE, the
When a JLab user
account is nearing its six month expiration deadline for password a password
change, the following automated email is sent as notification:
Dear
CUE User,
The
password for your account with the username (username)
will expire in x days on xx-xx-xxxx. Please change your password by that date,
either by logging in to any of the central Unix
systems (e.g. db1, jlabs1, jlabh1) and typing /apps/bin/jpasswd. Or, if you are
on-site, you may use the web interface at http://www.jlab.org/jpasswd
.
If
you let your password expire, you will no longer be able to log in to _any_ CUE
system. This includes all central interactive, batch, desktop, mail, calendar and web systems.
This
password policy is explained at:
http://cc.jlab.org/policies/PasswordRules.html.
Please
contact the Computer Center Helpdesk (helpdesk@jlab.org)
if you need further assistance.
Regards,
This message supplies
the username of the account to be suspended, the number of days before the
account is suspended, and the Calendar date that the account suspension will
occur. Upon receipt of this message it is imperative that you make the
suggested password change as directed, or risk losing access to your user
account. If you do not change your password before the stated date you will
lose access to your account and it will have to be re-activated. Any user
account that has been suspended as a result of password expiration can be
re-activated by contacting the helpdesk as directed in the Obtaining
Support section of this newsletter.
If you do not read your JLab email you will not see the
notification of the impending suspension of your account, so it is recommended
that you examine the contents of your JLab email periodically to prevent an
unnecessary account suspension. Those users who only
read email from an account outside of JLab, should create a mail forward for
their JLab user account which will direct their JLab email to their primary
email account. Instructions for setting up email forwarding for your JLab user
account can be found at http://cc.jlab.org/docs/services/email/mail_forward.html.
When changing your
JLab CUE password please remember to adhere to these rules as they are strictly
enforced by the jpasswd system:
Calendar Users’
UpdateSeveral months ago the
Corporate Time supports both Windows NT and Windows 2000. It can
be installed by clicking on the Start Button and going to the 'client installed
programs' section under the JLAB-CUE area. Instructions for JLab Calendar
Server client configuration can be found on the JLab Calendar web pages
provided by the
Linux users should visit the above calendar link for information regarding the
availability, configuration, and use of Corporate Time 5.1 on their Linux
systems.
Additionally don't forget that there is a secure web based client available for
all users that can be connected to from any web browser on the internet. The URL
to access the web provided calendar client is available at http://webcal.jlab.org.
There is no Corporate Time client available for Solaris or HPUX
systems. Users of these systems should use the above mentioned calendar web
client.
During the week of
The batch farm
consists of 175 dual processor Intel systems and 5 dual processor Sun Ultra1
systems. The Intel systems run Red Hat Linux 6.2 and the Sun systems run
Solaris 2.6. The processors in the Linux
systems range in speed from 400MHz to 1Ghz. Each
system has at least 128Mbytes of RAM and 18Gbytes of local disk space. The
newer 1GHz systems have 512Mbytes of RAM and 30Gbytes of local disk space.
Users should plan their work around the minimum values. Since each farm node
runs 3 jobs at a time, a user's job should not require more than 6Gbytes of
local disk space.
Each farm node has a
100mbit Ethernet connection to one of many 24-port switches. The 24-port
switches then have Gigabit Ethernet uplinks to a central Gigabit switch. The data movers and cache servers that hold
the data used by the batch farm are all directly connected to the Gigabit switch
via Gigabit Ethernet.
A commercial product
named LSF (Load Sharing Facility) manages the batch farm and schedules jobs for
execution. Users do not interface with LSF directly; instead they use a software
package written at JLab to interface with LSF. This interface is known as the
JobServer. Users submit jobs to LSF using the "jsub" command via the
JobServer. The JobServer does this by parsing the user's “jsub” command file
and creating one or more script files that it submits to LSF. The JobServer
will additionally stage any specified data files from JASMine (the mass storage
system managing the tape SILO) to the cache disks prior to execution. This
ensures that the batch farm does not waste CPU cycles by starting jobs that
will sit idle waiting for data from the tape SILO.
Jobs are first chosen
for execution based on the queue to which they were submitted. There are 4 job
queues: priority, production,
low_priority,
and idle queues. Jobs are selected from these queues for
execution respectively. This, however, is not absolute. LSF will occasionally
select jobs from lower priority queues to prevent the starvation of jobs from
lower priority queues.
Each queue has a
special purpose and restrictions associated with it. The priority queue is for
short jobs that a user wants executed as soon as possible. These jobs are
limited to 30 minutes of CPU usage on a dual 750MHz Pentium III or the
equivalent. Because of this limitation, the "TIME" keyword is
required in the "jsub" command file when using the priority queue.
The value cannot be greater than 30 because of the 30-minute CPU usage limit.
Jobs are killed if they exceed the number of minutes specified in the
"jsub" command file. If a priority job is executed on a system other
than a dual 750MHz Pentium III, the time is normalized to the processing power
of the system used.
The scheduling of
jobs in the production queue is based on a share algorithm which considers a
group or user's usage over a 24-hour time period. Currently CLAS is allocated
80% of the farm while Halls A and C share the remaining 20%. Halls A and C
share their 20% of the farm with a round robin algorithm which alternates the
execution of jobs between the two halls.
The low_priority queue is for jobs that users want started when
there are no runnable jobs in the priority or production queues, and that
cannot be suspended once they have started. These jobs are limited to 2880
minutes of a dual 750MHz Pentium III or the equivalent. Like the priority
queue, the low_priority queue requires the
"TIME" keyword in the user's "jsub" command file. Jobs are killed if they exceed the number of
minutes specified.
The idle queue is for
jobs that users want run when there are idle CPU cycles and that can be
suspended during execution. These jobs
do not have a time limit, but will be suspended to run jobs from the other
queues.
As stated earlier,
the JobServer will pre-stage any required data files from JASMine to a group of
cache disks before starting a user’s job. The group of cache disks used is one
that is reserved for the batch farm, therefore, the
pre-staging of files will have no effect on cache disks belonging to other
groups. Users must use the "INPUT_FILES" keyword in their "jsub"
command file if they want the JobServer to pre-stage their data files. If the
full path name provided starts with "/mss/" then the file is
pre-staged to the farm cache disks before the job is executed. If multiple
"/mss/" files are listed, they are handled in groups of 10. In other
words, a single “jcache” is issued for every 10 data files listed as
“INPUT_FILES”, this is important because a job will not begin until the
“jcache” for the job's required file has completed.
As an example, the “jsub”
command file below will create 15 farm jobs from a single “jsub,” one job for
each of the “INPUT_FILES” listed. The first 10 jobs created will be held in the
queue pending the completion of a “jcache” of the first 10 files. The last 5
jobs created will be held in the queue pending the completion of a “jcache” of
the last 5 files.
JOBNAME:
myjob
PROJECT:
clas
QUEUE:
production
COMMAND:
/home/me/bin/farm_script
OPTIONS:
-batch –debug
OS:
linux
INPUT_FILES:
/mss/xyz/data/file1 /mss/xyz/data/file2 /mss/xyz/data/file3 /mss/xyz/data/file4
/mss/xyz/data/file5
/mss/xyz/data/file6 /mss/xyz/data/file7 /mss/xyz/data/file8 /mss/xyz/data/file9
/mss/xyz/data/file10
/mss/xyz/data/file11 /mss/xyz/data/file12 /mss/xyz/data/file13
/mss/xyz/data/file14
/mss/xyz/data/file15
Because the files are
grouped together for pre-staging in the order in which they are listed, users
may want to use the ‘jls’ command to determine on which tapes the files are
located, then order the “INPUT_FILES” in the “jsub” command by tape. This will
decrease the number of tapes each “jcache” will require to complete and thus
reduce the chances of runnable jobs pending for a jcache requiring a heavily
used or ejected tape. If the ‘jls’ command output shows that a tape for a given
file is not available it would be better not to submit a job requiring that
file until the tape has been placed back into the SILO. Users should contact
the individual in charge of computing for their experiment or hall about having
ejected tapes inserted back into the SILO.
JASMine’s Farm Interactions JASMine (Jefferson
Lab’s Asynchronous Storage Manager) is a locally developed mass storage system
manager which manages the tape SILO, the tapes, the files on the tapes, and the
cache disks. Users interact with the system via the use of the “jput”, “jget”,
and “jcache” commands. The “jput”
command writes data to tape, while the “jget” and “jcache” commands retrieve
data from tape. The status of the JASMine system and user requests is found at http://cc.jlab.org/scicomp/JASMine/monitor/jasmine.html.
JASMine breaks down
user requests into jobs and a job is simply a file. So a “jcache” request for
10 files will generate 10 jobs, one for each file. JASMine does this to allow individual data
movers to select individual jobs from a request that it can process. If a
request requires 3 different tapes, 3 different data movers may work on that
request. Each data mover selects the job requiring the tape it has reserved. It
is not uncommon for multiple data movers to work on different jobs from a
request in parallel, resulting in faster processing.
JASMine has a
scheduler to order queued requests for execution. There are two priorities
associated with a user’s request, the priority and the share priority. The
priority is a numerical value given to a request that tells the system how to
order the execution of the requests. It works like the nice setting does on Unix. A smaller value indicates a higher priority. A share
priority is a numerical value given to a request that tells the system how to
order the execution of requests with the same priority. A smaller value
indicates a higher priority. The share priority is generated dynamically, and
is based on usage by the user and/or the host from which the request was
submitted during the previous 24-hour period. Usage currently refers to the
number of active or completed requests, not files. Requests from the farm currently have a
greater share and thus higher priorities to keep the farm moving.
How does a data mover
go about selecting jobs for execution? First the data movers will look for jobs
that require a tape that is already loaded in one of the data mover’s tape drives
or has already been claimed by the data mover. Next the data mover will look
for jobs that have not been claimed by another data mover, but are from a
request that is active (i.e. another data mover is processing part of the
request). If the data mover can process the request (i.e. it has the right tape
drive and enough stage disk space), then it is claimed and work on it begins.
Next the “jputs” are examined in order of their priority and share priority.
Finally, the remaining requests are ordered by their priority and share
priority.
JASMine automatically orders the jobs for execution
within a request by tape and position on tape. Users will not see any
performance gain by listing the requested files in order of tape and position
on tape. It may still be useful to make one request per tape so that an entire
request will not hang waiting for a tape that has been ejected from the SILO.
Users can check to see what tape a file is located on, and if that tape is
available by using the ‘jls’ command.
If the
performance of your Windows computer has deteriorated it may be a direct result
of limited hard disk space. There are some steps that can be taken to improve
this situation. The first step is to check to see whether the hard drive on the
system is too full. Ideally, you should have at least 250 megabytes (or more)
of free space available on the C: drive to allow for temporary file creation
and software installations.
To
determine the amount of space available on your Windows pc, open My Computer,
then right click the icon which represents the drive in question. Select Properties
from the pop-up menu, you’ll see a window similar to Figure 1.
Figure 1 (Windows 2000):
The first item to note is the
File System information. If your
file system is FAT, it is not as secure as an NTFS file system and should be
converted to NTFS. (If you are using Windows
95 or 98, do not convert the hard drive to NTFS.) To convert your file system to
NTFS, click on the Start button and select Run. In the dialog box type convert c: /fs:ntfs
(c: is the drive letter; if you are converting a different drive or
partition, substitute that drive letter for c: in this command).
A message will
indicate that the conversion will take place when the computer is next
rebooted. Close all programs and restart your pc; the boot process will pause
and onscreen messages will indicate that the conversion is taking place. Once
the conversion is complete, the pc will automatically be restarted again. If
you review the free space available on the drive, you will see a significant
difference; most likely 200 to 300 megabytes of additional space.
The second item to note is Free Space. As
previously mentioned, inadequate free space can cause programs to malfunction
or cease function. If your hard drive has less than 250 megabytes of free
space, you should review the contents of the hard drive for unnecessary files.
Using Windows Explorer, check the contents of c:\temp
and c:\tmp. Many installation
programs use these directories as a temporary repository for files and then
fail to clean up after themselves. Before you delete any files, be certain that
the files in c:\temp and c:\tmp are not your data files. If you didn’t create
the files, you should be able to delete them and never miss them.
You can
remove files in cache directories, too (e.g.,
c:\netscape\users\username\cache\). If
you applied a service pack or an update and backed up files in order to have
the back-out available, but it’s been months and you’ve determined no need to
back out of the service pack, you can delete folders named SP4 and SP5. If you
have questions about the necessity of a particular directory, please email the
Help Desk (helpdesk@jlab.org) and ask
before you delete it. We’ll be happy to help you identify folders and determine
the need to keep them.
After you have cleaned up
extraneous files, you should defragment
your hard drive. If you’re using Windows NT, there are no built in
defragmentation tools. However if your pc is configured for CUE, you can open
Windows Explorer, select apps on ‘Jlabapps’ (L:) and locate the directory named dklite. Open the directory and install
Diskkeeper Lite by double-clicking the Setup icon. If you have limited space on
c:, you should choose an alternate local drive as the
location for the dklite program files. Once the program is installed, close all
open program windows, then select Start, Programs, Executive Software, and
finally Diskkeeper. Diskkeeper Lite is an adequate tool for defragmenting a
hard drive; however it is not the most comprehensive defragmentation tool on
the market. Diskkeeper Lite may need to be run three or four times in order to
optimize the space available on your hard drive.
If you’re
using Windows 2000, close all open program windows. Open My Computer and then
right click the icon which represents the target drive. Select Properties from the pop-up menu and select
the Tools tab. Select Defragment Now. Highlight the disk drive you want to
optimize and select the Defragment button.
If you follow all these steps
and still don’t have adequate hard drive space available you may need to
purchase a new hard disk or repartition your system. If you need advice on what
steps to take please submit a problem report to the computer center from the
CCPR submission web page located at http://cc.jlab.org/helpdesk/.
Some time ago Pony Computers (from whom we had purchased
quite a number of PCs) was taken over by another company. That company is no
longer honoring the warranty agreements on the Pony systems. If you have a
hardware problem with a Pony system, please contact the
The Computer Center Web Pages provide solutions for
many documented problems encountered under Windows, Mac, Unix,
and Linux operating systems. You’ll find easy to follow instructions for most
common tasks including: Installation and/or system upgrades, computer
configuration for joining JLAB CUE, Connecting to JLAB mail services, and
Printer/OS compatibility for desktop selection of the site’s numerous HP
printers, just to name a few.
The Computer Center Homepage is located at http://cc.jlab.org/. Within the colored fields on the left and
right of the page you’ll find links to all of the helpful information
available. Check it out!
Are you a Windows 2000
user? The following link identifies HP printer
compatibility issues for the Win2K OS. Install ‘your’ local network printer by
following the steps provided to take full advantage of model’s available
features under this OS. NOTE: Incorrect network printer additions may produce
erratic results. http://cc.jlab.org/docs/services/printing/W2Kprinting.html
If you were the
recipient of an email message that contained a virus, you have probably seen a
message that looked similar to the following:
Section 1:
------------------Virus Warning Message
-----------------
Found virus Eicar_test_file
in file eicar.gif
The uncleanable
file eicar.gif is moved to /vw/qtine/virus/virYIARmaixP.
---- This virus was detected/cleaned
by JLAB's email filter ----
Section 2:
---------------------------------------------------------------------
---------------------------------------------------------------------
Section 3:
------------------Virus Warning
Message -----------------
eicar.gif is
removed from here because it contains a virus.
---- This virus was detected/cleaned
by JLAB's email filter ----
This is notification email automatically
generated by VirusWall, our email virus detection application. Notification
messages are composed of three sections:
§ Section 1 indicates that a
virus was detected in this mail message when it was received by VirusWall. It
shows the name of the virus detected and the name of the attached file in which
it was detected. The path indicates where in the quarantine area of the
filesystem the attachment (with the live virus) was saved.
§ Section 2 of the message
contains any body text that was contained in the original email message. This
is preserved because it is possible for a virus to be attached to a message
while the body contains legitimate correspondence.
§ Section 3 is the place holder
for the removed attachment.
No notification to the
Note that the email
virus scanner is independent of the Norton anti-virus software that is required
on all PCs on site. Every machine that is part of the JLAB domain will be
provided with automatic installation and update of the Norton software. If your
machine is not part of the domain, you are required to install and maintain the
package. Contact the Helpdesk for pointers to the software.
(1) Jefferson Lab's
computer security team will offer seminars on topics relating to computer
security. Seminar topics will range from technical presentations to policy discussions.
Seminars will be announced in news messages. If you have an item you would like
to see the subject of a seminar, please send email to security@jlab.org.
(2) We are currently
working to get a consistent configuration for ssh clients and servers.
Information on how to install sshd/ssh clients and
migration to protocol 2 will be made available when the configuration is
finalized.
(3) Gnutella based applications, like BearShare, are not allowed on JLab computers. Peer-to-Peer
file sharing applications are a security risk because they effectively bypass
the firewall, and they open the potential for Trojan files to be dropped onto
local file systems. Further, as we mentioned
in the July 2001 Newsletter, they allow the illegal transfer of copyrighted
material and are a violation of the acceptable use policy. Finally, many such
clients have "tag-a-long" applications bundled with the client,
including some that track computer usage and direct target pop-under
advertisements to users based on their usage.
The wireless network at Jefferson Lab has recently become
a production network service. It has been implemented to provide additional
avenues of access to computing resources and to extend network access to
locations where it would otherwise remain unavailable. The wireless network is
not viewed as a replacement for our existing network, simply a convenience for
our mobile users.
The following areas are currently under wireless network
coverage and are supported by the
The
a) "Station Name" should be set to your computer name.
b) "Network Operations" should be set to “Normal IEEE (=default)”.
c) "Transmit Rate" should be set to “Auto Rate Select (High)”.
Due to budget constraints the
Wireless Network Access
Points cannot be connected to the JLab network without prior approval from the
Requests for temporary wireless network connectivity for meetings and conferences can be fulfilled so long as they are made in a timely fashion. Such requests are evaluated on a case-by-case basis.
New Pager
Prefix is 584The process of converting our old pager numbers to our
new pager numbers with the prefix 584 was completed on Wednesday December 12th. Your old pager number continued to work until
January 31st. Reminder: if
you have programmed your voicemail to page you when you have a message waiting,
you will need to change it to reflect your new pager number.
For questions or comments please contact Dawn McGinnis at
extension 7206 or via email at mcginnis@jlab.org.
Submit a problem report or request for
assistance:
Note
that the quickest method is to submit an electronic request, as the report is
immediately assigned to a staff member and seen by many other staff.
After
business hours, for emergencies only involving major outages or interruptions
to the physics experimental program, contact the guards, who will contact our
on-call staff member.
Other
sources of information in CUE are the news messages available on login, and the
Computer Center’s web status and announcement pages. For news, simply type
“news” to get the latest unread messages, “news a b” for a brief listing of all
messages, or “news 100” to read message #100. The web page information can be
found at http://cc.jlab.org/announce/status.html.
For users who have purchased Dell, Micron, or
Gateway PCs under JLab’s ordering agreement, your machine includes 3-year
onsite hardware support. You can directly contact Dell at 1-888-560-8324; you
will need your 5-digit service code (a label on the back or bottom of your
machine). To contact Micron, call 1-800-249-1179: extension 59684 for Tech Support, extension
59028 for Customer Service, and extension 31205 for Sales Service. To contact
Gateway, call 1-800-846-2303 with your system’s serial number
The Computer Center Computing Weekly News mail archive
can be found at:
http://www.jlab.org/ccc/mail_archives/ALERTS/cc-weekly/CURRENT/.
The Computer Center Scientific Computing
Briefs mail archive can be found at:
http://www.jlab.org/ccc/mail_archives/ALERTS/jl-scicomp/CURRENT/.
The archive of
previous
This document is maintained by {helpdesk@jlab.org}
Copyright Jefferson Lab 2007