Connecting to the AVIDD Cluster

The only way to connect to the AVIDD cluster from your desktop or laptop is to use the so  called ``secure shell'' and  related utilities.

Standard telnet  and Kerberized telnet connections to the cluster are disabled, as are the Berkeley r-tools, i.e., rsh, rlogin  and the like. FTP and Kerberized FTP  do not work on the AVIDD cluster either, so you cannot transfer your data to it this way. You will have to use scp instead.

So in this section we're going to learn about using slogin and scp.

Slogin and scp are commonly distributed with Linux. But they are not distributed with Windows and proprietary UNIXes. I don't know about MacOS-X. Most researchers and students working with AVIDD will be either Linux or Windows users anyway. If you're a Linux user, you probably know all about slogin and scp already, so you can skip most of this section and go to chapter 4 right now. The only thing you need to know is that the front-end node on the AVIDD cluster is called bh1.avidd.iu.edu.

If you don't know all about slogin, or if you're a Windows user and do not know anything about it at all, read on.

The best way to get hold of slogin and scp under Windows is to install Cygwin. Cygwin  is a full Linux emulator that runs under Windows. In some ways it is even more than Linux, because it provides its users with seamless access to all Windows facilities and then gives them almost all of Linux on top. Cygwin  even provides access to NTFS  ACLs (the Cygwin commands that  manipulate NTFS ACLs are setfacl and getfacl) and to Microsoft Dfs. For example, the commands

$ cd //tqc.iu.edu/dfs/home
$ getfacl gustav
# file: gustav
# owner: gustav
# group: ovpit
user::rwx
group::r-x
other:r-x
mask:rwx
$ cd /cygdrive/c/winnt

put me in the Microsoft  Dfs  directory \\tqc.iu.edu\dfs\home first, then I list NTFS/Dfs ACLs on the sub-directory ``gustav'' and finally I go to C:\WINNT.

Cygwin comes with the secure shell and related utilities, X11 server and applications, GNU compilers, emacs, make, TeX, inetd utilities (you can run your own telnet and ftp servers), PostgreSQL data base, exim mail server and Apache WWW server. It's all there and it'll cost you nothing. If you wanted to buy all this functionality, you'd end up spending thousands of dollars on software.

How do you go about installing  Cygwin? It's easy. Go to

http://www.redhat.com/download/cygwin.html

and press ``Download Now!''. This will download  a small file called ``setup.exe''. When you execute this file, you will be guided through various steps, including the download of binaries and documentation, and then unpacking and installation of the whole package on your PC. You will use the same binary to install Cygwin upgrades and patches down the road.

You don't have to download and install the whole lot. It's up to you how much of the package you really need and want to have. But I have installed all of Cygwin on my Windows 2000 PC and use pretty much most of it too. The whole Cygwin will take about 900MB of disk space on your C drive. If you intend to work with IU clusters a lot, if you plan on doing a lot of scientific work and if you want to do it all from your Windows box but without having to go through a double-boot, I recommend that you install Cygwin.

I am one of these weird and perverted individuals who are not very fanatical about UNIX, Linux, Windows, Macs and the rest. To me it's all overpriced trash, unless it's free (like Linux or Cygwin), in which case it is free trash. Quite ruthlessly I'll reach for anything I can get my hands on, to get the job done as quickly and with as little fuss as possible. This usually implies a mixture of commercial software and freeware. Not all commercial software is worth spending money on, but then not all freeware is going to do the job either. You just have to mix and match, weighing your money, skills and requirements in the process.

So, let us assume that you already have Cygwin installed and there is going to be this little icon on your Windows desktop that looks like a black C with a green something inside it. Press on this  icon and, assuming that you have configured everything correctly (and this is going to take some tinkering) you'll get a window that looks rather slyly like a Linux window (even though it runs under Windows) with the bash prompt, e.g.,

gustav@WOODLANDS:../gustav 14:17:11 !516 $

My prompt tells me the name of the machine I'm on, the directory I'm in (but not the full path name, just the last segment in the path), the time of the day and the number of the command I am about to issue. But for the sake of brevity I'm going to truncate it just to the name of the machine and the dollar in the examples that follow.

In this course we are going to use IUB and IUPUI clusters. To connect to one or the other, you have to  slogin to either avidd-b.iu.edu (this is the Bloomington cluster) or to avidd-i.iu.edu (this is the IUPUI cluster). In both cases, you'll end up in the same home directory, which is going to be mounted on both clusters.

Let us begin with the Bloomington cluster. If this is your first connection, here is what it is going to look like:

WOODLANDS $ slogin avidd-b.iu.edu
gustav@avidd-b.iu.edu's password:
generating ssh file /N/B/gustav/.ssh/id_rsa ...
Generating public/private rsa key pair.
Created directory '/N/B/gustav/.ssh'.
Your identification has been saved in /N/B/gustav/.ssh/id_rsa.
Your public key has been saved in /N/B/gustav/.ssh/id_rsa.pub.
The key fingerprint is:
ed:84:29:8d:22:70:7d:5f:09:eb:c5:3b:ff:54:61:7b gustav@bh1
adding id to ssh file /N/B/gustav/.ssh/authorized_keys
[gustav@bh1 gustav]$

Observe that once you have made the connection you end up on the host called bh1. To be more precise, avidd-b.iu.edu evaluates to bh1.uits.indiana.edu and avidd-i.iu.edu evaluates to ih1.uits.iupui.edu. These two are called the head nodes of the two respective clusters. They are the nodes to which you connect from the outside in order to submit your AVIDD jobs. Such head nodes are also called front-end nodes.

The head nodes have more than just one network interface. The interfaces that correspond to avidd-i.iu.edu and avidd-b.iu.edu are on the public campus network. They are Gigi interfaces. These nodes can be seen on other networks too. There is a ``cluster'' network there, and a Myrinet network. The latter is used to run MPI jobs and to support GPFS, the General Parallel File System.

In order to make this first connection I had to type my AVIDD password explicitly. This is tedious, especially if you need to make new connections frequently and if you have a complicated password, and it is not very secure either, because the password travels over the network albeit in an encrypted form.

There is a simple way to change this by reconfiguring ssh on your PC and on the AVIDD cluster to work with DSA or RSA keys instead of passwords. This is more secure and more convenient too.

The procedure is as follows.

First you have to generate your own private/public key pair. You do this by calling a program  ssh-keygen. This program will generate the keys and it will place them in the .ssh directory in your Cygwin home (which is like Linux home; on my PC I have simply linked it to the Windows' ``My Documents'').

You can generate RSA or DSA  keys with ssh-keygen. DSA keys are more secure, so I recommend the latter. Issue the command:

WOODLANDS $ ssh-keygen -t dsa

The command  will ask you for the DSA passphrase. The passphrase  can be as long as you wish. Here is the first feature that makes the DSA system more secure than UNIX or Linux passwords, which are usually limited to just eight characters or so (if you have more than eight characters in your Linux password, the characters beyond eight may be ignored). Every character in your DSA passphrase matters, e.g., I have 43 characters in one of my favourite passphrases.

The private key will be stored on ~/.ssh/id_dsa and the public key will be stored on ~/.ssh/id_dsa.pub. You can show your public key to the world safely. Without its private partner it's useless. In particular you can transfer it to the AVIDD cluster and append it to the file ~/.ssh/authorized_keys. The easiest way to do this is to copy it from your local laptop window and then paste it into the AVIDD window, in which you are editing  authorized_keys. Make sure the pasted text is a single line. It may happen that the copy/paste process inserts newlines in the string. If it does, simply remove them. This is what my public  DSA key looks like:

ssh-dss AAAAB3NzaC1kc3MAAACBAKykdA8AG7Vazhia9fI+uKgsyQzQSCK5LhaQy9XwmEk80hJ/Pg3T
4m+yZ1CS93GM2Z2HXEIbCe39piNgg5d+0mhxaRHP48TUZhqX8pgU4vG89o/LqWmUSDAElbnyjL7VHfIl
LCZ465dTJezZpAYLz1B+JU20CKjN4y46rzJsMMznAAAAFQDv1+pusBscm1hq0/Gxiz8E7o+eGQAAAIBc
6fDEDraImCtSty124Wi7rEamNDIabswcObhMCm93HrO9VoNoO97A7c7shvsObbdfwUDCMtYSwkaeHB4o
VRR/ULL9FWcxzbv3HFw81PTx1CFcyL2+u8e/1d2itpAruTzcs0QZNQldBRjpMpUz52TSD89WV0ZE1Lox
58LKRy4ixAAAAIATq0aL2bbrXu2tK1QuMXYqFHSQUIXMWiQTW7ARJ8mu/EZ92MXvhRYBQYaSXkcq7HHq
qhpV3//sGhv28G5gxxFAIynD9xB7UxH44K0F8vl/KmF3Hldn74m3WWhn6+Xz6JOttRRZa7ZGAzrNkwu4
TD3k3y4hyw4M7p/fhftJ8/o0RA== gustav@WOODLANDS

Although the text printed above is spread over eight lines, it is, in fact, a single line.

Once you have done all this, you can carry out various transactions with the AVIDD cluster as follows.

WOODLANDS $ ssh-agent bash
WOODLANDS $ ssh-add
Enter passphrase for /home/gustav/.ssh/id_dsa:
Identity added: /home/gustav/.ssh/id_dsa (/home/gustav/.ssh/id_dsa)
WOODLANDS $ slogin avidd-b.iu.edu
[gustav@bh1 gustav]$ ^D
Connection to avidd-b.iu.edu closed.
WOODLANDS $ ssh avidd-b.iu.edu date
Thu Aug  7 15:10:46 EST 2003
WOODLANDS $ ssh avidd-b.iu.edu ls
src
WOODLANDS $ scp avidd-b.iu.edu:.bashrc bashrc-avidd
.bashrc                              100%  124     2.3KB/s   00:00
WOODLANDS $

Let me explain what happens here. First I have invoked  the program ssh-agent and asked it to execute my login shell, bash. The agent is going to hoard my keys and pass them on to any secure shell transactions transparently, i.e, without me having to type them in explicitly over and over. Once the ssh-agent has forked a new agent-supervised shell for me, I have invoked the command ssh-add in order  to add my keys to the agent's cache. This is the only time I actually have to type the passphrase. Now I have issued the slogin command and got right to avidd-b.iu.edu without having to type my passphrase. I can also issue ssh commands without having to type the passphrase and check the date or the content of my home directory on the AVIDD cluster. The last command, scp, transfers the content of my .bashrc file on the AVIDD cluster to bashrc-avidd on my local machine. Again, I am not asked for the password or for the passphrase. The ssh agent takes care of this behind my back.

A useful command to execute in the ssh agent supervised shell is

WOODLANDS $ xterm -sb -sl 300 -n avidd -T avidd -e slogin avidd-b.iu.edu &

This command brings up an X11 window on your display with a shell running on the AVIDD cluster. Observe that the xterm program runs locally on your PC, not on the cluster. Exiting the AVIDD shell closes the window automatically.

For this to work, you must have X11  server running on your PC. You can use the one that comes with Cygwin. Its latest version is very good, almost as good as quite expensive commercial offerings, although it still has a couple of glitches here and there and may even hang on you after a day-full of activity.

Here is how I use it on my home computer. First  I have copied a file /usr/X11R6/bin/startxwin.sh to my own private ~/bin and modified it to look as follows:

#!/bin/sh
export DISPLAY=127.0.0.1:0.0
PATH=/usr/X11R6/bin:$PATH
rm -f /tmp/.X11-unix/X0
XWin -multiwindow -clipboard &
emacs &
xclock &
exit

I have stripped numerous comment lines off this script to make it shorter. The script cleans the X11 socket from the /tmp/.X11-unix directory (the original script tries to remove the whole directory, but this can fail sometimes, especially if other users run X11 and Cygwin  programs on the same system too), then invokes the X11 Cygwin server, called XWin, with -multiwindow and -clipboard options. The first option combines X11 and  native Windows  displays into one, so that X11 applications can be managed the same way Windows applications are managed and so that you don't get a separate X11 root window in the background. The second option lets you copy  and paste between Windows and X11 applications. Then I call emacs and xclock and the script  exits. If everything works just fine, you should see the Emacs and the Clock windows pop up and you should also see a large X appear in the right corner of the task bar.

I usually run startxwin.sh under  the ssh-agent . This way every X11 application carries my DSA keys with it. I can invoke these applications from my Emacs shell.

I connect to the AVIDD cluster using the following command:

$ xterm -sb -sl 300 -e slogin -X avidd-b.iu.edu &

The -X option  will set up X11 environment for me automatically on the other side. The DISPLAY over there will be defined in terms of a socket local to the AVIDD head node. This file is readable to you only and it lives in the /tmp directory. Data sent to the socket will be encrypted and transmitted to your Cygwin X11 server, to be displayed on your screen. This is probably the safest way to use X11, because the data streams are going to be encrypted and because you don't enable access to your X11 display to all users on the AVIDD head node.

Note

Secure shell installed on the AVIDD cluster  is Open SSH version 3.6.1 patch 1. You get exactly the same secure shell with Cygwin, which should not be surprising, because Cygwin is Linux for Windows. But there are various other Secure Shells around, some free some commercial. I have Secure Shell 2 (ssh2) installed  on one of my systems, and it has a different format for its authorization file, public keys, etc. ssh2 format, copied directly from an ssh2 public key file and pasted onto the Open SSH authorization file will not work. You may even end up locking yourself off the AVIDD head node altogether.

The -X option will work only between OpenSSH on your workstation  and on the AVIDD cluster. It does not work on connecting to, e.g., SSH2 servers on other machines.