When you login on a head node of the AVIDD cluster, you land in your HOME directory, which is most likely to live on the NFS , which is currently served from bf4 (in Bloomington) and if4 (in Indianapolis).
NFS is a convenience file system, but NFS IO is notoriously poor. You will not notice it on simple file editing operations, but you will notice it if you try to run any IO intensive jobs against it. Another thing you'll notice is that there isn't all that much space there: a mere 100 GBs on each server, which is peanuts by HPC standards.
It is not much even by PC standards. You can buy a desktop with a 160 GBs drive nowadays, and larger drives are in the works. They don't cost much and, incidentally, if you don't have enough space for your every-day data on your desktop or on your laboratory machine, simply go to Circuit City and buy as much as you need. We're talking about a few hundred dollars at most. A few years down the road, we're going to have TBs of disk space in our desktops and hundreds of GBs in our laptops.
OK, but let's get back to today. If you have a very large data set and want to analyze it on the AVIDD cluster, the place to put the fileset is on GPFS. GPFS is mounted on /N/gpfsb in Bloomington and on /N/gpfsi in Indianapolis and there is about 1.7 TBs on each.
If you have so much data that only a portion of it is going to fit on GPFS, you should keep the rest on HPSS , the High Performance Storage System. HPSS provides several peta-bytes (PBs) of space on tape cartridges, which live inside two robotic Storagetek silos: one in Indianapolis and another one in Bloomington.
In the next three sections we are going to discuss first the GPFS and then the HPSS and finally the way to move data between one and the other.