The bigdat cluster has both a large attached storage system on it, and significant per-node HDD and SSD storage. Using these storage resources is not going to be heavily monitored and so user cooperation will be a must! Be nice!
A listing of the storage on the head node (bigdat.nmsu.edu) shows:
> df -h Filesystem Size Used Avail Use% Mounted on storage-ib:/home 10T 458M 10T 1% /home storage-ib:/nmsu 5.0T 34M 5.0T 1% /nmsu storage-ib:/data 60T 35M 60T 1% /data
These three filesystems are hosted on the storage node (named "storage"). The storage node has about 30TB still unallocated that we can allocate as needed.
/home is the home directories that you have, each of which is private. /nmsu and /data are both open, writeable filesystems for users to use as they need.
/nmsu is a small filesystem that is meant to host applications, tools, project code, and such things. It has several top-level directories already set up, so use them! If you install and application, put it under /nmsu/apps; if a tool, put it under /nmsu/tools; make sure they are installed in a way that everyone can use them.
/data is the large filesystem where you can put datasets you need for experiments or that you create from running experiments. It has no top-level directories -- I would suggest two practices: create a top-level directory of your own username if you need to have a place for personal experiments and data; BETTER: create a top-level PROJECT directory for some research project that you and your students are working on; this way you can all share that directory.
The compute nodes also each have significant local storage capacity that is open for you to use on experiments, and in addition they all mount the three directories above that are hosted on the storage node. So your applications can write to local storage or directly to the filesystems on the storage node.
A listing of the local storage on the "gpu01" compute node is:
[gpu01] > df -h Filesystem Size Used Avail Use% Mounted on .../HDD-DataHDD 5.4T 186M 5.1T 1% /data-hdd .../SSD-DataSSD 918G 200M 871G 1% /data-ssdAll of the other nodes are the same. These two filesystems represent the local hard disk and solid-state disk drives, respectively. They are writeable by everyone, so you can create whatever directories you need on them, and place data on them for experiments.
It is NOT expected that you will leave data on the nodes for extended periods of time. You should use the local storage as temporary data capacity, where you can put data for a set of experiments (or create data during the experiments), run them, and then remove the data off the compute nodes so that someone else can use the capacity. These are really there for speeding up data-intensive computations and for experimental result gathering. Use the storage node for long-term data storage.