Overview of available file systems
effective on July 13th, 2015
/scratch cleanup effective on Oct 1st, 2015
File system, Environment Variable for Access | Purpose | Typem overall net size and Bandwidth | Backup and snapshots | Intended lifetime, cleanup strategy | Quota |
|---|---|---|---|---|---|
/home $HOME | Store source files, input data, small files Globally accessible from login and compute nodes | GPFS, 100TB | backup to tape and snapshots | Account lifetime | 100GB per user |
/work $WORK | Result files and data sets Globally accessible from login and compute nodes | GPFS, 100TB | Backup upon request, at cost price. Snapshots | 3 years | 50GB* per group |
/scratch $SCRATCH | Temporary huge result files Accessible from the frontents and the compute nodes within one cluster. | Aries: GPFS 87TB Bellatrix: GPFS 200TB Castor: NFS 22TB Deneb: GPFS 350TB | No backup, no snapshots | 2 weeks Automatic deletion of files older than 2 weeks may happen without notice | no |
various $TMPDIR /tmp/${SLURM_JOB_ID} | Temporary, local file space for jobs on compute nodes. Not available on login nodes. | may vary | No backup, no snapshots | job execution content is deleted after job end | no |
* Space on work is charged for and, as such, the quota of the group depends on the amount of space purchased. There is no backup by default but a laboratory may request such a service. The price for backup will be the cost price to SCITAS.
The price for work is 300 CHF per TB for 3 years.
Each group is allocated, for free, 50GB.
Security of user data
The contents of the home file-system are backed-up on a daily basis with a six month retention period. The backed up data are held at a separate physical location to the original data.
The contents of the work file-system are not backed up by default
The scratch file-systems are not backed-up under any circumstances.
The scratch file-systems are only for short-lived files and, in the case of insufficient free space, files older than two weeks may be deleted without notice in order to ensure the usability of the cluster.
Files belonging to a former user
When a user no longer has a valid account on the clusters any files belonging to him on home are removed from the servers. They will remain on tape for 6 months after the user has left EPFL. The head of the laboratory is responsible for ensuring that these data are correctly managed. He or she can ask for a retrieval from tape.
The work file-system is divided by laboratory and, as such, it is the responsibility of the head of the laboratory to ensure that the data are correctly managed.
Once a user is no longer accredited, files belonging to them in scratch can be deleted without notice.
How to recover snapshots
A snapshot is the state of a system at a particular point in time. On our clusters, the home and work filesystems are snapshotted daily and snapshots are kept for one week. This is particularly useful in case a user removes a file by mistake.
Daily snapshots of the home and work filesystems can be found in /home/.snapshots, /work/.snapshots, respectively.