Home, raid and scratch

Posted by Administrator on Thursday, 16 June 2016
In the MMB clusters there are 3 kinds of storage:
  • Home directory
  • Raid disks
  • Scratch disks

Plus the offline systems:

  • Your workstation's local disk
  • Tapes or USB-disks

Online storage

Home directory

  • Network drive, accessible from all the machines
  • Daily backups and file history
  • Intended for small files
    • Mails, Homepage
    • Documents, articles, images
    • Small results or programs
  • Path: /~ from your account or, in general, /mmb/homes/<username>

IMPORTANT: Think that if you get out of space here, you can lose incoming mails at your "user@mmb.pcb.ub.es" address!!

RAIDs

  • Network drive, acessible from all the machines
  • Periodic backups
  • Intended for large files, but this is not a good working space!
    • Simulations
    • Genome wide studies
    • Recent results
  • Path: "/mmb/raid<X>/<username>"

There are few raids (raid3, raid6, raid7,...). The Systems Administrator will ask you how much space do you need and create you an account in the most suitable place.IMPORTANT: As raids are network drives, if you calculate something big that need to access to the remote disk, you will generate a lot of traffic in the local network, and this will piss-off your lab colleagues. Think in using the Scratch disks and transfer big datasets in low network usage hours (at lunch-time if its moderately sized, before leaving in the afternoon if it's bigger, at the weekends if it's huge...)

Scratch

  • Local disk in the cluster, only accessible from there
  • Backup not-guaranteed (but ask to the Administrator if something bad happens)
  • Intended for on-going work (specially with large files)
  • Path: usually "/scratch" from the local machine, but there could be special cases

Scratch is a local drive with a very fast access from the production machines. It avoids network usage and is very convenient for keeping the data while you are using it. It's not a safe place to save your results!!. Once you have finished to work with the files, compress them (tar/bzip...) and move them to the Raid. If some day you will need them, you can copy them back, but don't left all your data there as other people could need this space! 

Offline storage

Workstations local drive

We know it's convenient and easy to work with data there, but you can't not access it from outside the lab and they won't be backed up.IMPORTANT: Don't calculate big things on your workstations! The hard-drives there are not high-performance and you will broke them in short time! Also, any cluster has more processing power than your computer.

Tapes and USB-drives

You had finished a project and published it? Nice! Congratulations! I know don't want to delete the data just-in-case someday I need it, but seriously, do you really need to have 100GB in the raids just-in-case?Be clever and solidary with your lab mates and copy them to a offline disk. Administrator will tell you how, and also he will make a backup copy of your disk, just-in-case it falls from your hands when you pick it up.