Data storage

The types of storage available to Insigneo members are enumerated below.

CiCS-managed Research Storage

Valuable research data for all active Insigneo-related projects should be stored on CiCS-managed Research Storage unless that is appropriate.

See CiCS’s guide to:

  • Requesting new storage areas;

  • Accessing existing storage areas from different operating systems / machines;

  • Managing permissions.

Relevant storage areas

The most significant storage volumes that CiCS manage for Insigneo are:

  • bone_biomechanics1 (10TB) - for collaborators of Enrico Dall’Ara.

  • CCE_group1 (5TB) - for collaborators of Richard Clayton.

  • multisim2 (20TB) - for members of the IMSB Insigneo sub-group including those working on the MultiSim project.

  • polaris2 (5TB) - for members of the Polaris Insigneo sub-group.

  • viceconti_group1 (20TB) - for collaborators of Marco Viceconti.

How do I request access to these?

A relevant PI needs to contact the Insigneo tech team. Access can be read-only or read plus write and can be granted for an entire volume or just a subfolder.

How do I access the data from some machine?

See CiCS’s guide for instructions on accessing their Research Storage from local HPC (ShARC/Iceberg), shared workstations and your own machines.

In addition, the multisim2 area can be accessed from Beagle.

High-performance computing storage

Various filestores are available on the University’s HPC clusters, including certain (but not all) CiCS-Managed Research Storage areas.

These differ in terms of:

  • Speed (which may differ for small and large files);

  • Where they can be accessed from (from a single node; from mutiple nodes; from off-HPC);

  • Whether they are backed up;

  • Permissions management;

  • Whether files are deleted after a job or after a period of time.

See the main University HPC documentation for a guide to HPC storage options.

Google Drive

Advice on effective use of Google Drive will be added in due course.

Note that Google Drive, provided by the University’s GSuite for Education service, has several limitations you should be aware of with regards to:

  • The maximum individual file size that can be uploaded or synced;

  • The maximum volume of data that can be uploaded per day.

More information.

Insigneo NAS

Insigneo manage a Network-Attached Storage (NAS) server that provides 55TB of usable storage. This machine, NAS1 a.k.a. nas1-insigneo.sheffield.ac.uk, was historically used to provide a large amount of shared storage for various Insigneo members and Insigneo-related projects. It is regularly backed up to a second NAS server.

How do I gain access to NAS1?

A relevant PI needs to contact the Insigneo tech team. Access can be read-only or read plus write and can be granted for an entire volume or just a subfolder.

How do I access the data from some machine?

Data can be accessed from shared workstations and your own machines but not from ShARC/Iceberg/Beagle.

Web access is via https://nas1-insigneo.sheffield.ac.uk/. This could be slow for transferring data to/from NAS1.

A guide to accessing NAS1 without using the web interface (i.e. via Windows Explorer) will be provided in due course.

Beagle’s storage

Home directories

Each user has a home directory /home/USERNAME with a 1GB quota. Technical details: a 38GB EXT3 filesystem is mounted on /home.

Data directories

Each user also typically has a directory /data/USERNAME. Characteristics of /data:

  • faster than /home

  • 15TB of storage

  • no per-user or per-group quotas (so be considerate by not filling the entire 15TB with your own data)

  • Technical details: an XFS filesystem on an array of disks.