This is Lite Plone Theme
You are here: Home User guide Post-CXFS Changes

Post-CXFS Changes

Breakdown of the changes of how to use the COSMOS machines since the CXFS upgrade. What has been changed by the arrival of CXFS?

Previously, our filesystem setup was fractured between the UV1s (cosmos/universe) and the UV2 (cosmos2). On cosmos2 there was a local XFS filesystem mounted (projects/flagship), which provided high performance but only from cosmos2. It was available on cosmos/universe as only over an NFS mount and also only as read-only. This meant that you had to manage files on the XFS FS via an interactive session on Cosmos2.
There was also a filesystem that was available on all machines as read/write - /nfs/scratch. The disadvantage of this was that it was not very large and it wasn't very fast because it was mounted over NFS.

With CXFS there is no more fracture. All filesystems are present and read/writeable with the same high performance on all systems.

Filesystems and Paths

We have rethought the filesystem names to reflect their new roles in CXFS. The following shows the remapping of old -> new:

  • /nfs/space -> /fast/space (146TB)
  • /nfs/flagship -> /fast/flagship (60TB)
  • /nfs/scratch -> /slow/data (40TB)

The filesystems in /fast are intended for short term storage of large raw data from jobs. They can be considered to be the new 'scratch' space. /slow/data/ is intended for medium term data.
The nfs mounted home spaces remain the same.

UV2 Paritioned into Cosmos2 and Cosmic

Cosmos2 has been partitioned into two machines - Cosmos2 (with less nodes) and Cosmic. Cosmos2 now has 189 nodes of Intel Sandybridge (1512 cores) available for jobs and 11.03TB of shared memory. Cosmic contains the Xeon Phi cards (MICs). It has 29 Sandybridge nodes (232 cores) with 1.69TB of shared memory, and 25 MICs. Cosmic is available to run Xeon only jobs and Xeon+MIC jobs. Running natively on the MICs is not available.


All machines cosmos, universe, cosmos2, and cosmic are now open for user logins.

Jobs and Queues

Since you can access the storage from all systems there is no need to do file management sessions anymore. These actions can be done from the login nodes. Interactive sessions are still available however for application development, debugging, or for running interactive programming languages like python.
Queue names will also change very soon.

Plans for the Very Near Future

  • Automatic deletion of files older than 30 days on the fast storage - /fast/space and /fast/flagship. This will enable efficient and fair use of the space for all users. The spaces will also be without quotas.
  • There will be no automatic deletion of files on /slow/data/. It is intended for the medium term storage of useful data that needs to be kept or reused later, with the price of lower I/O performance. Quotas will be implemented for all users.
  • The queues for Moab will be simplified. There will be new queues with different names, but they will make the system easier to use allow more efficient utilisation of the resources.