This is Lite Plone Theme
You are here: Home System Status Power has been restored, partial service running

Power has been restored, partial service running

Dear users,

We have restored partial service after the scheduled power down today. Unfortunately, /nfs/scratch filesystem is still undergoing maintenance, as is universe, so just cosmos2 and cosmos are back in use.

You will also need to make sure your jobs do not use /nfs/scratch or else they will fail. Moab checks if the starting directory is unaccessible and puts such jobs in the Blocked queue (with "Deferred" status) from where they can be unblocked after /nfs/scratch comes back. The command to do that is "mjobctl -u ALL jobid", where jobid is the job that needs to be unblocked.

Please note that we will not release the holds automatically as we have no way of knowing if the jobs would conflict (overwrite, destroy, alter…) with something else you may have done after they went into Blocked.

Unfortunately that is all Moab can do: if you do "cd /nfs/scratch/..." in your job, Moab will not notice, the job will start, the "cd" command will fail and you will end up continuing wherever your job was before the "cd" failed. Please pay attention to this as it is easy for example to fill your home quota with accidents like that.

We are sorry that we were not able to restore full service yet and for the inconvenience this may cause.

Best regards,
COSMOS Management