Bulletin 186 - 2009 February 18

  1. Installation of new tape library for CSIRO Data Store
  2. Departed Users - CSIRO ASC
  3. File System Management Policies for ASC Systems
  4. New CSIRO ASC Software
  5. CSIRO ASC Software Upgrades

1. Installation of new tape library for CSIRO Data Store

As part of our continuing expansion of the capabilities of the Data Store to meet the ever-increasing demand, we are installing a new Sun/Storagetek SL8500 tape library, which will contain the T10000 tapes and drives which were commissioned in mid-2008. At the time, the Bureau kindly allowed us to install them temporarily in their library, as an interim arrangement.

Some tapes and drives from our existing Powderhorn library will also be relocated.

The process of placing the new equipment into production involves the relocation of five drives and around 5000 tape cartridges, and we expect to achieve this without significant interruption to the Data Store service.

However there will be a few times in this multi-stage process where we will be operating for a day or so with a reduced number of drives, thereby causing delays. These will be most noticeable when recalling files of over 100MB in size, which have not been used recently.

Please note that at no stage will files become unavailable.

During these periods, we will be asking users to postpone non-urgent file recalls in order to match the work load to the reduced capacity.

The process is expected to begin towards the end of February, and more accurate times will be advertised in the login message, and possibly via email to some groups of users.

[ page top ]



2. Departed Users - CSIRO ASC

CSIRO ASC plans to introduce the following policy for dealing with the files from departed users.

CSIRO ASC will run a weekly check, and when we detect the removal of a CSIRO NEXUS username $USERGONE in group $GROUP, then we will move all the files from the user's $HOME directory on cherax/Data Store to a directory /cs/datastore/$GROUP/${GROUP}u, and change ownership to the special group username ${GROUP}u, e.g. csdaru.

CSIRO ASC will reserve the right to consolidate files in this directory into larger archives, e.g. with the tar utility.

CSIRO ASC reserves the right to remove files for departed users in $HOME file systems on other systems, e.g. the clusters, relying on important files having been previously transferred to cherax/Data Store. Backups will be available for a while after files are removed.

Files and directories belonging to user $USERGONE in $WORKDIR areas will eventually be removed by flushing.

CSIRO ASC reserves the right to remove files and directories belonging to user $USERGONE in $DATADIR areas at any time after the username ceases to be a valid CSIRO NEXUS username.

These actions will be taken to preserve files, but to reduce the management issues for CSIRO ASC, and to free up resources for others. We would prefer to be informed in advance of the departure of users, and to have someone else take responsibility for the files. We will happily move files and directories, and change ownerships in these cases.

[ page top ]



3. File System Management Policies for ASC Systems

It is imperative that you understand the file systems management policies on ASC systems, including:

  • automated flushing/removal of files
  • backups are limited to a few file systems
  • file migration to tape on the CSIRO Data Store
  • no guarantee of any file recovery in the event of major disasters.

Copies of important files should be kept elsewhere.

Please see http://intra.hpsc.csiro.au/user.shtml for further information on CSIRO ASC systems. In particular, please read the 'Filesystems' section.

[ page top ]



4. New CSIRO ASC Software

  • Intel Trace Analyzer and Collector 7.2

    Intel Trace Collector and Intel Trace Analyzer are a combination of products used for profiling MPI code. (These products used to be called Vampir trace and vampir analyzer.) Intel Trace Collector is required to be linked with your code at compile time. The program is then run under the usual queueing system. A trace file is generated which can then be read by Intel Trace Analyzer.

    To access ITAC on burnet and cherax do:

      module load itac
    

    For usage instructions please see the software map - http://nf.nci.org.au/facilities/software/index.php?site=CSIRO

    The software has been installed on cherax, burnet, and can be accessed on partner systems.

[ page top ]



5. CSIRO ASC Software Upgrades

The following have been recently installed:

  • PGI Cluster Development Toolkit 8.0 for Linux (burnet, myhost)
  • Accelrys Materials Studio 4.4 (burnet, myhost)
  • R Programming Language version 2.8.1 (cherax, burnet)

[ page top ]





BoM Solar Help:

CSIRO ASC Help:

For urgent help at all times:
  • CSIRO users 0428 108 333
  • Bureau out of hours emergencies are managed through internal policy
HPCCC WWW Site: http://www.hpccc.gov.au/
CSIRO External ASC Site: http://www.hpsc.csiro.au/
CSIRO ASC Users' Site: http://intra.hpsc.csiro.au/

Comments to:


© Copyright 2010, CSIRO Australia
Use of this web site and information available from it is subject to our Legal Notice and Disclaimer and Privacy Statement