Bulletin 198 - 2010 February 15

1. NCI vayu system second stage
2. NCI CSIRO/CAWCR supplementary user guide
3. CSIRO Data Store - enhancement to dmget command
4. CSIRO ASC cherax $WORKDIR and flushing changes
5. CSIRO ASC systems - quotas
6. CSIRO upcoming conferences and workshops
7. VLSCI Blue Gene supercomputer

1. NCI vayu system second stage

NCI have informed registered users that technical problems are being overcome, and from today, the system capacity will be approximately doubled. The full second stage system (approximately 7 times the capacity of the initial system) is now expected to be available in mid-March. The upgrade has been planned such that no whole-system outages should be necessary.

[ page top ]

CSIRO and CAWCR researchers are welcome to register to use the CSIRO share of NCI. Core information about the NCI systems is on the NCI web-site. Supplementary information, targeting CSIRO and CAWCR researchers is now available at:

http://www.hpsc.csiro.au/userguides/vayu/

This includes tips on registration and file transfer. See also HPCbull 195.

[ page top ]

3. CSIRO Data Store - enhancement to dmget command

The local wrapper for the dmget command has been further enhanced.

A new --defer option has been provided. This allows you to specify files which you would like recalled for you overnight, for use the next day. There is no guarantee that these requests will be acted on, but if they are, then access (via a second normal dmget) ought to be much faster.

If the system is too busy then, these deferred requests will be ignored. This means that use of this option is not a replacement for a dmget just prior to file use, but it may well accelerate it.

Also new is a --recurse option which recalls all files in specified directories (and their subdirectories). So the following commands are all equivalent to each other:

        dmget --recurse mydir

        echo mydir | dmget --recurse

        dmfind mydir -state MIG -o -state OFL -o -state PAR | dmget

Finally, the local "-l" parameter is potentially incompatible with future SGI usage, and has been renamed to "--list". Continued use of "-l" will still work for now, but will result in a warning message.

Details are in the dmget man page.

4. CSIRO ASC cherax $WORKDIR and flushing changes

In order to get more performance in the file system dumps, the target for the dumps has been moved from a low-performance disc area to cherax's $WORKDIR.

This will add about another 480 Gbyte of data to the /work file system (capacity 2.8 Tbyte), which will currently fit, but will result in more frequent flushing of this area.

The file /work/flush.status shows the flush status, and there are login messages about flushing.

Recently, the status file showed:

Last flush of /work occurred Mon Jan 11 08:30:58 2010 UTC
93436 files older than Wed May 13 02:19:57 2009 UTC were deleted
7132 empty directories were deleted

This last flush removed files about 8 months old and older. We expect this period to drop to more like 3 months, with 7 days being the absolute minimum.

[ page top ]

5. CSIRO ASC systems - quotas

Message: please ask!

We have found that many users are close to their quota limits. We do not want users' workflow to be inhibited by quota limits. If you need an increase in your quota limits, then please ask. After counselling, we usually grant an increase. This applies also to usage on our partner systems, e.g. the NCI National Facility.

We put quotas on file systems in order to prevent one user's usage from having an impact on other users, for example by filling a file system, or by using more than an appropriate share of the resources.

Use the quota command (quota -v on the NCI systems) to see your usage file system usage.

Consult http://intra.hpsc.csiro.au/user/usage/ds/ to see your total Data Store usage.

Typically, we put quotas on space and inodes (number of files and directories) on areas like $HOME, $WORKDIR, $DATADIR and in areas like /var, where outputs from batch jobs reside.

Last year (see HPCbull 193 item 3), we imposed a default space quota of about 2 Tbyte on cherax $HOME: this applies only to the on-line disc space (there is about 6.8 Tbyte available), to stop one user from using most of that resource. There are no quotas on the total storage, and indeed, some users have over 100 Tbyte each.

However, we are reluctant to grant significant increases in the inode quotas for the cherax $HOME, since part of the protection for this is a full dump being done regularly. We do this on Sunday mornings, and the dump currently takes about 10 hours for 17 million inodes. Sixteen months ago, it took only about 5 hours for 13 million inodes. The time taken is dependent on the total number of inodes, and is growing at a greater rate than the increase in inode numbers.

If you have large numbers of inodes in cherax $HOME, consider consolidating them into tar archives, but please consult the Data Management Userguide at http://www.hpsc.csiro.au/userguides/ds/ or ASC staff first.

[ page top ]

6. CSIRO upcoming conferences and workshops

The Computational & Simulation Sciences Transformational Capability Platform (CSS-TCP) Annual Conference and Workshops which will be held on 16-18 March 2010 in Melbourne. http://intranet.csiro.au/multi/conferences/tcssworkshop/index.htm

The theme for this year's CMIS techfest conference is "Modelling Complex Systems". The conference will be held on 25-16 March 2010 in Hobart. http://www.csiro.au/events/Techfest10.html There are workshops on the preceding day, including a GPU cluster workshop.

[ page top ]

7. VLSCI Blue Gene supercomputer

Victorian Life Sciences to benefit from University of Melbourne and IBM Partnership

11 February 2010: The Premier of Victoria, the Hon. John Brumby today announced a partnership between the University of Melbourne and IBM.

Under the partnership, IBM will establish an IBM Collaboratory with leading life sciences and computational specialists at the University of Melbourne, and will provide a Blue Gene supercomputer offering high speed and large scale processing capacity, enabling scientists to address a wide range of complex problems.

http://www.vlsci.unimelb.edu.au/news_partnership.html

Victorian-based CSIRO staff working in the 'life sciences' may apply for access to the current VLSCI facility. There will be a mid-term call for applications in April, but applications can be submitted at any time.

http://www.vlsci.unimelb.edu.au/pcf/ras_grants.html

[ page top ]