Bulletin 144 - 2005 Aug 19

1. HPCCC SX-6 system upgrade to SUPER-UX R15.1
2. HPCCC SX-6 environment for SUPER-UX R15.1
3. HPCCC SX-6 NQS II batch parameter -l cpunum_job=n
4. HPCCC SX-6 sxqsub command
5. HPCCC SX-6 i/o buffer size - F_SETBUF, GFS and local disc
6. APAC05
7. cherax upgrade and downtime

Nodes sx618 to sx627 are now running on SUPER-UX R 15.1.

Nodes sx600, sx601, sx602 and sx605 are due to move to R15.1 on 22nd-23rd August.

No user issues that have been reported remain unsolved for running on the R15.1 nodes.

Users are urged to try their applications on the upgraded nodes as soon as possible, and to move their production work to the upgraded nodes as sufficient resources become available.

Users can find what nodes are available for a particular queue by running a command like

	qstato -S | grep ' bm3 '

The SUPER-UX 15.1 release has updated libraries, which should be used eventually by all applications on the SX-6s.

In brief, to compile and link with the R15.1 versions of libraries, use the command

        sxcross_upgrade

prior to compilation.

To simplify linking with third party libraries (e.g. netCDF), you can also use the command

        sxf90_new_site_options

to change the SX F90_SITE_OPTIONS to a new setting which we hope to make the default after a period of testing.

The sxcross_upgrade and sxf90_new_site_options commands work on the TX7s, gale and farrer, but are unlikely to work well on cherax during the transition to R15.1 because of packaging issues.

For more information, including alternate options, see

- look for SX-6 under the contents list, then "How do I cross- compile for SX during the transition to a new version of SUPER-UX?"

SUPER-UX R15.1 now uses the new NQS II parameter -l cpunum_job=n to control the number of CPUs assigned to a job.

For example, if you try running 4 background tasks in a job with -l cpunum_job set to 3, each task will run at best at 3/4 of full speed. The system will allocate at most 3 CPUs at a time to the job, even if more are free.

The sxqsub command on the cross environments for the SX-6s has been enhanced to allow correct treatment when one of the R15.1 transition '3' queues is requested, through constructs like

 sxqsub -q sx3 job.file

A detailed note was recently sent to all users via e-mail, and such notices are available on the www.hpccc.gov.au site.

Sometimes, users have used a setting to ensure that output from programs is available immediately rather than being buffered. This was used in difficult debugging cases, to ensure the latest output was received before a crash.

In a recent case, we found that a setting of F_SETBUF to 0 was made for the standard output of a program, and the performance was markedly impacted - in particular the use of GFS file systems was 2 to 8 times slower than the use of local disc. When the setting of F_SETBUF to 0 was deleted, the GFS case performance was comparable to the case using local disc.

Users are strongly advised to almost never use that setting of F_SETBUF to 0.

Users are also requested to move to using GFS rather local disc whenever possible - if you encounter performance problems using GFS compared with local disc, please contact HPCCC staff so that the cause can be investigated.

The APAC05 Conference and Exhibition on Advanced Computing, Grid Applications and eResearch runs from 26-30 September 2005 at the Royal Pines Resort, Gold Coast, Australia.

There are also workshops on Nimrod, the Access Grid, Grid Portals and the Globus Toolkit, and a dedicated student forum.

Details of the APAC05 conference, Empowering Research Communities, are given at www.apac.edu.au/apac05.

We would encourage users of HPC to attend.

The expansion of the main disc on cherax was completed on Saturday 13th August. /cs/datastore is now 2.7 Tbyte, and /work is now 0.4 Tbyte.

Speeds of up to 500 Mbyte/s have been seen on the primary disc areas since the upgrade. However, the system is supporting a high i/o load at present, averaging 100 Mbyte/s over 24 hour periods.

A new version of DMF was installed - new features are available which will be highlighted soon.

Thanks to Jeroen, who worked a very long Saturday, and much of Sunday.

There will be a further outage of cherax, the CSIRO Data Store and www.hpsc.csiro.au in a few weeks' time, to upgrade the operating system to SuSE SLES9 SP2.