Bulletin 175 - 2008 February 08

  1. SX-6 scheduling
  2. Creation of CSIRO Advanced Scientific Computing
  3. CSIRO Burnet Cluster Refresh
  4. Cherax storage maintenance
  5. Backups
  6. SX Cross Environment Modules
  7. Retirement - Bob Smart
  8. Seminar

Note: "CSIRO" items can apply to BoM users of cherax and burnet


1. SX-6 scheduling

A trial has been underway for over a month of a different way of allocating resources on the CSIRO SX-6 nodes.

Single-CPU jobs are allocated to the Resource Group 0, which is also used for system tasks.

This allows the other 7 CPUs to be used exclusively for multi-CPU jobs, and stops floods of single-CPU jobs from pushing out multi-CPU jobs.

It is planned to extend this new arrangement to Bureau nodes for testing.

To that end, would users who currently request 8 CPUs per job please reduce that request to a maximum of 7 CPUs per job (per node) before 1st March 2008. (In NEC NQSII terminology, a job is that portion of a request which is restricted to running on only one node).

After that, single-CPU Bureau queues, including operational single-CPU jobs, will be directed to Resource Group 0 for a trial period.

If the trial is successful, the plan is to extend the single-CPU queues to more nodes, and to make the queue assignments more uniform across the nodes.

(see http://www.hpccc.gov.au/hpccc/user_news_advice/queue_status/queue_status.shtml)

This should give better throughput for multi-CPU and multi-node jobs.

[ page top ]



2. Creation of CSIRO Advanced Scientific Computing

The HPSC and e-Science groups within CSIRO IM&T have been merged to create the Advanced Scientific Computing group. Dr Alf Uhlherr is the leader of the new group. CSIRO users can expect the ASC label to replace HPSC in the future. There will be 90 minute presentations (which includes time for questions) at various locations in Australia to help CSIRO users understand and take full advantage of CSIRO Advanced Scientific Computing capabilities. Venues are given below; further locations can be added on request.

  • North Ryde, 12th February
  • Clayton, 13th February
  • Hobart, 14th February
  • Black Mountain, 27th February

The ASC team also welcome the opportunity to meet for more in-depth discussion on the day with individual groups, concerning the ASC requirements for their research.

For further information, please contact Justin Baker on 03 8601 3801, or

[ page top ]



3. CSIRO Burnet Cluster Refresh
  1. Hardware Environment

    CSIRO HPSC has taken delivery of equipment to refresh the CSIRO clusters. The equipment will provide a more reliable head node and storage around the clusters.

    The plan is to:

    • merge the CMAR cluster 'nelson' into 'burnet'
    • decommission some nodes and processors on nelson, and double the memory of the remaining nodes
    • add new head and system management nodes to burnet, to separate the functions
    • add two storage servers running IBM's GPFS, to provide High-performance and reliable file services to the cluster
    • add about 5 Tbyte of storage. This will be configured mainly as additional $WORKDIR and $DATADIR for users.


  2. Software Environment

    The plan is to:

    • upgrade to SLES 10 operating systems on the head and compute nodes

    To ensure a smooth transition to the new Operating System environment, we ask that users please advise the HPSC of their critical and required software, including any software accessed via 'module load <package>/<revision>'. This will allow us to prioritize the porting and installation of software to make it available in a timely manner.

Call the helpdesk on 03 9669-8103 to discuss your requirements or email /

[ page top ]



4. Cherax storage maintenance

On 2nd January, cherax was shutdown for a period to carry out repairs to the disc sub-systems.

A major improvement was made to the IS220 sub-system, with one defective chassis replaced. This resulted in major performance improvements, particularly to the backup processes which wrote to these discs. The average time for one process dropped from 9 to 3 hours.

[ page top ]



5. Backups

The Data Storage Management Userguide at http://intra.hpsc.csiro.au/userguides/ds/ has been revised, principally to include more information about backups, and the circumstances under which files can be restored from them.

[ page top ]



6. SX Cross Environment Modules

Users will be aware that we recently installed Environment modules for SX Cross development and utilities (see System Change Notice 2008-A003). Environment modules are to be used to setup the user environment for SX cross software. It will replace the 'sxcross' utility. Users are encouraged to migrate from using sxcross to using Environment Modules. Please consult the 'Cross Systems' section of the HPCCC SX-6 User Guide for advice http://www.hpccc.gov.au/hpccc/userguides/sx/.

[ page top ]



7. Retirement - Bob Smart

Bob Smart is retiring from CSIRO on 14th February, having reached a magic age, and having spent 22 years working for CSIRO.

Bob joined CSIRO Division of Information Technology in 1986 in Carlton, and was involved in early efforts to join CSIRO to the Internet. Bob was instrumental in having CSIRO being the second organisation in Australia to be connected, and was responsible for setting up the csiro.au domain, giving CSIRO a whole sub-domain to itself at the national level.

Bob was part of the CSIRO Supercomputing Facilities Task Force in 1989, which led to the acquisition of CSIRO's first Cray, and he managed the network connections for that facility.

In the 1990s, Bob worked on various research projects around networking, and in 1999 joined the HPCCC, later to be part of the CSIRO HPSC team.

In recent years, Bob has been based in Sydney, and has particularly helped with the preparations for the computing needs of the ASKAP project.

We wish Bob well in his retirement.

If you wish to join in any of the farewell functions, please contact Rob Bell.

[ page top ]



8. Seminar

10:00 Friday 15th February 2008,
700 Collins St, Level 9E Conference Room

"Hedging the Weather needs an Intelligent Market"

Robert Smart from: Seagull Winds -- putting the Internet to work

Abstract

Modern radar systems deployed by the Bureau of Meteorology may be able to give an unbiased and reasonably accurate report of how much rain has fallen and where, in areas covered by such radar. This can form the basis of an insurance system to allow farmers and others in those areas to hedge against the weather. It could be done in a simple and old fashioned way, or we could do it properly by building an Intelligent Market. This talk will explain how such an Intelligent Market would operate and its advantages. Such a market would, in turn, create a demand for climate and weather expertise and technology and for the expansion of radar coverage. There are prospects for other Intelligent Markets which would naturally interact with the Rainfall Hedge market.

[ page top ]




BoM Solar Help:

CSIRO ASC Help:

For urgent help at all times:
  • CSIRO users 0428 108 333
  • Bureau out of hours emergencies are managed through internal policy
HPCCC WWW Site: http://www.hpccc.gov.au/
CSIRO External ASC Site: http://www.hpsc.csiro.au/
CSIRO ASC Users' Site: http://intra.hpsc.csiro.au/

Comments to:


© Copyright 2010, CSIRO Australia
Use of this web site and information available from it is subject to our Legal Notice and Disclaimer and Privacy Statement