We are preparing for acquisition or upgrade to meet
CSIRO's High Performance needs for mid 2007 and beyond.
We need benchmarks reflecting scientific needs. We are
conscious that code will change and the detail of data to be
analyzed will change. So don't let expected changes stop
you from offering code. This can include the use of standard
systems (like Gaussian), though we might ultimately be unable
to sort out the licensing problems to use non-free software
in benchmarks.
Up to 20% of the processing resources of cherax are available
for BoM use. There is still a lot of this allocation
available. Cherax can be used for data exchange and for
shared projects. So if you are a BoM user and you would like
an account on cherax please contact send an e-mail to
. For existing
HPCCC users, there is no need to fill in a new registration
form.
Cherax has changed for the better, but some things take a bit
of getting used to. There have also been things that needed
to be fixed. If you have problems with the new cherax please
don't hesitate to contact us: 03 9669 8103 or
.
Cherax now uses modules. Do "module --help"
or see our web page
http://intra.hpsc.csiro.au/userguides/faq/env_modules.php.
Note that "module avail" gives a succinct
list of what is available and "module whatis"
gives some information about each module.
Cherax's MPI runtime environment
The following environment variables have been set in
your environment to give a conservative memory
allocation for MPI buffers:
The specified amount of memory is allocated by *each*
process n times, where n is the number of processes!
Note: Without these settings, the buffer is determined
by the average available memory per processor - for
cherax this is 2GB for each buffer, which is quite
large.
If the buffer sizes are insufficient for your MPI
application you can override these variables in your
batch script. See 'man mpi' for more details.
Cherax's Batch System
Prologue and Epilogue:
In your batch job output, will now see a
'Prologue' and an 'Epilogue' section.
The Prologue section will show the credentials and
requested resources for your job as well as the cpuset
created for your job. The cpuset is used for managing
cpu and memory placement for your job, giving better
memory locality for processes. The Epilogue section
will show, in addition to credentials and requested
resources, the amount of resources actually used. This
information can be used to better estimate the required
resources for your job.
pbstop command:
A new command called pbstop is available which presents
a summary view of the batch system showing the jobs and
allocated cpus. If you are a user of burnet, you may
already be familiar with this command. Please do not
run pbstop for extended periods as the qstat commands
it runs can cause a significant overhead on the batch
system server if many users run them at once.
overload queue:
An 'overload' queue has been established to
allow, in situations where cherax is experiencing high
load, a relatively small amount of over-commitment of
CPUs. This will provide an overall better throughput
and turn-around for jobs. You must explicitly specify
'-q overload' in your qsub options to run your
jobs in the overload queue. Note: It is currently
limited to jobs that will complete in under one hour.
specifying memory for your batch job:
Use the PBS option: -l vmem=<x>gb to specify the
maximum memory required for all of the processes in
your job. The queues batch/short have a default vmem
value of 500mb and the overload queue a default of
200mb.
ls time format
Cherax's default time format output of "ls
-l" has changed. Use --time-style=locale for the
old behaviour.
improved interactive response
With the new improved Linux kernel, the incidences of
delayed responses to commands and character echo has
dropped dramatically. This long-standing annoyance with
the cherax service appears to be solved.
