|
|
||||
|
Bulletin 137 - 2005 Apr 28
1. Change of the www.hpsc.csiro.au WWW pages Late this week (or possibly early next week), we plan to make changes to the WWW page visibility. An entirely new form of pages will replace the existing pages at http://www.hpsc.csiro.au/
(The old pages will continue to be available from 2. Allowing migration of jobs on the SX-6 system There are advantages in allowing jobs to migrate between nodes on the SX-6 system. For example, earlier this week a user had a job return to the queues for an hour while operational jobs ran on the node that the job was assigned to. From 3rd May, we will work to enable job migration to be used on the bm, bmml, and bmmn queues (it is already enabled for the cs and csml queues). All jobs in these queues should include the flag -J n if migration is not desirable, e.g. if the job uses local disc on a node. Jobs that previously used the no-hold (otherwise known as the no-checkpoint) flag (-H n) to prevent migration should change to using just the no-migrate flag. Such jobs will then be saved over restarts. See the updated SX-6 Userguide and HPCBull 135, item 1.3 for more information. 3. Change of home base for the req (wreq) problem reporting system
The move of the problem reporting system known as wreq or req to See HPCbull 136.2 for further information. 4. pkgenv utility on the SX-6/TX7 system The pkgenv utility has been installed on the SX-6/TX7 system. It is now available for all users - it was restricted initially to CSIRO users. 5. New qsub local version The test qsub (qsubnew) command for the SX-6/TX7 system will replace the default version from Tuesday morning 10th May. This version fully supports the new -l cpunum_job=n parameter into the batch system for better scheduling. The value specified for jobs will be displayed by the erstatj command. 6. Downtime - CSIRO IBM clusters The CSIRO cluster burnet will be unavailable on Saturday 30th April from 09:00 to 14:00 for further software upgrades. 7. cherax and CSIRO Data Store status and upgrades The CSIRO Data Store usage has grown to from 57 to 81 Tbyte is less than three months, representing and annual growth of more than a factor of 5. This cannot be sustained with current budgets. Both copies of large files will from now on be stored on T9940 tape media - the higher capacity but slower access tapes. The boundary between 'large' and 'small' files will change with time as usage expands. Only one copy of some data uploaded from other sources will be kept. Users who delete (carefully!) large quantities of unwanted files will help to maintain performance. An extra T9940 tape drive has been installed and will be configured at the next re-boot. An extra 3 Tbyte of disc has been installed, and plans are underway for how this will be deployed. 8. Storage Resource Broker (SRB) CSIRO HPSC is now supporting the Storage Resource Broker (from the San Diego Supercomputer Center; see www.sdsc.edu/srb). SRB is client-server middleware which provides users with a unified view of their data holdings (which may be distributed across a network on heterogeneous hardware resources). It also provides a Meta-data-base accompanying the data. This combination enables users to access data resources based on their attributes and/or logical names rather than their physical location. The basic concept is that there are many SRB 'Zones' (each consisting of an SRB server and some data resources). These Zones can be 'federated' so that users can access seamlessly the resources of all of the Zones in the federation. Users of SRB can operate purely as clients to an existing Zone (and its federation partners), or if appropriate, host their own Zone and serve data resources. Here is a simple example in which a user puts data to two different SRB servers; one has a file-system called 'LOC-ZONE1' (hosted by the server in Zone 'ZONE1' and the other 'LOC-ZONE2' (hosted by the server in Zone 'ZONE2'); all the user has to know is the name of the file system (not where it is). In this example we use the Unix command-line interface (the S-commands client). % Sput -S LOC-ZONE1 file1.dat # Put data to SRB in ZONE1 % Sput -S LOC-ZONE2 file2.dat # Put data to SRB in ZONE2 % rm file1.dat file2.dat # Remove local copies of files % % Sget file1.dat file2.dat # Retrieve the files to local storage % Srm file1.dat file2.dat # Delete the files from SRB There are a range of other client-interfaces (HTML, Windows, Java, WSDL etc.) as well as the simple Unix S-commands. You will need to obtain and install client software on any local hosts that you use to interact with SRB. SRB has lots of functionality, including parallel data transfer (i.e. the file is sent in parallel streams resulting in better performance over long-fat pipes), file replication, third-party transfer and so on. HPSC is hosting an SRB Zone and providing data resources via the HPSC hierarchical file system. Our Zone can be federated with Zones exterior to CSIRO (such as APAC) as well as with Zones interior to CSIRO. For more information, contact Neil.Killeen@csiro.au and/or enjoy the SRB website at www.sdsc.edu/srb 9. Installation of Grace on farrer (CSIRO only)
At the request of a user, the scientific graphical display package Grace has been installed on farrer.hpsc.csiro.au. Try
Grace has some similarities with gnuplot, but with a GUI and much wider capabilities. For example it deals with netCDF files. It has grown out of a previous program called ACE/gr (also known as Xmgr). More information is available at the home website
|
|
Comments to: © Copyright 2010, CSIRO Australia Use of this web site and information available from it is subject to our Legal Notice and Disclaimer and Privacy Statement |