Networking Resources
for
Collaborative Research in the Southeast

Organized by:

AAAS
in collaboration with
SURA and NCSA


Developing a Heterogeneous Computing Environment
to Simulate Astrophysical Fluid Flows

Joel E. Tohline
HOMEPAGE: //www.phys.lsu.edu/faculty/tohline

Department of Physics & Astronomy
Louisiana State University


1. Lessons Learned (a brief, personal history)

['85-'87] Minnesota Supercomputing Center:
Cray 2
['87-'92] Cornell Theory Center:
IBM 3090 + FPS Array Processors + FPS Hypercube
Greatest successes during late '80s and early '90s:
  • T1 network line into LSU
  • NCSA Telnet
  • NCSA Mosaic
  • Louisiana Board of Regents grant to purchase 8K-node MasPar MP-1 and to establish CCLMS
  • Development of visualization tools, particularly at NCSA and CTC
[1992] Abandoned CTC:
  • Lack of commitment to a stable operating system
  • Lack of vision with regard to parallel architectures
  • Overcommitment to IBM!
['92-'93] Brief stints at PSC and NCSA:
Chasing TMC machines
['97-present] SDSC [NPACI] and Stennis [DoD MSRC}:
Cray T3E and SGI Onyx


2. Present Computing Environment and Strategy [A Success!]


3. Problems Remaining:

It's the Network, stupid !
Sometimes the network connection is so slow, it is impossible to get any work done! We regularly login to a machine at Goddard and connect to SDSC from there because it is quicker than a direct login.

How can we ever hope to perform routine analysis of our huge data sets remotely with the existing network situation?

Stable Parallel Architectures with Efficient Compilers
Disk Space
We are only allowed 10 MB of disk space on the T3E at SDSC. What a joke! Our compiled code with object files is 10 MB alone. This means we must maintain everything else (even diagnostic and continuation files) on scratch disk or download the data to our local disks (network!!) or use SDSC's archive server, which we've found to be unreliable.

The SGI has the opposite problem. There are no quotas on the SGI cluster, so sometimes the disk containing our home directory fills up. This screws up our automated imaging process. Also, sometimes their scratch disk fills up. (They must not have any automated routines to handle this, because we usually have to send e-mail to get the disk space fixed. And they're not very pleasant when asked to help out!)

Visualization Support
When we initially spent a full week at the SDSC in order to establish our HCE, we received relatively little help from experienced visualization staff. The vislab seemed to be geared to support the visualization needs of local researchers; they did not seem to even know what remote, batch processing was. (They also spend a disproportionate amount of time using their most powerful SGI hardware for conferences and tours!)
Staff Support
Every national center should require that 1/3 of its user-support staff spend at least 2 months each year at a remote (southeastern region?) site so that they can experience first-hand the frustrations associated with being a remote user.


4. The Future?

More Physics:
Incorporate more physics into simulation models.
New Simulation Tools:
Develop an implicit code to evolve systems on longer-than-dynamical timescales.
Diagnostic/Visualization Tools:
Develop time-dependent VRML tools. (Must have high bandwidth network!)
More Versatile Publication Medium:
Develop a truly on-line, browser-accessible publication medium that will permit those of us in simulation work to much more completely present the results of our research efforts.

5. Acknowledgments

This work has been supported, in part, by the U.S. National Science Foundation through grant AST-9528424 and, in part, by grants of high-performance-computing time at the San Diego Supercomputer Center and through the PET program of the NAVOCEANO DoD Major Shared Resource Center in Stennis, MS.