Report Navigation

GEaRS: High Performance Computing

LION-XC

LION-XC is the newest cluster built in partnership with 14 faculty members from 9 different departments. It consists of 140 compute servers, each with two dual-core Intel Xeon processors and either 8 or 16GB of RAM, for a total of 560 processor cores and 1.5TB of RAM. The entire cluster is interconnected with the Qlogic Infiniband high performance network.

LION-XA

Lion-XA is an experimental system currently being used as a testbed for parallel filesystems. Parallel filesystems differ from traditional filesystems; rather than have a single filesystem served from a single fileserver, the filesystem is distributed across multiple servers so that filesystem operations can take advantage of the aggregate performance of all of the servers in the filesystem. Parallel filesystems also help overcome the filesystem size limitations, thus helping to address significant increase in demand for storage space from faculty members in science, engineering and other disciplines.

The GEaRS group continues to investigate several of the most promising parallel filesystems available today to find one whose implementation will best serve the needs of the Penn State research community. These include, but are not limited to, GPFS, IBRIX, iGrid, Lustre, and CXFS.

Throughout the next year, 16 nodes of LION-XA will be moved from parallel file system testing and will be deployed as the new LION-XD test and development cluster.

LION-XB

LION-XB blends two distinct paradigms of high performance computing: 1) multiprocessor systems with large shared memory, and 2) distributed memory compute engine built with servers of fewer processors. LION-XB uses multi-core processors in a hybrid approach where a cluster model is followed, but the individual nodes of the cluster are mid-sized shared-memory systems. LION-XB is built from 4-socket servers that use dual-core AMD Opteron processors, yielding 8 processing elements in each server. Each server has 32GB of shared memory. The LION-XB cluster has 16 such servers, with an aggregate of 128 processing cores and 512GB of memory.

Using this hybrid approach, LION-XB simultaneously can accommodate both shared memory parallel programming (mostly using compiler directives) and distributed memory programming (using an MPI library). These distinct programming approaches are needed depending on the numerical algorithms and solution techniques that are applied to problems in various academic disciplines.

LION-XB makes use of the QLogic HTX InfiniBand adapters using the native bus used for communication between processors on AMD Opteron systems rather than using the traditional PCI bus. By having the InfiniBand adapter talk directly to the HTX bus, overhead is greatly reduced and performance increases tremendously. These InfiniBand adapters have a measured latency of 1.3 microseconds and a bandwidth of 930 MB/s. The QLogic HTX InfiniBand adapters scale with the number of processors within a system, reaching 11 million packets per second on an 8 core system like LION-XB.

The architecture of the individual nodes of LION-XB allows for more of the research computing user community to run parallel applications via shared memory parallel applications, in addition to the more traditional message passing codes.

LION-XO

LION-XO contains 124 compute servers with a total of 336 processors and 1.5TB of RAM. Aggregate local storage available as scratch space for running jobs is 25TB on this system. The processor count is broken down as follows: 32 SunFire v40z servers with 16GB of RAM, 12 SunFire v40z servers each with 32GB RAM, and 80 SunFire v20z servers each with 16GB of memory. The servers with 32GB of RAM allow GEaRS to address the needs of users who have research problems that require a large amount of memory to solve. This cluster was built in partnership with ten faculty members from eight different departments.

LION-XM

LION-XM is a cluster consisting of 176 compute servers with a total of 352 processors and 704GB of RAM, all inter-connected with the Myricom Myrinet high performance network. This cluster was built in partnership with fourteen faculty members from nine different departments.

LION-XL

LION-XL is a cluster consisting of 176 compute servers with a total of 352 processors and 704GB of RAM, all inter-connected with the Quadrics Elan3 high performance network. This cluster was built in partnership with eleven faculty members from six different departments.

Hammer

Hammer is an interactive login cluster that mainly targets users of scientific applications such as MATLAB, ANSYS, ABAQUS, etc. through their GUI interfaces. Hammer consists of seven Sun SunFire v40z servers, each with quad 2.6 GHz Opteron processors, 32GB of RAM and 584 GB of local scratch disk. This cluster allows the interactive use of applications in the solution of large scale research problems unable to be solved on more limited resources (e.g. departmental workstations).

Unisys

The Unisys ES7000 system targets the needs of researchers with large-scale shared-memory parallel code and also of those researchers who need to address an extraordinarily large amount of memory in a single system. The Unisys ES7000 consists of three machines: 1) a 16 Intel Itanium 2 processor login node with 32GB of RAM, 2) a 16 Intel Itanium 2 processor compute node with 32GB of RAM and 3) a 32 Intel Itanium 2 processor system with 128GB of RAM. These Unisys ES7000 systems also have more than 3TB of local scratch disk available to enable solving data intensive problems.

IBM Power System

The IBM p570 system is operated in partnership with the Institute for Computational Science (ICS) as a resource to support computing needs of the graduate Minor in Computational Science. This system consists of 6 dual-core IBM Power5 processors with 112GB of RAM. The system was made possible through an IBM Shared University Research (SUR) grant, an IBM program for promoting research in areas of mutual interest between IBM and the receiving institution.

Case Studies

The computing resources supported by GEaRS are at times tailored to provide solutions for various classes of problems being addressed by Penn State researchers. In particular, recent systems changes have targeted classes of problems that required more systems memory to be solved than previously available. The following are examples of how such problems have been addressed during the 2006-07 reporting period:

A focus area during this reporting period has been facilitating the solution of larger scale Finite Element Analysis (FEA) problems by taking advantage of recent improvements in GEaRS system capacities using applications such as ABAQUS, Nastran and LS-DYNA. Terry Johnson is an Aerospace Engineering Graduate Student who has been using hammer and lionxc to run very large ABAQUS jobs. In particular, the research involves evaluation of bistable structures for tip twist of helicopter rotor blades. The large memory and disk space of GEaRS compute clusters have been helpful in tackling the large problems addressed in this work.

Deepak Trivedi, a graduate student working with Dr. Gary Koopman and Dr. Chris Rahn, is using Hammer to solve a very large linear system of equations (dense matrix of dimensions 125000x125000) that come from a three-dimensional underwater acoustics problem. The Green's function in this problem has a three-level block Toeplitz structure. Fast iterative methods that exploit the matrix structure have been developed. However, since the equations have to be solved repeatedly for the same left hand side to perform design optimization, a one-time inversion is sufficient. This necessitates use of large memory while also making it computationally expensive. HPC systems have been effective in keeping this research project moving forward.

Boyan Neykov is a graduate student in mechanical and nuclear engineering. He is using Hammer and Fluent, a fluid-dynamics software package, to solve heat transfer related problems with nuclear fuel rod bundles.