High Performance Input/Output
           
Event Type Start Time End Time Rm # Chair  

 

Paper 3:30PM 4:00PM 36-37 Josep Torrellas (University of Illinois)
 
Title:
Fast Parallel Non-Contiguous File Access
  Speakers/Presenter:
Joachim Worringen (NEC C&C Research Lab), Jesper Larson Traff (NEC C&C Research Lab), Hubert Ritzdorf (NEC C&C Research Lab)

 

Paper 4:00PM 4:30PM 36-37 Josep Torrellas (University of Illinois)
 
Title:
Parallel netCDF: A High-Performance Scientific I/O Interface
  Speakers/Presenter:
Jianwei Li (ECE Department, Northwestern University), Wei-keng Liao (ECE Department, Northwestern University), Alok Choudhary (ECE Department, Northwestern University), Robert Ross (MCS Division, Argonne National Laboratory), Rajeev Thakur (MCS Division, Argonne National Laboratory), William Gropp (MCS Division, Argonne National Laboratory), Rob Latham (MCS Division, Argonne National Laboratory), Andrew Siegel (MCS Division, Argonne National Laboratory), Brad Gallagher (ASCI Flash Center, University of Chicago), Michael Zingale (UCO/Lick Observatory, University of California, Santa Cruz)

 

Paper 4:30PM 5:00PM 36-37 Josep Torrellas (University of Illinois)
 
Title:
Grid -Based Parallel Data Streaming implemented for the Gyrokinetic Toroidal Code
  Speakers/Presenter:
Scott Alan Klasky (PPPL), Stephane Ethier (PPPL), Zhihong Lin (UC Irvine), Kevin Martins (PPPL), Doug McCune (PPPL), Ravi Samtaney (PPPL)
             

 

     
  Session: High Performance Input/Output
  Title: Fast Parallel Non-Contiguous File Access
  Chair: Josep Torrellas (University of Illinois)
  Time: Thursday, November 20, 3:30PM - 4:00PM
  Rm #: 36-37
  Speaker(s)/Author(s):  
  Joachim Worringen (NEC C&C Research Lab), Jesper Larson Traff (NEC C&C Research Lab), Hubert Ritzdorf (NEC C&C Research Lab)
   
  Description:
  Many applications of parallel I/O perform non-contiguous file accesses, but only few file system interfaces support non-contiguous access. In contrast, the most commonly used parallel programming interface, MPI, supports parallel I/O through its MPI-IO interface. Within this interface, non-contiguous accesses are supported by the use of derived MPI datatypes. Unfortunately, current MPI-IO implementations suffer from low performance of such non-contiguous accesses when compared to the performance of the storage system for contiguous accesses although a considerable amount of work has been done in this area. In this paper we analyze an important bottleneck in current implementations of MPI-IO, and present a new technique termed listless i/o to perform non-contiguous access with MPI-IO. On the NEC SX-series of parallel vector computers, listless i/o is able to increase the bandwidth for non-contiguous file access by sometimes more than a factor of 500 when compared to the traditional approach.
  Link: Download PDF
   

 

     
  Session: High Performance Input/Output
  Title: Parallel netCDF: A High-Performance Scientific I/O Interface
  Chair: Josep Torrellas (University of Illinois)
  Time: Thursday, November 20, 4:00PM - 4:30PM
  Rm #: 36-37
  Speaker(s)/Author(s):  
  Jianwei Li (ECE Department, Northwestern University), Wei-keng Liao (ECE Department, Northwestern University), Alok Choudhary (ECE Department, Northwestern University), Robert Ross (MCS Division, Argonne National Laboratory), Rajeev Thakur (MCS Division, Argonne National Laboratory), William Gropp (MCS Division, Argonne National Laboratory), Rob Latham (MCS Division, Argonne National Laboratory), Andrew Siegel (MCS Division, Argonne National Laboratory), Brad Gallagher (ASCI Flash Center, University of Chicago), Michael Zingale (UCO/Lick Observatory, University of California, Santa Cruz)
   
  Description:
  Dataset storage, exchange, and access play a critical role in scientific applications. For such purposes netCDF serves as a portable, efficient file format and programming interface, which is popular in numerous scientific application domains. However, the original interface does not provide an efficient mechanism for parallel data storage and access.

In this work, we present a new parallel interface for writing and reading netCDF datasets. This interface is derived with minimal changes from the serial netCDF interface but defines semantics for parallel access and is tailored for high performance. The underlying parallel I/O is achieved through MPI-IO, allowing for substantial performance gains through the use of collective I/O optimizations. We compare the implementation strategies and performance with HDF5. Our tests indicate programming convenience and significant I/O performance improvement with this parallel netCDF (PnetCDF) interface.
  Link: Download PDF
   

 

     
  Session: High Performance Input/Output
  Title: Grid -Based Parallel Data Streaming implemented for the Gyrokinetic Toroidal Code
  Chair: Josep Torrellas (University of Illinois)
  Time: Thursday, November 20, 4:30PM - 5:00PM
  Rm #: 36-37
  Speaker(s)/Author(s):  
  Scott Alan Klasky (PPPL), Stephane Ethier (PPPL), Zhihong Lin (UC Irvine), Kevin Martins (PPPL), Doug McCune (PPPL), Ravi Samtaney (PPPL)
   
  Description:
  We have developed a threaded parallel data streaming approach using Globus to transfer multi-terabyte simulation data from a remote supercomputer to the scientist's home analysis/visualization cluster, as the simulation executes, with negligible overhead. Data transfer experiments show that this concurrent data transfer approach is more favorable compared with writing to local disk and then transferring this data to be post-processed. The present approach is conducive to using the grid to pipeline the simulation with post-processing and visualization. We have applied this method to the Gyrokinetic Toroidal Code (GTC), a 3-dimensional particle-in-cell code used to study micro-turbulence in magnetic confinement fusion from first principles plasma theory.
  Link: Download PDF