A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator

Dimitri Komatitsch, Seiji Tsuboi, Chen Ji, Jeroen Tromp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

72 Citations (Scopus)

Abstract

We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3%, and we reach a performance of 5 teraflops (30% of the peak performance) on 38% of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.

Original languageEnglish (US)
Title of host publicationProceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003
DOIs
StatePublished - Dec 1 2003
Event2003 ACM/IEEE Conference on Supercomputing, SC 2003 - Phoenix, AZ, United States
Duration: Nov 15 2003Nov 21 2003

Publication series

NameProceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003

Other

Other2003 ACM/IEEE Conference on Supercomputing, SC 2003
CountryUnited States
CityPhoenix, AZ
Period11/15/0311/21/03

Fingerprint

Earthquakes
Simulators
Earth (planet)
Bathymetry
Seismic waves
Wave propagation
Topography
Data storage equipment

All Science Journal Classification (ASJC) codes

  • Software

Cite this

Komatitsch, D., Tsuboi, S., Ji, C., & Tromp, J. (2003). A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator. In Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003 (Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003). https://doi.org/10.1145/1048935.1050155
Komatitsch, Dimitri ; Tsuboi, Seiji ; Ji, Chen ; Tromp, Jeroen. / A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator. Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003. 2003. (Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003).
@inproceedings{91225079dd7d4641b5530bb8575a2ec6,
title = "A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator",
abstract = "We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3{\%}, and we reach a performance of 5 teraflops (30{\%} of the peak performance) on 38{\%} of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.",
author = "Dimitri Komatitsch and Seiji Tsuboi and Chen Ji and Jeroen Tromp",
year = "2003",
month = "12",
day = "1",
doi = "https://doi.org/10.1145/1048935.1050155",
language = "English (US)",
isbn = "1581136951",
series = "Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003",
booktitle = "Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003",

}

Komatitsch, D, Tsuboi, S, Ji, C & Tromp, J 2003, A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator. in Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003. Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003, 2003 ACM/IEEE Conference on Supercomputing, SC 2003, Phoenix, AZ, United States, 11/15/03. https://doi.org/10.1145/1048935.1050155

A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator. / Komatitsch, Dimitri; Tsuboi, Seiji; Ji, Chen; Tromp, Jeroen.

Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003. 2003. (Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator

AU - Komatitsch, Dimitri

AU - Tsuboi, Seiji

AU - Ji, Chen

AU - Tromp, Jeroen

PY - 2003/12/1

Y1 - 2003/12/1

N2 - We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3%, and we reach a performance of 5 teraflops (30% of the peak performance) on 38% of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.

AB - We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3%, and we reach a performance of 5 teraflops (30% of the peak performance) on 38% of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.

UR - http://www.scopus.com/inward/record.url?scp=84877030797&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877030797&partnerID=8YFLogxK

U2 - https://doi.org/10.1145/1048935.1050155

DO - https://doi.org/10.1145/1048935.1050155

M3 - Conference contribution

SN - 1581136951

SN - 9781581136951

T3 - Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003

BT - Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003

ER -

Komatitsch D, Tsuboi S, Ji C, Tromp J. A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth simulator. In Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003. 2003. (Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003). https://doi.org/10.1145/1048935.1050155