Designing a High-Performance Cluster for Geophysical Fluid Dynamics Applications
This article discusses the design process of a high-performance cluster for geophysical fluid dynamics applications and showcases a successful cluster built with a grant from Knut and Alice Wallenberg Foundation. Key components of the cluster include 48 Intel P4 CPUs, 500 MB 800MHz Rdram, and SCI cards, run by NSC and delivered by South Pole.
- Uploaded on | 0 Views
-
jazmineger
About Designing a High-Performance Cluster for Geophysical Fluid Dynamics Applications
PowerPoint presentation about 'Designing a High-Performance Cluster for Geophysical Fluid Dynamics Applications'. This presentation describes the topic on This article discusses the design process of a high-performance cluster for geophysical fluid dynamics applications and showcases a successful cluster built with a grant from Knut and Alice Wallenberg Foundation. Key components of the cluster include 48 Intel P4 CPUs, 500 MB 800MHz Rdram, and SCI cards, run by NSC and delivered by South Pole.. The key topics included in this slideshow are Cluster design, geophysical fluid dynamics, high-performance computing, Intel P4 CPUs, SCI cards,. Download this presentation absolutely free.
Presentation Transcript
1. Designing a cluster for geophysical fluid dynamics applications Gran Brostrm Dep. of Oceanography, Earth Science Centre, Gteborg University .
2. Our cluster (me and Johan Nilsson, Dep. of Meterology, Stockholm University) Grant from the Knut & Alice Wallenberg foundation (1.4 MSEK) 48 cpu cluster Intel P4 2.26 Ghz 500 Mb 800Mhz Rdram SCI cards Delivered by South Pole Run by NSC (thanks Niclas & Peter)
3. What we study
4. Geophysical fluid dynamics Oceanography Meteorology Climate dynamics
5. Thin fluid layers Large aspect ratio
6. Highly turbulent Gulf stream: Re~10 12
7. Large variety of scales Parameterizations are important in geophysical fluid dynamics
8. Timescales Atmospheric low pressures: 10 days Seasonal/annual cycles: 0.1-1 years Ocean eddies: 0.1-1 year El Nino: 2-5 years. North Atlantic Oscillation: 5-50 years. Turnovertime of atmophere: 10 years. Anthropogenic forced climate change: 100 years. Turnover time of the ocean: 4.000 years. Glacial-interglacial timescales: 10.000-200.000 years.
9. Some examples of atmospheric and oceanic low pressures.
10. Timescales Atmospheric low pressures: 10 days Seasonal/annual cycles: 0.1-1 years Ocean eddies: 0.1-1 year El Nino: 2-5 years. North Atlantic Oscillation: 5-50 years. Turnovertime of atmophere: 10 years. Anthropogenic forced climate change: 100 years. Turnover time of the ocean: 4.000 years. Glacial-interglacial timescales: 10.000-200.000 years.
11. Normal state
12. Initial ENSO state
13. The ENSO state
14. The ENSO state
15. Timescales Atmospheric low pressures: 10 days Seasonal/annual cycles: 0.1-1 years Ocean eddies: 0.1-1 year El Nino: 2-5 years. North Atlantic Oscillation: 5-50 years. Turnovertime of atmophere: 10 years. Anthropogenic forced climate change: 100 years. Turnover time of the ocean: 4.000 years. Glacial-interglacial timescales: 10.000-200.000 years.
16. Positive NAO phase Negative NAO phase
18. Positive NAO phase Negative NAO phase
20. Timescales Atmospheric low pressures: 10 days Seasonal/annual cycles: 0.1-1 years Ocean eddies: 0.1-1 year El Nino: 2-5 years. North Atlantic Oscillation: 5-50 years. Turnovertime of atmophere: 10 years. Anthropogenic forced climate change: 100 years. Turnover time of the ocean: 4.000 years. Glacial-interglacial timescales: 10.000-200.000 years.
21. Temperature in the North Atlantic
22. Timescales Atmospheric low pressures: 10 days Seasonal/annual cycles: 0.1-1 years Ocean eddies: 0.1-1 year El Nino: 2-5 years. North Atlantic Oscillation: 5-50 years. Turnovertime of atmophere: 10 years. Anthropogenic forced climate change: 100 years. Turnover time of the ocean: 4.000 years. Glacial-interglacial timescales : 10.000-200.000 years.
23. Ice coverage, sea level
24. What model will we use?
25. MIT General circulation model
26. MIT General circulation model General fluid dynamics solver Atmospheric and ocean physics Sophisticated mixing schemes Biogeochemical modules Efficient solvers Sophisticated coordinate system Automatic adjoint schemes Data assimilation routines Finite difference scheme F77 code Portable
27. MIT General circulation model Spherical coordinates Cubed sphere
28. MIT General circulation model General fluid dynamics solver Atmospheric and ocean physics Sophisticated mixing schemes Biogeochemical modules Efficient solvers Sophisticated coordinate system Automatic adjoint schemes Data assimilation routines Finite difference scheme F77 code Portable
29. MIT General circulation model
30. MIT General circulation model
31. MIT General circulation model
32. MIT General circulation model
33. MIT General circulation model
34. MIT General circulation model
35. MIT General circulation model
36. MIT General circulation model
37. Some computational aspects
38. Some tests in INGVAR (32 AMD 900 Mhz cluster)
39. Experiments with 60*60*20 grid points
40. Experiments with 60*60*20 grid points
41. Experiments with 60*60*20 grid points
42. Experiments with 120*120*20 grid points
43. MM5 Regional atmospheric model
44. MM5 Regional atmospheric model
45. MM5 Regional atmospheric model
46. Choosing cpus, motherboard, memory, connections
47. Specfp (swim)
48. Run time on different nodes
49. Choosing interconnection (requires a cluster to test) Based on earlier experience we use SCI from Dolphinics (SCALI)
50. Our choice Named Otto SCI cards P4 2.26 GHz (single cpus) 800 Mhz Rdram (500 Mb) Intel motherboards (the only available) 48 nodes NSC (nicely in the shadow of Monolith)
51. Otto (P4 2.26 GHz)
52. Scaling Otto (P4 2.26 GHz) Ingvar (AMD 900 MHz)
53. Why do we get this kind of results?
54. Time spent on different subroutines 60*60*20 120*120*20
55. Relative time Otto/Ingvar
56. Some tests on other machines INGVAR: 32 node, AMD 900 MHz, SCI Idefix: 16 node, Dual PIII 1000 MHz, SCI SGI 3800: 96 Proc. 500 MHz Otto: 48 node, P4 2.26 Mhz, SCI ? MIT, LCS: 32 node, P4 2.26 Mhz, MYRINET
57. Comparing different system (120*120*20 gridpoints)
58. Comparing different system (120*120*20 gridpoints)
59. Comparing different system (60*60*20 gridpoints)
60. SCI or Myrinet? 120*120*20 gridpoints
61. SCI or Myrinet? 120*120*20 gridpoints (60*60*20 gripoints) (ooops, I used the ifc Compiler for these tests)
62. SCI or Myrinet? 120*120*20 gridpoints (60*60*20 gripoints) (ooops, I used the ifc Compiler for these tests) (1066Mhz rdram?)
63. SCI or Myrinet? (time spent in pressure calc.) 120*120*20 gridpoints (60*60*20 gripoints) (ooops, I used the ifc Compiler for these tests) (1066Mhz rdram?)
64. Conclusions Linux clusters are useful in computational geophysical fluid dynamics!! SCI cards are necessary for parallel runs >10 nodes. For efficient parallelization: >50*50*20 grid points per node! Few users - great for development. Memory limitations, for 48 proc. a 500 Mb, 1200*1200*30 grid points is maximum (eddy resolving North Atlantic, Baltic Sea). For applications similar as ours, go for SCI cards + cpu with fast memory bus and fast memory!!
65. Experiment with low resolution (eddies are parameterized)
68. Experiment with low resolution (eddies are parameterized)
69. Thanks for your attention
