Indian Government-Affiliated Research Laboratory Adopts Infortrend’s Storage for Weather Research and Forecasting

Indian Government-Affiliated Research Laboratory Adopts Infortrend’s Storage for Weather Research and Forecasting

A government-affiliated research laboratory is a prime centre in India specialized in fundamental and applied research in atmospheric and space sciences. To meet the computational needs of scientific research and simulations, the laboratory hosts on-premises HPC data center. For weather prediction and atmospheric research needs, the laboratory utilizes a numerical weather prediction framework — Weather Research and Forecasting (WRF) Model that relies heavily on HPC. The WRF process includes two phases: 1) ingesting the input data, i.e., large amounts of high-resolution images captured across multiple regions to provide initial conditions for the WRF; 2) running the forecasting model. For identifying patterns in the input data and analyzing the output data, the laboratory builds deep learning neural networks with the PyTorch framework and a Conda environment within a high-performance XFS file system. Recently, the size of the HPC cluster deployed by the research laboratory increased and the complexity of its WRF Model also raised. However, this increased demands for storage I/O throughput, as insufficient storage performance caused bottlenecks in efficient production of WRF’s atmospheric simulations, negatively influencing accuracy of weather prediction. To improve the speed for WRF output and optimize weather prediction accuracy, the laboratory needed to substitute the existing out-of-date storage with a new high performance storage solution.

Challenges

• Out-of-date storage systems provided extremely poor performance for I/O intensive workloads of the WRF Model, which negatively influenced the speed of generating weather simulations and the accuracy of weather prediction.
• Large data amounts generated for the WRF Model required ample storage capacity.

Solution

The research laboratory adopted Infortrend’s ultra-high performance EonStor GS 4000U hybrid flash NVMe storage with support of 100GbE RDMA and PCIe 4.0. It is loaded with 24 x 15.36TB NVMe SSD, delivering up to 1100K IOPS and 24/12 GB/s in throughput for performance demanding WRF Model. The solution is capable to handle the multiple tasks and computational processes involved in running simulation and analysis in WRF Model. To store the data generated during these processes, it provides 368.6TB of total storage capacity. Thanks to its compact 2U 24-bay form factor, it easily fits into the rack space of the IT server room. Deploying GS 4000U increased the speed of I/O operations for large and complex WRF simulations, which contributed to more accurate weather forecasts, a better understanding of atmospheric processes, and contributed to the success of meteorological research. The laboratory can further scale-out and scale-up storage infrastructure for the expansion of performance and capacity when such a need arises.

EonStor GS NVMe Hybrid Flash Storage for the Laboratory’s WRF Model

Why Infortrend

• Ultra-high performance of the U.2 NVMe hybrid flash storage GS 4000U ideally suitable for HPC workloads of WRF Model
• Support of 100GbE RDMA
• High capacity to store large amounts of data generated for WRF simulation
• Future-proof storage expandability by scale-out and scale-up