ORNL’s Exaflop Machine Frontier Keeps Top Spot, New Competitor Leonardo Breaks the Top10 List

Controller Master


The 60th edition of the TOP500 reveals that the Frontier system is still the only true exascale machine on the list.

With an HPL score of 1.102 EFlop/s, the Frontier machine at Oak Ridge National Laboratory (ORNL) did not improve upon the score it reached on the June 2022 list. That said, Frontier’s near-tripling of the HPL score received by second-place winner is still a major victory for computer science. On top of that, Frontier demonstrated a score of 7.94 EFlop/s on the HPL-MxP benchmark, which measures performance for mixed-precision calculation. Frontier is based on the HPE Cray EX235a architecture and it relies on AMD EPYC 64C 2 GHz processor. The system has 8,730,112 cores and a power efficiency rating of 52.23 gigaflops/watt. It also relies on gigabit ethernet for data transfer.

Frontier is the clear winner of the race to exascale, and it will require a lot of work and innovation to knock it from the top spot.

The Fugaku system at the Riken Center for Computational Science (R-CCS) in Kobe, Japan, previously held the top spot for two years in a row before being moved down by the Frontier machine. With an HPL score of 0.442 EFlop/s, Fugaku has retained its No. 2 spot from the previous list.

The LUMI system, which found its way to the No. 3 spot on the last list, has retained its spot. However, the system went through a major upgrade to keep it competitive. The upgrade doubled the machines size, which allowed it to achieve an HPL score of 0.309 EFlop/s.

The only new machine to grace the top of the list was the No. 4 Leonardo system at EuroHPC/CINECA in Bologna, Italy. The machine achieved an HPL score of 0.174 EFlop/s with 1,463,616 cores.

Here is a summary of the system in the Top 10:

  • Frontier is the No. 1 system in the TOP500. This HPE Cray EX system is the first US system with a performance exceeding one EFlop/s. It is installed at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, where it is operated for the Department of Energy (DOE). It currently has achieved 1.102 EFlop/s using 8,730,112 cores. The new HPE Cray EX architecture combines 3rd Gen AMD EPYC CPUs optimized for HPC and AI, with AMD Instinct 250X accelerators, and Slingshot-10 interconnect.
  • Fugaku, now the No. 2 system, is installed at the RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. It has 7,630,848 cores which allowed it to achieve an HPL benchmark score of 442 Pflop/s.
  • The upgraded LUMI system, another HPE Cray EX system installed at EuroHPC center at CSC in Finland is the No. 3 with a performance of 309.1 Pflop/s. The European High-Performance Computing Joint Undertaking (EuroHPC JU) is pooling European resources to develop top-of-the-range Exascale supercomputers for processing big data. One of the pan-European pre-Exascale supercomputers, LUMI, is located in CSC’s data center in Kajaani, Finland.
  • The new No. 4 system Leonardo is installed at a different EuroHPC site in CINECA, Italy. It is an Atos BullSequana XH2000 system with Xeon Platinum 8358 32C 2.6GHz as main processors, NVIDIA A100 SXM4 40 GB as accelerators, and Quad-rail NVIDIA HDR100 Infiniband as interconnect. It achieved a Linpack performance of 174.7 Pflop/s.
  • Summit, an IBM-built system at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, is now listed at the No. 5 spot worldwide with a performance of 148.8 Pflop/s on the HPL benchmark, which is used to rank the TOP500 list. Summit has 4,356 nodes, each one housing two POWER9 CPUs with 22 cores each and six NVIDIA Tesla V100 GPUs each with 80 streaming multiprocessors (SM). The nodes are linked together with a Mellanox dual-rail EDR InfiniBand network.
  • Sierra, a system at the Lawrence Livermore National Laboratory, CA, USA is at No. 6. Its architecture is very similar to the #5 system’s Summit. It is built with 4,320 nodes with two POWER9 CPUs and four NVIDIA Tesla V100 GPUs. Sierra achieved 94.6 Pflop/s.
  • Sunway TaihuLight, a system developed by China’s National Research Center of Parallel Computer Engineering & Technology (NRCPC) and installed at the National Supercomputing Center in Wuxi, which is in China’s Jiangsu province is listed at the No. 7 position with 93 Pflop/s.
  • Perlmutter at No. 8 is based on the HPE Cray “Shasta” platform and a heterogeneous system with AMD EPYC-based nodes and 1,536 NVIDIA A100 accelerated nodes. Perlmutter achieved 64.6 Pflop/s
  • Selene now at No. 9 is an NVIDIA DGX A100 SuperPOD installed inhouse at NVIDIA in the USA. The system is based on AMD EPYC processor with NVIDIA A100 for acceleration and a Mellanox HDR InfiniBand as network and achieved 63.4 Pflop/s.
  • Tianhe-2A (Milky Way-2A), a system developed by China’s National University of Defense Technology (NUDT) and deployed at the National Supercomputer Center in Guangzhou, China is now listed as the No. 10 system with 61.4 Pflop/s.

Other TOP500 Highlights
The data from this list shows that AMD processors are still the preferred choice for HPC systems. Frontier uses Gen AMD EPYC processors optimized for HPC and AI, as did the No. 3 LUMI system. That said, Xeon chips were also prevalent on the list. In fact, the new Leornardo system uses Xeon Platinum processors.

Once again, China and the United States took most of the entries on this list. While the US held strong at 126 machines on the TOP500, China dropped from 173 systems on the last list to 162 on this one. While these two countries make up nearly two-thirds of the machines on the list, it’s clear that other countries are working hard to bring about their own HPC innovations. In fact, taken as a whole continent, Europe accounted for 131 machines on this list as opposed to the 118 machines that appeared on the June 2022 TOP500.

The system interconnects shown on this list are very similar to previous ones, with small changes. Ethernet interconnects increased from 226 to 233 machines, and Infiniband dropped from 196 to 194 machines. Omnipath dropped to 36 machines from the previous 40, and the number of proprietary networks dropped from 6 systems to 4 on the new list.

GREEN500 Results

The system to take the top spot on the GREEN500 was the Henri system at the Flatiron Institute in the United States. Although the machine ranked at No. 405 on the TOP500 list, it had a successful showing in terms of its energy efficiency. The system has an efficiency score of 65.09 GFlops/Watts, 5,920 cores, and an HPL score of 2.038 PFlop/s.

Last list’s winner of the GREEN500 was the Frontier TDS machine, which has since moved down to the No. 2 spot. This system achieved an efficiency score of 62.68 GFlops/Watts, it has 120,832 total cores, and achieved an HPL score of 19.2 PFlop/s that earned it the No. 32 spot on the TOP500. Considering the Frontier TDS machine is basically just one rack identical to the ones used in the actual Frontier system, it would make sense that this machine is much more powerful than the No. 1 Henri system.

The No. 3 spot on the GREEN500 list saw an interesting development from the Adastra machine from France’s GENCI-CINES. On top of its high ranking in energy efficiency, this machine also captured the No. 11 spot on the TOP500. The Adastra system achieved an energy efficiency rating of 58.02 GFlops/Watts on top of its impressive HPL score of 46.1 PFlop/s.

However, in terms of both power and energy efficiency, the Frontier system once again shows some impressive results. Despite dropping from the No. 2 spot on the last GREEN500 to the No. 6 spot on this list, Frontier still provided incredible power for the amount of energy going in. The machine is able to produce 1.102 EFlop/s in HPL performance, all with an energy efficiency of 52.23 GFlops/Watts. Frontier is proof that the most powerful machines in the world don’t need to focus on performance at the expense of energy efficiency.

HPCG Results

The TOP500 list has incorporated the High-Performance Conjugate Gradient (HPCG) benchmark results, which provide an alternative metric for assessing supercomputer performance. This score is meant to complement the HPL measurement to give a fuller understanding of the machine.

The winner of this list, like the last list, is Fugaku with a score of 16.0 HPCG-petaflops. Unlike the last list, Frontier has submitted HPCG data and achieved an HPCG score of 14.054 HPCG-petaflops. This puts it at the No. 2 spot above LUMI, which scored 3.408 HPCG-petaflops.

HPL-MxP Results (formally HPL-AI)

The HPL-MxP benchmark seeks to highlight the convergence of HPC, and artificial intelligence (AI) workloads based on machine learning and deep learning by solving a system of linear equations using novel, mixed-precision algorithms that exploit modern hardware.

This year’s winner is the Frontier machine with its exciting 7.9 EFlop/s score on the HPL-AI benchmark test. Lumi was in second place with a score of 2.2 Eflop/s, followed by the Fugaku machine with a score of 2.0 Eflop/s