Tags:

Extending the ARM of ATLAS computing

16 July 2025 | By

With data from the ongoing LHC Run 3 continuing to mount, and the High-Luminosity LHC (HL-LHC) expected to deliver ten times more, the ATLAS experiment is being pushed to its computing limits. Traditional central processing units (CPUs) based on the widely used x86_64 architecture – common in servers and desktop computers – are struggling to meet these growing demands in the resource projections. To keep pace, the ATLAS Collaboration is looking beyond traditional computing solutions to meet its growing needs.

Technology,Computing,ATLAS — Figure 1: The HepScore23 (HS23) benchmark score (a proxy for throughput) per Watt as a function of the processor frequency for seven CPU models. The first two listed models are ARM CPUs. (Image: ATLAS Collaboration/CERN)

One promising alternative is ARM CPU architecture, which offers competitive performance with significantly lower power consumption. Predominantly used in mobile devices like smartphones and tablets, ARM chips are increasingly being adopted in cloud data centres. As of 2024, ARM CPUs held around 15% of the commercial cloud computing market – a number expected to rise significantly, driven by the boom of artificial intelligence where these CPUs are often used as host chips.

In 2022 and 2023, the ATLAS Collaboration successfully ported its entire software ecosystem, consisting of several million lines of code in C++ and Python, to run on ARM CPUs. As ATLAS software runs on the Linux operating system, already available for ARM CPUs, the porting effort primarily focused on adapting ATLAS-specific and high-energy-physics-specific code. For some packages this entailed updating the individual software build instructions and configuration, as well as applying small changes to how floating point exceptions are handled. The porting effort was accomplished with the support of the CERN EP/SFT group. All major ATLAS workflows for offline and online data processing, together with all external software packages, can now be used on ARM CPUs. Enabling ATLAS code on different platforms was found to make it more robust and numerically stable.

All major ATLAS workflows for offline and online data processing can now be used on ARM CPU architecture – which offers competitive performance and significantly lower power consumption.

After integrating ARM support into the automated software build system, ATLAS teams conducted a large physics validation campaign. This series of tests ensured that results obtained from ARM-CPU-processed simulation samples matched those from traditionally-processed x86_64 samples. During this validation campaign, access to ARM CPUs from the Worldwide LHC Computing Grid (WLCG) resources was scarce. Thus, the ATLAS computing grid workflow management system was enabled to run on cloud computing resources from Amazon and Google, using several hundred ARM CPUs for a limited period of time.

ATLAS production workflows have also been integrated into HepScore23, a benchmarking suite used to measure the processing power of WLCG resources. This allowed for direct comparisons between x86_64 and ARM processors, confirming that ARM CPUs offer comparable processing performance while consuming less energy – an important consideration, particularly in view of the HL-LHC’s demanding data processing requirements. A performance comparison of several ARM and x86_64 CPUs is shown in Figure 1 where the first two listed models are ARM CPUs.

ATLAS was the first experiment at the LHC to accept pledged WLCG computing resources based on ARM CPUs. As of now, several WLCG sites are each delivering between a few hundred and a thousand ARM CPUs each, as part of their contributions to ATLAS grid computing. ATLAS routinely processes simulation and reconstruction workflows on approximately 10,000 ARM CPU cores running concurrently (see Figure 2). While this represents a small fraction of the approximately 500,000 CPU cores routinely running ATLAS grid jobs, it’s an excellent start towards more energy efficient processing. Critically, it also positions ATLAS to tap into future large-scale computing facilities where ARM CPUs are expected to play a central role.