From Slots to Watts: High-Fidelity Power and Carbon Accounting per Experiment using HTCondor at the PIC Tier-1 Site
Jose Flix Molina at European HTCondor Workshop 2026
September 30, 2026
As scientific computing facilities face increasing pressure to quantify and reduce their environmental footprint, accurately attributing power consumption and carbon emissions to specific workloads becomes essential. This work presents a robust accounting methodology developed at the PIC WLCG Tier-1 site, leveraging historical HTCondor job logs to reconstruct real-time core utilization per experiment across the compute farm. By correlating slot occupation with out-of-band IPMI measurements from verified healthy nodes, we derived high-accuracy polynomial models that map active logical cores directly to server wattage for each CPU architecture. This parameterization establishes a reliable baseline that bypasses individual hardware sensor failures, further refined by empirical PDU-to-IPMI correction factors accounting for chassis design (multi-node shared versus single-node) and cooling infrastructure (air-cooled versus dielectric oil immersion).
Applying this model to historical batch data enables the precise reconstruction of continuous power draw (kW) and energy consumption distributions (Wh) for each supported scientific experiment. By integrating this compute power timeline with historical grid carbon intensity data from the open-source Wattnet platform, we quantify both the time-series and integrated $\text{CO}_2$ emissions generated by specific workloads. However, because scientific jobs inherently rely on reading and writing massive datasets, their environmental impact extends beyond CPU utilization. To achieve a fully holistic assessment, ongoing work expands these compute-centric metrics to include the energy overhead of the disk servers that store, touch, and serve data to the compute nodes during job execution. Combined with dynamic facility Power Usage Effectiveness (PUE) estimates and the embedded carbon footprint of the hardware lifecycle, this approach establishes a comprehensive framework for quantifying the true carbon cost of running data-intensive science tasks at the site.
