Software energy consumption and efficiency can be determined through a combination of similar top-down and bottom-up methods that are commonly used in other areas of computer information technology (CIT) environmental assessment. The top-down method involves an assessment of the entire system, identifying what percentage of energy required by that system is consumed by your particular technology, and extrapolating the impact. The bottom-up method involves a specific case study that is then generalized.
Assessing the impact of software is complicated by the fact that there probably isn’t one most energy efficient configuration, but a variety of configurations that consume more or less power in different contexts. For example, does the speed of the connection favor different systems? Additionally, the ways that software interacts with specific hardware resources that can be dispatched or allocated via software is also an important factor. This could include running processes on multiple cores, running processes on the CPU vs the GPU, the impact of cooling systems and hardware temperature, and overall runtime, among many other possibilities. All of the variations can be tested, but it makes a simple determination of software energy consumption difficult. Additionally, most web applications do not exist in a void and it is also worth considering the impact of shared resources and browser settings, like cache-ing. Finally, user behaviour is a crucial factor in how these systems operate in real world conditions.
The study, Sustainable software products – Towards assessment criteria for resource and energy efficiency (Kern, Hilty, Guldner, Maksimov, Filler, Groger, Naumann, 2018) uses this combination approach to propose a software specific method for evaluating environmental impact. This paper contains a very thorough overview of existing criteria. Their method specifically focuses on the operation of the software and they note that well established LCA approaches can be applied to the hardware and a general “internet energy intensity” measurement in kWh/GB can be applied to data over a network when the specific network hardware is unknown. These researchers identify 3 primary criteria to consider – resource efficiency, potential hardware operating life, and user autonomy.
Their method for measuring energy resource efficiency dedicates one machine as the system under test (SUT) and the software is run on that machine. A power meter measures the energy consumption of the SUT. Another computer, the workload generator or test runner, sends commands to that machine. A third computer collects the data from both the SUT and power meter, if needed. For each application a baseline measurement is taken, as well as the software in idle mode (if applicable), and the software running under a variety of typical usage scenarios.
The article How does docker affect energy consumption? Evaluating workloads in and out of Docker containers (Santos, McLean, Solinas, and Hindle, 2018) applies a similar method to measuring the differences between a Docker server and a “bare metal” linux server. The researchers measured the instantaneous wall power drawn by the test system via a Watts Up Pro device. They opted to measure all wall power, because they were interested in power usage of the entire system (CPU, memory, cooling, etc.). They determined the energy usage (i.e. power over time) by sampling the Watts Up Pro data and attached a timestamp to estimate to energy demands of different software implementations. The researchers ran their tests on the server in three different configurations, WordPress, Redis, and PostgresSQL, plus a baseline setup. Of particular relevance to us is their WordPress test. They used Tsung 1.6.0 (http://tsung.erlang-projects.org/1/01/about/), an open source load testing tool, to perform HTTP load stress tests on the wordpress server. Their paper contains detailed descriptions of their methods and the python code they used is also available, though not directly applicable to our work. Some of the articles cited in this paper look specifically at the energy use of virtualization and containers, which may be relevant if we are able to get access to the IDM website server and it turns out that it is not running on its own piece of hardware, but on a virtualized machine operating on a shared piece of hardware.
Overall both papers provide a strong roadmap for determining power consumption of different server configurations. In our particular case, once we finalize our system boundaries, we can select features from the various web designers and researchers who have proposed more environmentally sustainable web design techniques and apply similar tests.