Software with the fastest execution time isn’t necessarily the greenest.
Leland Teschler • Executive Editor
There once was an assumption among computer programmers that a faster program is also an energy efficient program. The idea transmuted over to software programming languages—surely a faster language was a greener one, the logic went.
That all changed in 2017 thanks to a group of researchers in Portugal who use what’s called the Computer Language Benchmarks Game (CLBG) as a way of ranking software languages in terms of energy efficiency. Conceived by programmer Doug Bagley in the early 2000s, the CLBG lays out a way of running, testing and comparing software by solving a set of well-known, diverse programming problems. Originally CLBG was devised to analyze runtime performance, but the Portuguese researchers realized it could also be used to study the energy efficiency of software.
Their work caught the imagination of people in the computer industry. The original 2017 paper received a lot of attention and comment, so much so that the researchers decided to update the results to reflect the energy efficiency of languages that have become more widely used since the original study took place.
The CLBG covers 13 benchmark problems. They include tasks such as switching from thread to thread while passing a token, a search for solutions to shape a packing puzzle, symmetrical thread rendezvous requests, operations on binary trees, and generating what’s called a Mandelbrot-set portable bitmap file.
The Portuguese researchers gathered information about energy consumption, execution time and peak memory usage for 27 languages. For measuring the energy consumption, researchers used Intel’s Running Average Power Limit (RAPL) tool, a feature of recent Intel processors that provides the energy consumption of the processor. Each benchmark executed 10 times, done to reduce the impact of cold starts and cache effects, and to analyze measurement consistency. Ditto for memory use.
As you might expect, compiled languages generally turned out to be the fastest and most energy efficient. In the original test, compiled languages consumed 120 J on average while executing the test cases while interpreted languages hit 2,365 J. The same tendencies emerged with respect to execution time: compiled languages required 5,103 msec on average to run the benchmarks compared to 87,614 msec for interpreted languages.
Pareto optimal sets for different sets of objectives, from the orginal 2017 study. Click image to enlarge.
The researchers say CPU-based energy consumption always represents the majority of the energy consumed. On average, for the compiled languages, this value represents 88.94% of the energy consumed, the remaining portion assigned to DRAM. The implication is that optimizing a program to reduce the CPU-based energy consumption will also decrease the energy associated with DRAM operations.
However, the researchers note that there is more variability in energy consumption figures for interpreted languages (min of 81.57%, max of 92.90%) when compared to compiled (min of 85.27%, max of 91.75%) or virtual machine languages (min of 86.10%, max of 92.43%).
With regard to whether or not the faster language is always the most energy efficient?, it is interesting to note that the top four most energy efficient languages keep their rank when sorted by execution time and with very small differences in both energy and time values. Additionally, it is common knowledge that C,C++, and Rust are known to be heavily optimized and efficient for execution performance.
Researchers also comment on the effect of memory usage on the memory’s energy consumption. Memory use can be categorized into continuous memory usage and peak memory usage. Researchers concluded there doesn’t seem to be a consistent correlation between the DRAM energy consumption and the peak memory use.
All things considered
Language assessments lead to the question of whether there is a “best” programming language considering energy consumption, execution time, and peak memory use, or if not, which are the best in each given scenario. To compare the languages using more than one quality at a time researchers sorted them using a multi-objective optimization algorithm known as Pareto optimization. Energy, time, and memory were the optimization objectives. The solution, called the Pareto optimal solution, in some cases turned out to be more than one programming language. Also, researchers didn’t rerun their Pareto optimization for the updated results obtained last year.
Researchers concluded that developers only concerned with execution time and energy consumption can almost always choose a best language for the job. Unfortunately, if memory is also a concern, the choice of language is no longer automatic. Developers will have to decide which are the most important qualities in the scenario at hand.
The original study found that C was the most energy efficient language overall with Rust a close second. The updated work now puts Rust in a tie with C for first place. More obscure languages make a stronger showing in the updated results. Julia is designed for numerical analysis and included built-in primitives for parallel computing. OCaml has elements of functional languages and object-oriented languages. Lisp is a functional programming languages designed for manipulating data strings. Haskell is also a functional programming language said to excel at handling high-load concurrent applications such as web backends. Chapel is a parallel programming language developed by supercomputer maker Cray. GO is a compiled language developed by Google which is syntactically similar to C but with memory safety, garbage collection, and concurrency. DW
Filed Under: ENGINEERING SOFTWARE, MOTION CONTROL