Sounding out OS noise
(page 3 of 4)
Noise inequality
Ratios can suggest tradeoffs. For each byte of collective communication an application uses, the system gets, for instance, 10 microseconds of noise.
Paying attention to such “performance versus usability tradeoffs,” Ferreira explains, is critical to designing and implementing large-scale operating systems. But many OS developers pursue complex OS modifications that ignore such tradeoffs and reduce noise only to degrade performance.
One notable instance had Cray reducing the Linux timer interrupt frequency to “well below normal,” Brightwell says. “But application performance was so poor that this kernel was essentially unusable.”
For Brightwell and his colleagues, successfully attacking the problem meant creating a new paradigm.
“Past work has sought to identify and limit the noise produced by operating systems,” says Karsten Schwan, who directs the Georgia Tech Center for Experimental Research in Computer Systems. “The paper by Brightwell, Ferreira and Bridges goes a step beyond by trying to understand what the effects are of different types of OS noise.”
Their key finding: Not all OS noise is created equal.
“Every type of noise has a distinct pattern characterized by a certain frequency and duration,” Ferreira explains. “Noise represents work that must be done by the OS. The work must be done every so often – its frequency – and takes time to accomplish – its duration.”
By studying how high-performance systems generate noise, the team found that certain applications absorb high-frequency, low-duration noise but amplify low-frequency, high-duration noise.
“We injected noise that simulated a certain type of OS work,” Ferreira says, to analyze how different frequencies and durations affect application performance. Under the old “one type fits all” paradigm, POP was considered a notorious amplifier. But Ferreira and his group discovered that POP amplifies only high-duration, low-frequency noise, not high-frequency, low-duration noise.
The CTH weapons-safety application was considered a highly tolerant absorber under the old paradigm. But the Sandia-UNM team discovered that CTH absorbs only high-frequency, low-duration noise and is highly sensitive to low-frequency, high-duration noise.
POP turned out to be the most noise-sensitive of the three applications. The researchers believe that was because, of the three programs, it spends the most time performing MPI-Allreduce, a noise-sensitive operation.
Steve Reinhardt, vice president of parallel performance research at Waltham, Mass.-based Interactive Supercomputing, says he ran CTH, POP, and SAGE as part of parallel system design studies at Cray Research and SGI. “Our intuition about what made performance good or bad and easy or hard is closely aligned with these new findings.”
