Computers as complex systems

Liz Bradley, professor at Colorado State University, and Joshua Garland, Omidyar Fellow at the Santa Fe Institute, have done fascinating work making the case that modern computers exhibit "complex and sometimes chaotic behavior" and therefore computer performance design and analysis would benefit significantly from the tools and methods used in nonlinear dynamics.

The idea that computers are nonlinear dynamical systems is "heresy" in the computer performance community, explains Bradley. "They think of them [computers] as random systems and the mathematics they use to model them. . . implicitly make the assumption of linearity and time invariance. And the system is neither linear nor time-invariant. What is inside them changes over time."

  Joshua Garland, at right. 

Joshua Garland, at right. 

What happens when the computer performance community uses linear models to predict performance and to direct engineering of new components to be used in a nonlinear system? Sometimes the results sometimes are unexpected hardware failures and expensive recalls. "Complexity can effectively overwhelm the predictive power of deterministic forecast models" used in predicting computer performance. 

"People in this community again are used to using these linear time invariant tools which are easy to use," Bradley explains in an interview with the Santa Fe Institute's Melanie Mitchell. "That’s a good reason to use them. They’re easy. But, if the system that you’re wanting to analyze is not amenable to that kind of analysis, it’s much more complicated, then we found ourselves in the position of coming into another community and saying, what you’re doing is wrong, which is never welcome. . . . Moreover the math we’re offering you is very hard to learn, and doesn’t always work."


  Liz Bradley

Liz Bradley

Bradley and Garland's paper, "On the importance of nonlinear modeling in computer performance prediction," is an exploration of the "disconnect" between the nature of a computer as a complex system and the commonly accepted models for predicting related future performance. Bradley and Garland also assess "when linear models are adequate for predicting computer performance and when they are not."

The paper's abstracts states an important distinction: 

Computers are deterministic dynamical systems. Among other things, that implies that one should be able to use deterministic forecast rules to predict their behavior. That statement is sometimes—but not always true. The memory and processor loads of some simple programs are easy to predict, for example, but those of more-complex programs like compilers are not.

The importance of performance prediction in engineering is highlighted in this excerpt from "On the importance of nonlinear modeling in computer performance prediction."

Accurate prediction is important in any number of applications. In a modern multi-core computer, for instance, an accurate forecast of processor load could be used by the operating system to balance the workload across the cores.

The traditional models used in the computer systems community use linear, time-invariant (and often stochastic) methods, e.g., autoregressive moving average (ARMA), multiple linear regression, etc. While these models are widely accepted—and for the most part easy to construct—they cannot capture the nonlinear interactions that have recently been shown to play critical roles in a computer’s performance. As computers become more and more complex, these interactions are beginning to cause problems—e.g., hardware design “improvements” that do not work as expected. Awareness about this issue is growing in the computer systems community, but the modeling strategies used in that field have not yet caught up with those concerns. . . .

The tools and methods of nonlinear dynamics "can get at the geometry and topology of a computer's performance patterns" and are "a much better way to study, understand, and (ultimately) design these critical systems" Bradley writes.



Computer systems are dynamical systems (2009) (Todd Mytkowicz, Amer Diwan, and Elizabeth Bradley) in Chaos: An Interdisciplinary Journal of Nonlinear Science 

Chaos in computer performance (2006) (Hugues Berry, Daniel Gracia Pérez and Olivier Temam) in Chaos: An Interdisciplinary Journal of Nonlinear Science 

Nonlinear Dynamics: Mathematical and Computational Approaches - Liz Bradley's MOOC at the Santa Fe Institute's Complexity Explorer. 

Predicting Computer Performance Dynamics (Joshua Garland, Elizabeth Bradley) in Advances in Intelligent Data Analysis X