Abstract
In this chapter, I review the main methods and techniques of complex systems science. As a first step, I distinguish among the broad patterns which recur across complex systems, the topics complex systems science commonly studies, the tools employed, and the foundational science of complex systems. The focus of this chapter is overwhelmingly on the third heading, that of tools. These in turn divide, roughly, into tools for analyzing data, tools for constructing and evaluating models, and tools for measuring complexity. I discuss the principles of statistical learning and model selection; time series analysis; cellular automata; agent-based models; the evaluation of complex-systems models; information theory; and ways of measuring complexity. Throughout, I give only rough outlines of techniques, so that readers, confronted with new problems, will have a sense of which ones might be suitable, and which ones definitely are not.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
12. References
Gamma E, Helm R, Johnson R, Vlissides J. 1995. Design patterns: elements of reusable object-oriented software. Addison-Wesley J, Reading, MA 1995.
Anderson RW. 1988. Random-walk learning: a neurobiological correlate to trial-and-error. In Progress in neural networks, pp. 221–244. Ed. OM Omidvar, J Dayhoff. Academic Press, Boston.
Mueller S, Marchettod J, Airaghi S, Koumoutsakos P. 2002. Optimization based on bacterial chemotaxis. IEEE Trans Evolut Comput 6:16–29.
Simon HA. 1962. The architecture of complexity: Hierarchic systems. Proc Am Philos Soc 106:467–482 (reprinted as chap. 8 of [221]).
Simon HA. 1955. On a class of skew distribution functions. Biometrika 42:425–440.
Turing A. 1952. The chemical basis of morphogenesis. Philos Trans Roy Soc B 237:37–72.
Strong SP, Freedman B, Bialek W, Koberle R. 1998. Adaptation and optimal chemotactic strategy for E. coli. Phys Rev E 57:4604–4617 (http://arxiv.org/abs/adap-org/9706001).
Alon U, Surette MG, Barkai N, Leibler S. Robustness in bacterial chemotaxis. Nature 397:168–171.
Yi T-M, Huang Y, Simon MI, Doyle J. 2000. Robust perfect adaptation in bacterial chemotaxis through integral feedback control. Proc Natl Acad Sci USA 97:4649–4653.
Badii R, Politi A. 1997. Complexity: hierarchical structures and scaling in physics. Cambridge UP, Cambridge.
Fontana W, Buss, LW. 1994. “Arrival of the fittest”: towards a theory of biological organization. Bull Math Biol 56:1–64 (http://www.santafe.edu/~walter/Papers/arrival.US.ps.gz).
Zurek WH, ed. 1990. Complexity, entropy, and the physics of information. Addison-Wesley, Reading, MA.
Frisch U. 1995. Turbulence: the legacy of A.N. Kolmogorov. Cambridge, Cambridge UP
Cross MC, Hohenberg P. 1993. Pattern formation out of equilibrium. Rev Mod Phys 65:851–1112.
Ball P. 1999. The self-made tapestry: pattern formation in nature. Oxford UP, Oxford.
Holland JH. 1992. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. 2nd ed. MIT Press, Cambridge (1st ed. 1975, U Michigan P, Ann Arbor).
Mitchell M. 1996. An introduction to genetic algorithms. MIT Press, Cambridge.
Gintis H. 2000. Game theory evolving: a problem-centered introduction to modeling strategic interaction. Princeton UP, Princeton.
Hofbauer J, Sigmund K. 1988. The theory of evolution and dynamical systems: mathematical aspects of selection. Cambridge UP, Cambridge.
Fischer KH, Hertz JA. 1988. Spin glasses. Cambridge Studies in Magnetism. Cambridge UP, Cambridge.
Stein DL. 2003. Spin glasses: still complex after all these years? In Quantum decoherence and entropy in complex systems. Ed. T Elze. Springer, Berlin.
Vapnik VN. 2000. The nature of statistical learning theory, 2nd ed. Springer, Berlin.
Akaike H. 1973. Information theory and an extension of the maximum likelihood principle. In Proceedings of the second international symposium on information theory, pp. 267–281. Ed. BN Petrov, F Caski. Akademiai Kiado, Budapest (repr. in [287, pp. 199–213].
Akaike H. 1977. On entropy maximization principle. In Applications of statistics, pp. 27–41. Ed. PR Krishnaiah. North-Amsterdam, Holland
Schwarz G. 1978. Estimating the dimension of a model. Ann Stat 6:461–464.
van de Geer S. 2000. Empirical processes in M-estimation. Cambridge UP, Cambridge.
Biggers ED. 1928. Behind that curtain. Grosset and Dunlap, New York.
Valiant LF. 1984. A theory of the learnable. Commun Assoc Comput Machinery 27:1134–1142.
Shao X, Cherkassky V, Li W. 2000. Measuring the VC-dimension using optimized experimental design. Neural Comput 12:1969–1986.
Meir R. 2000. Nonparametric time series prediction through adaptive model selection. Machine Learning 39:5–34.
Hastie T, Tibshirani R, Friedman J. 2001. The elements of statistical learning: data mining, inference, and prediction. Springer, New York.
Ripley BD. 1996. Pattern recognition and neural networks. Cambridge UP, Cambridge.
Wahba G. 1990. Spline models for observational data. Society for Industrial and Applied Mathematics, Philadelphia.
Anthony M, Bartlett PL. 1999. Neural network learning: theoretical foundations. Cambridge UP, Cambridge.
Zapranis A, Refenes A-P. 1999. Principles of neural model identification, selection and adequacy: with applications to financial econometrics. Springer, London.
Engel A, Van den Broeck C. 2001. Statistical mechanics of learning. Cambridge UP, Cambridge.
Breiman L, Friedman J, Olshen R, Stone C. 1984. Classification and regression trees. Wadsworth, Belmont, CA.
Gigerenzer G, Todd PM, ABC Research Group. 1999. Simple heuristics that make us smart. Oxford UP, Oxford.
Herbrich R. 2002. Learning kernel classifiers: theory and algorithms. MIT Press, Cambridge.
Cristianini N, Shawe-Taylor J. 2000. An introduction to support vector machines: and other kernel-based learning methods. Cambridge UP, Cambridge.
Pearl J. 2000. Causality: models, reasoning, and inference. Cambridge UP, Cambridge.
Shafer G. 1996. The art of causal conjecture. MIT Press, Cambridge.
Spirtes P, Glymour C, Scheines R. 2001. Causation, prediction, and search, 2nd ed. MIT Press, Cambridge.
Dayan P, Hinton GE, Neal RM, Zemel, RS. 1995. The Helmholtz machine. Neural Comput 7:889–904 (http://www.cs.utoronto.ca/~hinton/absps/helmholtz.htm).
Domingos P. 1999. The role of Occam’s Razor in knowledge discovery. Data Mining and Knowledge Discovery 3:409–425 (http://www.cs.washington.edu/home/pedrod/dmkd99.pz.gz).
Klein JL. 1997. Statistical visions in time: a history of time series analysis, 1662–1938. Cambridge UP, Cambridge.
Dirac PAM. 1935. Principles of quantum mechanics. Clarendon Press, Oxford.
Knight FB. 1975. A predictive view of continuous time processes. Ann Probability 3:573–596.
Knight FB. 1992. Foundations of the prediction process. Oxford Studies in Probability, Vol. 1. Clarendon Press, Oxford.
Mayo DG. 1996. Error and the growth of experimental knowledge. U Chicago P, Chicago.
Gray RM. 1988. Probability, random processes, and ergodic properties. Springer, New York (http://ee-www.stanford.edu/~gray/arp.html)
Basawa IV, Scott DJ. 1983. Asymptotic optimal inference for non-ergodic models. Springer, Berlin.
West BJ, Deering B. 1995. The lure of modern science: fractal thinking. World Scientific, Singapore.
Press WH, Teukolsky SA, Vetterling WT, and Flannery BP. 1992. Numerical recipes in c: the art of scientific computing, 2nd ed. Cambridge UP, Cambridge.
Shumway RH, Stoffer DS. 2000. Time series analysis and its applications. Springer Texts in Statistics. Springer, New York.
Wiener N. 1961. Cybernetics: or, control and communication in the animal and the machine, 2nd ed. MIT Press, Cambridge (1st ed. 1948, Wiley, New York).
Hubbard BB. 1996. The world according to wavelets: the story of a mathematical technique in the making. A.K. Peters, Wellesley.
Honerkamp J. 1994. Stochastic dynamical systems: concepts, numerical methods, data analysis. Transl. Katja Lindenberg. VCH, New York.
Box GEP, Jenkins GM. 1970. Time series analysis, forecasting, and control. Holden-Day, Oakland, CA.
Durbin J, Koopman SJ. 2001. Time series analysis by state space methods. Oxford UP, Oxford.
Eyink GL. 1998. Linear stochastic models of nonlinear dynamical systems. Phys Rev E 58:6975–6991.
Barndorff-Nielsen OE, Jensen JL, Sorensen M. 1990. Parametric modelling of turbulence. Philos Trans Roy Soc A 332:439–455.
Eyink GL, Alexander FJ. 1998. Predictive turbulence modeling by variational closure. J Stat Phys 91:221–283.
Beran J. 1994. Statistics for long-memory processes. Chapman and Hall, New York.
Embrechts P, Maejima M. 2002. Selfsimilar processes. Princeton UP, Princeton.
Bosq D. 1998. Nonparametric statistics for stochastic processes: estimation and prediction, 2nd ed. Springer, Berlin.
Algoet P. 1992. Universal schemes for prediction, gambling and portfolio selection. Ann Probability 20:901–941. See also an important Correction, Ann Probability 23:474–478, 1995.
Takens F. 1981. Detecting strange attractors in fluid turbulence. In Symposium on dynamical systems and turbulence, pp. 366–381. Ed. DA Rand and LS Young. Springer, Berlin.
Kantz H, Schreiber T. 1997. Nonlinear time series analysis. Cambridge UP, Cambridge.
Judd K, Mees A. 1998. Embedding as a modeling problem. Physica D 120:273–286.
Small M, Tse CK. 2004. Optimal embedding parameters: a modelling paradigm. Physica D 194:283–296 (http://arxiv.org/abs/physics/0308114).
Kennel MB, Brown R, Abarbanel HDI. 1992. Determining minimum embedding dimension using a geometric construction. Phys Rev A 45:3403–3411.
Sprott JC. 2003. Chaos and time-series analysis. Oxford UP, Oxford.
Smith LA. 1988. Intrinsic limits on dimension calculations. Phys Lett A 133:283–288.
Fraser AM, Swinney HL. 1986. Independent coordinates for strange attractors from mutual information. Phys Rev A 33:1134–1140.
Cellucci CJ, Albano AM, Rapp PE. 2003. Comparative study of embedding methods. Phys Rev E 67:162–210.
Letellier C, Aguirre LA. 2002. Investigating nonlinear dynamics from time series: the influence of symmetries and the choice of observables. Chaos 12:549–558.
Wiener N. 1949. Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications. Technology Press of MIT, Cambridge.
Kolmogorov AN. 1941. Interpolation und extrapolation von stationären zufälligen folgen. Bull Acad Sci USSR Math 3:3–14 (in Russian with German summary).
Kalman RE. 1960. A new approach to linear filtering and prediction problems. ASME Trans J Basic Eng 82D:35–50.
Kalman RE, Bucy RS. 1961. New results in linear filtering and prediction. ASME Trans J Basic Eng 83D:95–108.
Bucy RS. 1994. Lectures on discrete time filtering. Springer, Berlin.
Stratonovich RL. 1968. Conditional markov processes and their application to the theory of optimal control, 2nd ed., revised by the author. Transl. RN and NB McDonough, with a preface by R Bellman. Elsevier, New York. (1st ed. 1966, Uslovnyye markovskiye protessy i ikh primeneiye k teorri optimal’nogo upravleniya, Moscow UP, Moscow).
Kushner HJ. 1967. Dynamical equations for optimal nonlinear filtering. J Differ Eqs 3:179–190.
Lipster RS, Shiryaev AN. 2001. Statistics of random processes, 2 vols., 2nd ed. Transl. AB Aries. Springer, Berlin (first published 1974, Statistika sluchainykh protessov, Nauka, Moscow).
Tanizaki H. 1996. Nonlinear filters: estimation and applications, 2nd ed. Springer, Berlin.
Darling RWR. 1998. Geometrically intrinsic nonlinear recursive filters I: algorithms. Technical Report 494, Statistics Department, University of California-Berkeley (http://www.stat.berkeley.edu/tech-reports/494.abstract).
Darling RWR. 1998. Geometrically intrinsic nonlinear recursive filters ii: foundations. Technical Report 512, Statistics Department, University of California-Berkeley (http://www.stat.berkeley.edu/tech-reports/512.abstract).
Eyink GL. 2000. A variational formulation of optimal nonlinear estimation. Methodology and Computing in Applied Probability. submitted (http://arxiv.org/abs/physics/0011049).
Ahmed NU. 1998. Linear and nonlinear filtering for scientists and engineers. World Scientific, Singapore.
Chomsky N. 1956. Three models for the description of language. IRE Trans Inf Theory 2:113–124.
Charniak E. 1993. Statistical language learning. MIT Press, Cambridge.
Manning CD, Schütze H. 1999. Foundations of statistical natural language processing. MIT Press, Cambridge.
Elliott RJ, Aggoun L, Moore JB. 1995. Hidden markov models: estimation and control. Applications of Mathematics: Stochastic Modelling and Applied Probability, Vol. 29. Springer, New York.
Baldi P, Brunak S. 2001. Bioinformatics: the machine learning approach, 2nd ed. MIT Press, Cambridge.
Neal RM, Hinton GE. 1998. A view of the EM algorithm that justifies incremental, sparse, and other variants. In Learning in Graphical Models, pp. 355–368. Ed. MI Jordan, Kluwer Academic, Dordrecht.
Rissanen J. 1983. A universal data compression system. IEEE Trans Inf Theory 29:656–664.
Willems F, Shtarkov Y, Tjalkens T. 1995. The context-tree weighting method: basic properties. IEEE Trans Inf Theory 41:653–664.
Ron D, Singer Y, Tishby N. 1996. The power of amnesia: learning probabilistic automata with variable memory length. Machine Learning 25:117–149.
Bühlmann P, Wyner AJ. 1999. Variable length Markov chains. Ann Stat 27:480–513 (http://www.stat.berkeley.edu/tech-reports/479.abstract1).
Kennel MB, Mees AI. 2002. Context-tree modeling of observed symbolic dynamics. Phys Rev E 66:056209.
Crutchfield JP, Young K. 1989. Inferring statistical complexity. Phys Rev Lett 63:105–108.
Jaeger H. 2000. Observable operator models for discrete stochastic time series. Neural Comput 12:1371–1398 (http://www.faculty.iu-bremen.de/hjaeger/pubs/oom/neco00.pdf).
Littman ML, Sutton RS, Singh S. 2002. Predictive representations of state. In Advances in neural information processing, pp. 1555–1561. Ed. TG Dietterich, S Becker, Z Ghahramani, Systems 14. MIT Press, Cambridge (http://www.eecs.umich.edu/~baveja/Papers/psr.pdf).
Shalizi CR, Shalizi KL. 2004. Blind construction of optimal nonlinear recursive predictors for discrete sequences. In Uncertainty in artificial intelligence: proceedings of the twentieth conference, pp. 504–511. Ed. M Chickering, J Halpern. AUAI Press, Arlington, VA (http://arxiv.org/abs/cs.LG/0406011).
Salmon WC. 1971. Statistical explanation and statistical relevance. With contributions by RC Jeffrey and JG Greeno. U Pittsburgh P.
Salmon WC. 1984. Scientific explanation and the causal structure of the world. Princeton UP, Princeton.
Singh S, Littman, ML, Jong NK, Pardoe D, Stone P. 2003. Learning predictive state representations. In Proceedings of the twentieth international conference on machine learning (ICML-2003), pp. 712–719. Ed. T Fawcett, N Mishra. AAAI Press, New York (http://www.eecs.umich.edu/~baveja/Papers/ICMLfinal.ps.gz).
Upper DR. 1997. Theory and algorithms for hidden markov models and generalized hidden markov models. PhD thesis, University of California, Berkeley (http://www.santafe.edu/projects/CompMech/ or papers/TAHMMGHMM.html).
Dupont P, Denis F, Esposito Y. 2004. Links between probabilistic automata and hidden Markov models: probability distributions, learning models and induction algorithms. Pattern Recognit Forthcoming (http://www.info.ucl.ac.be/people/pdupont/pdupont/postscript/Links_PA_HMM_preprint.ps.gz)
Jaeger H. 1999. Characterizing distributions of stochastic processes by linear operators. Technical Report 62, German National Center for Information Technology (http://www.faculty.iu-bremen.de/hjaeger/pubs/oom_distributionsTechRep.pdf).
Jaeger H. 2000. Modeling and learning continuous-valued stochastic processes with OOMs. Technical Report 102, German National Center for Information Technology (http://www.faculty.iu-bremen.de/hjaeger/pubs/jaeger.00.tr.contoom.pdf).
Crutchfield JP. 1992. Unreconstructible at any radius. Phys Lett A 171:52–60.
Bollt EM, Stanford T, Lai Y-C, Zyczkowski K. 2000. Validity of threshold-crossing analysis of symbolic dynamics from chaotic time series. Phys Rev Lett 85:3524–3527.
Bollt EM, Stanford T, Lai Y-C, Zyczkowski K. 2001. What symbolic dynamics do we get with a misplaced partition? On the validity of threshold crossing analysis of chaotic time-series. Physica D 154:259–286.
Kitchens BP. 1998. Symbolic dynamics: one-sided, two-sided and countable state markov shifts. Springer, Berlin.
Kennel MB, Buhl M. 2003. Estimating good discrete partitions from observed data: symbolic false nearest neighbors. Phys Rev Lett 91:084102 (http://arxiv.org/abs/nlin.CD/0304054).
Hirata Y, Judd K, Kilminster D. 2004. Estimating a generating partition from observed time series: Symbolic shadowing. Phys Rev E 70:016215.
Crutchfield JP, Packard NH. 1983. Symbolic dynamics of noisy chaos. Physica D 7:201–223.
Moore C. 1997. Majority-vote cellular automata, Ising dynamics, and P-completeness. J Stat Phys 88:795–805 (http://arxiv.org/abs/cond-mat/9701118).
Moore C, Nordahl MG. 1997. Lattice gas prediction is P-complete. Electronic preprint (http://arxiv.org/abs/nlin.CG/9704001).
Hardy J, Pomeau Y, de Pazzis O. 1976. Molecular dynamics of a classical lattice gas: transport properties and time correlation functions. Phys Rev A 13:1949–1960,.
Frisch U, Hasslacher B, Pomeau Y. 1986. Lattice-gas automata for the Navier-Stokes equation. Phys Rev Lett 56:1505–1508.
Rothman DH, and Zaleski S. 1997. Lattice-gas cellular automata: simple models of complex hydrodynamics. Cambridge UP, Cambridge.
Fisch R, Gravner J, Griffeath D. 1991. Threshold-range scaling of excitable cellular automata. Stat Comput 1:23–39 (http://psoup.math.wisc.edu/papers/tr.zip).
Nilsson M, Rasmussen S, Mayer B, Whitten D. 2003. Constructive molecular dynamics (MD) lattice gases: 3-D molecular self-assembly. In New constructions in cellular automata, pp. 275–290. Ed. D Griffeath, C Moore. Oxford UP, Oxford.
Nilsson M, Rasmussen S. 2003. Cellular automata for simulating molecular self-assembly. Discr Math Theor Comput Sci AB (DMCS):31–42 (http://dmtcs.loria.fr/proceedings/html/dmAB0103.abs.html).
Bartlett MS. 1955. An introduction to stochastic processes, with special reference to methods and applications. Cambridge UP, Cambridge.
Jacquez JA, Koopman JS, Simon CP, Longini IM. 1994. The role of the primary infection in epidemics of HIV-infection in gay cohorts. J Acq Immune Def Synd Hum Retrovirol 7:1169–1184.
Koopman J, Jacquez J, Simon C, Foxman B, Pollock S, Barth-Jones D, Adams A, Welch G, Lange K. 1997. The role of primary HIV infection in the spread of HIV through populations. J AIDS 14:249–258.
Budd T. 2000. Understanding object-oriented programming with Java, 2nd ed. Addison-Wesley, Reading, MA.
Resnick M. 1994. Turtles, termites and traffic jams: explorations in massively parallel microworlds. MIT Press, Cambridge.
Brown JS, Duguid P. 2000. The social life of information. Harvard Business School Press P, Boston.
Bonabeau E, Dorigo M, Theraulaz G. 1999. Swarm intelligence: from natural to artificial systems. Oxford UP, Oxford.
Lerman K. Design and mathematical analysis of agent-based systems. E-print, Information Sciences Institute, University of Southern California, 2000 (http://www.isi.edu/~lerman/papers/fmw00_abstract.html).
Ossowski S. 2000. Co-ordination in artificial agent societies: social structure and its implications for autonomous problem-solving agents. Springer, Berlin.
Wooldridge M. 2000. Reasoning about rational agents. MIT Press, Cambridge.
Jonker CM, Snoep JL, Treur J, Westerhoff HV, Wijngaards WAC. 2002. Putting intentions into cell biochemistry: An artificial intelligence perspective. J Theor Biol 214:105–134.
Chaikin PM, Lubensky TC. 1995. Principles of condensed matter physics. Cambridge UP, Cambridge.
Hammersley JM, Handscomb DC. 1964. Monte Carlo methods. Chapman and Hall, London.
Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. 1953. Equations of state calculations by fast computing machines. J Chem Phys 21:1087–1092.
Brémaud P. 1999. Markov chains: gibbs fields, monte carlo simulation, and queues. Springer, Berlin.
Beckerman M. 1997. Adaptive cooperative systems. Wiley, New York.
Jordan MI, ed. 1998. Learning in graphical models. Kluwer Academic, Dordrecht.
Young HP. 1998. Individual strategy and social structure: an evolutionary theory of institutions. Princeton UP, Princeton.
Sutton J. 1998. Technology and market structure: theory and history. MIT Press, Cambridge.
Epstein IR, Pojman JA. 1998. An introduction to nonlinear chemical dynamics: oscillations, waves, patterns, and chaos. Oxford UP, Oxford.
Winfree AT. 1987. When time breaks down: the three-dimensional dynamics of electrochemical waves and cardiac arrhythmias. Princeton UP, Princeton.
Varela FJ, Maturana HR, Uribe R. 1974. Autopoiesis: the organization of living systems, its characterization and a model. Biosystems 5:187–196.
Luhmann N. 1984/1995. Social systems. Transl. J Bednarz Jr, with D Baecker, and foreword by EM Knodt. Stanford UP, Stanford. Originally published as Soziale systeme: grundriss einer allgemeinen theorie. Suhrkamp-Verlag, Frankfurt am Main.
McMullin B. 1997. The case of the independent test. Santa Fe Inst Bull 12(2).
McMullin B, Varela FJ. 1997. Rediscovering computational autopoesis. Technical Report no. 97-02-012, Santa Fe Institute (http://www.santafe.edu/research/publications/wpabstract/199702012).
Mitchell M, Hraber PT, Crutchfield JP. 1993. Revisiting the edge of chaos: evolving cellular automata to perform computations. Complex Syst 7:89–130. (http://www.cse.ogi.edu/~mm/revedge.pdf).
Schuessler AA. 1999. Ecological inference. Proc Natl Acad Sci USA 96:10578–10581.
Atkinson AC, Donev AN. 1992. Optimum experimental designs. Clarendon Press, Oxford.
Borowiak DS. 1989. Model discrimination for nonlinear regression models. Marcel Dekker, New York.
Newman MEJ, Palmer RG. 2003. Modeling extinction. Oxford UP, Oxford (http://arxiv.org/abs/adap-org/9908002).
Cover TM, Thomas JA. 1991. Elements of information theory. Wiley, New York.
Kullback S. 1968. Information theory and statistics, 2nd ed. Dover, New York.
Kulhavy R. 1996. Recursive nonlinear estimation: a geometric approach. Springer, Berlin.
Khinchin AI. 1957. Mathematical foundations of information theory. Transl. RA Silverman, MD Friedman. Dover, New York. Originally published as two Russian articles in Uspekhi Matematicheskikh Nauk (7:3–20, 1953; 9:17–75, 1956).
Ruelle D. 1989. Chaotic evolution and strange attractors: the statistical analysis of time series for deterministic nonlinear systems. Cambridge UP, Cambridge. Notes prepared by Stefano Isola.
Beck C, Schlögl F. 1993. Thermodynamics of chaotic systems: an introduction. Cambridge UP, Cambridge.
Victor JD. 2000. Asymptotic bias in information estimates and the exponential (Bell) polynomials. Neural Comput 12:2797–2804.
Kraskov A, Stögbauer H, Grassberger P. 2003. Estimating mutual information. Phys Rev E. Submitted (http://arxiv.org/abs/cond-mat/0305641).
Kontoyiannis I, Algoet M. Sukhov YuM, and Wyner AJ. 1998. Nonparametric entropy estimation for stationary processes and random fields, with applications to english text. IEEE Trans Inf Theory 44:1310–1327 (http://www.dam.brown.edu/people/yiannis/PAPERS/suhov2.pdf).
Benedetto D, Caglioti E, Loreto V. 2002. Language trees and zipping. Phys Rev Lett 88:048702 (http://arxiv.org/abs/cond-mat/0108530).
Khmelev DV, Teahan WJ. 2003. Comment on “Language trees and zipping.” Phys Rev Lett 90:089803 (http://arxiv.org/abs/cond-mat/0205521).
Goodman J. 2002. Extended comment on “Language trees and zipping.” Electronic pre-print (http://arxiv.org/abs/cond-mat/0202383).
MacKay DJC. 2003. Information theory, inference, and learning algorithms. Cambridge UP, Cambridge (http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html).
Durbin R, Eddy S, Krogh A, Mitchison G. 1998. Biological sequence analysis: probabilistic models of proteins and nucleic acids. Cambridge UP, Cambridge.
Rieke F, Warland D, van Steveninck RDR, Bialek W. 1997. Spikes: exploring the neural code. MIT Press, Cambridge.
Abbott LF, Sejnowski TJ, eds. 1998. Neural codes and distributed representations: foundations of neural computation. MIT Press, Cambridge.
Billingsley P. 1965. Ergodic theory and information. Wiley, New York.
Katok AB, Hasselblatt B. 1995. Introduction to the modern theory of dynamical systems. Cambridge UP, Cambridge.
Solomonoff RJ. 1964. A formal theory of inductive inference. Inf Control 7:1–22, 224–254 (http://world.std.com/~rjs/pubs.html).
Li M, Vitanyi PMB. 1993. An introduction to Kolmogorov complexity and its applications. Springer, New York.
Zurek WH. 1998. Algorithmic randomness, physical entropy, measurements, and the demon of choice. E-print, arxiv.org (http://arxiv.org/abs/quant-ph/9807007).
Bennett CH. 1985. Dissipation, information, computational complexity and the definition of organization. In Emerging syntheses in science, pp. 215–234. Ed D Pines. Santa Fe Institute, Santa Fe, NM.
Bennett CH. 1986. On the nature and origin of complexity in discrete, homogeneous locally-interacting systems. Found Phys 16:585–592.
Bennett CH. 1990. How to define complexity in physics, and why. In Complexity, entropy, and the physics of information, pp. 137–148. Ed. WH Zurek. Addison-Wesley, Reading, MA.
Lloyd S, Pagels H. 1988. Complexity as thermodynamic depth. Ann Phys 188:186–213.
Gács P, Tromp JT, and Vitanyi PMB. 2001. Algorithmic statistics. IEEE Trans Inf Theory 47:2443–2463 (http://arxiv.org/abs/math.PR/0006233).
Rissanen J. 1978. Modeling by shortest data description. Automatica 14:465–471.
Rissanen J. 1989. Stochastic complexity in statistical inquiry. World Scientific, Singapore.
Hraber PT, Korber BT, Wolinsky S, Erlich H, Trachtenberg E. 2003. HLA and HIV infection progression: application of the minimum description length principle to statistical genetics. Technical Report 03-04-23, Santa Fe Institute (http://www.santafe.edu/research/publications/wpabstract/200304023).
Grassberger P. 1986. Toward a quantitative theory of self-generated complexity. Int J Theor Phys 25:907–938.
Shalizi CR. 2001. Causal architecture, complexity and self-organization in time series and cellular automata. PhD thesis, University of Wisconsin-Madison (http://bactra.org/thesis/).
Shaw R. 1984. The dripping faucet as a model chaotic system. Aerial Press, Santa Cruz, CA.
Lindgren K, Nordahl MG. 1988. Complexity measures and cellular automata. Complex Syst 2:409–440.
Li W. 1991. On the relationship between complexity and entropy for Markov chains and regular languages. Complex Syst 5:381–399.
Arnold D. 1996. Information-theoretic analysis of phase transitions. Complex Syst 10:143–155.
Bialek W, Nemenman I, Tishby N. 2001. Predictability, complexity and learning. Neural Comput 13:2409–2463 (http://arxiv.org/abs/physics/0007070).
Crutchfield JP, Young K. 1990. Computation at the onset of chaos. In Complexity, entropy, and the physics of information, pp. 223–269. Ed. WH Zurek. Addison-Wesley, Reading, MA.
Shalizi CR, Crutchfield JP. 2001. Computational mechanics: Pattern and prediction, structure and simplicity. J Stat Phys 104:817–879 (http://arxiv.org/abs/cond-mat/9907176).
Shalizi CR. 2003. Optimal nonlinear prediction of random fields on networks. Discr Math Theor Comput Sci AB (DMCS):11–30 (http://arxiv.org/abs/math.PR/0305160).
Shalizi, CR, Shalizi KL, Haslinger R. 2004. Quantifying self-organization with optimal predictors. Phys Rev Lett 93:118701 (http://arxiv.org/abs/nlin.AO/0409024).
Lewis HR, Papadimitriou CH. 1998. Elements of the theory of computation. Prentice-Hall, Upper Saddle River, NJ.
Clarke RW, Freeman MP, Watkins NW. 2003. Application of computational mechanics to the analysis of natural data: an example in geomagnetism. Phys Rev E 67:0126203 (http://arxiv.org/abs/cond-mat/0110228).
Chandler D. 1987. Introduction to modern statistical mechanics. Oxford UP, Oxford.
Reed WJ, Hughes BD. 2002. From gene families and genera to incomes and Internet file sizes: why power laws are so common in nature. Phys Rev E 66:067103.
Bak P, Tang C, Wiesenfeld K. 1987. Self-organized criticality: An explanation of 1/f noise. Phys Rev Lett 59:381–384.
Jensen HJ. 1998. Self-organized criticality: emergent complex behavior in physical and biological systems. Cambridge UP, Cambridge.
Carlson JM, Doyle J. 1999. Highly optimized tolerance: A mechanism for power laws in designed systems. Phys Rev E 60:1412–1427.
Carlson JM, Doyle J. 2000. Highly optimized tolerance: robustness and design in complex systems. Phys Rev Lett 84:2529–2532.
Newman MEJ, Girvan M, Farmer JD. 2002. Optimal design, robustness, and risk aversion. Phys Rev Lett 89:028301 (http://arxiv.org/abs/cond-mat/0202330).
Shirky C. 2003. Power laws, weblogs, and inequality. In Extreme democracy. Ed. M Ratcliffe, J Lebkowsky. Forthcoming. First published online February 2003 (http://www.shirky.com/writings/powerlaw_weblog.html).
Drenzer D, Farrell H. 2004. The power and politics of blogs. Persp Politics. Submitted (http://www.utsc.utoronto.ca/~farrell/blogpaperfinal.pdf).
Crutchfield JP, Shalizi CR. 1999. Thermodynamic depth of causal states: objective complexity via minimal representations. Phys Rev E 59:275–283 (http://arxiv.org/abs/cond-mat/9808147).
Huberman BA, Hogg T. 1986. Complexity and adaptation. Physica D 22:376–384.
Wolpert DH, Macready WG. 2000. Self-dissimilarity: an empirically observable measure of complexity. In Unifying themes in complex systems. Ed. Y Bar-Yam. Perseus Books, Boston (http://www.santafe.edu/research/publications/wpabstract/199712087).
Sporns O, Tononi G, Edelman GM. 2000. Connectivity and complexity: the relationship between neuroanatomy and brain dynamics. Neural Networks 13:909–992 (http://php.indiana.edu/~osporns/nn_connectivity.pdf).
Sporns O, Tononi G, Edelman GM. 2000. Theoretical neuroanatomy: Relating anatomical and functional connectivity in graphs and cortical connection matrices. Cerebral Cortex 10:127–141.
Sporns O, Tononi G. 2002. Classes of network connectivity and dynamics. Complexity 7:28–38 (http://php.indiana.edu/~osporns/complexity_2002.pdf).
Bates J, Shepard H. 1993. Measuring complexity using information fluctuation. Phys Lett A 172:416–425 (http://physics.unh.edu/people/profiles/bates_shepard.pdf).
Boffetta G, Cencini M, Falcioni M, Vulpiani A. 2002. Predictability: A way to characterize complexity. Phys Rep 356:367–474 (http://arxiv.org/abs/nlin.CD/0101029).
Feldman DP, Crutchfield JP. 1998. Measures of statistical complexity: why? Phys Lett A 238:244–252 (http://hornacek.coa.edu/dave/Publications/MSCW.html).
Axelrod R, Cohen MD. 1999. Harnessing complexity: organizational implications of a scientific frontier. Free Press, New York.
Flake GW. 1998. The computational beauty of nature: computer explorations of fractals, chaos, complex systems and adaptation. MIT Press, Cambridge.
Holland JH. 1998. Emergence: from chaos to order. Addison-Wesley, Reading.
Simon HA. 1996. The sciences of the artificial, 3rd ed. MIT Press, Cambridge.
Boccara N. 2004. Modeling complex systems. Springer, Berlin.
Hand D, Mannila H, Smyth P. 2001. Principles of data mining. MIT Press, Cambridge.
Kearns MJ, Vazirani UV. 1994. An introduction to computational learning theory. MIT Press, Cambridge.
Vapnik VN. 1979/1982. Estimation of dependencies based on empirical data. Transl. S Kotz. Springer, Berlin. From Vosstanovlyenie Zavicimostei po Empiricheckim Dannim, Nauka, Moscow.
Vapnik VN. 1998. Statistical learning theory. Wiley, New York.
Nelles O. 2001. Nonlinear system identification: from classical approaches to neural networks and fuzzy models. Springer, Berlin.
Tukey JW. 1977. Exploratory data analysis. Addison-Wesley, Reading, MA.
Tufte ER. 1983. The visual display of quantitative information. Graphics Press, Cheshire, CT.
Tufte ER. 1990. Envisioning information. Graphics Press, Cheshire, CT.
Tufte ER. 1997. Visual explanations: images and quantities, evidence and narrative. Graphics Press, Cheshire, CT.
Abarbanel HDI. 1996. Analysis of observed chaotic data. Springer, New York.
Casdagli M, Eubank S, eds. 1992. Nonlinear modeling and forecasting. Addison-Wesley, Reading, MA.
Weigend AS, Gershenfeld NA, eds. 1993. Time series prediction: forecasting the future and understanding the past. Addison-Wesley, Reading, MA.
Azencott R, Dacunha-Castelle D. 1984/1986. Series of irregular observations: forecasting and model building. Springer, Berlin. Originally published as Series d’observations irregulieres, Masson, Paris.
Taniguchi M, Kakizawa Y. 2000. Asymptotic theory of statistical inference for time series. Springer, Berlin.
Amari S, Nagaoka H. 1993/2000. Methods of information geometry. American Mathematical Society, Providence, RI. Transl. D Harada from Joho Kika no Hoho, Iwanami Shoten Publishers, Tokyo.
Ay N. 2002. An information-geometric approach to a theory of pragmatic structuring. Ann Probability 30:416–436 (http://www.mis.mpg.de/preprints/2000/prepr5200-abstr.html).
Ay N. 2001. Information geometry on complexity and stochastic interaction. Technical Report 95-2001, Max Planck Institute for Mathematics in the Sciences (http://www.mis.mpg.de/preprints/2001/prepr9501-abstr.html).
Cutler CD, Kaplan DT, eds. 1997. Nonlinear dynamics and time series: building a bridge between the natural and statistical sciences. American Mathematical Society, Providence, RI.
Tong H. 1990. Nonlinear time series: a dynamical systems approach. Oxford UP, Oxford.
Pearson RK. 1999. Discrete-time dynamic models. Oxford UP, New York.
Stengel RF. 1994. 1986. Optimal control and estimation. Dover, New York. Originally published as Stochastic optimal control: theory and application. Wiley, New York.
Lind D, Marcus B. 1995. An introduction to symbolic dynamics and coding. Cambridge UP, Cambridge.
Gershenfeld N. 1999. The nature of mathematical modeling. Cambridge UP, Cambridge.
Ilachinski A. 2001. Cellular automata: a discrete universe. World Scientific, Singapore.
Chopard B, Droz M. 1998. Cellular automata modeling of physical systems. Cambridge UP, Cambridge.
Toffoli T, Margolus N. 1987. Cellular automata machines: a new environment for modeling. MIT Press, Cambridge.
Farmer JD, Toffoli T, Wolfram S, eds. 1984. Cellular automata: proceedings of an interdisciplinary workshop, Los Alamos, NM 87545, March 7–11. North-Holland, Amsterdam. Also published in Physica D 10(1–2), 1984.
Gutowitz H, ed. 1991. Cellular automata: theory and experiment. MIT Press, Cambridge. Also published in Physica D 45(1–3), 1990.
Manneville P, Boccara N, Vichniac GY, Bidaux R, eds. 1990. Cellular automata and modeling of complex systems: proceedings of the winter school, Les Houches, France, February 21–28, 1989. Springer, Berlin.
von Neumann J. 1966. Theory of self-reproducing automata, Ed. and completed by AW Burks. U Illinois P, Urbana.
Burks AW, ed. 1970. Essays on cellular automata. U Illinois P, Urbana.
Poundstone W. 1984. The recursive universe: cosmic complexity and the limits of scientific knowledge. William Morrow, New York.
Langton CG, ed. 1988. Artificial life. Addison-Wesley, Reading, MA.
Wolfram S. 1994. Cellular automata and complexity: collected papers. Addison-Wesley, Reading, MA (http://www.stephenwolfram.com/publications/books/ca-reprint/).
Griffeath D, Moore C, eds. 2003. New constructions in cellular automata. Oxford UP, Oxford.
Rivet J, Boon J. 2001. Lattice gas hydrodynamics. Cambridge UP, Cambridge.
Doolen GD, ed. 1989. Lattice gas methods for partial differential equations: a volume of lattice gas reprints and articles. Addison-Wesley, Reading, MA.
Doolen GD, ed. 1991. Lattice gas methods: theory, applications, and hardware. MIT Press, Cambridge. Also published in Physica D 47(1–2), 1991.
Axelrod R. 1984. The evolution of cooperation. Basic Books, New York.
Varela FJ, Bourgine P, eds. 1992. Toward a practice of autonomous systems: proceedings of the first European conference on artificial life. MIT Press, Cambridge.
Epstein JM, Axtell R. 1996. Growing artificial societies: social science from the bottom up. MIT Press, Cambridge.
Epstein JM. 1999. Agent-based computational models and generative social science. Complexity 4(5):41–60.
Macy MW, Willer R. 2002. From factors to actors: Computational sociology and agent-based modeling. Ann Rev Sociol 28:143–66.
Gimblett R, ed. 2001. Integrating geographic information systems and agent-based modeling techniques for understanding social and ecological processes. Oxford UP, Oxford.
Kohler TA, Gumerman GJ, eds. 2000. Dynamics in human and primate societies: agent-based modeling of social and spatial processes. Santa Fe Institute Studies in the Sciences of Complexity. Oxford UP, Oxford.
Bonabeau E. 1997. From classical models of morphogenesis to agent-based models of pattern formation. Artificial Life 3:191–211.
Camazine S, Deneubourg J-L, Franks NR, Sneyd J, Theraulaz G, and Bonabeau E. 2001. Self-organization in biological systems. Princeton UP, Princeton.
Gardiner CW. 1990. Handbook of stochastic methods for physics, chemistry, and the natural sciences, 2nd ed. Springer, Berlin.
Newman MEJ, Barkema GT. 1999. Monte Carlo methods in statistical physics. Clarendon Press, Oxford.
MacKeown PK. 1997. Stochastic simulation in physics. Springer, Singapore.
King G. 1997. A solution to the ecological inference problem: reconstructing individual behavior from aggregate data. Princeton UP, Princeton.
Shannon CE. 1948. A mathematical theory of communication. Bell Syst Tech J 27:379–423. Reprinted in The mathematical theory of communication. Ed. Shannon CE, Weaver W. U Illinois P, Urbana, 1963 (http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html).
Hartley RVL. 1928. Transmission of information. Bell Syst Tech J, pp. 535–563.
Gray RM. 1990. Entropy and information theory. Springer, New York (http://wwwee.stanford.edu/~gray/it.html).
Peliti L, Vulpiani AV, eds. 1988. Measures of complexity: proceedings of a conference held in Rome, September 30–October 2, 1987. Springer, Berlin.
Edmonds BH. 1997. Hypertext bibliography of measures of complexity (http://www.cpm.mmu.ac.uk/~bruce/combib/).
Grünwald P. 2005. A tutorial introduction to the minimum description length principle. In Advances in minimum description length: theory and applications. Ed. P Grünwald, IJ Myung, M Pitt. MIT Press, Cambridge (http://arxiv.org/abs/math.ST/0406077).
Skouras S. 2001. Decisionmetrics: A decision-based approach to econometric modeling. Technical Report 01-11-064, Santa Fe Institute (http://www.santafe.edu/research/publications/wpabstract/200111064).
Vidyasagar M. 1997. A theory of learning and generalization: with applications to neural networks and control systems. Springer, Berlin.
William of Ockham. 1964. Philosophical writings: a selection. Transl. with an introduction by Philotheus Boehner. Bobbs-Merrill, Indianapolis. Originally published in various European cities during the early 1300s.
Lehmann EL. 1997. Testing statistical hypotheses, 2nd ed. Springer Texts in Statistics. Springer, Berlin.
Churchland PS, Sejnowski TJ. 1992. The computational brain. MIT Press, Cambridge.
Giunti M. 1997. Computation, dynamics, and cognition. Oxford UP, Oxford.
Margolus N. 1999. Crystalline computation. In Feynman and computation: exploring the limits of computers, pp. 267–305. Ed. AJG Hey. Perseus Books, Reading, MA (http://arxiv.org/abs/nlin.CG/9811002).
Hirotugu Akaike. 1998. Selected papers of Hirotugu Akaike. Ed. E Parzen, K Tanabe, G Kitagawa. Springer, Berlin.
Shannon CE, Weaver W, eds. 1963. The mathematical theory of communication. U Illinois P, Urbana.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer Inc.
About this chapter
Cite this chapter
Shalizi, C.R. (2006). Methods and Techniques of Complex Systems Science: An Overview. In: Deisboeck, T.S., Kresh, J.Y. (eds) Complex Systems Science in Biomedicine. Topics in Biomedical Engineering International Book Series. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-33532-2_2
Download citation
DOI: https://doi.org/10.1007/978-0-387-33532-2_2
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30241-6
Online ISBN: 978-0-387-33532-2
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)