When I look at the English definition of the word "Chaos", I get the following definition : "Behavior so unpredictable as to appear random, owing to great sensitivity to small changes in conditions."
Part 1: Due to such definitions, for years, I personally thought that a Stochastic Process was synonymous with "Chaotic" behavior. For example, consider a Random Walk (with drift, without drift) :
$$X_{t} = X_{t-1} + \epsilon_{t}, \quad \epsilon_{t} \sim N(0, 1)$$ $$X_{t} = X_{t-1} + drift + \epsilon_{t}, \quad \epsilon_{t} \sim N(0, 1)$$
- the same Random Walk model will produce different results each times its run (i.e. random)
- the next result of the same Random Walk is unpredictable
- adding different "drift" terms to the Random Walk model (i.e. modifying changes in conditions) will result in different results as well.
Thus, I thought that for all intent and purposes a Random Walk is "Chaotic" - for example, individual trajectories of Random Walks can appear to be pretty chaotic (R code for demonstration):
library(ggplot2)
library(latex2exp)
# Generate and plot random walks without drift
df1 <- data.frame()
for (i in 1:100) {
df1 <- rbind(df1, data.frame(Time = 1:100, Value = cumsum(rnorm(100)), Walk = paste("Walk", i)))
}
p1 <- ggplot(df1, aes(x=Time, y=Value, group=Walk)) +
geom_line() +
theme_bw() +
theme(legend.position="none") +
labs(title = TeX("$X_{t} = X_{t-1} + \\epsilon_{t}$ (Random Walk without Drift)"))
# Generate and plot random walks with drift
df2 <- data.frame()
for (i in 1:100) {
df2 <- rbind(df2, data.frame(Time = 1:100, Value = cumsum(rnorm(100)) + 0.1*(1:100), Walk = paste("Walk", i)))
}
p2 <- ggplot(df2, aes(x=Time, y=Value, group=Walk)) +
geom_line() +
theme_bw() +
theme(legend.position="none") +
labs(title = TeX("$X_{t} = X_{t-1} + drift + \\epsilon_{t}$, drift = 0.1 (Random Walk with Drift)"))
But looking more into this, I notice that Stochastic Processes are not considered Chaotic.
Part 2: Recently, I was reading about the "Lorenz System" - a series of differential equations that is considered to be a Chaotic System as it is apparently highly sensitive to initial conditions - yet for a given set of initial conditions, the system behaves deterministically:
$$\begin{align*} \frac{dx}{dt} &= \sigma(y - x) \\ \frac{dy}{dt} &= rx - y - xz \\ \frac{dz}{dt} &= xy - \beta z \end{align*}$$
It took me a while, but using the Euler Method, I think I was able to plot the Lorenz System for different sets of initial conditions (R code):
library(ggplot2)
library(dplyr)
# simulate and plot Lorenz system
plotLorenz <- function(s, r, b, color) {
dt <- 0.01
vec <- c(1, 1, 1) # initial conditions for x, y, z
t <- 0
# compute next state of Lorenz system
stepLorenz <- function(vec) {
x <- vec[1]
y <- vec[2]
z <- vec[3]
dx <- s * (y - x)
dy <- x * (r - z) - y
dz <- x * y - b * z
vec[1] <- vec[1] + dx * dt
vec[2] <- vec[2] + dy * dt
vec[3] <- vec[3] + dz * dt
t <- t + dt
return(vec)
}
# Simulate Lorenz system
results <- matrix(nrow=10000, ncol=3)
for (i in 1:10000) {
vec <- stepLorenz(vec)
results[i,] <- vec
}
df <- as.data.frame(results)
names(df) <- c("X", "Y", "Z")
df$id <- seq_len(nrow(df))
# X vs Y
p1 <- ggplot(df, aes(x=X, y=Y)) +
geom_path(color=color) +
theme_bw() +
ggtitle(paste("Lorenz Curve: X vs Y for S=", round(s, 2), ", r=", round(r, 2), ", b=", round(b, 2)))
# Y vs Z
p2 <- ggplot(df, aes(x=Y, y=Z)) +
geom_path(color=color) +
theme_bw() +
ggtitle(paste("Lorenz Curve: Y vs Z for S=", round(s, 2), ", r=", round(r, 2), ", b=", round(b, 2)))
# X vs Z
p3 <- ggplot(df, aes(x=X, y=Z)) +
geom_path(color=color) +
theme_bw() +
ggtitle(paste("Lorenz Curve: X vs Z for S=", round(s, 2), ", r=", round(r, 2), ", b=", round(b, 2)))
return(list(p1, p2, p3))
}
plots1 <- plotLorenz(10, 28, 8/3, "red")
plots2 <- plotLorenz(10, 30, 8/3, "blue")
plots3 <- plotLorenz(10, 32, 8/3, "green")
I can see small changes in the behavior of the Lorenz System when the initial conditions are modified - but to be honest, the changes in the Random Walks are far more noticeable.
My Question: Given that the alleged Non-Chaotic system (i.e. Stochastic Random Walk) looks ALOT more chaotic than the actual Chaotic system (i.e. Lorenz System) - mathematically, how do we define and quantify "Chaos" in mathematics?
I have heard about the Lyapunov Exponent, which can be used to used to measure how significantly the trajectories of a system change when the initial conditions are modified (i.e. serving as a proxy/indicator for Chaos) - but I have only seen this concept being applied in situations such as Lorenz System and not Random Walks.
Can the Lyapunov Exponent be applied to Random Walks and Stochastic Processes? Can Stochastic Processes be considered as Chaotic? Can chaos be mathematically measured and quantified? For example - given some mathematical equations: How would someone decide if these equations correspond to a Chaotic System?
Thanks!
Lawvere in his 1984's Functorial Remarks on the General Concept of Chaos for one defines, for a given adjunction $U \dashv H: D \leftrightharpoons C$, a morphism $U(X)\xrightarrow{\varphi}Y$ to be $U$-chaotic iff $X\xrightarrow{H(\varphi)}H(Y)$ is an epimorphism, generalizing from the case with $C$ a category of 'spaces' and $D=C^T$ for $T$ a monoid in $C$
Then, there is a a category $Stoch \ (\subsetneq Meas)$ of measurable spaces with 'probabilistic mappings'/'transition probabilities' as morphisms, such that "2. Construction.", "3. Theorem 1." and "4. The Kleisli category" of Giry give us that $Stoch$ is monoidal, [with the one-point-space as tensor unit,] and $(\mathbb{R}_{>0}, *)$ a monoid in it, so there should be a lot to look into
$Stoch \dashv Meas$ and $Stoch \dashv Stoch^{(\mathbb{R}_{>0}, *)}$
(presumably) with $Y$'s equal to $[0, 1]$, or $\mathbb{R}_{\geq 0}$, or $\mathbb{R}$, and pertinent structure in all cases
For differential stuff probably one should take a look at Shiebler's $EucMeas$