What does it mean for something to be linear?

Question

I used to be told that "linear" refers to things that follow the concept of a "line" but recently I have been introduced to concepts that use the term "linear" in ways that aren't as easily comprehensible such as "linear generator" or "linear predictor".

I'm not looking for the specific meaning of those terms but what intuiton can you share regarding how you treat something when its described to be "linear", say you didn't know what linear code meant but I approached you about linear code what sort of image would you start with?

Thank you.

Answer 1

Your question reminds me of a very good electrical circuits professor, far back in the eighties of the 20th century. The first day we met him, he started by describing how we should study a scientific topic (and I'd have followed more scrupulously his advices) and then explained us the various hypotheses which lay at the basis of basic circuit theory. Talking of linearity, He said:

"A physical system is called "linear" if it satisfies the superposition principle".

A formal description

Definition 1 (Newcomb [1], §1.1, p.7). A system is a binary relation (in the set-theoretical sense) between two sets of variables $X$ and $Y$, i.e. it is a subset of their cartesian product $X\times Y$.
Thus a system $\mathscr{S}$ generally has the following structure $$ \mathscr{S}=\big\{ (x, y)\in X\times Y: x\mathscr{R} y\big\} $$ It is customary to write $x \mathscr{R} y$ in the equivalent form $(x,y)\in \mathscr{S}$. Now, in order to define what a superposition principle means we require that both $X$ and $Y$ are vector spaces (defined on the same field $\mathbb{K}$ in order to simplify the exposition) in order to formalize the intuitive fact that "causes" and "effects" can be "added" and "multiplied by constants". Now we are ready to define what the superposition principle is:

Definition 2. A system $\mathscr{S}$ satisfies the superposition principle if

1. $$ \begin{split} x_1 &\mathscr{R} y_1\:\big[ (x_1,y_1)\in\mathscr{S}\big]\\ x_2 &\mathscr{R} y_2\:\big[ (x_2,y_2)\in\mathscr{S}\big]\\ \end{split}\implies(x_1+x_2)\mathscr{R}(y_1+y_2)\:\big[(x_1+x_2,y_1+y_2)\in\mathscr{S}\big] $$
2. $$ x \mathscr{R} y\:\big[(x,y)\in \mathscr{S}\big]\implies cx \mathscr{R}cy\:\big[(cx,cy)\in \mathscr{S}\big]\quad \forall c\in\Bbb K $$

Now, if we look at the second one of the above relations and remember the meaning of the relation $(x,y)\in \mathscr{S}$ we may guess because systems for which the superposition principle holds are called linear: precisely we see that a relation $\mathscr{S}$ for which the superposition principle holds contains each line passing through each of its point $(x,y)$ and thus in this sense is "linear". This is very explicitly seen in the simplest case where $\dim X=\dim Y=1$ and $\Bbb{K}=\Bbb{R}$, depicted here:

Final note

• Definition 1 is very general and possibly includes almost every physically conceivable system, since it does not even require the relation between causes and effect to be a function, i.e. it does not require the uniqueness of the effect corresponding to a given cause.
  $X$ and $Y$ may be any vector space of physical quantities and the system may be time-(or any other parameter) variant, it may present hysteresis or any other "wild" behavior: therefore it is reasonable to suppose that definition 2 may possibly cover the core content of the linearity concept.
  
• Roughly speaking, the superposition principle says that there exist systems such that sending at their input two different "causes" makes their output behave as the "join" of the output effects you would have if you consider each single input. Thus perhaps the very concept of linearity has simply been distilled over the years in order to give a mathematically sound description of the superposition principle.

References

[1] Newcomb, Robert W., Linear multiport synthesis, New York: McGraw-Hill, 1966, pp. 397, (see its citation classic review).

Answer 2

A function $f: X \rightarrow Y$ from some set $X$ to another $Y$ is linear if

$$f(\alpha x) = \alpha f(x)$$ and $$ f(x_1+x_2)= f(x_1)+f(x_2)$$ for all $x_1,x_2\in X$ and all scalar multipliers $\alpha$.

It seems plausible for me to assume that you are refering to such functions that are linear "predictors".

Answer 3

$\textbf{IN MATHEMATICS}$

In mathematics, a linear map or linear function f(x) is a function that satisfies the following two properties (as @M1183 has mentioned already):

1. Additivity: $f(x + y) = f(x) + f(y).$

2. Homogeneity of degree 1: $f(αx) = αf(x) \ \ (\forall α)$

Also, in mathematics, a linear differential equation is a differential equation that is defined by a linear polynomial in the unknown function and its derivatives, that is an equation of the form $$a_0(x)y+a_1(x)y'+a_2(x)y''+\cdots a_n(x)y^{(n)}+b(x)=0$$ where $ a_{0}(x), ..., a_{n}(x) \text{ and } b(x)$ are arbitrary differentiable functions that do not need to be linear, and $y', y'', ..., y^{(n)}$ are the successive derivatives of an unknown function $y$ of the variable $x$.

---

$\textbf{IN PHYSICS}$

In physics, linearity is a property of the differential equations governing many systems; for instance, the Maxwell equations or the diffusion equation.

Linearity of a homogenous differential equation means that if two functions $f$ and $g$ are solutions of the equation, then any linear combination $af + bg$ is, too.

---

$\textbf{IN ELECTRONICS}$

In electronics, the linear operating region of a device, for example a transistor, is where a dependent variable (such as the transistor collector current) is directly proportional to an independent variable (such as the base current). This ensures that an analog output is an accurate representation of an input, typically with higher amplitude (amplified). A typical example of linear equipment is a high fidelity audio amplifier, which must amplify a signal without changing its waveform. Others are linear filters, linear regulators, and linear amplifiers in general.

---

$\textbf{IN ART}$

Linear is one of the five categories proposed by Swiss art historian Heinrich Wölfflin to distinguish "Classic", or Renaissance art, from the Baroque. According to Wölfflin, painters of the fifteenth and early sixteenth centuries (Leonardo da Vinci, Raphael or Albrecht Dürer) are more linear than "painterly" Baroque painters of the seventeenth century (Peter Paul Rubens, Rembrandt, and Velázquez) because they primarily use outline to create shape. Linearity in art can also be referenced in digital art. For example, hypertext fiction can be an example of nonlinear narrative, but there are also websites designed to go in a specified, organized manner, following a linear path.

---

$\textbf{IN MUSIC}$

In music the linear aspect is succession, either intervals or melody, as opposed to simultaneity or the vertical aspect.

---

$\textbf{P.S.}$ To conclude, there is no general way of approach, but all approaches in different fields may have something associated with "line" and this is what they may have in common.

Answer 4

A linear space or set

I did not really learn this point until grad school. I always thought that a linear set just had to resemble a line, however there is another technical requirement. All linear sets must contain the zero vector. Visually, I think of a line or a plane in 3-D that is "pinned" at the origin. This is in stark contast with the definition of a "line" we learn in primary school -- $\{y(x)=mx+b \ | \ x\in\mathbb{R}\}$ is only actually linear when $b=0$! When $b\neq 0$, the proper term is affine.

A bit more on the analysis side -- these types of sets typically have very well-behaved (and relatively easily computable) projection operators.

A linear functional

When talking about a "bounded linear functional" (or, equivalently, a continuous linear functional), I always think of lines pinned at zero with varying slope. The intuition here relies heavily on the Hahn-Banach theorem which, roughly speaking, states that every bounded linear function looks like $\langle \cdot | m \rangle$ for some $m$ in your space. If your space is $\mathbb{R}$, then this is equivalent to the linear function described above (remember $b$ must be zero!) -- the inner product is just multiplication in $\mathbb{R}$. Even when working in more general spaces, I still think of $m$ as the ``slope'' of the functional, even though it is a vector.

If you have an unbounded linear functional, things get weird. These guys do not have a nice representation theorem like Hahn-Banach. I always try to remember two things about this class of operator: (1) they can only appear in infinite-dimensoinal spaces. (2) I think of a classical example -- differentiation. (this is a tangent, but here is a brief justificaiton of why differentiation is not always continuous/bounded -- Consider the space of functions on $[0,1]$ with the $\sup$ norm. Continuous operators should map bounded sequences to bounded sequences, but differentiation maps the bounded sequence $\{x^n\}_{n\in\mathbb{N}}$ to an unbounded sequence.)

Linear code/algorithm

A linear code or algorithm is a rather broad notion. It could be confusing, since a "linear algorithm" might do a task that is nonlinear or hard to even formulate in a vector space. The only requirement to call a code/algorithm linear is determined by its runtime. An algorithm which takes an input of size $n$ is linear if it requires $\mathcal{O}(n)$ basic operations (add/subtract/multiply/divide -- see here for more on big-O notation and runtime analysis) to complete its task. This is in contrast with algorithms which may require a different amount of time to complete, e.g. a quadratic algorithm requires $\mathcal{O}(n^2)$ operations.

If I'm writing a code, and visually I can see nested for-loops, then most of the time the code is not linear. If I hear someone say "We have a linear algorithm to compute X" then I'm usually impressed, since it means they found a pretty efficient way to complete the task X.

Answer 5

Well, to answer your question on the things that follow the word linear, I think it just means operating in one dimension, relating to lines or just that the specific thing is "simple". Or, a linear system is something that doesn't just exist in math, but in real visual space. And something that you can physically interact with.

Answer 6

If I were asked for an elevator pitch about the concept of "linear" I would start with the better known concept of proportionality. You can fill out that idea with different kinds of mathematical examples, depending on the level of the elevator mates you are enlightening.

In the function $f(x) = ax$, when you double the input the output doubles. For $f(x) = ax + b$ the linearity is a little subtler: when the difference in the input doubles so does the difference in the output.

Those examples work for vector spaces too, when you think of $x$ and $b$ as vectors and $a$ as a matrix (so a linear transformation). That covers the examples of linear differential equations when you rewrite them as differential operators on a vector space of functions.

In fact, vector spaces were invented to provide the abstract framework for "linearity", which is captured by the ability to add vectors and multiply them by scalars. So when I see a reference to a "linear something" I am pretty sure there's a vector space lurking.

Linear codes are not examples of proportionality (which deals with the scalar multiplication) but are examples where the ability to add vectors comes into play: code words can be thought of as vectors, and the vector sum of codewords is again a codeword.

It's often useful when thinking about a new concept to consider things that aren't it. The squaring and exponential functions ar nonlinear. So are physical problems like falling objects when air resistance matters.

Answer 7

I think your question is somehow unanswerable in this forum. Anyway here are my thoughts:

• Sometimes an object called linear object if all material of it assembled/moving along/like a line.
• Sometimes an object called linear if you stretch out/in along a line its global shape doesn't change. e.g. a line, 2d-plane, 3d-space and more generally a vector space.

• Sometimes an object called linear if there is no loop or circle. (This is something like this 2 phase logic: if it is not p then it is q.).

• A function called linear if it preserve linearity of a linear object. e.g. $f(ax+y)=af(x)+f(y).$

All of above cases are non-precise explanation.