Linear vs Nonlinear Mathematical Models
We use mathematical models to represent the relationship between different variables. These models help us make predictions, understand patterns, and make decisions. There are two broad categories of mathematical models, linear and nonlinear. In this article, we will explore the differences between these two types of forms and when each is appropriate to use.
Linear Models
Linear models are used to represent flat relationships between variables. A one-dimensional straight line, two-dimensional flat surface (called planes), and higher-dimensional hyperplanes are all examples of flat relationships that can be modeled with a linear model. The graph of a linear function, which models a linear dependency, is forever flat and does not bend. In other words, if the relationship between two variables is such that a change in one variable results in a proportional change in the other, we can use a linear model to represent this relationship.
The formula for a linear function is very simple to write down. In a linear function, the variables appear in the formula as themselves, with no powers or roots, and are not embedded in any other functions, such as denominators of fractions, sine, cosine, exponential, logarithmic, or other calculus functions. They can only be multiplied by scalars (real or complex numbers, not vectors or matrices), and added to or subtracted from each other.
A linear function that depends on three variables, x1, x2, and x3, can be written as:
y = w1x1 + w2x2 + w3x3 + bIn this equation, the parameters w1, w2, and w3 are scalar numbers that combine the variables to produce the outcome of y after adding the bias term b. The outcome is produced as a result of linear interactions between the variables x1, x2, and x3, and is said to depend linearly on these variables.
Nonlinear Models
Nonlinear models, on the other hand, are used to represent relationships that are not flat. If a relationship between two variables is such that a change in one variable does not result in a proportional change in the other, we need a nonlinear model to represent this relationship. Nonlinear models are used to represent curved relationships, where the graph of the function bends. If data points congregate around bending curves or surfaces, they are generated by nonlinear functions.
The formula for a nonlinear function is easy to spot. One or more variables appear in the formula with a power other than one, or multiplied or divided by other variables, or embedded in some other calculus functions, such as sines, cosines, exponentials, logarithms, etc.
We can come up with all kinds of nonlinear functions, and the possibilities related to what we can do and how much of the world we can model using nonlinear interactions are limitless. In fact, neural networks are successful because of their ability to pick up on the relevant nonlinear interactions between the data features.
When to Use Linear or Nonlinear Models
Choosing whether to use a linear or nonlinear model depends on the nature of the relationship between the variables being modeled. If the relationship is flat or near-flat, a linear model is appropriate. For example, if we’re trying to model the relationship between height and weight, a linear model is appropriate, since the relationship between these two variables is nearly proportional.
However, if the relationship is curved, a nonlinear model is necessary. For example, if we’re trying to model the relationship between the price of a product and the number of units sold, a nonlinear model may be more appropriate. In this case, we might expect that as the price of the product decreases, the number of units sold will increase.
Further reading: I highly recommend reading: Essential Math for AI .
Chapter 2. Data, Data, Data
