Liyuan Chen · Follow
4 min read · Sep 26, 2024
--
Definition
Definition 1: We say f : ℝⁿ → ℝ is a convex or concave up function, if for any two points x, y in its domain and any t ∈ [0, 1], we have:
f(tx + (1 — t)y) ≤ tf(x) + (1 — t)f(y).
Definition 2: We say f : ℝⁿ → ℝ is a concave or concave down function, if for any two points x, y in its domain and any t ∈ [0, 1], we have:
f(tx + (1 — t)y) ≥ tf(x) + (1 — t)f(y).
Geometric Interpretation
A function f is convex or concave up if the line segment between any two points on the graph of the function lies above or on the graph. The curve is U-shape, bowl-like open upward.
A function f is concave or concave down if the line segment between any two points on the graph of the function lies below or on the graph. The curve is ∩-shape, bowl-like open downward.
In 2D and 3D space
Here are examples of convex and concave functions in 2D and 3D space:
We can see that the concepts of convex and concave functions directly generalize from single-variable functions (in 2D space) to multi-variable functions (in spaces of 3 or more dimensions).
First and Second-order characterizations of Convex Function
If a function f : ℝⁿ → ℝ is differentiable, then it is convex if and only if for any two points x, y in its domain, it holds:
f(y) ≥ f(x) + ∇f (𝑥)ᵀ (y - x)
If a function f : ℝⁿ → ℝ is twice differentiable, then it is convex if and only if for every point x in its domain, it holds:
Hessian matrix ∇²f (𝑥) is positive semidefinite.
Curvature describes how much a geometric object deviates from being flat or straight.
Curvature in 2D Space
For a function f(x), curvature is defined as: k(x) = f ′′(x)/(1 + f (x)²)³/².
We can observe that the sign of the curvature k(x) is determined by the sign of f ′′(x), which indicates the concavity of the function as shown in Figure 3:
- Positive curvature (f ′′(x) > 0, k > 0): The function is concave up
(U-shape), meaning the curve lies above its tangent line locally. - Negative curvature (f ′′(x) < 0, k < 0): The function is concave down
(∩-shape), meaning the curve lies below its tangent line locally. - Zero curvature (f ′′(x) = 0, k = 0): Linear or inflection point
The magnitude of the curvature |k| measures how sharply a curve bends. The larger magnitude bends more sharply, in other words, it deviates faster from its tangent line.
Curvature in 3D Space
In 3D space, the Gaussian curvature K of a surface at a point p is the product of two principal curvatures (maximum and minimum curvatures k₁ and k₂) at the point p: Kɢ = k₁k₂.
Note that the definition of convexity in 3D is quite different from 2D. In 3D a convex object is for any two points within the object, the straight line segment connecting them lies entirely within the object, which means it doesn’t have any cave-in. For example, a football is a convex object. And convex surfaces (e.g. spheres) always have positive curvatures when using outward-pointing normal vectors.
We can see that the concept of curvature cannot be directly generalized from single-variable functions (in 2D space) to multi-variable functions (in spaces of 3 or more dimensions). In 3D, beyond the relationship between second derivatives, curvature also relates to outward-pointing normals and the embedding of surfaces to represent the convexity of the 3D object as a whole.
Therefore, the sign of curvature does not represent the convexity and concavity of the graph of a function as it does in 2D. Instead, it relates to the convexity of the 3D object itself.
Curvature in Hyperspace
In manifolds of higher-dimensional spaces (n > 3), the curvature becomes much more complex (e.g. Riemann Curvature) and is hard to visualize.
- In 2D, second-derivative f ′′(x), along with curvature, can provide information about the concavity of a function’s graph:
a) When f ′′(x) > 0, the function is concave up (U-shape)
b) When f ′′(x) < 0, the function is concave down (∩-shape) - In 3D geometry, curvature that refers to the convexity of a 3D object does not provide information about the concavity of a function's graph. However, we still use the term "curvature" to describe the concavity of a function's graph in a 2D plane within a 3D coordinate system, particularly in the context of optimization.
- In 3D and higher dimensions, we typically use the Hessian matrix (formed by all the second partial derivatives ∂²f / ∂x², ∂²f / ∂y², ∂²f / ∂x∂y) to analyze the concavity of the function’s graph. If you’re interested, you can check my article about the Hessian matrix :)
NOTE: All images are by the author.