Mastering Lagrange Multipliers: Constrained Optimization

This study guide covers the Method of Lagrange Multipliers, a powerful technique for finding the local maxima and minima of a function subject to equality constraints.

Learning Objectives

After studying this chapter, you should be able to:

• Formulate the Lagrange system of equations for a given objective function and constraint.
• Solve optimization problems involving one constraint using the multiplier $\lambda$.
• Extend the method to solve optimization problems involving two constraints using multipliers $\lambda$ and $\mu$.
• Interpret the geometric significance of the gradient vectors at a constrained extremum.

Key Terms & Glossary

• Objective Function ($f$): The function you want to maximize or minimize (e.g., profit, surface area).
• Constraint Function ($g$): The equation that restricts the possible values of the variables (e.g., budget limits, fixed volume).
• Lagrange Multiplier ($\lambda$): A scalar constant that relates the gradient of the objective function to the gradient of the constraint.
• Level Curve: A curve along which a function $f(x, y)$ has a constant value $c$.

The "Big Idea"

At the point where a function $f$ reaches its maximum or minimum subject to a constraint $g(x, y) = k$, the level curve of $f$ is tangent to the constraint curve $g$. Geometrically, this means their gradient vectors point in the same (or opposite) direction. Since the vectors are parallel, one must be a scalar multiple of the other: $\nabla f = \lambda \nabla g$.

Formula / Concept Box

[!IMPORTANT] The method of Lagrange multipliers only works if $\nabla g \neq 0$ at the point of interest. If the gradient of the constraint is zero, the method may fail.

Hierarchical Outline

1. Theoretical Foundation
   • Gradient Alignment: Gradients $\nabla f$ and $\nabla g$ are parallel at extrema.
   • Smoothness Requirements: Functions must have continuous partial derivatives.
2. The Single Constraint Process
   • Step 1: Identify $f(x, y)$ and $g(x, y) = k$.
   • Step 2: Set up the system: $f_x = \lambda g_x$, $f_y = \lambda g_y$, $g(x, y) = k$.
   • Step 3: Solve for $x, y, \lambda$.
   • Step 4: Evaluate $f$ at all solution points to find absolute max/min.
3. Multiple Constraints
   • Introduce a second multiplier $\mu$ for the second constraint $h(x, y, z) = c$.

Visual Anchors

Problem-Solving Flowchart

Geometric Representation (TikZ)

Definition-Example Pairs

• Constraint Function ($g$): A restriction on the domain of the objective function.
  • Example: If you are building a box with a fixed volume of $12 ft^3, the constraint is g(x, y, z) = xyz - 12 = 0$.
• Objective Function ($f$): The quantity you are attempting to optimize.
  • Example: If you want to minimize the cost of the box materials, the objective function is $f(x, y, z) = C_1(xy) + C_2(yz) + C_3(xz)$.

Worked Examples

Example: Minimizing $f(x, y) = x^2 + y^2$ subject to $xy = 1$

1. Identify Functions:

• $f(x, y) = x^2 + y^2$
• $g(x, y) = xy = 1$

2. Set up Gradients:

• \nabla f = \langle 2x$, $2y \rangle
• \nabla g = \langle y$, $x \rangle

3. Form the System:

1. $2x = \lambda y$
2. $2y = \lambda x$
3. $xy = 1$

4. Solve:

• From (1), $\lambda = \frac{2x}{y}$. Substitute into (2):
• $2y = (\frac{2x}{y})x \implies 2y^2 = 2x^2 \implies y^2 = x^2 \implies y = \pm x$.
• Plug into (3): $x(x)$= 1 \implies x^2 = 1 \implies x = \pm 1$$.
• If $x=1$, then $y=1$. If $x=-1$, then $y=-1$.
• Points: $(1, 1)$ and $(-1, -1)$.

5. Evaluate:

• $f(1, 1) = 1^2 + 1^2 = 2$
• $f(-1, -1) = (-1)^2 + (-1)^2 = 2$
• The minimum value is 2.

Checkpoint Questions

1. What is the geometric relationship between the gradient of $f$ and the gradient of $g$ at a constrained extremum point?
2. Why do we need to include the constraint equation $g(x, y) = k$ in our system of equations?
3. If a problem has two constraints, $g$ and $h$, what does the gradient equation $\nabla f = \lambda \nabla g + \mu \nabla h$ imply about the relationship between the three gradient vectors?

▶Click to reveal answers

1. They are parallel (one is a scalar multiple of the other).
2. Because the gradient equation provides only the relationship between variables; the constraint equation ensures the solution actually lies on the allowed path/surface.
3. It implies that $\nabla f$ lies in the plane spanned by $\nabla g$ and $\nabla h$.

Muddy Points & Cross-Refs

• $\lambda = 0$: If you find $\lambda = 0$, it means the constrained extremum is also an unconstrained critical point of $f$.
• Multiple Solutions: Always check all $(x, y)$ pairs found. Lagrange multipliers identify candidates for extrema; you must compare values to determine which is the max and which is the min.
• Deep Dive: For inequality constraints (e.g., $g(x,y)$\leq k$$), look up the Karush-Kuhn-Tucker (KKT) conditions in advanced optimization texts.