Advanced Applications and Mathematical Connections

4. Optimisation and Mathematical Modelling

Learning outcomes
  • I can model optimisation problems using vectors and matrices.
  • I can analyse systems with multiple variables.
  • I can evaluate mathematical models.
  • I can justify modelling choices.
  • I can solve authentic optimisation problems.

Introduction

Many important problems in mathematics, science, engineering, economics, and technology involve finding the best possible solution.

A company may want to maximise profit while limiting production costs. An engineer may need to minimise the weight of a structure while maintaining its strength. A transport company may want to reduce travel distances while meeting delivery requirements. A scientist may need to determine the combination of variables that produces the most accurate mathematical model.

These are examples of optimisation problems.

Optimisation involves finding the maximum or minimum value of a quantity while satisfying any necessary conditions or restrictions.

Vectors and matrices are particularly useful because they allow us to represent many variables and constraints in a compact, organised mathematical form.

In this lesson, we will investigate how to construct optimisation models, analyse systems involving multiple variables, solve linear and quadratic optimisation problems, evaluate mathematical assumptions, and apply optimisation to authentic situations.

 

What Is Optimisation?

Optimisation is the process of finding the best possible value of an objective, subject to any relevant constraints.

The objective may be to:

  • Maximise profit, output, efficiency, or performance.

  • Minimise cost, energy consumption, travel distance, error, or waste.

  • Find the best combination of resources.

  • Determine the most accurate parameters for a mathematical model.

An optimisation problem usually contains three important components.

Decision variables are quantities we can choose or control.

Objective function is the mathematical expression we want to maximise or minimise.

Constraints are restrictions that the solution must satisfy.

For example, suppose a company produces two products.

Let:

x = number of units of Product A.

y = number of units of Product B.

If each unit of A earns $30 profit and each unit of B earns $20 profit, the objective function is:

\[ \boxed{P=30x+20y} \]

The company wants to maximise P.

However, production may be limited by available labour, materials, or machine time.

These limitations become constraints.

 

Representing Optimisation Problems Using Vectors

When an optimisation problem contains several decision variables, we can organise them into a vector.

For three variables:

\[ \boxed{ \mathbf{x}= \begin{bmatrix} x_1\\x_2\\x_3 \end{bmatrix} } \]

The objective can also be represented using a vector of coefficients:

\[ \mathbf{c}= \begin{bmatrix} c_1\\c_2\\c_3 \end{bmatrix} \]

For a linear objective function:

\[ P=c_1x_1+c_2x_2+c_3x_3 \]

we can write:

\[ \boxed{P=\mathbf{c}^{T}\mathbf{x}} \]

Here, cᵀ is the transpose of the coefficient vector.

This is the dot product of the coefficient vector and the decision-variable vector.

Worked Example 1: A Profit Model

A manufacturer produces three products.

Product

Profit per unit

A

$12

B

$18

C

$25

The company produces:

  • 40 units of A.

  • 30 units of B.

  • 20 units of C.

Represent the production and profit information using vectors, then calculate total profit.

The production vector is:

\[ \mathbf{x}= \begin{bmatrix} 40\\30\\20 \end{bmatrix} \]

The profit coefficient vector is:

\[ \mathbf{c}= \begin{bmatrix} 12\\18\\25 \end{bmatrix} \]

Total profit:

\[ P=\mathbf{c}^{T}\mathbf{x} \]
\[ P=12(40)+18(30)+25(20) \]
\[ =480+540+500 \]
\[ \boxed{P=\$1520} \]

This example demonstrates how a dot product can calculate an objective function involving several variables.

 

Representing Constraints Using Matrices

Suppose a factory produces three products using two limited resources.

Each product requires different quantities of labour and material.

Resource

Product A

Product B

Product C

Available

Labour hours

2

3

4

120

Material units

5

2

3

150

Let:

x₁ = number of units of A.

x₂ = number of units of B.

x₃ = number of units of C.

The labour constraint is:

\[ 2x_1+3x_2+4x_3\leq120 \]

The material constraint is:

\[ 5x_1+2x_2+3x_3\leq150 \]

These can be written as:

\[ \boxed{A\mathbf{x}\leq\mathbf{b}} \]

where:

\[ A= \begin{bmatrix} 2&3&4\\ 5&2&3 \end{bmatrix} \]
\[ \mathbf{x}= \begin{bmatrix} x_1\\x_2\\x_3 \end{bmatrix} \]
\[ \mathbf{b}= \begin{bmatrix} 120\\150 \end{bmatrix} \]

The inequality is interpreted component by component.

We also require:

\[ x_1\geq0,\quad x_2\geq0,\quad x_3\geq0 \]

because production quantities cannot be negative.

General Linear Optimisation Model

A common form of a linear programming problem is:

\[ \boxed{\text{Maximise }\mathbf{c}^{T}\mathbf{x}} \]

subject to:

\[ \boxed{A\mathbf{x}\leq\mathbf{b}} \]

and:

\[ \boxed{\mathbf{x}\geq\mathbf{0}} \]

This mathematical structure is widely used in operations research, manufacturing, transportation, logistics, and resource allocation.

 

Feasible Regions

The feasible region is the set of all solutions that satisfy every constraint.

In a two-variable problem, the feasible region can be represented graphically.

For example, consider:

\[ x+y\leq8 \]
\[ 2x+y\leq12 \]
\[ x\geq0,\quad y\geq0 \]

The first inequality describes points on or below the line:

x + y = 8

The second inequality describes points on or below:

2x + y = 12

The non-negativity constraints restrict the solution to the first quadrant.

 

Every point inside or on the boundary of the feasible region is an allowable solution.

Points outside the region violate at least one constraint.

 

Linear Programming

Linear programming is the process of optimising a linear objective function subject to linear constraints.

When there are two decision variables, many problems can be solved graphically.

An important result is that if a linear objective has a maximum or minimum over a nonempty, bounded polygonal feasible region, at least one optimal solution occurs at a vertex.

This is called the corner-point principle.

Worked Example 2: Maximising Factory Profit

A factory manufactures two products.

Product A earns $40 profit per unit.

Product B earns $30 profit per unit.

The constraints are:

\[ 2x+y\leq100 \]
\[ x+2y\leq80 \]
\[ x\geq0,\quad y\geq0 \]

Find the maximum possible profit.

Step 1: Write the objective function

\[ \boxed{P=40x+30y} \]

Step 2: Find the vertices of the feasible region

The vertices include:

(0, 0)

(50, 0)

(0, 40)

We must also find the intersection of:

2x + y = 100

x + 2y = 80

From the first equation:

y = 100 − 2x

Substitute into the second:

x + 2(100 − 2x) = 80

x + 200 − 4x = 80

−3x = −120

\[ x=40 \]

Then:

\[ y=100-2(40)=20 \]

The intersection is:

(40, 20)

Step 3: Evaluate the objective at each vertex

Vertex

Profit calculation

Profit

(0, 0)

40(0) + 30(0)

$0

(50, 0)

40(50) + 30(0)

$2,000

(0, 40)

40(0) + 30(40)

$1,200

(40, 20)

40(40) + 30(20)

$2,200

Step 4: Identify the maximum

The greatest profit is $2,200.

Therefore:

\[ \boxed{x=40,\quad y=20} \]
\[ \boxed{P_{\max}=\$2200} \]

The factory should produce 40 units of A and 20 units of B.

Step 5: Interpret the result

Both resource constraints are fully used:

\[ 2(40)+20=100 \]
\[ 40+2(20)=80 \]

This means the optimal solution uses all available quantities of both constrained resources.

 

Integer Programming

Some optimisation variables must be whole numbers.

For example:

  • Number of vehicles.

  • Number of employees.

  • Number of machines.

  • Number of products.

  • Number of classrooms.

A solution such as 3.7 vehicles is not meaningful.

An integer programming problem includes restrictions requiring selected variables to be integers.

Worked Example 3: Whole-Number Production

A workshop produces chairs and tables.

Let:

x = number of chairs.

y = number of tables.

The profit is:

\[ P=25x+40y \]

The constraints are:

\[ x+2y\leq9 \]
\[ 2x+y\leq10 \]
\[ x,y\geq0 \]

and x and y must be integers.

The intersection of the two boundary lines is:

x + 2y = 9

2x + y = 10

Multiply the first equation by 2:

2x + 4y = 18

Subtract the second:

3y = 8

\[ y=\frac83 \]

Then:

\[ x=9-2\left(\frac83\right)=\frac{11}{3} \]

The continuous intersection is:

\[ \left(\frac{11}{3},\frac83\right) \]

But fractional chairs and tables are not allowed.

We must examine feasible integer combinations.

Chairs

Tables

Profit

3

3

$195

4

2

$180

2

3

$170

1

4

$185

5

0

$125

0

4

$160

The highest profit among these feasible combinations is $195. To verify global optimality, note that for each integer y = 0, 1, 2, 3, 4, the maximum feasible x is respectively 5, 4, 4, 3, 1. Their best profits are $125, $140, $180, $195, and $185.

Therefore:

\[ \boxed{x=3,\quad y=3} \]
\[ \boxed{P_{\max}=\$195} \]

This illustrates why integer constraints can change an optimisation problem.

 

Optimisation with Several Variables

Many real problems involve more than two variables.

Suppose a delivery company uses three types of vehicles.

Let:

x₁ = number of small vehicles.

x₂ = number of medium vehicles.

x₃ = number of large vehicles.

Suppose operating costs are:

$80, $120, and $180 per vehicle.

The cost function is:

\[ C=80x_1+120x_2+180x_3 \]

In vector form:

\[ \boxed{C=\mathbf{c}^{T}\mathbf{x}} \]

where:

\[ \mathbf{c}= \begin{bmatrix} 80\\120\\180 \end{bmatrix} \]

The company may need to satisfy capacity requirements:

\[ 2x_1+4x_2+7x_3\geq50 \]

and a vehicle availability constraint:

\[ x_1+x_2+x_3\leq15 \]

The objective is to minimise cost.

This model is an integer linear programming problem if all vehicle counts must be whole numbers.

In higher-dimensional problems, graphical methods become impractical.

Algorithms such as the simplex method, interior-point methods, and branch-and-bound are used to solve appropriate optimisation problems.

 

Quadratic Optimisation

Not all optimisation problems have linear objective functions.

A quadratic objective function contains squared variables or products of variables.

For example:

\[ \boxed{f(x,y)=x^2+y^2} \]

This function measures the squared distance of a point (x, y) from the origin.

To minimise it without additional constraints:

\[ x=0,\quad y=0 \]

Therefore:

\[ \boxed{f_{\min}=0} \]

Matrix Form of a Quadratic Function

A quadratic function can be written using matrices:

\[ \boxed{f(\mathbf{x})=\mathbf{x}^{T}Q\mathbf{x}+\mathbf{c}^{T}\mathbf{x}+d} \]

where Q is a square matrix.

For example:

\[ Q= \begin{bmatrix} 2&0\\ 0&3 \end{bmatrix} \]

and:

\[ \mathbf{x}= \begin{bmatrix} x\\y \end{bmatrix} \]

Then:

\[ \mathbf{x}^{T}Q\mathbf{x}=2x^2+3y^2 \]

Quadratic optimisation is used in:

  • Engineering design.

  • Portfolio optimisation.

  • Control systems.

  • Statistical modelling.

  • Machine learning.

 

Optimisation Using Calculus

Calculus provides methods for finding maximum and minimum values of differentiable functions.

For a function of one variable:

\[ f(x) \]

stationary points occur where:

\[ \boxed{f'(x)=0} \]

These points may be maxima, minima, or neither.

Worked Example 4: Minimising a Cost Function

Suppose the cost of operating a machine is modelled by:

\[ C(x)=2x^2-40x+300 \]

where x represents an adjustable operating setting.

Find the value of x that minimises cost.

Step 1: Differentiate

\[ C'(x)=4x-40 \]

Step 2: Set the derivative equal to zero

\[ 4x-40=0 \]
\[ 4x=40 \]
\[ \boxed{x=10} \]

Step 3: Confirm a minimum

\[ C''(x)=4 \]

Since:

\[ C''(x)>0 \]

the function has a minimum at x = 10.

Step 4: Calculate the minimum cost

\[ C(10)=2(10)^2-40(10)+300 \]
\[ =200-400+300 \]
\[ \boxed{C_{\min}=100} \]

The minimum cost is 100 cost units at operating setting x = 10.

 

Optimisation with Multiple Variables

For a function involving several variables:

\[ f(x,y) \]

we use partial derivatives.

The gradient is:

\[ \boxed{ \nabla f= \begin{bmatrix} \frac{\partial f}{\partial x}\\ \frac{\partial f}{\partial y} \end{bmatrix} } \]

At an interior stationary point of a differentiable function:

\[ \boxed{\nabla f=\mathbf{0}} \]

Worked Example 5: Minimising a Two-Variable Function

Find the minimum of:

\[ f(x,y)=x^2+y^2-4x-6y+20 \]

Step 1: Calculate partial derivatives

\[ \frac{\partial f}{\partial x}=2x-4 \]
\[ \frac{\partial f}{\partial y}=2y-6 \]

Step 2: Set both equal to zero

\[ 2x-4=0 \]
\[ x=2 \]
\[ 2y-6=0 \]
\[ y=3 \]

Step 3: Evaluate the function

\[ f(2,3)=2^2+3^2-4(2)-6(3)+20 \]
\[ =4+9-8-18+20 \]
\[ \boxed{f(2,3)=7} \]

Step 4: Confirm the minimum

Complete the square:

\[ f(x,y)=(x-2)^2+(y-3)^2+7 \]

Since squared terms cannot be negative:

\[ \boxed{f_{\min}=7} \]

at:

\[ \boxed{(x,y)=(2,3)} \]

This is the global minimum.

 

The Hessian Matrix

For functions involving multiple variables, second derivatives can be organised into a matrix called the Hessian.

For f(x, y):

\[ \boxed{ H= \begin{bmatrix} \frac{\partial^2 f}{\partial x^2}&\frac{\partial^2 f}{\partial x\partial y}\\ \frac{\partial^2 f}{\partial y\partial x}&\frac{\partial^2 f}{\partial y^2} \end{bmatrix} } \]

For:

\[ f(x,y)=x^2+y^2-4x-6y+20 \]

the Hessian is:

\[ H= \begin{bmatrix} 2&0\\ 0&2 \end{bmatrix} \]

This matrix is positive definite.

Therefore, the stationary point is a strict local minimum.

Because the function is a convex quadratic, it is also the global minimum.

Interpreting the Hessian

At a stationary point:

  • A positive definite Hessian indicates a strict local minimum.

  • A negative definite Hessian indicates a strict local maximum.

  • An indefinite Hessian indicates a saddle point.

  • A semidefinite Hessian may require further analysis.

The Hessian connects multivariable calculus directly to matrix algebra.

 

Least-Squares Optimisation

A major application of optimisation is finding a mathematical model that fits observed data.

Suppose a linear model is:

\[ \hat y=mx+b \]

We want to choose m and b so that predictions are close to the actual observations.

The least-squares objective is:

\[ \boxed{SSE=\sum_{i=1}^{n}(y_i-\hat y_i)^2} \]

The goal is to minimise SSE.

Matrix Representation

A linear model can be written as:

\[ \boxed{A\boldsymbol{\beta}\approx\mathbf{y}} \]

where:

  • A is the design matrix.

  • β is the vector of model parameters.

  • y is the vector of observed values.

The least-squares problem is:

\[ \boxed{\min_{\boldsymbol{\beta}}\|A\boldsymbol{\beta}-\mathbf{y}\|^2} \]

When A has linearly independent columns, the unique least-squares solution satisfies:

\[ \boxed{A^TA\boldsymbol{\beta}=A^T\mathbf{y}} \]

These are called the normal equations.

If AᵀA is invertible:

\[ \boxed{\boldsymbol{\beta}=(A^TA)^{-1}A^T\mathbf{y}} \]

In practical numerical computing, QR factorisation or singular value decomposition is often preferred to explicitly forming this inverse.

Worked Example 6: Fitting a Line

Suppose the observations are:

x

y

1

2

2

3

3

5

Find the least-squares regression line.

We use:

\[ \hat y=mx+b \]

The means are:

\[ \bar x=\frac{1+2+3}{3}=2 \]
\[ \bar y=\frac{2+3+5}{3}=\frac{10}{3} \]

The slope is:

\[ m=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2} \]

Numerator:

\[ (1-2)\left(2-\frac{10}{3}\right) +(2-2)\left(3-\frac{10}{3}\right) +(3-2)\left(5-\frac{10}{3}\right) \]
\[ =\frac43+0+\frac53=3 \]

Denominator:

\[ (1-2)^2+(2-2)^2+(3-2)^2=2 \]

Therefore:

\[ m=\frac32=1.5 \]

The intercept is:

\[ b=\bar y-m\bar x \]
\[ =\frac{10}{3}-1.5(2) \]
\[ =\frac{10}{3}-3=\frac13 \]

The regression equation is:

\[ \boxed{\hat y=1.5x+\frac13} \]

This line minimises the sum of squared vertical residuals for the observations.

 

Gradient Descent

Some optimisation problems are too large or complicated to solve directly.

Gradient descent is an iterative method that adjusts variables in the direction opposite to the gradient.

The update rule is:

\[ \boxed{\mathbf{x}_{k+1}=\mathbf{x}_k-\alpha\nabla f(\mathbf{x}_k)} \]

where:

  • xₖ is the current estimate.

  • α is the learning rate or step size.

  • ∇f is the gradient.

The gradient points in the direction of steepest local increase, so moving against it can reduce the objective function.

Worked Example 7: Gradient Descent

Consider:

\[ f(x)=x^2 \]

The derivative is:

\[ f'(x)=2x \]

Start with:

x₀ = 5

Use learning rate:

α = 0.1

The update rule is:

\[ x_{k+1}=x_k-0.1(2x_k) \]
\[ =0.8x_k \]

First iteration:

\[ x_1=0.8(5)=4 \]

Second iteration:

\[ x_2=0.8(4)=3.2 \]

Third iteration:

\[ x_3=0.8(3.2)=2.56 \]

The values move toward zero, where f(x) reaches its minimum.

Choosing a Learning Rate

If the learning rate is too small, convergence may be slow.

If it is too large, the algorithm may overshoot the minimum or fail to converge.

Gradient descent is widely used in machine learning, optimisation, and scientific computing.

 

Constrained Optimisation Using Lagrange Multipliers

Some optimisation problems involve equality constraints.

For example, we may want to maximise a function:

\[ f(x,y) \]

subject to:

\[ g(x,y)=c \]

The method of Lagrange multipliers introduces a new variable λ.

At a suitable constrained optimum:

\[ \boxed{\nabla f=\lambda\nabla g} \]

together with:

\[ g(x,y)=c \]

Worked Example 8: Maximum Product with Fixed Sum

Find the maximum value of:

\[ f(x,y)=xy \]

subject to:

\[ x+y=10 \]

with x, y ≥ 0.

The gradients are:

\[ \nabla f= \begin{bmatrix} y\\x \end{bmatrix} \]
\[ \nabla g= \begin{bmatrix} 1\\1 \end{bmatrix} \]

Set:

\[ \nabla f=\lambda\nabla g \]

Therefore:

y = λ

x = λ

Hence:

x = y

Using the constraint:

x + y = 10

2x = 10

x = 5

y = 5

Therefore:

\[ \boxed{f_{\max}=5(5)=25} \]

We can verify this result using substitution:

\[ y=10-x \]
\[ f(x)=x(10-x)=10x-x^2 \]

This is a downward-opening parabola with maximum at x = 5.

The maximum product occurs when the two nonnegative numbers are equal.

 

Mathematical Modelling in Engineering

Engineering optimisation often involves balancing competing requirements.

For example, a structural engineer may want to minimise the mass of a component while ensuring that it can support a specified load.

A general engineering optimisation model might be:

\[ \boxed{\min f(\mathbf{x})} \]

subject to:

\[ g_i(\mathbf{x})\leq0 \]

and:

\[ h_j(\mathbf{x})=0 \]

where the decision vector x may contain dimensions, material quantities, or operating parameters.

The constraints may represent:

  • Maximum stress.

  • Minimum strength.

  • Maximum displacement.

  • Available materials.

  • Safety requirements.

  • Manufacturing limitations.

Example: Minimising Material Cost

Suppose a rectangular container has a fixed volume:

\[ V=lwh \]

The surface area of a closed rectangular container is:

\[ S=2lw+2lh+2wh \]

If material cost is proportional to surface area, minimising material cost means minimising S.

The constraint is:

\[ lwh=V \]

This is a nonlinear constrained optimisation problem.

For a closed rectangular box with a fixed positive volume and no other restrictions, the minimum surface area occurs when:

\[ \boxed{l=w=h} \]

The optimal shape is a cube.

This result illustrates how mathematical modelling can guide engineering design.

 

Optimisation in Transportation and Logistics

Transport companies frequently solve optimisation problems involving:

  • Vehicle capacity.

  • Delivery demand.

  • Fuel consumption.

  • Travel time.

  • Driver availability.

  • Storage limitations.

Worked Example 9: A Transportation Model

A supplier has two warehouses and two stores.

Warehouse A can supply 40 units.

Warehouse B can supply 60 units.

Store 1 requires 50 units.

Store 2 requires 50 units.

Shipping costs per unit are:

Route

Store 1

Store 2

Warehouse A

$4

$6

Warehouse B

$5

$3

Let:

x₁₁ = units shipped from A to Store 1.

x₁₂ = units shipped from A to Store 2.

x₂₁ = units shipped from B to Store 1.

x₂₂ = units shipped from B to Store 2.

The objective is:

\[ \boxed{C=4x_{11}+6x_{12}+5x_{21}+3x_{22}} \]

Constraints:

Warehouse A:

\[ x_{11}+x_{12}=40 \]

Warehouse B:

\[ x_{21}+x_{22}=60 \]

Store 1:

\[ x_{11}+x_{21}=50 \]

Store 2:

\[ x_{12}+x_{22}=50 \]

All variables must be nonnegative.

Step 1: Express all variables using one variable

Let:

\[ x_{11}=t \]

Then:

\[ x_{12}=40-t \]
\[ x_{21}=50-t \]
\[ x_{22}=10+t \]

Non-negativity requires:

\[ 0\leq t\leq40 \]

Step 2: Substitute into the cost function

\[ C=4t+6(40-t)+5(50-t)+3(10+t) \]

Expand:

\[ C=4t+240-6t+250-5t+30+3t \]

Simplify:

\[ \boxed{C=520-4t} \]

Step 3: Minimise cost

Since the coefficient of t is negative, cost decreases as t increases.

Therefore, choose the largest feasible value:

\[ t=40 \]

Step 4: Calculate shipment quantities

\[ x_{11}=40 \]
\[ x_{12}=0 \]
\[ x_{21}=10 \]
\[ x_{22}=50 \]

Step 5: Calculate minimum cost

\[ C=4(40)+6(0)+5(10)+3(50) \]
\[ =160+0+50+150 \]
\[ \boxed{C_{\min}=\$360} \]

The optimal shipping plan is:

  • 40 units from Warehouse A to Store 1.

  • 10 units from Warehouse B to Store 1.

  • 50 units from Warehouse B to Store 2.

 

Multi-Objective Optimisation

Some problems have more than one objective.

For example, a company may want to:

  • Minimise cost.

  • Maximise quality.

  • Minimise environmental impact.

  • Maximise customer satisfaction.

These objectives may conflict.

A cheaper solution may produce lower quality.

A more environmentally sustainable solution may have higher initial costs.

Weighted Objective Functions

One approach combines objectives using weights.

Suppose:

C = cost.

E = environmental impact.

We may minimise:

\[ \boxed{F=\alpha C+\beta E} \]

where α and β represent the relative importance of each objective.

The quantities should be scaled appropriately before combining them, especially when measured in different units.

Pareto Optimality

A solution is Pareto optimal if no objective can be improved without making at least one other objective worse.

Rather than producing a single best solution, multi-objective optimisation may produce a collection of reasonable trade-offs.

This collection is called the Pareto frontier.

 

Evaluating Mathematical Models

A mathematically correct solution is not necessarily a useful real-world solution.

A model must also be evaluated in terms of its assumptions, limitations, and practical meaning.

Important Questions

1. Are the assumptions reasonable?

For example, does the model assume constant costs when costs actually change with production volume?

2. Are the variables clearly defined?

Every variable should have a precise interpretation and appropriate units.

3. Are all important constraints included?

A production model may be unrealistic if it ignores storage, labour availability, or maintenance.

4. Is the objective appropriate?

Maximising short-term profit may not be the same as maximising long-term sustainability.

5. Are the data reliable?

Incorrect estimates of cost, demand, or capacity may lead to poor decisions.

6. Is the solution practical?

A mathematically optimal solution may require fractional machines, impossible schedules, or unavailable resources.

7. How sensitive is the solution?

Would a small change in input data produce a very different answer?

 

Sensitivity Analysis

Sensitivity analysis examines how changes in model parameters affect the optimal solution.

Suppose a company maximises:

\[ P=40x+30y \]

If the profit per unit of Product B increases from $30 to $50, the objective becomes:

\[ P=40x+50y \]

The best production combination may change.

Worked Example 10: Sensitivity of an Optimal Solution

Return to the feasible vertices from Worked Example 2:

(0, 0), (50, 0), (0, 40), and (40, 20).

The new profit function is:

\[ P=40x+50y \]

Evaluate:

Vertex

New profit

(0, 0)

$0

(50, 0)

$2,000

(0, 40)

$2,000

(40, 20)

$2,600

Therefore:

\[ \boxed{P_{\max}=\$2600} \]

The optimal production quantities remain:

\[ \boxed{x=40,\quad y=20} \]

Although the profit increased, the optimal production combination did not change.

This demonstrates that some optimisation solutions are stable under certain changes in model parameters.

 

Choosing an Appropriate Optimisation Method

Different problems require different mathematical techniques.

Problem type

Common method

Linear objective, linear constraints

Linear programming

Whole-number decision variables

Integer programming

Smooth one-variable function

Differentiation

Smooth multivariable function

Partial derivatives and gradients

Quadratic objective with suitable constraints

Quadratic programming

Equality-constrained differentiable problem

Lagrange multipliers

Fitting a linear model to data

Least squares

Large differentiable optimisation problem

Gradient-based methods

Several competing objectives

Multi-objective optimisation

Complex discrete routing or scheduling

Integer optimisation or specialised algorithms

The best method depends on the mathematical structure of the problem.

 

Real-World Applications

Manufacturing and Production

Optimisation helps determine production quantities, resource allocation, staffing, and operating schedules to reduce costs or maximise output.

 

Transportation and Logistics

Mathematical models determine efficient shipping plans, delivery schedules, and vehicle assignments while satisfying demand and capacity constraints.

 

Engineering Design

Engineers optimise structures and systems to reduce weight, cost, and energy consumption while maintaining strength, safety, and performance.

 

Artificial Intelligence

Machine learning algorithms optimise model parameters to reduce prediction errors and improve performance on relevant tasks.

 

Energy Systems

Optimisation helps balance energy production, storage, demand, and operating costs in electrical networks and renewable energy systems.

 

Common Mistakes

Mistake 1: Confusing decision variables with fixed parameters

Decision variables are quantities we choose. Parameters are values supplied by the model.

Mistake 2: Writing an objective function without defining the variables

Every variable must have a clear interpretation.

Mistake 3: Forgetting constraints

An unconstrained maximum may be unrealistic or unbounded.

Mistake 4: Using the wrong inequality direction

For example, a maximum resource limit requires ≤, while a minimum demand requirement often requires ≥.

Mistake 5: Forgetting non-negativity restrictions

Negative production quantities or vehicle counts are usually not meaningful.

Mistake 6: Assuming every feasible solution is optimal

A feasible solution satisfies the constraints, but it may not maximise or minimise the objective.

Mistake 7: Assuming the optimum always occurs at a corner

The corner-point principle applies to linear objectives over appropriate polyhedral feasible regions. Nonlinear objectives may have interior optima.

Mistake 8: Ignoring integer restrictions

A fractional solution may not be practical when variables represent indivisible objects.

Mistake 9: Confusing local and global optima

A local optimum is best within a nearby region. A global optimum is best over the entire feasible set.

Mistake 10: Assuming every stationary point is a minimum

Stationary points may be minima, maxima, or saddle points.

Mistake 11: Applying matrix formulas without checking dimensions

Matrix multiplication requires compatible dimensions.

Mistake 12: Assuming the inverse of AᵀA always exists

The inverse requires the columns of A to be linearly independent.

Mistake 13: Choosing a learning rate without considering convergence

An inappropriate step size can slow or destabilise gradient descent.

Mistake 14: Ignoring model assumptions

A mathematically valid answer may not represent real conditions accurately.

Mistake 15: Treating an optimal solution as permanently optimal

Changes in prices, demand, resource availability, or constraints may change the best solution.

 

Key Terms

  • Optimisation: Process of finding the best value of an objective under specified conditions.

  • Mathematical Model: Mathematical representation of a real-world system.

  • Decision Variable: Quantity selected or controlled in an optimisation problem.

  • Parameter: Fixed numerical value used in a model.

  • Objective Function: Function to be maximised or minimised.

  • Constraint: Restriction on allowable solutions.

  • Feasible Solution: Solution satisfying every constraint.

  • Feasible Region: Set of all feasible solutions.

  • Optimal Solution: Feasible solution producing the best objective value.

  • Linear Programming: Optimisation involving linear objectives and constraints.

  • Integer Programming: Optimisation requiring some or all decision variables to be integers.

  • Corner-Point Principle: Property that a bounded linear programming problem with an optimum has an optimal vertex solution.

  • Dot Product: Scalar obtained by multiplying and summing corresponding vector components.

  • Constraint Matrix: Matrix containing coefficients of linear constraints.

  • Quadratic Optimisation: Optimisation involving quadratic objective functions.

  • Gradient: Vector of first-order partial derivatives.

  • Stationary Point: Point where the gradient is zero, when the gradient exists.

  • Hessian Matrix: Matrix of second-order partial derivatives.

  • Positive Definite Matrix: Symmetric matrix for which xᵀAx is positive for every nonzero vector x.

  • Convex Function: Function whose graph lies below or on the line segment connecting any two points on its graph.

  • Local Minimum: Minimum within a neighbourhood of a point.

  • Global Minimum: Smallest function value over the entire feasible domain.

  • Least Squares: Method minimising the sum of squared residuals.

  • Residual: Difference between an observed value and a predicted value.

  • Design Matrix: Matrix representing predictor variables in a statistical model.

  • Gradient Descent: Iterative optimisation method using the negative gradient.

  • Learning Rate: Step size used in gradient-based optimisation.

  • Lagrange Multiplier: Auxiliary variable used to analyse constrained optima.

  • Sensitivity Analysis: Study of how changes in parameters affect model results.

  • Multi-Objective Optimisation: Optimisation involving several objectives.

  • Pareto Optimality: Condition where improving one objective necessarily worsens another.

  • Pareto Frontier: Set of non-dominated trade-off solutions.

  • Model Validation: Process of checking whether a model adequately represents the intended system.

 

Key Takeaways

  • Optimisation involves maximising or minimising an objective while satisfying constraints.

  • Decision variables can be organised into vectors.

  • Linear objective functions can be expressed using dot products.

  • Systems of linear constraints can be represented using matrices.

  • The feasible region contains every solution satisfying all constraints.

  • Linear programming optimises linear objectives subject to linear constraints.

  • For suitable two-variable linear programming problems, optimal solutions can be found by evaluating vertices.

  • Integer programming is necessary when some variables must represent whole-number quantities.

  • Quadratic optimisation involves squared variables and can be represented using matrix expressions.

  • Differentiation and partial derivatives help identify stationary points.

  • The Hessian matrix helps classify stationary points of multivariable functions.

  • Least-squares optimisation is used to fit mathematical models to observed data.

  • Gradient descent provides an iterative method for minimising differentiable objective functions.

  • Lagrange multipliers help solve equality-constrained optimisation problems.

  • Transportation and production problems can be modelled using matrices and linear constraints.

  • Multi-objective optimisation considers trade-offs between competing goals.

  • Sensitivity analysis reveals how model results respond to changing assumptions.

  • A mathematically optimal solution must still be evaluated for realism, feasibility, and usefulness.

 

Check Your Understanding

Questions

Question 1: Objective Functions

A company produces two products.

Product A earns $15 profit per unit.

Product B earns $25 profit per unit.

Write an objective function for total profit.

Question 2: Vector Representation

A manufacturer produces three products with profits of $10, $20, and $30 per unit.

Write the profit coefficient vector.

Question 3: Dot Product

Given:

\[ \mathbf{c}= \begin{bmatrix} 5\\8\\12 \end{bmatrix} \]

and:

\[ \mathbf{x}= \begin{bmatrix} 10\\6\\4 \end{bmatrix} \]

Calculate cᵀx.

Question 4: Matrix Constraints

Write the following system in matrix form:

\[ 2x+3y\leq60 \]
\[ 4x+y\leq80 \]

Question 5: Feasibility

Determine whether the point (5, 10) satisfies:

\[ 2x+y\leq25 \]
\[ x+3y\leq40 \]

Question 6: Linear Programming

Maximise:

\[ P=5x+4y \]

subject to:

\[ x+y\leq10 \]
\[ x\leq6 \]
\[ y\leq8 \]
\[ x,y\geq0 \]

Find the maximum value of P.

Question 7: Integer Programming

Explain why a company purchasing delivery trucks may require an integer programming model.

Question 8: Quadratic Optimisation

Find the minimum value of:

\[ f(x)=x^2-8x+20 \]

Question 9: Multivariable Optimisation

Find the minimum of:

\[ f(x,y)=x^2+y^2-6x-4y+15 \]

Question 10: Gradient Descent

For:

\[ f(x)=x^2 \]

start at x = 4 with learning rate α = 0.25.

Calculate the next two iterations.

Question 11: Least Squares

Actual values are:

10, 15, 20

Predicted values are:

12, 14, 19

Calculate the sum of squared errors.

Question 12: Transportation Costs

A company ships:

  • 20 units at $4 per unit.

  • 30 units at $6 per unit.

  • 10 units at $8 per unit.

Calculate total shipping cost using a dot product.

Question 13: Sensitivity Analysis

A product earns $20 profit per unit.

The company produces 50 units.

Calculate total profit.

Then calculate the new profit if profit per unit increases to $24.

Question 14: Model Evaluation

Explain three reasons why a mathematically optimal solution might not be suitable in practice.

Question 15: Integrated Optimisation Challenge

A company produces two products, A and B.

Profit per unit:

A: $50

B: $40

Production constraints:

\[ 2x+y\leq100 \]
\[ x+2y\leq80 \]
\[ x,y\geq0 \]

a) Define the decision variables.

b) Write the objective function.

c) Express the constraints in matrix form.

d) Find the vertices of the feasible region.

e) Calculate profit at each vertex.

f) Determine the optimal production quantities.

g) Calculate maximum profit.

h) Explain two assumptions of the model.

i) Explain how the model could be improved.

 

Fully Worked Answers

Answer 1: Objective Functions

Let:

x = number of units of A.

y = number of units of B.

Profit from A:

15x

Profit from B:

25y

Therefore:

\[ \boxed{P=15x+25y} \]

The objective is to maximise P.

Answer 2: Vector Representation

The profit coefficient vector is:

\[ \boxed{ \mathbf{c}= \begin{bmatrix} 10\\20\\30 \end{bmatrix} } \]

Answer 3: Dot Product

\[ \mathbf{c}^{T}\mathbf{x}=5(10)+8(6)+12(4) \]
\[ =50+48+48 \]
\[ \boxed{\mathbf{c}^{T}\mathbf{x}=146} \]

Answer 4: Matrix Constraints

The system is:

\[ 2x+3y\leq60 \]
\[ 4x+y\leq80 \]

Therefore:

\[ \boxed{ \begin{bmatrix} 2&3\\ 4&1 \end{bmatrix} \begin{bmatrix} x\\y \end{bmatrix} \leq \begin{bmatrix} 60\\80 \end{bmatrix} } \]

Answer 5: Feasibility

Substitute:

x = 5

y = 10

First constraint:

\[ 2(5)+10=20 \]

Since:

20 ≤ 25

the first constraint is satisfied.

Second constraint:

\[ 5+3(10)=35 \]

Since:

35 ≤ 40

the second constraint is satisfied.

Therefore:

\[ \boxed{(5,10)\text{ is feasible.}} \]

Answer 6: Linear Programming

Objective:

\[ P=5x+4y \]

Constraints:

\[ x+y\leq10,\quad x\leq6,\quad y\leq8 \]

The vertices are:

(0, 0)

(6, 0)

(6, 4)

(2, 8)

(0, 8)

Evaluate:

Vertex

P

(0, 0)

0

(6, 0)

30

(6, 4)

46

(2, 8)

42

(0, 8)

32

Therefore:

\[ \boxed{P_{\max}=46} \]

at:

\[ \boxed{(x,y)=(6,4)} \]

Answer 7: Integer Programming

Delivery trucks cannot normally be purchased in fractional quantities.

A mathematical solution such as 4.6 trucks is not physically meaningful.

Therefore, the number of trucks must be restricted to nonnegative integers.

An integer programming model includes these restrictions.

Answer 8: Quadratic Optimisation

Given:

\[ f(x)=x^2-8x+20 \]

Differentiate:

\[ f'(x)=2x-8 \]

Set equal to zero:

\[ 2x-8=0 \]
\[ x=4 \]

Evaluate:

\[ f(4)=16-32+20 \]
\[ \boxed{f_{\min}=4} \]

Since the coefficient of x² is positive, the parabola opens upward and this is the global minimum.

Answer 9: Multivariable Optimisation

Given:

\[ f(x,y)=x^2+y^2-6x-4y+15 \]

Partial derivatives:

\[ \frac{\partial f}{\partial x}=2x-6 \]
\[ \frac{\partial f}{\partial y}=2y-4 \]

Set equal to zero:

\[ 2x-6=0\Rightarrow x=3 \]
\[ 2y-4=0\Rightarrow y=2 \]

Evaluate:

\[ f(3,2)=9+4-18-8+15 \]
\[ \boxed{f(3,2)=2} \]

Complete the square:

\[ f(x,y)=(x-3)^2+(y-2)^2+2 \]

Therefore:

\[ \boxed{f_{\min}=2} \]

at:

\[ \boxed{(3,2)} \]

Answer 10: Gradient Descent

Given:

\[ f(x)=x^2 \]

Derivative:

\[ f'(x)=2x \]

Update rule:

\[ x_{k+1}=x_k-0.25(2x_k) \]
\[ =0.5x_k \]

Start:

x₀ = 4

First iteration:

\[ x_1=0.5(4)=2 \]

Second iteration:

\[ x_2=0.5(2)=1 \]

Therefore:

\[ \boxed{x_1=2,\quad x_2=1} \]

Answer 11: Least Squares

Residuals:

10 − 12 = −2

15 − 14 = 1

20 − 19 = 1

Square:

4, 1, 1

Add:

\[ SSE=4+1+1 \]
\[ \boxed{SSE=6} \]

Answer 12: Transportation Costs

Cost vector:

\[ \mathbf{c}= \begin{bmatrix} 4\\6\\8 \end{bmatrix} \]

Quantity vector:

\[ \mathbf{x}= \begin{bmatrix} 20\\30\\10 \end{bmatrix} \]

Total cost:

\[ C=\mathbf{c}^{T}\mathbf{x} \]
\[ =4(20)+6(30)+8(10) \]
\[ =80+180+80 \]
\[ \boxed{C=\$340} \]

Answer 13: Sensitivity Analysis

Original profit:

\[ P=20(50) \]
\[ \boxed{P=\$1000} \]

New profit:

\[ P=24(50) \]
\[ \boxed{P=\$1200} \]

Increase:

\[ 1200-1000=200 \]

The total profit increases by $200, assuming the production quantity remains unchanged.

Answer 14: Model Evaluation

Three possible reasons are:

Unrealistic assumptions: The model may assume constant costs or unlimited demand when these conditions do not hold.

Missing constraints: The model may ignore labour availability, safety requirements, storage capacity, or environmental regulations.

Practical limitations: The solution may require fractional quantities, unavailable equipment, or unrealistic operating schedules.

A useful optimisation model must represent the important features of the real system.

Answer 15: Integrated Optimisation Challenge

Part a: Define variables

Let:

x = number of units of Product A.

y = number of units of Product B.

Part b: Objective function

\[ \boxed{P=50x+40y} \]

The objective is to maximise P.

Part c: Matrix constraints

\[ \boxed{ \begin{bmatrix} 2&1\\ 1&2 \end{bmatrix} \begin{bmatrix} x\\y \end{bmatrix} \leq \begin{bmatrix} 100\\80 \end{bmatrix} } \]

with:

\[ x,y\geq0 \]

Part d: Find the vertices

On the x-axis:

2x ≤ 100

Therefore:

x ≤ 50

Vertex:

(50, 0)

On the y-axis:

2y ≤ 80

Therefore:

y ≤ 40

Vertex:

(0, 40)

The origin is:

(0, 0)

Now solve:

2x + y = 100

x + 2y = 80

From the first equation:

y = 100 − 2x

Substitute:

x + 2(100 − 2x) = 80

x + 200 − 4x = 80

−3x = −120

\[ x=40 \]

Then:

\[ y=100-2(40)=20 \]

Intersection:

(40, 20)

Therefore, the vertices are:

(0, 0), (50, 0), (0, 40), (40, 20).

Part e: Evaluate profit

Vertex

Profit

(0, 0)

$0

(50, 0)

$2,500

(0, 40)

$1,600

(40, 20)

$2,800

Part f: Optimal production

\[ \boxed{x=40,\quad y=20} \]

Part g: Maximum profit

\[ P=50(40)+40(20) \]
\[ =2000+800 \]
\[ \boxed{P_{\max}=\$2800} \]

Part h: Two assumptions

The model assumes that profit per unit remains constant regardless of production quantity.

It also assumes that the two resource constraints represent the important limitations on production.

Part i: Improvements

The model could be improved by including additional constraints such as labour availability, storage, customer demand, and machine maintenance.

It could also include changing production costs, uncertain demand, and sensitivity analysis.

These improvements would make the model more realistic and useful for decision-making.

 

Final Reflection

Optimisation and mathematical modelling connect abstract mathematical ideas with practical decision-making.

Vectors provide an organised way to represent decision variables. Matrices allow many constraints to be written and analysed efficiently. Linear programming identifies optimal solutions under linear restrictions, while calculus and quadratic optimisation extend these ideas to nonlinear relationships.

Least-squares methods and gradient descent demonstrate how optimisation is used to construct mathematical models from data. Engineering, transportation, economics, and artificial intelligence all rely on these techniques.

However, finding a mathematical optimum is only one part of solving a real-world problem. A successful model must also use realistic assumptions, appropriate constraints, reliable data, and meaningful interpretations.

The central idea is that optimisation uses mathematical models to identify the best available decisions while respecting the limitations and requirements of real systems.