Module 4: Algebraic Systems and Numerical Solvers
Learn how discretized CFD equations become systems of algebraic equations. Explore matrix representations, boundary conditions, direct and iterative solvers, residuals, convergence, conditioning, and how to implement the foundations of a CFD solver in C++.
Introduction
In Module 3, we learned how to convert continuous differential equations into discrete algebraic equations using finite differences. We approximated derivatives, studied truncation error, and derived a discrete form of the one-dimensional heat-conduction equation.
But obtaining a discrete equation is only part of the problem. A practical CFD simulation must solve thousands, millions, or even billions of coupled algebraic equations to determine the unknown pressure, velocity, temperature, and other physical quantities.
This module explains what happens between discretizing a governing equation and obtaining a numerical solution.
We will start with a simple one-dimensional heat-conduction problem, construct its algebraic equations, express them in matrix form, and examine different numerical methods for solving them. We will also learn how boundary conditions enter the system, what residuals mean, why convergence matters, and how these concepts translate into C++ code.
The objective is to understand the numerical machinery behind a CFD solver rather than treating a commercial solver as a black box.
Learning Objectives
By the end of this module, you should be able to:
- Explain how discretized differential equations form systems of algebraic equations.
- Represent a numerical problem using matrix notation, .
- Derive a matrix system for one-dimensional steady heat conduction.
- Distinguish direct solvers from iterative solvers.
- Understand Jacobi, Gauss–Seidel, and successive over-relaxation methods.
- Explain residuals, convergence criteria, and stopping conditions.
- Recognize the importance of matrix sparsity, diagonal dominance, and conditioning.
- Implement a basic iterative solver in C++.
- Understand how numerical linear algebra fits into a larger CFD algorithm.
1. From Discretized Equations to an Algebraic System
Consider the steady, one-dimensional heat-conduction equation in a homogeneous solid with constant thermal conductivity and no internal heat generation:
As discussed in Module 3, using the central finite-difference approximation gives
Multiplying both sides by gives
Rearranging,
Equivalently,
These equations state that the temperature at an interior grid point equals the average of its two neighboring temperatures.
However, each equation contains multiple unknown temperatures. We must solve all the equations together to obtain the temperature distribution.
A simple numerical example
Suppose a one-dimensional wall has:
- Length: .
- Left boundary temperature: .
- Right boundary temperature: .
- Four equally spaced intervals, giving .
There are five grid points:
The boundary temperatures are known:
The unknowns are the three interior temperatures:
Applying the discrete equation at each interior point gives
We now have three equations and three unknowns. This is a linear algebra problem.
Solving the equations gives
The temperature decreases linearly from the hot wall to the cold wall, as expected from the physics.
The important idea is that a differential equation has been transformed into a finite collection of algebraic equations that a computer can solve.
2. Matrix Representation: The Form Ax = b
Writing individual equations is manageable for three unknowns. For a real CFD mesh, there may be millions of unknowns, so we need a systematic representation.
A linear system can be written as
where:
- is the coefficient matrix.
- is the vector of unknowns.
- is the right-hand-side vector.
For our heat-conduction example,
The matrix contains the coefficients associated with each unknown.
The temperature vector contains the unknown interior temperatures, and the right-hand-side vector contains contributions from the prescribed boundary temperatures.
What does each row represent?
Each row corresponds to one discrete equation.
The first row,
represents the equation at grid point 1.
The second row,
represents the equation at grid point 2.
The third row,
represents the equation at grid point 3.
Each row connects one unknown to its neighbors. This local connectivity is fundamental to the structure of CFD matrices.
General matrix notation
For unknowns, the system can be expressed as
In a CFD problem, the unknown vector might represent temperature, pressure, one velocity component, or a collection of coupled variables.
For example, a simple thermal problem may have one temperature unknown per cell. A more complex incompressible-flow solver may need to handle pressure and multiple velocity components, with additional equations connecting these quantities.
3. How Boundary Conditions Enter the Matrix
Boundary conditions are not merely settings supplied to the solver. They influence the algebraic equations and therefore the numerical solution.
Consider the first interior equation:
Since the left boundary temperature is known,
we substitute the boundary value into the equation:
The boundary temperature becomes a contribution to the right-hand side.
Similarly, at the last interior point,
for a grid whose interior unknowns are indexed from 1 to .
This is one common way to incorporate prescribed-temperature, or Dirichlet, boundary conditions into a matrix system.
Dirichlet boundary condition
A Dirichlet condition prescribes the value of the unknown at a boundary.
For temperature,
where is the specified wall temperature.
Examples include a wall maintained at or a cooling surface maintained at .
Neumann boundary condition
A Neumann condition prescribes a derivative or flux.
For example, a prescribed temperature gradient is
where denotes the direction normal to the boundary.
For a stationary solid with Fourier heat conduction,
so specifying heat flux is closely related to specifying a normal temperature gradient.
An insulated wall is often represented by zero normal heat flux:
For constant, nonzero thermal conductivity, this implies
A Neumann condition usually modifies the discrete equation at the boundary rather than simply inserting a known temperature into the right-hand side.
Why this matters in CFD
An incorrect boundary treatment can produce a plausible-looking but physically incorrect result.
For example, accidentally prescribing a temperature where a heat flux was intended changes the physical problem. Similarly, incorrectly handling a velocity boundary can alter the flow rate or disturb mass conservation.
Boundary conditions must therefore be consistent with both the physics and the discrete equations.
4. Understanding Matrix Structure
Not all matrices are equally expensive to solve. The pattern of nonzero coefficients is particularly important in CFD.
Dense matrices
A dense matrix contains nonzero values in most or all positions.
For a system with unknowns, a dense matrix stores approximately coefficients.
If , the matrix contains entries. Storing those entries as 64-bit floating-point numbers requires approximately 800 MB for the matrix alone.
For large CFD simulations, this can become prohibitively expensive.
Sparse matrices
A sparse matrix contains mostly zeros.
In the one-dimensional heat-conduction problem, each interior equation connects only to the previous point, the current point, and the next point. The matrix is therefore tridiagonal:
Each row contains only a small number of nonzero entries.
A three-dimensional finite-volume cell usually interacts directly with a limited number of neighboring cells. For a simple structured mesh, that might mean six face-neighbors plus the cell itself. More general meshes have varying numbers of neighbors.
This local interaction creates sparse systems.
Why sparsity matters
Sparse storage avoids wasting memory on zeros. Sparse algorithms also avoid unnecessary arithmetic involving zero coefficients.
For large CFD problems, the difference between dense and sparse methods can determine whether a simulation is practical.
A production CFD solver therefore usually does not store a large coefficient matrix as a conventional two-dimensional array. It may use compressed sparse row storage, specialized matrix structures, or matrix-free operations.
For our first C++ learning examples, however, we can initially use simple vectors and loops before introducing more advanced sparse data structures.
5. Direct Solvers and Iterative Solvers
Once we have assembled the system
we need a numerical method to solve it.
Two broad approaches are direct solvers and iterative solvers.
Direct solvers
A direct solver uses a sequence of algebraic operations to obtain the solution, subject to floating-point rounding error.
For example, Gaussian elimination transforms a system into an equivalent triangular system that can be solved by back substitution.
For a small system,
we can solve the second equation for
Substituting into the first equation,
which gives
A computer generalizes these operations to much larger systems.
Common direct methods include:
- Gaussian elimination.
- LU factorization.
- Cholesky factorization for suitable matrices.
- Specialized direct methods for structured systems.
Direct solvers can be very effective for small and medium-sized problems, but their memory and computational costs can become large for huge systems. Sparse direct solvers exploit the zero structure of the matrix, although factorization may introduce additional nonzero entries called fill-in.
Iterative solvers
An iterative solver begins with an initial estimate and repeatedly improves it.
Instead of computing the exact solution through a single finite sequence of elimination operations, it generates a sequence of approximations:
The superscript indicates the iteration number.
The method stops when the solution estimate satisfies a specified convergence criterion or when a maximum iteration limit is reached.
Common iterative methods include:
- Jacobi iteration.
- Gauss–Seidel iteration.
- Successive over-relaxation (SOR).
- Conjugate gradient for suitable systems.
- GMRES for more general systems.
- BiCGSTAB for certain nonsymmetric systems.
The choice depends on the matrix properties, computational resources, and the requirements of the physical problem.
Comparison
| Feature | Direct solver | Iterative solver |
|---|---|---|
| Basic approach | Algebraic elimination or factorization | Repeatedly improves an estimate |
| Stopping rule | Finite factorization and solve operations | Residual or iteration criterion |
| Memory use | Can be high for large sparse systems | Often lower for sparse systems |
| Large CFD systems | May be costly | Often preferred |
| Sensitivity to matrix properties | Depends on factorization and conditioning | Depends strongly on method and matrix properties |
| Main limitation | Factorization cost and memory | May converge slowly or fail to converge |
The distinction is not that direct solvers are always accurate and iterative solvers are inaccurate. Both are subject to numerical errors, and iterative solvers can produce highly accurate solutions when used appropriately.
6. Jacobi Iteration: Solving One Equation at a Time
The Jacobi method is one of the simplest iterative methods. It is useful for learning because each new estimate is calculated using values from the previous iteration.
Consider a linear system with diagonal coefficients :
Separating the diagonal term gives
Rearranging,
The Jacobi update formula is
Every new value uses the previous iteration’s values.
Jacobi iteration for heat conduction
For our interior heat-conduction equations,
the update becomes
For the first and last interior points, the prescribed boundary temperatures are included in the right-hand side.
For the example with boundary temperatures and , the system is
Starting with
the first iteration gives
Thus,
For the next iteration,
giving
The estimate is moving toward the final solution:
It does not reach the exact solution in just two iterations. The process continues until the solution satisfies the stopping criterion.
A key implementation detail
The Jacobi method requires two sets of values:
- The old solution, .
- The new solution, .
All updates must use the old vector. If the new values are written directly into the old vector while traversing the grid, the algorithm is no longer standard Jacobi iteration.
This distinction matters when implementing numerical algorithms in C++.
7. Gauss–Seidel Iteration
Gauss–Seidel uses newly calculated values as soon as they become available.
Its update formula is
The first summation uses values already updated during the current iteration. The second uses values that have not yet been updated and therefore still belong to the previous iteration.
For our one-dimensional heat-conduction example, sweeping from left to right gives
Starting from zero, the first iteration is
The update has propagated information from the hot boundary through the grid during a single sweep.
For this particular example and update ordering, Gauss–Seidel makes more progress per sweep than Jacobi. This does not mean it is always faster for every matrix or implementation. Convergence depends on the system, and parallel implementations may favor different algorithms.
8. Successive Over-Relaxation (SOR)
Successive over-relaxation extends Gauss–Seidel by multiplying the update correction by a relaxation factor, .
Let be the value obtained from a standard Gauss–Seidel update. Then SOR uses
The parameter controls the amount of correction applied.
- If , SOR reduces to Gauss–Seidel.
- If , the update is under-relaxed.
- If , the update is over-relaxed; for certain matrix classes, this can accelerate convergence.
An unsuitable relaxation factor can slow convergence or cause divergence. The useful range and optimal value depend on the matrix and problem.
In CFD, relaxation is also used to stabilize coupled or nonlinear solution procedures. The relaxation used in an outer CFD iteration is related in spirit to SOR, but it is not necessarily the same algorithm as applying SOR to a linear system.
9. Residuals: How Do We Know Whether the Equations Are Satisfied?
A solver needs a way to measure how well the current estimate satisfies the algebraic system.
For
the residual vector is commonly defined as
Some software uses the opposite sign, . The sign convention changes the direction of the residual, but not its magnitude.
If the exact algebraic solution is reached, then
For an approximate solution, the residual is generally nonzero.
Residual of a single equation
Consider
If our current estimate is and , the left-hand side is
The residual using is
The equation is not yet satisfied.
If we improve the estimate to and , then
and the residual becomes zero for this equation.
A system with many equations has a residual for every row. We usually combine them into a single measure.
Common residual norms
The norm is
The infinity norm is
The norm measures the overall magnitude of the residual vector, while the infinity norm reports the largest absolute residual component.
A solver may use one of these measures or another appropriate norm to determine convergence.
Absolute and relative residuals
An absolute residual depends on the scale of the equations. A relative residual compares the current residual with a reference scale, for example,
when .
This helps compare convergence across some differently scaled systems. However, the precise definition of a solver’s reported residual may include normalization based on initial residuals, matrix coefficients, or other quantities. Always check the convention used by the software.
If the right-hand side is zero, the formula above is undefined. In that case, a different normalization or an absolute criterion may be appropriate.
Why residuals matter in CFD
During a CFD simulation, residuals help monitor how well the discrete equations are being satisfied.
A decreasing residual often indicates progress, but a low residual alone does not prove that the solution is physically correct. The solution may still have:
- Incorrect boundary conditions.
- An inadequate mesh.
- Incorrect material properties.
- Significant discretization error.
- Poor mass or energy conservation.
- A physically inappropriate turbulence or multiphase model.
Residuals are one important diagnostic, not a complete verification and validation strategy.
10. Convergence Criteria and Stopping Conditions
An iterative solver should not continue indefinitely. We need a rule for deciding when to stop.
A common criterion is
where is a prescribed tolerance.
For example,
means the chosen residual measure must fall below before the residual-based stopping criterion is satisfied.
The meaning of this number depends on the residual definition and the scaling of the problem. A tolerance of is not automatically necessary or appropriate for every CFD equation.
A robust implementation may stop when either:
- The residual reaches the specified tolerance.
- A maximum number of iterations is reached.
If the maximum iteration count is reached first, the solver should report that it did not meet the convergence criterion.
Monitoring solution changes
Another useful diagnostic is the change between successive estimates:
A small change can indicate that the iteration is stabilizing. However, small changes do not necessarily mean that the equations are satisfied. An iteration can stagnate far from the solution.
Therefore, residuals and solution changes should be interpreted together.
Linear convergence versus CFD convergence
It is important to distinguish two levels of convergence.
Linear solver convergence means that the iterative algorithm has reduced the residual of a particular algebraic system.
Overall CFD convergence concerns the behavior of the coupled governing equations as the simulation progresses.
A CFD solver may repeatedly solve its linear systems accurately while the outer flow solution is still changing. Conversely, an overly loose linear-solver tolerance can make the outer iteration inefficient or unreliable.
A practical simulation should monitor equation residuals and relevant physical quantities, such as mass imbalance, pressure drop, drag, lift, heat transfer, and other quantities of interest.
11. Why Do Some Systems Converge Slowly?
The behavior of an iterative solver depends on the mathematical properties of the matrix.
Diagonal dominance
A matrix is strictly diagonally dominant by rows if
for every row .
In simple terms, the magnitude of each diagonal coefficient exceeds the sum of the magnitudes of the other coefficients in that row.
Strict diagonal dominance is a useful sufficient condition for convergence of Jacobi and Gauss–Seidel methods, although it is not a necessary condition. Other matrix properties can also guarantee convergence.
For example, the matrix
is strictly diagonally dominant because
This matrix has a strong diagonal structure that is often favorable for basic iterative methods.
Conditioning
Conditioning describes how sensitive a problem’s solution is to perturbations in its input data.
For a nonsingular matrix, the 2-norm condition number is
A large condition number indicates that small perturbations in the matrix or right-hand side may cause relatively large changes in the solution.
A poorly conditioned system can therefore make accurate numerical computation more difficult.
Mesh refinement can affect conditioning. For example, discretizing diffusion equations on very fine grids often produces systems that are more challenging for simple iterative methods. The exact behavior depends on the discretization, boundary conditions, geometry, and scaling.
Why more iterations may not be the answer
If a solver converges slowly, increasing the maximum iteration count may help, but it does not address every underlying cause.
Possible improvements include:
- Improving the matrix scaling.
- Choosing a more suitable iterative method.
- Using a preconditioner.
- Applying a multigrid method.
- Reviewing mesh quality and boundary conditions.
- Checking whether the physical model or discretization produces an ill-conditioned system.
In modern CFD, preconditioning and multigrid methods are particularly important because they can substantially accelerate convergence for large systems.
12. Preconditioning: Making a System Easier to Solve
Suppose we need to solve
A preconditioner is an operator, commonly denoted , that approximates some useful properties of while being cheaper to apply or invert than the original system.
One form of a preconditioned system is
The solution remains the same when the transformation is valid and applied consistently, but the new system may be easier for an iterative method to solve.
A good preconditioner can reduce the number of iterations needed to reach a target residual.
An intuitive analogy
Imagine walking through a mountain range to reach a destination. A poor route may involve many steep climbs and sharp turns. A better route changes the way you approach the same destination, making the journey easier without changing where you need to end up.
Preconditioning is similar: it changes the numerical problem’s representation to improve the efficiency of the solution process.
Common preconditioning approaches include diagonal (Jacobi) preconditioning, incomplete LU factorization, and more advanced methods used with Krylov-subspace solvers.
The effectiveness of a preconditioner depends on the equation and the solver. It is not a universal shortcut that improves every system equally.
13. From One Equation to Coupled CFD Equations
The heat-conduction example contains one unknown temperature at each interior grid point. A fluid-flow simulation is more complicated because multiple physical quantities are coupled.
For incompressible flow, the principal governing equations include continuity,
and momentum conservation,
Here, velocity influences momentum transport, while pressure helps enforce the incompressibility constraint.
When these equations are discretized, they produce algebraic systems involving the discrete velocity and pressure fields.
In a simplified description, the coupled system can be written as
where:
- represents the discretized momentum operator.
- represents a discrete pressure-gradient operator.
- represents a discrete divergence operator.
- represents the discrete velocity unknowns.
- represents the discrete pressure unknowns.
This is a schematic representation of a coupled incompressible-flow system, not the exact matrix assembled by every CFD solver. The pressure block and right-hand-side structure depend on the formulation and treatment of boundary conditions.
Pressure–velocity coupling
Pressure is not simply another independent transported scalar in incompressible flow. Its role is closely connected to satisfying the continuity equation.
Algorithms such as SIMPLE and PISO address this pressure–velocity coupling through correction or prediction steps. These algorithms are commonly used in finite-volume CFD solvers, although their precise implementation differs across software and problem types.
A practical CFD code may therefore contain several nested numerical procedures:
- An outer loop updates the nonlinear flow solution.
- Momentum equations are assembled and solved.
- A pressure-correction equation is constructed and solved.
- Velocity and pressure are corrected.
- Additional equations, such as energy or turbulence transport, are solved.
- Convergence and physical conservation are monitored.
The linear solvers introduced in this module are building blocks inside this larger procedure.
14. Nonlinear Equations and Linearization
Not every discretized CFD problem leads directly to a single linear system.
The momentum equation contains the convective term
Because the velocity appears in both the transported quantity and the transport velocity, this term is nonlinear.
A nonlinear discrete system can be represented abstractly as
where contains the unknown fields and represents the residuals of the discrete governing equations.
A common strategy is to construct a sequence of linear systems that approximate the nonlinear problem around the current estimate.
For example, a Newton-type method uses the linearized system
where is the Jacobian matrix of derivatives of the residuals with respect to the unknowns.
The estimate is then updated:
CFD solvers may instead use segregated methods, Picard-type linearization, under-relaxation, or specialized pressure–velocity algorithms.
The important distinction is that solving a linear system is not always the same as solving the complete physical problem. A nonlinear CFD calculation may require repeatedly assembling and solving linearized systems until the coupled equations converge.
15. Implementing Jacobi Iteration in C++
We will now implement a basic Jacobi solver for the one-dimensional heat-conduction example.
The goal is educational: to understand how a numerical algorithm translates into loops, arrays, residual calculations, and a stopping criterion.
We will solve the three-equation system
The expected solution is
Create a file named jacobi_heat.cpp.
#include <algorithm>
#include <cmath>
#include <iomanip>
#include <iostream>
#include <vector>
int main()
{
// Three interior temperature unknowns.
const int n = 3;
// Matrix A and right-hand-side vector b.
const std::vector<std::vector<double>> A = {
{ 2.0, -1.0, 0.0 },
{-1.0, 2.0, -1.0 },
{ 0.0, -1.0, 2.0 }
};
const std::vector<double> b = {
100.0, 0.0, 0.0
};
// Initial guess.
std::vector<double> x(n, 0.0);
// Separate vector for the next iteration.
std::vector<double> xNew(n, 0.0);
const int maxIterations = 1000;
const double tolerance = 1.0e-10;
bool converged = false;
std::cout << std::scientific
<< std::setprecision(6);
for (int iteration = 1;
iteration <= maxIterations;
++iteration)
{
// Jacobi update: every new value uses only x.
for (int i = 0; i < n; ++i)
{
double sum = 0.0;
for (int j = 0; j < n; ++j)
{
if (j != i)
{
sum += A[i][j] * x[j];
}
}
xNew[i] = (b[i] - sum) / A[i][i];
}
// Compute the residual norm using xNew.
double residualSquared = 0.0;
for (int i = 0; i < n; ++i)
{
double Ax = 0.0;
for (int j = 0; j < n; ++j)
{
Ax += A[i][j] * xNew[j];
}
const double residual = b[i] - Ax;
residualSquared += residual * residual;
}
const double residualNorm =
std::sqrt(residualSquared);
// Report progress.
if (iteration == 1 || iteration % 10 == 0)
{
std::cout << "Iteration " << iteration
<< ", residual L2 = "
<< residualNorm << '\n';
}
// Accept the new estimate.
x = xNew;
if (residualNorm < tolerance)
{
converged = true;
std::cout << "Converged at iteration "
<< iteration << '\n';
break;
}
}
std::cout << "\nTemperature solution:\n";
for (int i = 0; i < n; ++i)
{
std::cout << "T[" << i + 1 << "] = "
<< x[i] << '\n';
}
if (!converged)
{
std::cout << "\nWarning: maximum iterations reached "
"without meeting the tolerance.\n";
return 1;
}
return 0;
}
Understanding the implementation
Step 1: Store the matrix.
The matrix is represented using a vector of vectors. This is simple and readable for a small example. It is not an efficient sparse-matrix representation for a large CFD simulation.
Step 2: Create an initial guess.
The initial temperature estimate is zero at every interior point. This is not the physical solution; it is merely the starting point for the iteration.
Step 3: Calculate the new values.
The nested loops implement the Jacobi formula. The inner loop sums the contributions from all off-diagonal matrix entries.
Step 4: Calculate the residual.
The code evaluates
and calculates the norm:
Step 5: Test convergence.
The solver stops when the residual norm falls below the specified tolerance. If it reaches the maximum iteration count first, it reports that convergence was not achieved.
Step 6: Report the temperatures.
The final estimates should approach , , and degrees Celsius for the three interior grid points.
Compile and run
Using a C++ compiler such as GCC:
g++ -std=c++20 -O2 -Wall -Wextra -pedantic jacobi_heat.cpp -o jacobi_heat
On Linux or WSL, run:
./jacobi_heat
On Windows PowerShell, if the executable is named jacobi_heat.exe, run:
.\jacobi_heat.exe
The output should show the residual decreasing and the solution approaching the expected temperatures.
A note about this implementation
This example prioritizes clarity rather than performance. It stores a dense matrix even though the heat-conduction matrix is sparse.
A production solver would usually avoid storing all the zero coefficients and would use a suitable sparse representation. It might also use a more efficient linear solver or preconditioner.
For this small system, Jacobi is sufficient to demonstrate the principles. For large CFD systems, plain Jacobi is often too slow to use as the only solver.
16. Improving the C++ Solver
The educational implementation can be improved in several ways.
Validate the matrix
The Jacobi update divides by each diagonal coefficient:
Therefore, the implementation should reject a zero or numerically unsuitable diagonal entry instead of dividing by it.
It should also verify that the matrix and right-hand-side dimensions match.
Use sparse storage
For a one-dimensional tridiagonal system, storing three diagonals is enough. There is no need to store the entire matrix.
For more general sparse systems, compressed sparse row (CSR) storage is a common option. It stores the nonzero coefficients and their column indices alongside row offsets.
Use appropriate stopping criteria
A practical solver may use a relative residual, an absolute residual, or a combination of criteria.
For example, a stopping condition could be based on
This combines absolute and relative tolerances. Special handling is needed when the right-hand-side norm is zero, and the exact criterion should match the solver’s design.
Add tests
A numerical solver should be tested against cases where the correct solution is known.
For the heat-conduction problem, the discrete solution is exactly linear for uniform spacing and constant conductivity when there is no internal heat generation and the boundary temperatures are prescribed.
For a larger grid, compare the numerical values with
Then calculate the maximum absolute error:
For this particular problem, the central-difference discretization reproduces the linear solution to floating-point precision, assuming the equations and boundary conditions are assembled correctly. This is an excellent first verification test, although it does not test every possible behavior of a solver.
17. What This Means for a Real CFD Project
The simple heat-conduction example is a foundation for a more substantial solver.
A sensible development path is to add complexity one step at a time.
Stage 1: Verify the algebraic solver
- Solve small systems with known analytical solutions.
- Compare the computed result with the expected result.
- Test convergence and maximum-iteration handling.
- Test invalid matrix dimensions and diagonal coefficients.
- Record residual histories.
Stage 2: Introduce a larger diffusion problem
Discretize a one-dimensional heat equation on increasingly fine grids. Assemble the corresponding algebraic system and solve it with a suitable method.
Compare the result with the analytical temperature distribution.
Stage 3: Use sparse matrix storage
Replace the dense matrix with a representation that stores only nonzero coefficients. Compare memory requirements and computational cost as the number of unknowns increases.
Stage 4: Add finite-volume discretization
Construct equations from flux balances over control volumes. This makes the code more naturally aligned with the conservation-based structure of many CFD solvers.
Stage 5: Introduce transport and fluid flow
Once diffusion and algebraic solution are reliable, add advection–diffusion, transient terms, and then more complex flow equations.
For incompressible flow, pressure–velocity coupling requires additional numerical machinery beyond solving an isolated scalar equation.
Stage 6: Build verification into the project
For every new feature, define a benchmark and measurable acceptance criteria before trusting the results.
Possible checks include:
- Analytical solutions.
- Grid-refinement studies.
- Mass conservation.
- Energy conservation.
- Residual convergence.
- Comparison with trusted reference data.
This development sequence helps separate implementation errors from discretization errors and physical-model errors.
18. Common Mistakes to Avoid
Mistake 1: Assuming a small residual guarantees a correct physical answer.
A small residual indicates that a particular algebraic system is being satisfied according to the selected residual definition. It does not guarantee that the governing equations, boundary conditions, mesh, or physical models are appropriate.
Mistake 2: Updating Jacobi values in place.
Standard Jacobi iteration requires a separate vector for the next estimate. Updating the same vector during the sweep changes the algorithm.
Mistake 3: Ignoring boundary contributions.
Boundary conditions affect the assembled equations. Leaving out a prescribed boundary contribution can change the physical solution.
Mistake 4: Choosing a tolerance without considering scaling.
Residuals depend on how the equations are formulated and normalized. A tolerance that is suitable for one equation may not be suitable for another.
Mistake 5: Using a dense matrix for a large CFD problem.
A dense matrix wastes memory and computation when most coefficients are zero. Sparse storage and algorithms are important for scalability.
Mistake 6: Treating every linear solver as interchangeable.
Different algorithms have different requirements and convergence properties. Conjugate gradient, for example, is designed for symmetric positive-definite systems, while GMRES can handle more general nonsymmetric systems.
Mistake 7: Confusing linear-solver convergence with overall simulation convergence.
A linear system can be solved accurately at every outer iteration even while the overall nonlinear CFD solution continues to change.
Key Takeaways
- Discretization converts differential equations into algebraic equations.
- A linear system is commonly represented as .
- Boundary conditions influence the coefficient matrix and right-hand-side vector.
- CFD matrices are generally sparse because discrete equations couple each unknown to a limited set of neighboring unknowns.
- Direct solvers use algebraic elimination or factorization, while iterative solvers repeatedly improve an estimate.
- Jacobi uses values from the previous iteration; Gauss–Seidel immediately reuses updated values.
- SOR introduces a relaxation factor that may accelerate convergence when chosen appropriately.
- Residuals measure how closely an estimate satisfies the discrete equations.
- A low residual does not by itself prove physical accuracy or conservation.
- Matrix properties, conditioning, preconditioning, and the choice of solver strongly affect computational performance.
- Real CFD calculations often involve nonlinear equations and coupled linear systems rather than one isolated matrix equation.
- Reliable solver development requires analytical benchmarks, automated tests, and systematic verification.
What’s Next?
In Module 5: Finite Volume Method from First Principles, we will move from finite-difference approximations toward the conservation-based framework widely used in CFD.
We will derive the finite-volume method by integrating a governing equation over a control volume, examine face fluxes, and construct discrete equations that conserve quantities across neighboring cells.
The key question will be: how can a numerical method preserve the physical conservation laws while representing fluid flow on a finite computational mesh?