PINNs: Neural Networks That Do Their Physics Homework

The most capable machines on Earth don't understand 2 + 2

Ask ChatGPT what 2 + 2 is. It says 4. Ask it something harder, 47 + 58, and it still says 105. This makes us puny humans think “Oh, it understand mathematics like we do!”

Well, sorry, but it doesn't, and the real reason is far stranger than "it just memorized the answer." When researchers pried open a language model to watch it add two numbers, they didn't find a memorized table, and they didn't find the carrying method we learned in third grade. Instead, they found that the model had quietly invented its own algorithm (something about representing numbers as points on a spinning helix and rotating the helices to add - yeah, very complex and random, a bit out of scope of this blog but we could cover it later on :)). Nobody designed this. Nobody taught it. It grew this method on its own, out of nothing but a diet of internet text.

And here's the part that should genuinely unsettle you. When you ask the model how it added the numbers, it confidently describes the ordinary schoolbook method; line them up, add the digits, carry the one. But that is not what it did. The machine cannot even accurately report its own reasoning. It runs one alien procedure and narrates a different, human-sounding one (hmm, AI lies… another potential blog topic).

The lesson isn't that AI is dumb. These systems are astonishing. The lesson is how they're capable: they learn patterns, not principles. A language model has no list of arithmetic axioms and theorems just lying there in its memory. It is a statistics-based-machine that approximates a “right answer” and fails in ways that indicate upon it own “pattern-based method”. Some famous errors include being off by exactly 10, a fingerprint of its guessing structure.

Taking this gap to a larger scale, we land upon the problem this blog addresses. A modern language model has ingested every physics textbook ever written, and yet it doesn't know that water flows downhill (at least, not in the way physics knows it). It knows what sentences about flowing water tend to look like, but that’s not the same as knowing the law and having a feel of the physics, the way us humans do.

Why this is the situation across almost all of science

Science doesn’t want correct-sounding sentences - it wants numbers, especially getting those numbers right in situations nobody has ever measured. So the question sharpens into something concrete and surprisingly deep:

How do you get a neural network to actually obey the laws of physics, instead of merely imitating data that happens to follow them?

The first obstacle is data. Deep learning's superpower is gorging on oceans of data. But walk into a physics lab, wind tunnel, or a surgical operation theatre, and data stops being free. One data point might mean a full engine test run, a destroyed rare-material sample, a patient's MRI scan, or a day of ship time out at sea. You do not get a million examples… you may only get twelve :(

And twelve data points handed to a flexible neural network is a recipe for quiet a disaster, as there are infinite possibilities. If I asked you to model a graph that passes through twelve specific datapoints, you may draw a haphazardly oscillating graph, or perhaps a fairly symmetrical line, depending on your mood on that day. Similarly, the network would certainly pick the wrong one, some nonsense that nails every measurement and defines no real relation between them. Nothing in ordinary machine learning tells it which of those infinite curves is the physically sensible one. The model is only focused on fitting the data - it has no sense of whether the datapoints are supposed to be following a physics law.

But scientists have something ImageNet never did. Before collecting a single measurement, they already know the answer has to obey a law. We know heat flows from hot to cold. We know momentum is conserved. We know, often, the exact governing equation. That knowledge is free, hard-won over centuries, but standard machine learning throws it in the bin.

Physics-Informed Neural Networks are, at their heart, a single stubborn idea: stop throwing it away. Hand the law to the network as part of what it gets graded on.

"But if we know the law, why not just solve it ourselves?"

Reasonable question. If the physics is known, why drag a neural network into it at all — why not just solve the equation?

Because knowing the law is not the same as being able to write down the answer. A physical law almost never arrives as a tidy formula you can plug numbers into. It arrives as a differential equation — a rule about how a quantity and its rates of change relate to each other. It tells you the relationship the answer must satisfy; it does not tell you the answer.

While first-year calculus might explain how to solve simple ODEs, physic consists of unsolvable and exhaustive differential equations. A term where the unknown multiplies its own derivative, an oddly shaped domain, a few extra dimensions, and something remarkable happens: the equation becomes provably impossible to solve with any formula. Not "difficult". Impossible. For example, the Navier-Stokes Equations, Boltzmann equation, etc.

These aren't exotic edge cases. These are the equations that actually run the world, and for most of them, humanity's only option has been to compute an approximate answer, laboriously and expensively. That gap between knowing the law and being able to evaluate its consequences is exactly where PINNs step in.

A sixty-second refresher on neural networks

(Comfortable with neural nets and loss functions? Skip ahead. Rusty? This is all you need for the rest of the post.)

A neural network is literally just a very flexible function. Stack some simple linear steps, wrap each one in a little squishing "activation" so the whole thing can bend and curve, and you get a shape-shifter that can approximate almost any function — provided you tune its internal knobs, called weights, correctly. This tuning happens through a loss function that measures how wrong the network currently is — a single number, bigger when it's more wrong, lesser when it gets closer to the actual answer. Then an algorithm called back-propagation (which is nothing more mysterious than the chain rule from high-school calculus) works out which direction to nudge each knob to make that “loss function” number smaller. Nudge, re-grade, repeat. The network rolls downhill on its own wrongness until it can't get any lower.

That's the whole game. Define what "wrong" means, then roll downhill. Hold onto that sentence, because the entire PINN idea is just a clever new definition of "wrong."

Never seen a neural network before? Give this 3b1b Neural Networks video a go. Come back after; the rest will click.

Watching a PINN work: your phone's charging bar

You already know the phenomenon we're about to model. You plugged in your phone at 5%, noticed that it quickly reach 80% in half an hour, and then wretchedly watched it crawl the rest of the way. And don’t even get me started on that treacherous journey from 99% to 100%. Time never went slower.

That maddening slowdown is a physical law in disguise. A simple model of charging says: the rate at which the battery fills is proportional to how far it still has to go. Early on, the gap to full is huge, so it charges fast. Near the top, the gap is tiny, so it inches along. In one line:

(Real batteries are messier than this — actual charging circuits do something more elaborate near the top. We're using the clean version that captures the essential behavior.)

Now here's our data: Imagine a friend texts you their battery percentage every couple of minutes while their phone charges — and then gets bored and stops after the first few minutes. You're left with maybe eight dots, all bunched up at the start. You want the whole curve, including the long slow climb your friend never bothered to send.

Attempt 1: a plain neural network

Train an ordinary network to fit those eight dots. It fits them perfectly — of course it does, that's what they're good at. But look at what it does between and beyond them:

It wiggles. It shoots off in some absurd direction. It behaves like a maniac everywhere you didn't give it a dot — because nothing was constraining it there. This is the "infinitely many curves" problem made real: a few dots, endless curves that fit them, and the network grabbed a ridiculous one.

Attempt 2: teach it the law

Here's the move that changes everything.

We know the law the answer must obey:

So we take the network's own output and using the very same chain-rule machinery that trains it, and compute its derivative, dV/dt. Then we ask a simple question: does the network's actual slope match the slope the law demands?

If the network is physically correct, the two sides of the equation match and their difference is zero. If it's wrong, that difference is a number telling us exactly how badly it's breaking the law at that point.

And crucially, we can ask this question anywhere — even at times where we have no measurement at all. We scatter a few hundred of these check-points across the whole time axis. These are called collocation points: locations where we test whether the physics holds, as opposed to data points, where we actually know the answer. We have eight data points. We can have as many collocation points as we like, because checking the physics costs us no measurements - just a bit of arithmetic.

Now we redefine "wrong." The network is graded on two things added together:

1. Data loss: how far its predictions sit from the eight real dots.

2. Physics loss: how badly it violates the law across the hundreds of collocation points.

Add them. Roll downhill, exactly as before.

The wild wiggles are gone. Any curve that fits the eight dots but misbehaves elsewhere now gets punished, because misbehaving means a large law-violation at the collocation points. The only curve that can satisfy both the data and the law everywhere is the true one and the network, rolling downhill on its new definition of wrong, finds it. It correctly predicts the entire charging curve, including the long slow crawl your friend never texted you, from eight early dots and a single line of physics.

What actually just happened (the engine and the harness)

Look closely at what did the work, because it's easy to give the neural network too much credit.

The network did not discover the law of charging. We handed it that law. The equation is the engine, i.e. a precision instrument built by physicists long before any of this existed. The neural network is just a harness bolted on top: a flexible, searchable shape that gradient descent bent until it fit the engine's rules.

Almost all of the intelligence in this system lives in the equation, not the network. And that is precisely why it works with so little data. The network never had to learn physics from scratch, an enormous job that would need mountains of data. It only had to learn the one specific solution that obeys a law it was handed for free. That's a far smaller task, and eight dots is enough for it.

The part that makes this genuinely useful: reality never matches the textbook

So far this might still sound like an elaborate (and stupidly unnecessary) way to solve an equation. Here's the twist that makes PINNs actually worth having.

Real physical systems never obey the clean theoretical equation exactly. Your specific phone has a slightly aged battery, a charger that delivers not-quite-rated current, a temperature in the room, resistances and quirks in its particular circuitry. The textbook equation is an idealization. The real battery deviates from it, measurably.

Now notice what a PINN is actually balancing. It has two graders pulling on it at once: the physics loss, whispering "obey the ideal law," and the data loss, insisting "but also match what this real battery actually did." When the real system deviates from theory, those two forces disagree slightly — and the network settles into a curve that honors the physics while bending to fit the real measurements. It quietly absorbs all the little system-specific deviations that the pure equation never knew about: the aging, the temperature, the manufacturing tolerances, the sensor biases.

Think about what that means. You didn't have to know any of those imperfections. You didn't measure the room temperature or the battery's age or model the charger's exact behavior. The data points carried that information implicitly, and the network folded it in on its own. The result is a model tuned to this system as it truly behaves, and not as the textbook imagines it.

This is exactly the situation in the wild. The classic version of this story is inferring a real object's cooling rate: the theoretical law tells you the shape of how something cools, but the actual rate depends on this specific object's material, surface, and surroundings — messy details you'd rather not measure one by one. Feed a PINN a handful of real temperature readings plus the law of cooling, and it hands you back a reasonable close estimate to the cooling behaviour of that object, deviations and all.

PINNs are, in a real sense, personalised networks

Here's the memorable way to hold this: a PINN is a personalised neural network.

The network you train on your phone's battery is not the same network you'd train on mine. Same equation, same eight lines of code, but different data, and therefore a different trained network, tuned to a different physical object. Mine might charge a hair slower because my battery's older; the PINN trained on my data learns my battery, yours learns yours. The physics is universal; the network that results is bespoke.

That's a genuinely different mindset from most of deep learning, where you train one giant model to serve everyone. A PINN is the opposite — a small, custom-fit model that marries a universal law to one specific system's real behavior. It's part solver, part personal biographer of the thing you pointed it at.

"Uhm, hold on a second. I could have solved that battery equation by hand."

If you've taken calculus, you may be a little annoyed right now. That charging equation does have a clean pen-and-paper solution. We built an entire neural network to do a homework problem.

Guilty. The battery was training wheels; a problem small enough that you could see every moving part at once. The point was never the battery. The point is that the exact same recipe keeps working when the equation stops being solvable by anyone at all. So let's break something.

The Schrödinger equation, for a molecule that actually exists

You met the Schrödinger equation in first year. For a single hydrogen atom with one lonely electron, it can be solved exactly. Those tidy orbital shapes we memorized, the round s and the dumbbell p, are that solution. Beautiful, closed-form, done.

Now for the Helium atom, add one more electron, and the exact solution vanishes. No one has ever written down a closed-form wavefunction for helium. No one ever will. The reason is brutal and geometric: every electron you add drags the equation into three more dimensions. The wave function describing an n-electron molecule doesn't live in our familiar 3D space - it lives in a 3n-dimensional space.

Here's why that detonates the classical approach. The traditional way to solve a differential equation you can't crack by hand is to chop its domain into a fine grid and grind out the answer point by point. But the number of grid points needed grows exponentially with the number of dimensions. Just ten grid points along each of thirty dimensions is 10^30 points. You could not store that if you turned every atom on Earth into a hard drive. Classical methods don't get slow here. They hit a wall built out of arithmetic itself.

Why the PINN idea survives where grids die

A neural network never lays down a grid. It represents the unknown wave function as one continuous, flexible function, and it only ever checks the physics at sampled points, sidestepping the exponential explosion that annihilates grid methods. Represent the unknown physical quantity as a network, and train it against the equation: the same idea you just watched tame a battery is now a serious research frontier in quantum chemistry, aimed at molecules no exact mathematics can reach.

The takeaway is the one that makes this whole field worth caring about: the toy you just watched on a charging bar is a scaled-down cousin of a technique people are seriously using to attack problems that have defeated pen-and-paper mathematics for a century.

The honest part: where this breaks

A technique is only as trustworthy as the limitations its own inventors are willing to state out loud. Here are the ones that matter.

It is not a replacement for classical solvers. The paper that launched PINNs says this in plain language. Fifty years of finite-element and spectral methods are mature, fast, and come with guarantees a neural network simply doesn't have. For a great many everyday problems, the old tools are the right tools. PINNs are a new instrument, not a coronation.

One network solves one problem. The battery PINN we trained knows exactly one battery at exactly one charging rate. Change the rate and you retrain from scratch. It behaves less like a reusable "model" and more like a single bespoke calculation that happens to use gradient descent.

Training can be fragile. Balancing the two loss terms, data against physics, is finicky. Lean too hard on the physics and the loss landscape can turn into a jagged minefield where gradient descent gets stuck. Harder equations can make training unstable in ways the old grid methods never suffered.

By default, it hands you no error bars. A classical method can often tell you how wrong it might be. A plain PINN gives you a confident answer and no built-in sense of its own uncertainty (genuinely dangerous if you're relying on it for anything that matters). The original authors flagged this as an open problem, and to a real extent it still is.

Garbage law in, garbage answer out. The whole method rests on you knowing the correct governing equation. Feed it the wrong physics and it will faithfully, confidently produce a wrong answer that obeys your wrong law. It does not notice that your physics was incomplete. It trusts you completely.

The one-sentence version

Strip away everything and this is all a PINN is: redefine "wrong" to include "disobeys physics," then roll downhill.

That's it. It's almost embarrassingly simple, which is exactly what makes it beautiful, and exactly why, years after it was first proposed, people are still finding fresh corners of science to point it at. It solves equations no formula can crack, it sips data instead of guzzling it, and the part worth remembering longest - it doesn't just parrot the textbook. It bends the universal law to fit the messy, specific, imperfect real system in front of it, becoming a personalized model of that object and no other.

(For your reference, here is the original paper in which the concept of PINNs was introduced: PINNs-arxiv)