Claude Opus 5.5 cut PSP math tables from 4.9 MB to 10.5 KB

Claude Opus 5.5 took about an hour to work out the circuit behind the eight special math functions of the Sony PSP’s vector unit. Its 10.5 KB of coefficients replace the 4.9 MB of correction tables the PPSSPP emulator needed to match the hardware, so future PPSSPP versions will ship a few megabytes lighter and run accurate emulation of these functions by default. The new code is bit-exact on every one of the 2^32 float inputs, according to a September 24 post on the PPSSPP blog that includes Claude’s own, very lightly edited writeup of how it got there.

The switch to on-by-default comes down to speed. The old path cost some performance on top of the memory, and the interpolator beats it on every function measured: sin, the slowest under the tables, now takes 5.9 ns instead of 14.6 ns, log2 drops from 7.1 ns to 5.1 ns, exp2 from 6.3 ns to 3.9 ns, and rcp, rsqrt and asin all land at 3.6 ns. The vrot instruction, which returns sine and cosine together for rotation matrices, gets about 10% faster again because the pair shares its argument reduction.

The functions in question are reciprocal, reciprocal square root, square root, exp2, log2, sin, cos and arc-sine, with the trig functions using 4.0 for a full circle instead of 2π. None of them returns the correctly rounded IEEE result, and while what they return has long been known, how the chip computes it was not. PPSSPP already had them bit-exact thanks to fp64, a long-term contributor, who built an easy base approximation, added correction deltas for every 64 inputs plus lists of exceptions, and shipped the result as about 4.9 MB of .dat files loaded from assets/vfpu.

What made an hour enough

Claude’s hour rested almost entirely on that earlier work. fp64’s code agrees with the hardware on every input and runs in a second on a laptop, which made it an exact oracle: Claude could dump every output of a function, for example all 2^23 mantissas of 1/x over [1, 2), and test ideas against them freely. A new hardware test on a real PSP (pspautotests cpu/vfpu/exact) confirmed the oracle first and turned up a few mismatches in PPSSPP’s other CPU backends along the way. Checking a hypothesis against all 2^32 inputs took 30 seconds, so every idea got the full test.

Curve fitting went nowhere. A piecewise quadratic for 1/x matched about 93% of outputs whatever the chunk size or degree, and float coefficients evaluated through the VFPU’s own dot product reached about 92%. Henrik suggested the functions might be built from vdot operations, since on the Dreamcast the inner-product unit turned out to be reused by the function approximator, but a single vdot of float coefficients also stalled at 92%. The misses had one shape: about one 24-bit ulp, a step of the last bit, in a band one ulp wider than the output’s truncation window.

So Claude stopped asking how close a formula could get and asked whether any coefficients could reproduce every output exactly. Each output then becomes an interval constraint on the unknowns, and intervals intersect cheaply. “Least squares hid the discrete structure. Interval feasibility exposed it,” the writeup says.

One squarer for eight functions

Within every run of 64 consecutive inputs, the output is a straight line, and runs share a slope in groups of exactly 2^16 inputs. The top 7 bits of the mantissa therefore pick one of 128 segments, with the slope equal to the true derivative at the segment’s centre: the textbook layout of a hardware quadratic interpolator, a small table indexed by the top bits plus arithmetic on the rest. What was left over after a smooth fit was a one-ulp sawtooth, the same in every function, and its climb rate matched the slope of each function’s squared term (0.28 ulps per interval against an expected 0.273 for rcp). The squared term is floored to whole ulps on its own before being added, which no polynomial fit can see.

Knowing that did not say how the floor worked. Truncated squarers, split squarers and two-step products all hit the same wall at 99.65% of outputs, and several “impossible” results were bugs in Claude’s own search code: a coefficient stepped too coarsely to ever hit its true value, a constant subtracted twice, and a zsh quirk that made one parameter sweep silently test nothing.

The break came from a coincidence in a sweep’s output. Neighbouring segments of rcp printed identical best-fit squared coefficients to four decimals, segments 115 to 120 all at 143.9636. Identical decimals meant identical integer data, so the squared coefficient is a small integer, and the integer sequences turned out to be byte-for-byte the same across functions: exp2’s segment 32 matches rcp’s segment 116. One squaring circuit serves all of them. With 109 different coefficient values in the data, Claude could solve for that circuit’s output directly and found it rounds the squared offset up to a multiple of 256. That model fit all 512 segments of rcp, rsqrt, sqrt and exp2, and matched fp64’s code over all 2^32 inputs on the first run, apart from one bug in the wrapper arithmetic.

Two functions needed one more rule each. sin fit only once it was indexed from the other end of the quarter wave, which amounts to computing it as the cosine of the complementary angle; the giveaway was that its few wrong-way steps sit one input later than rcp’s. log2 was the hardest: above an input of 4 and below 1, the datapath trims the coefficients to match the coarser output precision. For negative exponents the squared term disappears entirely, which is why that path had looked like plain linear interpolation, and truncating toward zero also covers the region just below 1.0 that fp64’s code handled as a special case, sign of −0 included.

A design from the textbooks

The writeup places the chip in the published literature on hardware function evaluation: Piñeiro, Oberman, Muller and Bruguera’s minimax quadratic interpolator, or NVIDIA’s multifunction interpolator (Oberman and Siu, 2005), which covers much the same set of functions. The PSP version has a 128-entry ROM per function, a squarer that sees only the top 10 bits of the offset, and coefficient widths cut to what the output needs. That last trade is a standard one: a paper in MDPI’s Electronics describes quadratic interpolators for reciprocal, square root, reciprocal square root and sine built with truncated squarers, with coefficient lengths minimized to shrink the lookup tables and truncated columns to cut area, delay and power.

One question about the PSP’s design remains open. Whether the final adder is shared with vdot, as on the Dreamcast, can’t be told from outputs alone; the separately truncated products are compatible with it.