Testing PenFit for structural bias.
PenFit was deliberately stress-tested to see whether its scoring architecture favoured a retirement approach even when the underlying questionnaire evidence was neutral and symmetric.
The profiles were generated across all five valid response levels specifically to test the model rather than represent a real population.
Symmetric stress-test results
The goal is not to force six identical outcomes. The check looks for structural dominance, an impossible-to-win strategy, unequal neutral starting points or excessive dependence on one characteristic.
| Retirement approach | Ranked first | Highest winning alignment | Audit result |
|---|---|---|---|
| Flexible Income | 17.4% | 82.3 | PASS |
| Secure Income | 21.2% | 90.3 | PASS |
| Target Income | 14.2% | 85.8 | PASS |
| Flexible and Secure Income Mix | 19.8% | 86.0 | PASS |
| Flexible Income then Secure Income Later | 11.6% | 84.8 | PASS |
| Flexible Income with Secure Income Build-Up | 15.8% | 86.0 | PASS |
What changed to control bias
The working calibration was changed only where a specific structural issue had been identified.
Each of the 12 retirement values appears twice and every answer uses the same +2 / +1 / 0 / −1 / −2 evidence scale.
Strategy-specific starting advantage was removed. An all-A/B response gives every approach 57.9.
A strategy is not rewarded simply because its natural score range moves more as answers change.
No strategy bonus, equal-win target or automatic 100% top score is imposed.
Double-counting and sensitivity checks
The audit also challenged whether correlated characteristics such as retained capital, liquidity and reversibility were giving one economic property too many votes.
Correlated-characteristic test
Capping overlapping security/flexibility characteristic families still allowed all six strategies to rank first. First-place frequencies remained approximately 12.6%–21.8%, with maximum winning scores within about ±4.8 points of their mean.
13 leave-one-out tests
Each retirement characteristic was removed in turn. No single characteristic made any strategy impossible to select, so no single characteristic appears to be driving the result.