dspp wrote:Sorcery wrote:
I get the impression that Ridley is critical of Ferguson's model, but then everybody is! It's certainly a problem when Ferguson's model produces different results on separate runs, it strongly implies buggy code. Can't really trust any model that is not bug free in that way. It is absolutely basic to be able to get repeatable results.
I've not looked in detail at anybody's model, but I would expect all medium-complexity (or higher) models to be stochastic in nature, i.e. all runs would be different. Low-complexity deterministic models are surely only for kindergarten use ? Higher-complexity models should be both stochastic and recursive.
- dspp
Getting repeatable results is an initial check that the code is correct. Unrepeatable results are a strong indicator that the code is incorrect.
Perhaps accessing memory beyond it's declared dimensions or use of an un-initialised variable, that kind of thing.
Highly complex models whether stochastic or not, should produce repeatable answers. If you want to vary the inputs in a controlled way as in the Monte Carlo method, that's code that sits on top of a model's code that produces repeatable results. However even there unless you are are using genuinely random seeds for random number generation, the results should always be repeatable.
I seem to remember Electronicfur or another user on one of these boards was attempting to check Ferguson's code, who also didn't think that unrepeatable results ought to be tolerated.