"All models are wrong, but some are useful."
2.3 Parsimony
Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.
2.4 Worrying Selectively
Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about safety from mice when there are tigers abroad.
— Science and Statistics on Journal of the American Statistical Association3, 1976.
Now it would be very remarkable if any system existing in the real world could be exactly represented by any simple model. However, cunningly chosen parsimonious models often do provide remarkably useful approximations. For example, the law $PV = nRT$ relating pressure $P$, volume $V$ and temperature $T$ of an “ideal” gas via a constant $R$ is not exactly true for any real gas, but it frequently provides a useful approximation and furthermore its structure is informative since it springs from a physical view of the behavior of gas molecules. For such a model there is no need to ask the question “Is the model true?”. If “truth” is to be the “whole truth” the answer must be “No”. The only question of interest is “Is the model illuminating and useful?”.
— Robustness in the strategy of scientific model building, 1978.
Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.
… all models are approximations. Essentially, all models are wrong, but some are useful. However, the approximate nature of the model must always be borne in mind….
— Empirical Model-Building and Response Surfaces, 1987.
It has been said that “all models are wrong but some models are useful.” In other words, any model is at best a useful fiction — there never was, or ever will be, an exactly normal distribution or an exact linear relationship. Nevertheless, enormous progress has been made by entertaining such fictions and using them as approximations.
… So since all models are wrong, it is very important to know what to worry about; or, to put it in another way, what models are likely to produce procedures that work in practice (where exact assumptions are never true).
— Statistical Control: By Monitoring and Feedback Adjustment, 1997.
All models are approximations. Assumptions, whether implied or clearly stated, are never exactly true. All models are wrong, but some models are useful. So the question you need to ask is not “Is the model true?” (it never is) but “Is the model good enough for this particular application?”
— Statistical Control By Monitoring and Adjustment, 2009.
References