I used to run the Modeling and Simulation Division at the U.S. Army Night Vision and Electronics Sensors Directorate. Models and simulations are extremely useful in making estimates of how imaging systems, photonic systems, lens systems, and devices perform and to help understand more fully how they work. In some cases, a more basic understanding of device design is uncovered. I personally have done groundbreaking work using models and simulations that are used to this day in imaging system design.

The flipside of modeling and, in particular, simulation is that many researchers use techniques to provide incremental progress in applied optics research. A close colleague of mine routinely says that “all simulations are wrong.” The trick to providing useful simulations is to quantify the simulation errors. If the errors are within acceptable bounds (and can be verified and validated), then the simulation is useful. How useful the simulation is to researchers, in journal terms, can be called significance.

I have identified two primary reasons for simulations and simulation papers to lack significance. The first is that the simulation is an incremental improvement on previous research. That is, a simulation might take a previous experiment and match the results with some small extension that is not very significant. The second reason is that a simulation might provide results, however, there is no clear way to determine the accuracy and error of a simulation. The simulation is not verified (meaning the behavior has been independently deemed to be appropriate for the physical system) and is not validated (that is, backed up by real independent results) over the correct range of input parameters. The corroboration of simulations by actual measurements is crucial. Authors should recognize when their simulations agree with and deviate from actual measurements, and they should adequately explain any deviations.

A significant simulation paper might be one in which a simulation has been performed (and described accurately) with some limited hardware validation, but the overall results are surprising and unexpected to the reader. And still, it is clear that the paper is correct from the content. Simulations are particularly credible when they have been improved by the conditions of the measurements or the equipment used during the measurements. Incremental simulation papers are not interesting, not cited, and not read.

I have provided direction to our Deputy Editors and Topical Editors that an incremental simulation paper can be rejected without review. Some questions they will consider when making this decision are the following:

I hope that these questions are useful, not only to the editors, but to the authors who consider submitting a simulation paper to Applied Optics. The journal still wants your simulation papers, but only the ones that are significant.