MPS Guidelines for Reporting Results of Computational Experiments

The guidelines that are used by the Mathematical Programming Society are based on

Crowder, H.P., Dembo, R.S., and Mulvey, J.M. "On reporting Computational Experiments with Mathematical Software", ACM Transactions on Mathematical Software, 5, 1979, pp. 193-203.

In the wake of the various claims made about Karmarkar's algorithm, a draft of a followup report was published:

Jackson, R.H.F., Boggs, P.T., Nash, S.G., and Powell, S. "Report of the Ad Hoc Committee to Revise the Guidelines for Reporting Computational Experiments for Mathematical Programming", COAL Newsletter, Mathematical Programming Society, No. 18, March, 1989.

The final version of this report was published as:

Jackson, R.H.F., Boggs, P.T., Nash, S.G., and Powell, S. "Guidelines for reporting results of computational experiments. Report of the ad hoc committee", Mathematical Programming 49, 1990/91, pp. 413-426.

Some thoughts on how computationally papers can be classified are given in:

Greenberg, H.J. "Computational testing: Why, how, and how much", ORSA Journal on Computing, Vol. 2, No. 1, 1990, pp. 94-97.

The journal Operations Research references an article "Reporting Computational Experience in Operations Research", Vol. 29, No. 2, on the guidelines authors for that journal should use. I don't have the full reference on that particular paper. Maybe someone else can fill that in.

-Irv Lustig
Director of Numerical Optimization
CPLEX Optimization, Inc.
irv@dizzy.cplex.com
http://www.cplex.com/

>The journal Operations Research references an article "Reporting Computational >Experience in Operations Research", Vol. 29, No. 2, on the guidelines authors >for that journal should use. I don't have the full reference on that >particular paper. Maybe someone else can fill that in.

That's enough information to look it up. It's an editorial, not listed in the table of contents:

H. D. Ratliff and W. Pierskalla, "Reporting Computational Experience in _Operations Research_", _Operations Research_ v29,n2, March-April, 1981, xi--xiv.

Pierskalla was Editor and Ratliff was Area Editor for Optimization at the time. The guidelines themselves were actually written by Florian, Fox, Crowder, Dembo, and Mulvey, so I suspect that they recap the TOMS criteria.

Matthew Saltzman
Clemson University Math Sciences
mjs@clemson.edu