Global Optimization Test Result Presentation

• the guidelines On reporting computational experiments used by the Mathematical Programming Society,
• the paper Designing and reporting on computational experiments with heuristic methods (123K) by R. S. Barr, B. L. Golden, J. P. Kelly, M. G. C. Resende, and W. R. Stewart.
• my list of links to Global Optimization Test Problems

My current wishes for reporting results are:

Report for each problem the type of information used (e.g., function values, gradients, Hessians, their interval enclosures or underestimating functions) and how often you used it (a) until you found the global optimizer for the first time, and (b) until your stopping test decided that there is no better feasible point, and quits. If the global optimum is not found, or if the best point is not known, indicate the lowest function value reached. If the best point found is not yet in the database, give coordinates and function value, and indicate whether you believe it to be the global minimizer, and why.

Report times used in units 1 = time for 1000 objective function (and constraint) evaluations. (This seems more useful than the traditions of either reporting absolute times, or of reporting times relative to the Shekel 5 function, which may be completely unrelated to the kind of expressions used in the current function. There is some discussion on the choice of units to something that is independent of the programming style of researchers. So it may well be that this is changing soon. So before you do your final tests and submit timings, please check this page again to see whether anything has changed.)

For probabilistic algorithms, run the method at least 10 times (for smaller problems 100 times), and report the quotient q = -log (probabilility of missing the global minimizer) / (time used). Here the natural logarithm should be used, and to make the figures more machine independent, the time used should again be measured in units 1 = time for 1000 function evaluations. The probability of success of the method run repeatedly for the time equivalent to 1000*T function evaluations is then 1-exp(-qT).

Be specific about the program name and version, the parameter settings used to tune your algorithm, starting points used (if there is a choice in your algorithm) and the stopping rule used. In particular, for each particular problem collection, all problems should be handled by the same set of parameters and stopping rules. If you only test part of a collection, you should give your reasons. If you use test problems from several collections, please make separate tables.