Skip to main content
Log in

Journal of Heuristics - Policies on Heuristic Search

These general policies apply to manuscripts submitted to the Journal of Heuristics and that belong to the general areas of heuristic search for optimization, including but not limited to metaheuristics, hyperheuristics, and matheuristics.

Metaphor-based methodologies

  1. The optimization literature is rich with heuristic search methodologies for both discrete and continuous spaces. Proposing new paradigms is only acceptable if they contain innovative basic ideas, such as those that are embedded in classical frameworks like genetic algorithms, tabu search, and simulated annealing. The Journal of Heuristics avoids the publication of articles that repackage and embed old ideas in methods that are claimed to be based on metaphors of natural or manmade systems and processes. These so-called “novel” methods employ analogies that range from intelligent water drops, musicians playing jazz, imperialist societies, leapfrogs, kangaroos, all types of swarms and insects and even mine blast processes (Sörensen, 2013). If a researcher uses a metaphor to stimulate his or her own ideas about a new method, the method must nevertheless be translated into metaphor-free language, so that the strategies employed can be clearly understood, and their novelty is made clearly visible. (See items 2 and 3 below.) Metaphors are cheap and easy to come by. Their use to “window dress” a method is not acceptable.
     
  2. The Journal of Heuristics is interested in advancing the area of heuristic search by publishing articles that, as mentioned by Sörensen (2013), adequately frame the methodology being applied within the existing optimization literature. Adequately framing a method entails deconstructing it and describing its components, measuring their contribution, and making connections to other procedures where these and/or similar components appear. Contributions must provide a clear explanation on how the components were adapted to the specific problem that is being solved. Implementations should be explained by employing standard optimization terminology, where a solution is called a “solution” and not something else related to some obscure metaphor (e.g., harmony, flies, bats, countries, etc.). In short, the journal embraces a component-based view of heuristic search.
     
  3. The Journal of Heuristics fully endorses Sörensen’s view that metaphor-based “novel” methods should not be published if they cannot demonstrate a contribution to their field. Renaming existing concepts does not count as a contribution. Even though these methods are often called “novel”, many present no new ideas, except for the occasional marginal variant of an already existing methodology. These methods should not take the journal space of truly innovative ideas and research. Since they do not use the standard optimization vocabulary, they are unnecessarily difficult to understand.
     
  4. The Journal of Heuristics considers new methodologies only if they are scientifically tested by following the principles outlined by Hooker (1995). Scientific testing entails the construction of controlled experiments to isolate the effects of algorithmic components as well as to investigate how problem characteristics influence the behavior of those components. The journal considers that there is little gain for the scientific community for yet another search method whose polished implementation is narrowly tested on benchmark instances of a single problem class.

Competitive testing and up-the-wall game

  1. The Journal of Heuristics does not endorse the up-the-wall-game (Burke, et al. 2009). The idea of the up-the-wall game is to develop and apply a proposed search procedure to existing benchmark problems in order to compare it with other players. The goal is to get further “up the wall” than the other players. Although some competition among researchers or research groups could stimulate innovation, the ultimate goal of science is to understand (Burke, et al. 2009). True innovation in heuristic-search research is not achieved from yet another method that performs better than its competitors if there is no understanding as to why the method performs well (Sörensen, 2013).
     
  2. The Journal of Heuristics favors the publication of meaningful insights over procedures that are tuned to perform better than others on a set of benchmark instances. In other words, the journal finds no value in conclusions stating that procedure X outperformed procedure Y if there is no insight related as to why this happened (Sörensen, 2013). Competitive testing fails to yield insight in the performance of algorithms (Hooker, 1995). The journal strives to assess the value of experimental results by their contribution to our understanding of heuristic search instead of whether they show that the polished implementation of a proposed method is able to win a race against the state of the art.

Development of customized solutions

  1. The need for developing a customized solution to a problem must be justified. General-purpose solvers based on exact and heuristic methodologies should be tried first if the goal of the project is to solve a specific problem that requires a search procedure. If these general-purpose optimizers perform adequately for the application being considered, there is no need for a specialized procedure.
     
  2. When the contribution is centered on developing a customized solution for a particular problem (e.g., those submitted to the area of Real-World Applications), considerable effort must be made to assess solution quality. Acceptable practices include but are not limited to measuring optimality gaps with lower or upper bounds and comparing solutions against known results or against results found with general-purpose optimizers. It is not acceptable to simply compare several versions of the same proposed solution method.

Statistically valid experiments and parameter tuning

  1. The Journal of Heuristics requires that the authors conduct statistically valid computational experiments in order to support their statements about the performance of proposed procedures. Statistical validity refers to both the design of experiments and the analysis of the data. Barr, et al. (1995) present guidelines on how to design and perform statistically valid experiments.
     
  2. For procedures that require parameter tuning, the available data must be partitioned into a training and a test set. Tuning should be performed in the training set only. Procedures that are tuned to solve a particular set of problems and that are not able to demonstrate their merit outside the chosen set of instances are of little interest.

References

Barr, R. S., B. L. Golden, J. P. Kelly, M. G. C. Resende and W. R. Stewart (1995) “Designing and Reporting on Computational Experiments with Heuristic Methods,” Journal of Heuristics, vol. 1, no. 1, pp. 9-32

Burke, E. K., T. Curtois, M. Hyde, G. Kendall, G. Ochoa, S. Petrovic, and J. A. Vazquez-Rodriguez (2009) “Towards the Decathlon Challenge of Search Heuristics,”
http://www.cs.stir.ac.uk/~goc/papers/GECCO09Decwk1005-ochoaATS.pdf (this opens in a new tab)

Hooker, J. N. (1995) “Testing Heuristics: We Have It All Wrong,” Journal of Heuristics, vol. 1, no. 1, 33-44.

Sörensen, K. (2013) “Metaheuristics – The Metaphor Exposed,” International Transactions in Operational Research, http://onlinelibrary.wiley.com/doi/10.1111/itor.12001/abstract (this opens in a new tab)

Navigation