A Memory-Based RASH Optimizer

Brunato, Mauro and Battiti, Roberto and Pasupuleti, Srinivas (2006) A Memory-Based RASH Optimizer. Technical Report DIT-06-023, Ingegneria e Scienza dell'Informazione, University of Trento.

Abstract

This paper presents a memory-based Reactive Affine Shaker (M-RASH) algorithm for global optimization. The Reactive Affine Shaker is an adaptive search algorithm based only on the function values. M-RASH is an extension of RASH in which good starting points to RASH are suggested online by using Bayesian Locally Weighted Regression (B-LWR). Both techniques use the memory about the previous history of the search to guide the future exploration but in very different ways. RASH compiles the previous experience into a local search area where sample points are drawn, while locally-weighted regression saves the entire previous history to be mined extensively when an additional sample point is generated.
Because of the high computational cost related to the B-LWR model, it is applied only to evaluate the potential of an initial point for a local search run. The experimental results, focussed onto the case when the dominant computational cost is the evaluation of the target $f$ function, show that M-RASH is indeed capable of leading to good results for a smaller number of function evaluations.