Fast and Scalable Local Kernel Machines

Segata, Nicola and Blanzieri, Enrico (2009) Fast and Scalable Local Kernel Machines. UNSPECIFIED. (Unpublished)

Download (498Kb) | Preview


    A computationally efficient approach to local learning with kernel methods is presented. The Fast Local Kernel Support Vector Machine (FaLK-SVM) trains a set of local SVMs on redundant neighbourhoods in the training set and an appropriate model for each query point is selected at testing time according to a proximity strategy. Supported by a recent result by Zakai and Ritov (2009) relating consistency and localizability, our approach guarantees high generalization ability by dividing the separation function in local optimization problems that can be handled very efficiently. The introduction of a fast local model selection further speeds-up the learning process. Learning and complexity bounds are derived for FaLK-SVM, and the empirical evaluation of the approach (with datasets up to 3 million points) showed that it is much faster and more accurate and scalable than state-of-the-art accurate and approximated SVM solvers at least for non high-dimensional datasets. More generally, we show that locality can be an important factor to sensibly speed-up learning approaches and kernel methods, differently from other recent techniques that tend to dismiss local information in order to improve scalability.

    Item Type: Departmental Technical Report
    Department or Research center: Information Engineering and Computer Science
    Subjects: Q Science > QA Mathematics > QA075 Electronic computers. Computer science
    Uncontrolled Keywords: Locality, Kernel Methods, Local Learning Algorithms, Support Vector Machines, Memory-Based Learning
    Report Number: DISI-09-072
    Repository staff approval on: 20 Jan 2010

    Actions (login required)

    View Item