학술논문

Using Gradient Based Multikernel Gaussian Process and Meta-Acquisition Function to Accelerate SMBO
Document Type
Conference
Source
2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI) Tools with Artificial Intelligence (ICTAI), 2019 IEEE 31st International Conference on. :440-447 Nov, 2019
Subject
Computing and Processing
Robotics and Control Systems
Signal Processing and Analysis
SMBO, black box optimization, hypergradient, metalearning
Language
ISSN
2375-0197
Abstract
Automatic machine learning (automl) is a crucial technology in machine learning. Sequential model-based optimisation algorithms (SMBO) (e.g., SMAC, TPE) are state-of-the-art hyperparameter optimisation methods in automl. However, SMBO does not consider known information, like the best hyperparameters high possibility range and gradients. In this paper, we accelerate the traditional SMBO method and name our method as accSMBO. In accSMBO, we build a gradient-based multikernel Gaussian process with a good generalisation ability and we design meta-acquisition function which encourages that SMBO puts more attention on the best hyperparameters high possibility range. In L2 norm regularised logistic loss function experiments, our method exhibited state-of-the-art performance.