학술논문
一种基于高斯过程与粒子群算法的CNN超参数自动搜索混合模型优化算法 / A hybrid-model optimization algorithm based on the Gaussian process and particle swarm optimization for mixed-variable CNN hyperparameter automatic search
Document Type
Academic Journal
Author
Source
信息与电子工程前沿（英文版） / Frontiers of Information Technology & Electronic Engineering. 24(11):1557-后插4
Subject
Language
Chinese
ISSN
2095-9184
Abstract
卷积神经网络(CNN)在许多实际应用领域中有着快速发展.然而,CNN性能很大程度上取决于其超参数,而为CNN配置合适的超参数通常面临着以下3个挑战:(1)不同类型CNN超参数的混合变量编码问题;(2)评估候选模型的昂贵计算成本问题;(3)确保搜索过程中收敛速率和模型性能问题.针对上述问题,提出一种基于高斯过程(GP)和粒子群优化算法(PS0)的混合模型优化算法(GPPS0),用于自动搜索最优的CNN超参数配置.首先,设计一种新的编码方法高效编码CNN中不同类型的超参数.其次,提出一种混合代理辅助(HSA)模型降低评估候选模型的高计算成本.最后,设计一种新的激活函数改善模型性能并确保收敛速率.在图像分类基准数据集上进行了大量实验,验证GPPS0优于最先进的方法.以金属断口诊断为例,验证GPPS0算法在实际应用中的有效性.实验结果表明,GPPS0仅需0.04和1.70 GPU天即可在CIFAR-10和CIFAR-100数据集上实现95.26％和76.36％识别准确率.
Convolutional neural networks(CNNs)have been developed quickly in many real-world fields.However,CNN's performance depends heavily on its hyperparameters,while finding suitable hyperparameters for CNNs working in application fields is challenging for three reasons:(1)the problem of mixed-variable encoding for different types of hyperparameters in CNNs,(2)expensive computational costs in evaluating candidate hyperparameter configuration,and(3)the problem of ensuring convergence rates and model performance during hyperparameter search.To overcome these problems and challenges,a hybrid-model optimization algorithm is proposed in this paper to search suitable hyperparameter configurations automatically based on the Gaussian process and particle swarm optimization(GPPSO)algorithm.First,a new encoding method is designed to efficiently deal with the CNN hyperparameter mixed-variable problem.Second,a hybrid-surrogate-assisted model is proposed to reduce the high cost of evaluating candidate hyperparameter configurations.Third,a novel activation function is suggested to improve the model performance and ensure the convergence rate.Intensive experiments are performed on image-classification benchmark datasets to demonstrate the superior performance of GPPSO over state-of-the-art methods.Moreover,a case study on metal fracture diagnosis is carried out to evaluate the GPPSO algorithm performance in practical applications.Experimental results demonstrate the effectiveness and efficiency of GPPSO,achieving accuracy of 95.26％and 76.36％only through 0.04 and 1.70 GPU days on the CIFAR-10 and CIFAR-100 datasets,respectively.
Convolutional neural networks(CNNs)have been developed quickly in many real-world fields.However,CNN's performance depends heavily on its hyperparameters,while finding suitable hyperparameters for CNNs working in application fields is challenging for three reasons:(1)the problem of mixed-variable encoding for different types of hyperparameters in CNNs,(2)expensive computational costs in evaluating candidate hyperparameter configuration,and(3)the problem of ensuring convergence rates and model performance during hyperparameter search.To overcome these problems and challenges,a hybrid-model optimization algorithm is proposed in this paper to search suitable hyperparameter configurations automatically based on the Gaussian process and particle swarm optimization(GPPSO)algorithm.First,a new encoding method is designed to efficiently deal with the CNN hyperparameter mixed-variable problem.Second,a hybrid-surrogate-assisted model is proposed to reduce the high cost of evaluating candidate hyperparameter configurations.Third,a novel activation function is suggested to improve the model performance and ensure the convergence rate.Intensive experiments are performed on image-classification benchmark datasets to demonstrate the superior performance of GPPSO over state-of-the-art methods.Moreover,a case study on metal fracture diagnosis is carried out to evaluate the GPPSO algorithm performance in practical applications.Experimental results demonstrate the effectiveness and efficiency of GPPSO,achieving accuracy of 95.26％and 76.36％only through 0.04 and 1.70 GPU days on the CIFAR-10 and CIFAR-100 datasets,respectively.