학술논문

A review of differentiable digital signal processing for music and speech synthesis
Document Type
article
Source
Frontiers in Signal Processing, Vol 3 (2024)
Subject
digital signal processing
machine learning
audio synthesis
automatic differentiation
neural networks
Electrical engineering. Electronics. Nuclear engineering
TK1-9971
Language
English
ISSN
2673-8198
Abstract
The term “differentiable digital signal processing” describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music and speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably, which is further supported by a web book containing practical advice on differentiable synthesiser programming (https://intro2ddsp.github.io/). Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.