e-Article
딥러닝 기반 영상 콘텐츠의 메타데이터 생성에 관한 연구 / A Study on Deep Learning Based Metadata Generation of Video Contents
Document Type
Dissertation/ Thesis
Author
Source
Subject
Language
Korean
Abstract
본 논문에서는 서버-클라이언트 구조에서 사용자 요청에 따라 영상 콘텐츠 메타데이터를 생성하는 시스템을 제안한다. 시스템은 크게 영상 학습 및 분석부, 학습데이터 구축부, 메타데이터 생성부 세 부분으로 나뉘며, 이 중 메타데이터 생성부와 학습데이터 구축부는 서버 역할로, 클라이언트와 REST 기반의 정보교환이 이루어진다. 영상 학습 및 분석부에서는 딥러닝 기반 얼굴 및 객체 인식 알고리즘이 수행되며, 메타데이터 생성부는 이를 이용하여 얼굴 및 객체 정보, 샷 경계를 추출하고 그 결과를 반환한다. 그리고 학습데이터 구축부는 딥러닝 모델을 업데이트하기 위한 학습데이터를 반자동으로 구축하는 역할을 수행하며, 얼굴 및 객체 인식, 추적, 클러스터링 기능을 포함한다. 제안하는 시스템을 구현하기 위해 924개의 클래스에 대한 초기 학습데이터를 구축하였으며, 학습데이터 구축부와 메타데이터 생성부에서는 REST 통신을 위해 각 부분별로 데이터 교환 내용 및 URI를 지정하였다. 또한, 일부 기능을 별도의 스레드로 동작하도록 하여, 영상 분석과 같이 응답 대기 시간이 오래 걸리는 경우에 처리가 다 끝나지 않아도 요청 시 현재의 진행률을 반환하도록 하였다. 그 후 처리가 끝나면, 복잡한 구조의 내용을 nested JSON 형태로 보낸다. 해당 내용에는 프레임 번호, 객체 정보 등이 포함된다. 실험은 초기 생성한 학습데이터를 이용하여 얼굴 인식 정확도를 측정하고, 서버-클라이언트 간 요청 및 응답 테스트를 수행함으로써 이루어졌다. 학습데이터 중 중복되거나 비슷한 데이터를 제거함으로써 정확도가 향상되었고, 각 기능에 따른 클라이언트 요청메시지를 올바르게 파싱하고, 해당 기능을 수행하여 응답함을 확인하였다.
In this paper, we proposed a system in which server-client architectures generate image content metadata according to user requests. The system is largely divided into three parts: the Video Learning and Analysis Unit, the Learning Data Building Unit, and the Metadata Generation Unit. the Metadata Generation Unit and the Learning Data Building Unit are server roles, and information exchange based on the REST (Representational State Transfer) is conducted with the client. Deep Learning-based face and object recognition algorithms are performed by the Video Learning and Analysis Unit, and the Metadata Generation Unit uses them to obtain face and object information, shot boundaries, and return the results. In addition, the Learning Data Building Unit plays a role of semi-automatically building learning data to update the deep learning model by face and object recognition, tracking, and clustering. In order to implement the proposed system, initial learning data was obtained for 924 classes, and the data exchange content and URI were specified for each part for REST communication. In addition, some functions were required to operate as separate threads so that the response latency, such as image analysis, is returned to the current progress on request, when the processing was not complete. Then, when the processing is complete, the contents of the complex structure are structured and sent in the form of nested JSON. The experiment was conducted by measuring face recognition accuracy using initially obtained learning data and conducting requests and response tests between servers and clients. It was confirmed that accuracy was improved by removing duplicate or similar data from the learning data, that client request messages based on each function were parsed correctly, and that the function was performed to respond.
In this paper, we proposed a system in which server-client architectures generate image content metadata according to user requests. The system is largely divided into three parts: the Video Learning and Analysis Unit, the Learning Data Building Unit, and the Metadata Generation Unit. the Metadata Generation Unit and the Learning Data Building Unit are server roles, and information exchange based on the REST (Representational State Transfer) is conducted with the client. Deep Learning-based face and object recognition algorithms are performed by the Video Learning and Analysis Unit, and the Metadata Generation Unit uses them to obtain face and object information, shot boundaries, and return the results. In addition, the Learning Data Building Unit plays a role of semi-automatically building learning data to update the deep learning model by face and object recognition, tracking, and clustering. In order to implement the proposed system, initial learning data was obtained for 924 classes, and the data exchange content and URI were specified for each part for REST communication. In addition, some functions were required to operate as separate threads so that the response latency, such as image analysis, is returned to the current progress on request, when the processing was not complete. Then, when the processing is complete, the contents of the complex structure are structured and sent in the form of nested JSON. The experiment was conducted by measuring face recognition accuracy using initially obtained learning data and conducting requests and response tests between servers and clients. It was confirmed that accuracy was improved by removing duplicate or similar data from the learning data, that client request messages based on each function were parsed correctly, and that the function was performed to respond.