Recent developments in advanced generative deep learning techniques have led to considerable progress in deepfake technology. CNN-based deepfake detection approaches have demonstrated superior performance. The ability to learn meaningful representations generated by convolutional multilayer nonlinear structures is the key to success. However, the black-box nature of such approaches has been a major concern for exploring hidden and complex characteristics as well as potential limitations of CNN-based models. To gain insights into the scope of the deepfake detection task, we investigate the effectiveness of handcrafted feature-based methods for deepfake video detection. First, we experiment with six top-performing handcrafted descriptors to extract the discriminating image features and then train SVMs on the extracted features to learn a suitable model. We also study the effect of selecting specific facial components on the detection performance. Specifically, we consider features extracted from the left eye, right eye, mouth, and entire face. Moreover, we propose a combination of these features and highlight the importance of this combination in terms of detection performance. Experimental results show that the SIFT feature descriptor achieves the best performance on deepfake videos generated by the neural texture technique, with a detection accuracy of 83.50%, which is better than deep learning-based methods. This is in contrast to the conventional understanding that deep learning methods systematically outperform handcrafted feature-based approaches. In addition, the obtained results on the FaceForensics++ dataset highlight the benefit of using some facial components to further boost the detection performance. Moreover, motivated by the effectiveness of the LBPTOP and SIFT in the deepfake detection task, we combined the LBPTOP and SIFT to best characterize the specific spatiotemporal inconsistencies commonly found in fake videos for boosting deepfake detection performance. Finally, we show the strengths and weaknesses of methods based on handcrafted features for deepfake detection and provide directions for future research.
Film grain is often a desirable feature in video production, creating a natural appearance and contributing to the expression of creative intent. Film grain, however, does not compress well with modern video compression standards, such as Versatile Video Coding (VVC) also known as ITU-T H.266 and ISO/IEC 23090-3. Indeed, within various filtering and lossy compression steps, film grain is suppressed without the possibility of recovering it. One option to alleviate this problem is to use lower quantization parameters to better preserve fine details such as film grain. However, this may strongly increase the bitrate. In some scenarios, information on film grain can be communicated as metadata through for instance an SEI message specified by Versatile Supplemental Enhancement Information (VSEI, also known as ITU-T Recommendation H.274 and ISO/IEC 23002-7). Thus, film grain is often modeled and removed prior to compression, and it is then synthesized at the decoder side with the aid of appropriate metadata. In addition, film grain can also be used as a tool to mask coding artifacts introduced by the compression. Different approaches have been studied for film grain modeling. In the context of the novel VVC standard, a frequency filtering solution to parameterize and synthesize film grain can be used. This paper provides an overview of such film grain VVC-compatible technology, including parameterization, signaling and decoder side synthesis. Thus, in this paper, an approach based on the frequency filtering is firstly summarized. Then, a quantitative and qualitative simulations are preformed to show the benefits of film grain parameterization in terms of the bitrate savings for the same perceived quality.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.