Assessment framework for deepfake detection in real-world situations

EURASIP Journal on Image and Video Processing

Table 4 AUC (%) scores of cores of the Capsule-Forensics, denoted as CapsuleNet, and XceptionNet methods tested on unaltered and distorted variants of FFpp

Methods	TrainScheme	Unaltered	JPEG				DL-Comp				Gaussian Noise				Po-Gau Noise
			95	60	30	AVG	High	Med	Low	AVG	5	10	30	AVG
CapsuleNet	FFpp-Raw	99.20	97.91	76.48	59.60	78.00	55.24	54.50	50.92	53.55	61.80	59.73	51.26	57.60	55.63
	FFpp-Full	94.52	94.95	92.18	84.50	90.54	86.83	60.98	55.69	67.83	89.03	78.54	57.95	75.17	64.87
	FFpp-Augmix	98.6	98.68	79.67	57.62	78.66	71.10	53.51	51.61	58.74	75.11	61.93	56.80	64.61	59.19
	FFpp-DAug	93.06	92.90	91.24	88.90	91.01	90.35	81.96	70.00	80.77	92.50	88.78	79.99	87.09	86.63
	FFpp-SDAug	98.16	97.97	96.36	94.08	96.14	93.81	71.41	59.74	74.99	97.05	93.89	83.51	91.48	87.06
XceptionNet	FFpp-Raw	99.56	76.77	56.00	54.20	62.32	50.16	50.37	50.10	50.21	50.12	51.00	50.36	50.49	51.02
	FFpp-Full	99.02	99.00	94.78	87.86	93.88	94.36	54.88	55.78	68.34	59.00	55.09	54.08	56.06	51.43
	FFpp-Augmix	99.15	87.12	63.38	59.58	70.03	77.86	62.07	55.76	65.23	77.37	68.45	56.99	67.60	62.00
	FFpp-DAug	89.51	89.47	89.27	89.00	89.25	89.49	88.71	86.16	88.12	89.43	89.30	88.22	88.98	88.97
	FFpp-SDAug	98.44	98.25	97.36	96.12	97.24	98.03	87.76	82.74	89.51	97.37	95.88	91.71	94.99	94.57

Methods	TrainScheme	Gaussian Blur				Gamma Correction					Resize				Overall Average
		3	7	11	AVG	0.1	0.75	1.3	2.5	AVG	x4	x8	x16	AVG
CapsuleNet	FFpp-Raw	67.19	58.22	52.26	59.22	50.50	98.86	99.17	96.12	86.16	67.48	52.18	53.10	57.59	65.35
	FFpp-Full	85.72	58.83	56.05	66.87	56.02	93.86	93.87	85.44	82.30	84.65	65.34	52.02	67.34	75.01
	FFpp-Augmix	90.86	54.08	50.67	65.20	76.17	98.57	98.42	94.53	91.92	89.62	61.39	50.58	67.20	71.06
	FFpp-DAug	91.79	86.00	79.95	85.91	67.39	92.40	93.13	91.83	86.19	88.42	77.06	55.22	73.57	84.09
	FFpp-SDAug	96.86	90.32	80.31	89.16	60.17	97.68	98.18	96.91	88.24	93.54	79.22	58.05	76.94	86.16
XceptionNet	FFpp-Raw	68.76	55.61	50.70	58.36	54.66	98.66	99.57	70.45	80.84	68.60	55.80	50.45	58.28	60.08
	FFpp-Full	96.36	70.51	54.50	73.79	51.38	98.91	98.84	88.91	84.51	93.47	75.30	60.55	76.44	75.50
	FFpp-Augmix	90.45	62.58	53.00	68.68	93.45	99.33	98.32	85.87	94.24	64.64	54.57	50.00	56.40	70.36
	FFpp-DAug	89.22	87.62	85.28	87.37	69.08	89.42	89.35	87.74	83.90	88.31	81.30	63.89	77.83	85.91
	FFpp-SDAug	98.31	97.35	94.51	96.72	80.48	98.25	98.44	97.75	93.73	97.30	86.26	67.14	83.57	92.63

The suffix +DAug denotes that the model is trained with the proposed augmentation chain but without the stochastic manner. The suffix +SDAug denotes that the model is trained with the stochastic degradation-based augmentation technique. In this table, Bold font denotes the highest score