Assessment framework for deepfake detection in real-world situations

EURASIP Journal on Image and Video Processing

Table 2 AUC (%) scores of the Capsule-Forensics, denoted as CapsuleNet, XceptionNet, and UIA-VIT methods tested on unaltered and distorted variants of FFpp and Celeb-DFv2 test set respectively. Raw, C23, and Full refer to different quality settings of the FFpp. DL-Comp refers to deep learning-based compression [60] and ‘High’ refers to the high-quality compressed image

Methods	TrainSet	Unaltered	JPEG				DL-Comp				Gaussian Noise				Po-Gau Noise
			95	60	30	AVG	High	Med	Low	AVG	5	10	30	AVG
CapsuleNet	FFpp-Raw	99.20	97.91	76.48	59.60	78.00	55.24	54.50	50.92	53.55	61.80	59.73	51.26	57.60	55.63
	FFpp-C23	96.32	95.09	95.76	74.91	88.59	56.96	57.42	81.57	65.32	84.51	78.56	58.63	73.90	70.59
	FFpp-Full	94.52	94.95	92.18	84.50	90.54	86.83	60.98	55.69	67.83	89.03	78.54	57.95	75.17	64.87
	CelebDFv2	99.14	99.32	98.88	93.07	97.09	99.01	96.77	88.95	94.91	95.24	63.27	59.20	72.57	87.06
XceptionNet	FFpp-Raw	99.56	76.77	56.00	54.20	62.32	50.16	50.37	50.10	50.21	50.12	51.00	50.36	50.49	48.98
	FFpp-C23	98.64	98.01	95.99	82.77	92.26	96.11	56.25	55.71	69.36	71.35	55.84	50.87	59.35	51.48
	FFpp-Full	99.02	99.00	94.78	87.86	93.88	94.36	54.88	55.78	68.34	59.00	55.09	54.08	56.06	51.43
	CelebDFv2	99.73	99.78	99.59	97.76	99.04	96.23	90.23	75.46	87.31	94.85	69.87	52.50	72.41	86.87
UIA-VIT	FFpp-Raw	99.38	99.30	95.16	84.92	93.13	89.19	57.49	56.75	67.81	96.86	89.10	72.32	86.09	82.97
UIA-VIT	FFpp-C23	97.99	97.94	96.86	91.92	95.57	88.72	65.32	59.32	71.12	97.02	93.05	74.50	88.19	85.37

Methods	TrainSet	Gaussian Blur				Gamma Correction					Resize				Overall Average
		3	7	11	AVG	0.1	0.75	1.3	2.5	AVG	x4	x8	x16	AVG
CapsuleNet	FFpp-Raw	67.19	58.22	52.26	59.22	50.50	98.86	99.17	96.12	86.16	67.48	53.18	53.10	57.92	65.41
	FFpp-C23	85.21	53.94	52.04	63.73	52.08	95.06	96.72	92.91	84.19	79.33	64.62	50.33	64.76	73.41
	FFpp-Full	85.72	58.83	56.05	66.87	56.02	93.86	93.87	85.44	82.30	84.65	65.34	52.02	67.34	75.01
	CelebDFv2	99.01	91.04	77.52	89.19	79.60	98.52	99.40	94.62	93.04	89.22	66.98	61.94	72.71	86.58
XceptionNet	FFpp-Raw	68.76	55.61	50.70	58.36	54.66	98.66	99.57	70.45	80.84	68.60	55.80	50.45	58.28	60.08
	FFpp-C23	95.38	59.92	52.33	69.21	58.43	98.34	98.64	91.35	86.69	92.55	70.27	56.00	72.94	74.97
	FFpp-Full	96.36	70.51	54.50	73.79	51.38	98.91	98.84	88.91	84.51	93.47	75.30	60.55	76.44	75.50
	CelebDFv2	98.77	91.81	79.94	90.17	74.97	99.53	99.74	97.49	92.93	96.01	72.21	63.03	77.08	86.49
UIA-VIT	FFpp-Raw	98.81	86.71	72.62	86.05	57.35	99.05	99.04	89.14	86.15	98.44	87.14	61.37	82.32	85.49
UIA-VIT	FFpp-C23	91.19	65.74	64.08	73.67	57.34	96.55	96.28	88.32	84.62	94.24	81.35	60.24	78.61	84.39