Beyond EER: Multi-Dimensional Evaluation of Identity Leakage in Speaker De-Identification

Published

September 1, 2026

Author(s)

Seungmin Seo, Oleg Aulov, P Phillips, Kevin Mangold, Jonathan Eskin

Abstract

Speaker de-identification (SDID) aims to preserve privacy by concealing speaker identity while maintaining speech utility. However, current evaluations often reduce privacy to a single dimension—biometric verification performance—typically measured by Equal Error Rate (EER). This narrow focus ignores critical leakage channels, such as soft biometric inference, embedding-level re-identification, and structural template similarity, which threaten the unlinkability and irreversibility of biometric references. We propose a holistic evaluation framework across five complementary metrics: (i) EER, (ii) soft biometric leakage score , (iii) cumulative match characteristic (CMC) re-identification analysis, (iv) canonical correlation analysis (CCA) and Procrustes embedding alignment, and (v) intelligibility via word error rate and semantic similarity. Evaluating five SDID systems from the IARPA ARTS program, we demonstrate that these metrics capture independent dimensions of identity leakage. Our results indicate that reliance on a single metric can misrepresent the privacy properties of an SDID system.

Proceedings Title

IEEE/IAPR INTERNATIONAL JOINT CONFERENCE ON BIOMETRICS (IJCB 2026)

Conference Dates

September 1-4, 2026

Conference Location

Rome, IT

Pub Type

Conferences

Keywords

speaker identification, speaker de-identification, speaker privacy, identity leakage

Citation

Seo, S. , Aulov, O. , Phillips, P. , Mangold, K. and Eskin, J. (2026), Beyond EER: Multi-Dimensional Evaluation of Identity Leakage in Speaker De-Identification, IEEE/IAPR INTERNATIONAL JOINT CONFERENCE ON BIOMETRICS (IJCB 2026), Rome, IT (Accessed September 18, 2026)

Additional citation formats