Fusing Elasticsearch with neural networks to identify personal data

Ryan Turner

Senior ML Researcher

Twitter

Rakshit Wadhwa

Senior Software Engineer

Twitter

A critical aspect of incorporating data privacy is the process of classifying personal or sensitive data. The personal data protection (PDP) annotation system provides a solution to automatically classify columns of data in databases at Twitter. Listen in as we explore the annotation system that combines Elasticsearch queries with a neural network to provide probabilistically calibrated predictions on PDP data types for every column.