De-Identification Packages in R

In the context of healthcare data where governance dimensions such as ethics and security can be unclear and prone to interpretations concepts of privacy and anonymity (reffering to GDPR in the EU, HIPAA in the US, PIPEDA in Canada) has now become part of business-as-usual considerations.

General Data Protection Regulation (GDPR)

To protect users’ personally identifiable information (PII-name, emails, physical address, IP address, health information, income, etc) European Union has pput in place GDPR regulation that hold businesses to a higher standard when it comes to how they collect, store, and use the data. The ultimate goal of GDPR is to challenge the data privacy approach of organizations across the world to give EU citizens control over their personal data .

Health Insurance Portability and Accountability Act (HIPAA)

HIPAA is specifically designed for healthcare purposes in 1996 with the perspective of reducing the cost of health care by creating a standardised process for financial transactions and admin. HIPAA also sets out requirements intended to protect different types if sensitive personal health information (PHI) by implementation of data governance procedures. It also specifies the circumstances where healthcare providers may disclose this information to third parties with or without patients’ consent.

The Personal Information Protection and Electronic Documents Act (PIPEDA)

Under Canadian PIPEDA, personal information includes any type of subjective or objective information, about an identifiable person. This includes information in any form, such as:

• Personally identifiable information (PII) including age, name, ID numbers, income, ethnic origin, or blood type
• Subjective opinions, evaluations, as well as intentions (for example, to acquire goods or services, or change jobs), comments, social status, or disciplinary actions
• Factual and background details such as employee files, credit records, loan records, medical records, existence of a dispute between a consumer and a merchant. PIPEDA requires organizations to obtain the data owners’ consent when they collect, use or disclose the information. Personal information can only be used for the purposes for which it was collected in case an organization is going to use it for another purpose, they must obtain consent again. Personal information must be protected by appropriate safeguards. Data owners have the right to access their personal information held by an organization and challenge its accuracy.

Deidentification

Deidentification is the process of removing personally identifiable information (PII) from the raw data. This procedure ensures compliance to legal and ethical standards of data collection, storage, and analysis. PII can include items such as nominal information, social security numbers, health insurance number, and addresses. The procedures and methodology of deidentification is important as it protects the privacy of individuals and prevents any biased use or interpretation of the data.

Risk of re-identification and K-anonymity

A shared database provides K-anonymity protection if the information about each person in the release cannot be distinguished from at least k-1 individuals whose information is also included in the release.

Automated De-Identification Tools

Several automated de-identification programs have been developed in recent years, which can be grouped into three categories:

• Tools for efficiently handling direct identifiers in record-level data,
• Tools for reducing the risk of re-identification from indirect identifiers in record-level data
• Tools for handling aggregate data sets.

Here we are going to focus on some R packages for handling direct identifiers and reducing the risk of re-identification.

Package ‘detector’

The detector package aimes to make identification of data containing PIIs quick, easy, and scalable. It provides high-level functions that can take vectors and dataframes and return important summary statistics in a convenient dataframe foramt. The function detect() is able to detect the different types of PII (e.g. name, social security number, etc.).

if (packageVersion("devtools") < 1.6) {
  install.packages("devtools")
}

#devtools::install_github("paulhendricks/detector")
library(dplyr, warn.conflicts = FALSE)
library(generator)
library(detector)



n <- 10
set.seed(1)
nominal_data <- 
  data.frame(name = r_full_names(n), 
             snn = r_national_identification_numbers(n), 
             dob = r_date_of_births(n), 
             email = r_email_addresses(n), 
             ip = r_ipv4_addresses(n), 
             phone = r_phone_numbers(n), 
             credit_card = r_credit_card_numbers(n), 
             lat = r_latitudes(n), 
             lon = r_longitudes(n), 
             stringsAsFactors = FALSE)
knitr::kable(nominal_data, format = "markdown")

nominal_data %>% 
  detect %>% 
  knitr::kable(format = "markdown")

Package ‘anonymizer’

Package ‘anonymizer’ proposes several options for replacing PIIs with a random unique identifier (Hendricks, 2015). The package can be installed from CRAN or from GitHub depending on your version of R.

#devtools::install_github("paulhendricks/anonymizer")
#install.packages("anonymizer")
library(anonymizer)

non_nominal_data=nominal_data

non_nominal_data[] <- lapply(nominal_data, anonymize, .algo = "crc32")
non_nominal_data %>% 
  knitr::kable(format = "markdown")

|name |snn |dob |email |ip |phone |credit_card |lat |lon | |:——–|:——–|:——–|:——–|:——–|:——–|:———–|:——–|:——–| |f057ecc4 |af429f4f |537f4b33 |e58444a5 |4ed2ca90 |d8ea79bc |d31ff3ae |f0356e76 |b875afeb | |1861927c |838951aa |7b246030 |e0c00f2c |d984e4f6 |e1573c5b |a4252767 |e76dc4be |a14e9a62 | |ba183ba4 |5218cb3b |fa1ed41b |ccc49859 |88a4d3ab |8cca0f9b |84a3aba1 |75c4729d |5b01ffdf | |5de95b08 |fee13b97 |db563d5d |1eccc0ae |474ff3c1 |f18f8cc8 |65f5aef8 |8097921a |18ef073f | |4cdeb6ed |f626cd3c |79159036 |a5832098 |1f661be7 |863347bb |887b9ea4 |981432b6 |c76533d7 | |978c9b5c |b30ee80e |946360cf |3263a481 |5ed07ddd |7e63f48b |4f53569b |1601f541 |72769401 | |c0b887d8 |efc98126 |e46c96ac |dc930c1c |b1a4ebda |01f546dd |9ea6c5f6 |192ac290 |0ceb1823 | |71207b6b |9aff7755 |62a88642 |43ecafc2 |a0da760e |38c225cd |9581f39f |379adeda |89ea2f87 | |d43e1a57 |78fae7be |f4717ee2 |d77b1e73 |9b020da0 |fcf212d7 |21bb4eff |0326a6c5 |62b54aa4 | |af5cdcde |050a5f2f |39989dbb |4d0bf0df |55a1fad0 |d7e7c064 |aa90f82a |f5a02b03 |e5856780 |

Package ‘deidentifyr’

Another package that can be used for data deidentification is ‘deidentifyr.’ This package aims to avoid the potential recovery of hashed PIIs by using a longer SHA-256 hash to generate a unique ID code (Wilcox, 2019). This package is not yet on CRAN, but can be installed from the author’s GitHub. The functtion ‘deidentify()’ will generate a unique ID from personally identifying information. Because the IDs are generated with the SHA-256 algorithm, they are a) very unlikely to be the same for people with different identifying information, and b) nearly impossible to recover the identifying information from.

#devtools::install_github('wilkox/deidentifyr')
library(deidentifyr)
set.seed(1234)
n <- 10
SSNs <- sample(10000000:99999999, n)
DOBs <- lubridate::today() - lubridate::dyears(sample(18:99, n, replace = T))

days_in_hospitals <- sample(1:100, n, replace = T)

nominal_patient_data <- data.frame(SSN = SSNs, DOB = DOBs, 
                           days_in_hospital = days_in_hospitals)
nominal_patient_data%>% 
  knitr::kable(format = "markdown")

masked_patient_data <- deidentify(nominal_patient_data, SSN, DOB)

masked_patient_data%>% 
  knitr::kable(format = "markdown")

sexes <- sample(c("F", "M"), n, replace = T)
nominal_patient_data2 <- data.frame(SSN = SSNs, DOB = DOBs, sex = sexes)

nominal_patient_data2%>% 
  knitr::kable(format = "markdown")

masked_patient_data2 <- deidentify(nominal_patient_data2, DOB, SSN)

masked_patient_data2%>% 
  knitr::kable(format = "markdown")

combined_data <- merge(masked_patient_data, masked_patient_data2, by = "id")
combined_data%>% 
  knitr::kable(format = "markdown")

Package ‘digest’

The third package that can be used in the context of deidentification is ‘digest.’ This package generates hashed character strings based on a variety of algorithms (e.g. md5, sha-1, sha-256, crc32, xxhash, murmurhash, spookyhash and blake3 algorithms). The digest package provides a principal function digest() that creats hash digests of arbitrary R objects permitting easy deidentification of R language objects not limited to vectors and dataframes (Eddelbuettel et al., 2020).

#install.packages("digest")
#install.packages("rbenchmark")
library(digest)
library(rbenchmark)

#vectorisation
md5 <- getVDigest()
digest2 <- base::Vectorize(digest)
x <- rep(letters, 1e3)
rbenchmark::benchmark(
  vdigest = md5(x, serialize = FALSE),
  Vectorize = digest2(x, serialize = FALSE),
  vapply = vapply(x, digest, character(1), serialize = FALSE),
  replications = 5
)[,1:4]

set.seed(1234)
n <- 10
SSNs <- sample(10000000:99999999, n)
DOBs <- lubridate::today() - lubridate::dyears(sample(18:99, n, replace = T))

days_in_hospitals <- sample(1:100, n, replace = T)

nominal_patient_data <- data.frame(SSN = SSNs, DOB = DOBs, 
                                   days_in_hospital = days_in_hospitals)
nominal_patient_data%>% 
  knitr::kable(format = "markdown")

masked_patient_data=nominal_patient_data

masked_patient_data$SSN=md5(as.vector(masked_patient_data$SSN))
masked_patient_data$DOB=md5(as.vector(masked_patient_data$DOB))
masked_patient_data%>% 
  knitr::kable(format = "markdown")

Package ‘duawranglr’

Similar to above this package offers a set of functions to help users create shareable data sets from raw data files with protected elements. Relying on a master crosswalk files that lists variables to be restricted, package functions (e.g. deid_dua) warn users about possible violations of data usage agreement and prevent writing protected elements.

suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(readr))
library(duawranglr)
dua_cw_file <- system.file('extdata', 'dua_cw.csv', package = 'duawranglr')
admin_file <- system.file('extdata', 'admin_data.csv', package = 'duawranglr')
set_dua_cw(dua_cw_file)

see_dua_options(level = c('level_ii', 'level_iii'))

set_dua_level('level_ii', deidentify_required = TRUE, id_column = 'sid')

see_dua_level(show_restrictions = TRUE)

see_dua_level(show_restrictions = TRUE)

df <- read_dua_file(admin_file)
df%>% 
  knitr::kable(format = "markdown")

tmpdir <- tempdir()
df <- deid_dua(df, write_crosswalk = TRUE, id_length = 20, crosswalk_filename = file.path(tmpdir, 'tmp.csv'))
df%>% 
  knitr::kable(format = "markdown")

Package ‘easySdcTable’

This package provides functions to create shareable datasets based on K-anonymity. The main function, ProtectTable(), performs row deletion according to a frequency rule predefined by K-anonymity on a dataset. The functions, protectLinkedTables () or runArgusBatchFile () for linked tables (relational databases) or batch tables that generalize the same structure.

library(easySdcTable)

## Loading required package: SSBtools

## Loading required package: Matrix

set.seed(1234)
n <- 1000
SSNs <- sample(10000000:99999999, n)
DOBs <- lubridate::today() - lubridate::dyears(sample(1:100, n, replace = T))
Region <- sample(c("a","b","c", "d","e","f","g"), n, replace = T)
Sex <- sample(c("m","w"), n, replace = T)
days_in_hospitals <- sample(1:100, n, replace = T)
nominal_patient_data <- data.frame(SSN = SSNs, DOB = DOBs, days_in_hospital = days_in_hospitals, Region=Region, Sex=Sex)
K_ano_patient_data=ProtectTable(nominal_patient_data,dimVar = c("Region","Sex"), freqVar = c("days_in_hospital"), method="HITAS") 

head(K_ano_patient_data$data)%>% 
  knitr::kable(format = "markdown")

• Paul Hendricks (2015). “Package ‘detector’.”
• Paul Hendricks (2015). “anonymizer: Anonymize Data Containing Personally Identifiable Information.”
• Wilcox (2019). “deidentify: Deidentify a dataset.”
• Eddelbuettel , et. al (2020). “Package ‘digest’.”
• Benjamin Skinner (2020) “Package ‘duawranglr’.”
• Cassidy Kantoris (2020) Deidentifying Data
• Sarah Kooper, Anja Sautmann, and James Turrito(2020) J-PAL Guide to De-Identifying Data
• Øyvind Langsrud (2020) “Package ‘easySdcTable’.”
• Martin Monkman(2020) Data Science with R: A Resource Compendium