Even though local data is not directly shared, standard Federated Learning (FL) architectures (see Figure 1) still raise important privacy and security concerns that hinder their practical adoption. Privacy risks can emerge through the messages exchanged during training, the final model (if published), or even the model’s predictions.
Notably, recent studies have shown that adversaries (like our purple devil in Figure 1) can extract private information from local datasets using the information exchanged between nodes. This is especially relevant when using gradient-based methods, which are of particular concern in the TRUMPET project. If adversaries gain access to these updates, they can carry out targeted privacy attacks, such as inferring whether a specific record was part of the training data, or reconstructing sensitive attributes of individual records.
TRUMPET and sensitive medical data use cases
These privacy concerns are especially relevant in TRUMPET’s three main use cases, which involve collaboration between hospitals. The data in these cases is highly sensitive, containing detailed patient information related to different types of cancer and treatment responses.
A non-exhaustive list of attributes in the patient records includes:
- Demographic information (e.g., age, sex, smoking habits)
- Social and psychological status (e.g., living situation, occupation)
- Comorbidities (e.g., diabetes, cardiovascular disease)
- Laboratory test results (e.g., presence of gene alterations)
- Treatment history (e.g., treatment types and timelines)
- Disease progression (e.g., stage progression and dates)
Privacy metrics in TRUMPET
Given the breadth of sensitive information associated with each patient record, any data leakage would constitute a serious privacy breach. In some cases—such as those involving genetic alterations—the privacy risks can even extend to a patient’s relatives. Some attributes, especially demographic or social ones, might be publicly available or easier for an attacker to guess. If an adversary knows part of a patient’s information and gains access to a model trained on that data, they could infer additional sensitive features (e.g., whether the patient has a cardiovascular condition).
To assess and mitigate these risks, TRUMPET proposes integrating dedicated privacy metrics into the FL workflow. These metrics are designed to measure how much private information could still be inferred from model updates. By doing so, we aim to quantify the risks introduced by practical inference attacks, enabling hospitals to actively evaluate and manage their exposure while participating in FL collaborations.
Example privacy attacks and threats
To illustrate the privacy threats measured in TRUMPET, let’s consider a simplified example involving a lung cancer risk prediction task. In this scenario, a network of hospitals collaborates to train a model without sharing raw patient data. Each hospital trains its own local model and sends updates (e.g., gradients or parameters) to a central server. The server aggregates these updates to build a more accurate global model. However, if the aggregator acts as an adversary, it can analyze the updates to infer sensitive details.
For instance, consider a record for a hypothetical patient, Alice. Her record includes demographic information (e.g., sex, smoking habits) and sensitive medical data (e.g., presence of cardiovascular disease, EGFR gene mutations). EGFR mutations are linked to abnormal cell growth and several cancers, including lung cancer. Her cancer status serves as the label in this classification task.
Figure 2 illustrates two attack scenarios:
- Attribute Inference: The adversary already knows that Alice is female, smokes a certain amount daily, and has a cardiovascular condition. With access to gradients or model updates, they try to infer whether Alice has lung cancer or an EGFR mutation.
- Membership Inference: In this attack, the adversary wants to know whether Alice’s record was used in training. Suppose the adversary knows that Alice is 47 years old. In the general population, 350 out of 100,000 people aged 45–49 have a certain cancer (see https://www.cancer.gov/about-cancer/causes-prevention/risk/age). But if the adversary discovers that Alice’s record was used in training a balanced dataset (i.e., 50% cancer cases), they may now estimate a 0.5 probability that she has cancer—significantly higher than the baseline. Even without certainty, the adversary gains meaningful insight, posing a clear privacy risk.
Alberto Pedrouzo-Ulloa, Fernando Pérez-González (UVIGO)