Hugging Face's logo

LiquidAI
/
LFM2.5-Encoder-350M-PII-Detector

Token Classification
Transformers
Safetensors
lfm2
liquid
lfm2.5
bidirectional
masked-lm
encoder
pii
ner
privacy
multilingual
custom_code
Model card Files
xet
 Community
2
yungisimon commited on
Commit
984783a
·
verified ·
1 Parent(s): 64b744b

v3a: +public train splits (in-distribution) -> #1 on all 4 clean benchmarks

Browse files
Files changed (4) hide show
  1. README.md +6 -6
  2. leaderboard.png +2 -2
  3. model.safetensors +1 -1
  4. modeling_lfm2_tc.py +1 -1
README.md CHANGED
@@ -54,7 +54,7 @@ _clean · synthetic · tests PII-vs-NER discrimination_
54
 
55
  | Model | P | R | F1 | F2 |
56
  |---|---|---|---|---|
57
- | trained:pii-detect-v2-pkg | 0.326 | 0.746 | 0.453 | 0.593 |
58
  | regex baseline | 0.259 | 0.579 | 0.358 | 0.464 |
59
  | openai/privacy-filter | 0.169 | 0.595 | 0.264 | 0.396 |
60
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.160 | 0.513 | 0.244 | 0.356 |
@@ -67,9 +67,9 @@ _clean · synthetic English_
67
 
68
  | Model | P | R | F1 | F2 |
69
  |---|---|---|---|---|
 
70
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.887 | 0.735 | 0.804 | 0.761 |
71
  | OpenMed/privacy-filter-nemotron | 0.762 | 0.773 | 0.767 | 0.770 |
72
- | trained:pii-detect-v2-pkg | 0.829 | 0.695 | 0.756 | 0.718 |
73
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.722 | 0.686 | 0.703 | 0.693 |
74
  | iiiorg/piiranha-v1 | 0.633 | 0.490 | 0.552 | 0.513 |
75
  | openai/privacy-filter | 0.455 | 0.462 | 0.459 | 0.461 |
@@ -80,11 +80,11 @@ _clean · human gold · English legal (ECHR)_
80
 
81
  | Model | P | R | F1 | F2 |
82
  |---|---|---|---|---|
 
83
  | OpenMed/privacy-filter-nemotron | 0.728 | 0.625 | 0.673 | 0.643 |
84
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.984 | 0.406 | 0.575 | 0.460 |
85
  | openai/privacy-filter | 0.933 | 0.383 | 0.543 | 0.434 |
86
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.995 | 0.325 | 0.490 | 0.375 |
87
- | trained:pii-detect-v2-pkg | 0.860 | 0.176 | 0.293 | 0.210 |
88
  | iiiorg/piiranha-v1 | 0.721 | 0.160 | 0.262 | 0.189 |
89
  | regex baseline | 0.000 | 0.000 | 0.000 | 0.000 |
90
 
@@ -93,12 +93,12 @@ _clean · human gold · multilingual EU legal_
93
 
94
  | Model | P | R | F1 | F2 |
95
  |---|---|---|---|---|
 
96
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.527 | 0.194 | 0.283 | 0.222 |
97
  | iiiorg/piiranha-v1 | 0.465 | 0.181 | 0.261 | 0.206 |
98
  | openai/privacy-filter | 0.372 | 0.186 | 0.248 | 0.207 |
99
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.274 | 0.140 | 0.185 | 0.155 |
100
  | OpenMed/privacy-filter-nemotron | 0.081 | 0.180 | 0.111 | 0.144 |
101
- | trained:pii-detect-v2-pkg | 0.583 | 0.032 | 0.061 | 0.040 |
102
  | regex baseline | 0.000 | 0.000 | 0.000 | 0.000 |
103
 
104
 
@@ -112,8 +112,8 @@ _⚠ in-distribution for OpenMed-nemotron & nvidia GLiNER (read as ceilings)_
112
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.894 | 0.518 | 0.656 | 0.566 |
113
  | iiiorg/piiranha-v1 | 0.841 | 0.538 | 0.656 | 0.580 |
114
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.815 | 0.533 | 0.644 | 0.572 |
 
115
  | openai/privacy-filter | 0.699 | 0.484 | 0.572 | 0.516 |
116
- | trained:pii-detect-v2-pkg | 0.843 | 0.427 | 0.567 | 0.474 |
117
  | regex baseline | 0.645 | 0.227 | 0.335 | 0.260 |
118
 
119
  ### AI4PRIVACY (660 samples)
@@ -126,7 +126,7 @@ _⚠ in OUR backbone's pretraining — contaminated for us; reference only_
126
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.399 | 0.614 | 0.484 | 0.554 |
127
  | OpenMed/privacy-filter-nemotron | 0.341 | 0.541 | 0.418 | 0.484 |
128
  | openai/privacy-filter | 0.316 | 0.492 | 0.385 | 0.443 |
129
- | trained:pii-detect-v2-pkg | 0.343 | 0.284 | 0.311 | 0.294 |
130
  | regex baseline | 0.269 | 0.147 | 0.190 | 0.162 |
131
 
132
 
 
54
 
55
  | Model | P | R | F1 | F2 |
56
  |---|---|---|---|---|
57
+ | trained:pii-detect-v3a | 0.304 | 0.742 | 0.431 | 0.576 |
58
  | regex baseline | 0.259 | 0.579 | 0.358 | 0.464 |
59
  | openai/privacy-filter | 0.169 | 0.595 | 0.264 | 0.396 |
60
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.160 | 0.513 | 0.244 | 0.356 |
 
67
 
68
  | Model | P | R | F1 | F2 |
69
  |---|---|---|---|---|
70
+ | trained:pii-detect-v3a | 0.815 | 0.850 | 0.832 | 0.843 |
71
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.887 | 0.735 | 0.804 | 0.761 |
72
  | OpenMed/privacy-filter-nemotron | 0.762 | 0.773 | 0.767 | 0.770 |
 
73
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.722 | 0.686 | 0.703 | 0.693 |
74
  | iiiorg/piiranha-v1 | 0.633 | 0.490 | 0.552 | 0.513 |
75
  | openai/privacy-filter | 0.455 | 0.462 | 0.459 | 0.461 |
 
80
 
81
  | Model | P | R | F1 | F2 |
82
  |---|---|---|---|---|
83
+ | trained:pii-detect-v3a | 0.897 | 0.746 | 0.815 | 0.772 |
84
  | OpenMed/privacy-filter-nemotron | 0.728 | 0.625 | 0.673 | 0.643 |
85
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.984 | 0.406 | 0.575 | 0.460 |
86
  | openai/privacy-filter | 0.933 | 0.383 | 0.543 | 0.434 |
87
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.995 | 0.325 | 0.490 | 0.375 |
 
88
  | iiiorg/piiranha-v1 | 0.721 | 0.160 | 0.262 | 0.189 |
89
  | regex baseline | 0.000 | 0.000 | 0.000 | 0.000 |
90
 
 
93
 
94
  | Model | P | R | F1 | F2 |
95
  |---|---|---|---|---|
96
+ | trained:pii-detect-v3a | 0.499 | 0.473 | 0.486 | 0.478 |
97
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.527 | 0.194 | 0.283 | 0.222 |
98
  | iiiorg/piiranha-v1 | 0.465 | 0.181 | 0.261 | 0.206 |
99
  | openai/privacy-filter | 0.372 | 0.186 | 0.248 | 0.207 |
100
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.274 | 0.140 | 0.185 | 0.155 |
101
  | OpenMed/privacy-filter-nemotron | 0.081 | 0.180 | 0.111 | 0.144 |
 
102
  | regex baseline | 0.000 | 0.000 | 0.000 | 0.000 |
103
 
104
 
 
112
  | LiquidAI demo (LFM2.5-PII-Demo) | 0.894 | 0.518 | 0.656 | 0.566 |
113
  | iiiorg/piiranha-v1 | 0.841 | 0.538 | 0.656 | 0.580 |
114
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.815 | 0.533 | 0.644 | 0.572 |
115
+ | trained:pii-detect-v3a | 0.807 | 0.516 | 0.629 | 0.556 |
116
  | openai/privacy-filter | 0.699 | 0.484 | 0.572 | 0.516 |
 
117
  | regex baseline | 0.645 | 0.227 | 0.335 | 0.260 |
118
 
119
  ### AI4PRIVACY (660 samples)
 
126
  | VAGOsolutions/SauerkrautLM-GLiNER | 0.399 | 0.614 | 0.484 | 0.554 |
127
  | OpenMed/privacy-filter-nemotron | 0.341 | 0.541 | 0.418 | 0.484 |
128
  | openai/privacy-filter | 0.316 | 0.492 | 0.385 | 0.443 |
129
+ | trained:pii-detect-v3a | 0.339 | 0.442 | 0.384 | 0.416 |
130
  | regex baseline | 0.269 | 0.147 | 0.190 | 0.162 |
131
 
132
 
leaderboard.png CHANGED

Git LFS Details

  • SHA256: cfe7f418e61fcdedf69ac762b493a2de0edfef5b62a0f01e95c13804034e4408
  • Pointer size: 131 Bytes
  • Size of remote file: 136 kB

Git LFS Details

  • SHA256: a3d0ae7f2e3f9b736dbc9c9e70377174bbdc2ff8ccf8215ed3f8922f124258b0
  • Pointer size: 131 Bytes
  • Size of remote file: 130 kB
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:83b023af61276b5c2c50e422ae83f83065caa3c66dbd0a75d72ed5a4c381c174
3
  size 1250657676
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c863eb56a6c0ddcf1ed1fc83d14ac07a4f182e0f1aeba6fd583c65978ee766d4
3
  size 1250657676
modeling_lfm2_tc.py CHANGED
@@ -354,7 +354,7 @@ class Lfm2BidirModel(PreTrainedModel):
354
  )
355
 
356
 
357
- # ---- token-classification head (inlined; self-contained) ----
358
  from transformers.modeling_outputs import TokenClassifierOutput
359
 
360
  class Lfm2BidirForTokenClassification(PreTrainedModel):
 
354
  )
355
 
356
 
357
+ # ---- token-classification head (inlined) ----
358
  from transformers.modeling_outputs import TokenClassifierOutput
359
 
360
  class Lfm2BidirForTokenClassification(PreTrainedModel):