Hey, I’m using BertForSequenceClassification + Pytorch Lightning-Flash for a text classification task. I want to add additional features besides the text (e.g. categorical features). From what I understand, I need to override BertForSequenceClassification “forward” method and change the final class…

Correction: my response above is useful in case one wants to add additional text features to the tokens. However, if you want to combine text features with other features (like categorical or numerical ones) - which was actually the question above (apologies) - it makes sense to concatenate the fin…

@nielsr in the notebook it doesn’t look like the classification head is actually fitted against the data? I might be missing something, but it looks like it takes a pretrained model, adds a classification head and then uses it directly for inference without actually training it? I’m trying to do exa…

Combine BertForSequenceClassificaion with Additional Features

🤗Transformers

nielsr June 4, 2021, 9:26am 2

See this response where I explain how to modify BERT to add additional POS (part-of-speech) features to tokens to perform named-entity recognition.

Topic		Replies	Views
How to concatenate additional features to the last layer of Bert 🤗Transformers	6	2662	January 14, 2024
How to use additional input features for NER? Beginners	27	16298	June 5, 2023
Adding new features to Bert for NER 🤗Transformers	1	1233	May 7, 2022
Options for feature addition 🤗Transformers	0	1063	November 24, 2022
Adding additional features to BERT model Models	0	1081	July 18, 2022

Combine BertForSequenceClassificaion with Additional Features

Related topics