# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("drt/srtk-scorer")
model = AutoModelForMaskedLM.from_pretrained("drt/srtk-scorer")Quick Links
SRTK Scorer
This model is a trained scorer for SRTK. It is used to compare the similarity between a query and the expansion path at the time of subgraph retrieval.
Training Information
It is initialized with roberta-base. It is trained jointly on the following datasets:
It achieves an answer coverage rate of 0.9728 on SimpleQuestionsWikidata (depth 1) 0.8501 on WebQSP test set (depth 2) with a beam width of only 2!
Usage Example
First install the package:
pip install srtk
Then you can retrieve subgraphs with the help of this scorer:
srtk retrieve -i data/wikidata-simplequestions/intermediate/scores_test.jsonl \
-o artifacts/subgraphs/wikidata-simple-contrast \
-e http://localhost:1234/api/endpoint/sparql \
--scorer-model-path drt/srtk-scorer \
--scorer --beam-width 2 --max-depth 1 --evaluate
Limitations
As both SimpleQuestionsWikidata and SimpleDBpediaQA contain only one-hop relations, the model tends to stop at one-hop when you retrieve subgraphs on Wikidata and DBpedia. We will release a updated version of the model that is trained on a more diverse dataset in the future.
License
MIT
- Downloads last month
- 7
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="drt/srtk-scorer")