Hello Hugging Face team and community,
We are Qwalora LLC, an independent cloud engineering, cybersecurity and applied security research company in New Jersey. Our PARALLAX program studies when attacker-controlled text in a retrieved document leads an open-weight instruction model to follow it instead of its system instructions.
Public showcase, with an in-browser Ed25519 verification demo on synthetic records:
What exists today
- An evaluation harness with a deterministic rule-based evaluator, trust-fenced prompts, hash-chained per-item records, resumable runs, and offline Ed25519 signing. It has been exercised end to end on simulated outputs only. No capable model has been run and we report no findings.
- The code and synthetic corpus are private while security, licensing and methodological review finish. We intend to publish them before any results.
What we plan (not yet run)
- A protocol that is pre-specified and under independent methodological review: 48 synthetic tasks, 10 fixed attack variants plus benign controls, 3 injection positions, one temperature-0 generation per item, a seeded rerun to measure nondeterminism, and a separate sampled-decoding arm. It will be frozen and its hash published before any run whose outputs are reported as results. A short bounded benchmark and pilot runs, reported only as engineering measurements, come first.
- Models: Qwen2.5-1.5B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and Llama-3.1-8B-Instruct, with exact revisions, weight hashes, chat-template hashes and decoding settings recorded on every record.
Compute
Our preliminary estimate, pending benchmarking, is on the order of 40 GPU-hours on a single 24 GB GPU for four models (about 3,500 generations per model, up to 256 new tokens each). We will replace it with measurements from a short bounded benchmark before any formal request.
Our question
The Community GPU Grant form is in a compute Space’s settings, and our organization can host only static Spaces on the free tier. What is the supported way for an open evaluation run by a small company, with all outputs published, to request compute, and is a grant something the team would consider for it?
We intend to publish everything on the Hub: code, the synthetic corpus, signed per-item records, and a report that includes null and inconclusive results.
Thank you,
Franck Kengne, Chief Technology Officer, Qwalora LLC
partners@qwalora.com