Compute guidance for an open RAG prompt-injection evaluation by a small company

Hello Hugging Face team and community,

We are Qwalora LLC, an independent cloud engineering, cybersecurity and applied security research company in New Jersey. Our PARALLAX program studies when attacker-controlled text in a retrieved document leads an open-weight instruction model to follow it instead of its system instructions.

Public showcase, with an in-browser Ed25519 verification demo on synthetic records:

What exists today

  • An evaluation harness with a deterministic rule-based evaluator, trust-fenced prompts, hash-chained per-item records, resumable runs, and offline Ed25519 signing. It has been exercised end to end on simulated outputs only. No capable model has been run and we report no findings.
  • The code and synthetic corpus are private while security, licensing and methodological review finish. We intend to publish them before any results.

What we plan (not yet run)

  • A protocol that is pre-specified and under independent methodological review: 48 synthetic tasks, 10 fixed attack variants plus benign controls, 3 injection positions, one temperature-0 generation per item, a seeded rerun to measure nondeterminism, and a separate sampled-decoding arm. It will be frozen and its hash published before any run whose outputs are reported as results. A short bounded benchmark and pilot runs, reported only as engineering measurements, come first.
  • Models: Qwen2.5-1.5B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and Llama-3.1-8B-Instruct, with exact revisions, weight hashes, chat-template hashes and decoding settings recorded on every record.

Compute
Our preliminary estimate, pending benchmarking, is on the order of 40 GPU-hours on a single 24 GB GPU for four models (about 3,500 generations per model, up to 256 new tokens each). We will replace it with measurements from a short bounded benchmark before any formal request.

Our question
The Community GPU Grant form is in a compute Space’s settings, and our organization can host only static Spaces on the free tier. What is the supported way for an open evaluation run by a small company, with all outputs published, to request compute, and is a grant something the team would consider for it?

We intend to publish everything on the Hub: code, the synthetic corpus, signed per-item records, and a report that includes null and inconclusive results.

Thank you,
Franck Kengne, Chief Technology Officer, Qwalora LLC
partners@qwalora.com