Text Generation
Transformers
Safetensors
GGUF
MLX
English
qwen3
lora
regulatory
compliance
escalation
decision-gate
flowx
conversational
Eval Results (legacy)
text-generation-inference
Instructions to use flowxai/sentinel-gate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use flowxai/sentinel-gate with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="flowxai/sentinel-gate") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("flowxai/sentinel-gate") model = AutoModelForCausalLM.from_pretrained("flowxai/sentinel-gate", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - MLX
How to use flowxai/sentinel-gate with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("flowxai/sentinel-gate") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flowxai/sentinel-gate with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flowxai/sentinel-gate:Q4_K_M # Run inference directly in the terminal: llama cli -hf flowxai/sentinel-gate:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flowxai/sentinel-gate:Q4_K_M # Run inference directly in the terminal: llama cli -hf flowxai/sentinel-gate:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf flowxai/sentinel-gate:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf flowxai/sentinel-gate:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf flowxai/sentinel-gate:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf flowxai/sentinel-gate:Q4_K_M
Use Docker
docker model run hf.co/flowxai/sentinel-gate:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use flowxai/sentinel-gate with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "flowxai/sentinel-gate" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/sentinel-gate", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/flowxai/sentinel-gate:Q4_K_M
- SGLang
How to use flowxai/sentinel-gate with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "flowxai/sentinel-gate" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/sentinel-gate", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "flowxai/sentinel-gate" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/sentinel-gate", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use flowxai/sentinel-gate with Ollama:
ollama run hf.co/flowxai/sentinel-gate:Q4_K_M
- Unsloth Desktop
- Pi
How to use flowxai/sentinel-gate with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/sentinel-gate"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "flowxai/sentinel-gate" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use flowxai/sentinel-gate with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "flowxai/sentinel-gate"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "flowxai/sentinel-gate" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flowxai/sentinel-gate", "messages": [ {"role": "user", "content": "Hello"} ] }' - Docker Model Runner
How to use flowxai/sentinel-gate with Docker Model Runner:
docker model run hf.co/flowxai/sentinel-gate:Q4_K_M
- Lemonade
How to use flowxai/sentinel-gate with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flowxai/sentinel-gate:Q4_K_M
Run and chat with the model
lemonade run user.sentinel-gate-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use flowxai/sentinel-gate with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/sentinel-gate"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default flowxai/sentinel-gate
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use flowxai/sentinel-gate with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "flowxai/sentinel-gate"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "flowxai/sentinel-gate" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download inference_contract/schema_sentinel_v1.json from flowxai/sentinel-gate: direct link, hf CLI and curl.
- Browser
- Download file 5.8 kB
-
https://huggingface.co/flowxai/sentinel-gate/resolve/main/inference_contract/schema_sentinel_v1.json
- Command line
-
hf download hf://flowxai/sentinel-gate/inference_contract/schema_sentinel_v1.json
-
curl -L -o schema_sentinel_v1.json https://huggingface.co/flowxai/sentinel-gate/resolve/main/inference_contract/schema_sentinel_v1.json
5.8 kB
| { | |
| "$schema": "http://json-schema.org/draft-07/schema#", | |
| "$id": "https://huggingface.co/flowxai/sentinel-gate/inference_contract/schema_sentinel_v1.json", | |
| "title": "FlowX Sentinel Gate output (schema_sentinel_v1)", | |
| "description": "The single JSON object the Sentinel Gate emits for one regulated-decision case. The gate decides ESCALATE (route to a human) vs DECIDE (safe to automate). When action=ESCALATE, escalation_category is one of six ids and a category-specific block is present; when action=DECIDE, escalation_category is null. Prompt version sentinel_sys_v1. Note: additionalProperties is intentionally true because the category-specific block key varies by category (policy_violations, missing_preconditions, boundary_analysis, confidence_factors, conflicting_signals, external_dependency) and DECIDE cases carry decision/rationale keys.", | |
| "type": "object", | |
| "additionalProperties": true, | |
| "required": [ | |
| "action", | |
| "escalation_category", | |
| "confidence_score", | |
| "audit_trail" | |
| ], | |
| "properties": { | |
| "action": { | |
| "type": "string", | |
| "description": "The gate decision. ESCALATE routes the case to a human; DECIDE marks it safe to automate. This is the field to gate on (perfect on the held-out set).", | |
| "enum": ["ESCALATE", "DECIDE"] | |
| }, | |
| "escalation_category": { | |
| "description": "The human-routing label for an escalation, or null when action=DECIDE. A secondary routing hint (~0.61 accuracy on true-escalate), not the gate.", | |
| "type": ["string", "null"], | |
| "enum": [ | |
| "MISSING_REQUIRED_DOCUMENTATION", | |
| "POLICY_VIOLATION", | |
| "BOUNDARY_CONDITION", | |
| "INSUFFICIENT_CONFIDENCE", | |
| "CONFLICTING_SIGNALS", | |
| "EXTERNAL_DEPENDENCY", | |
| null | |
| ] | |
| }, | |
| "confidence_score": { | |
| "type": "number", | |
| "description": "Calibrated confidence in the decision, 0.0-1.0. For ESCALATE this is typically the confidence that the case is safe to automate (low), so a low score supports escalation; for DECIDE it is the confidence in the auto-decision (high).", | |
| "minimum": 0.0, | |
| "maximum": 1.0 | |
| }, | |
| "confidence_reasoning": { | |
| "type": "string", | |
| "description": "One to three sentences explaining the confidence_score and why the case was escalated or auto-decided." | |
| }, | |
| "human_action_required": { | |
| "type": "string", | |
| "description": "For ESCALATE: the concrete next step a human owner must take (who does what). For DECIDE: the string \"NONE\"." | |
| }, | |
| "audit_trail": { | |
| "type": "array", | |
| "description": "Ordered, append-only log of the reasoning steps and policy gates evaluated, for compliance review.", | |
| "items": { "type": "string" }, | |
| "minItems": 1 | |
| }, | |
| "policy_violations": { | |
| "type": "object", | |
| "description": "Category-specific block for POLICY_VIOLATION. Keyed by violation id; each entry names the policy, regulation, restriction, and consequence.", | |
| "additionalProperties": true | |
| }, | |
| "missing_preconditions": { | |
| "type": "object", | |
| "description": "Category-specific block for MISSING_REQUIRED_DOCUMENTATION. Keyed by the missing precondition; each entry names required_by, regulation, severity, and reason.", | |
| "additionalProperties": true | |
| }, | |
| "boundary_analysis": { | |
| "type": "object", | |
| "description": "Category-specific block for BOUNDARY_CONDITION. Names the policy_threshold, the shipment/case value, distance_from_threshold, and an assessment of the edge case.", | |
| "additionalProperties": true | |
| }, | |
| "confidence_factors": { | |
| "type": "object", | |
| "description": "Category-specific block for INSUFFICIENT_CONFIDENCE. Lists ambiguous_signals and why_uncertain.", | |
| "additionalProperties": true | |
| }, | |
| "conflicting_signals": { | |
| "type": "array", | |
| "description": "Category-specific block for CONFLICTING_SIGNALS. The competing sources/values that disagree.", | |
| "items": { "type": "object", "additionalProperties": true } | |
| }, | |
| "external_dependency": { | |
| "type": "object", | |
| "description": "Category-specific block for EXTERNAL_DEPENDENCY. Names what the decision is awaiting and the blocking_gate.", | |
| "additionalProperties": true | |
| }, | |
| "escalation_path": { | |
| "type": "string", | |
| "description": "Optional routing hint naming the specialist queue or workflow that should own the escalation." | |
| }, | |
| "policy_gates_passed": { | |
| "type": "array", | |
| "description": "Optional list of policy gates that were checked and passed before the decision (present on some ESCALATE edge cases and on DECIDE cases).", | |
| "items": { "type": "string" } | |
| }, | |
| "decision": { | |
| "type": "string", | |
| "description": "For DECIDE cases: the automated outcome selected (e.g. ROUTE_APPROVED)." | |
| }, | |
| "selected_route": { | |
| "type": "string", | |
| "description": "For DECIDE cases where a route/option is chosen: the selected option." | |
| }, | |
| "rationale": { | |
| "type": "string", | |
| "description": "For DECIDE cases: the plain rationale for auto-deciding (some records use confidence_reasoning for this)." | |
| } | |
| }, | |
| "allOf": [ | |
| { | |
| "if": { "properties": { "action": { "const": "DECIDE" } } }, | |
| "then": { "properties": { "escalation_category": { "type": "null" } } } | |
| }, | |
| { | |
| "if": { "properties": { "action": { "const": "ESCALATE" } } }, | |
| "then": { | |
| "properties": { | |
| "escalation_category": { | |
| "type": "string", | |
| "enum": [ | |
| "MISSING_REQUIRED_DOCUMENTATION", | |
| "POLICY_VIOLATION", | |
| "BOUNDARY_CONDITION", | |
| "INSUFFICIENT_CONFIDENCE", | |
| "CONFLICTING_SIGNALS", | |
| "EXTERNAL_DEPENDENCY" | |
| ] | |
| } | |
| } | |
| } | |
| } | |
| ] | |
| } | |