ilessio-aiflowlab/project_def_rpga

DEF-RPGA — Adversarial Camouflage via 3D Gaussian Splatting

Part of the ANIMA Intelligence Compiler Suite by Robot Flow Labs.

Paper

3D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generation arXiv: 2507.01367

Authors: Tianrui Lou, Xiaojun Jia, Siyuan Liang, Jiawei Liang, Ming Zhang, Yanjun Xiao, Xiaochun Cao

Architecture

DEF-RPGA uses 3D Gaussian Splatting as a differentiable attack substrate for generating multi-view robust physical adversarial camouflage. The pipeline:

1. Scene Representation: 3DGS point cloud with SH coefficients
2. Attack Target: Optimize only zero-order SH (SH0) color terms
3. Differentiable Rendering: CUDA rasterizer (diff-gaussian-rasterization)
4. Adversarial Objective: Suppress detector confidence (paper Eq. 6)
5. Regularization: Cross-view consistency, NPS (Non-Printability Score), primary-color regularization
6. Background Hardening: Min-max perturbation to filter non-robust features (paper Eq. 9)

Exported Formats

Format	File	Use Case
PyTorch (.pth)	pytorch/def_rpga_v1.pth	Training, fine-tuning
SafeTensors	pytorch/def_rpga_v1.safetensors	Fast loading, safe
ONNX	onnx/def_rpga_v1.onnx	Cross-platform inference
TensorRT FP16	tensorrt/def_rpga_v1_fp16.trt	Edge deployment (Jetson/L4)
TensorRT FP32	tensorrt/def_rpga_v1_fp32.trt	Full precision inference

Usage

import torch
from anima_def_rpga.scene.gaussian_state import GaussianState
from anima_def_rpga.scene.checkpoint_loader import load_gaussian_state

# Load trained adversarial camouflage
state = load_gaussian_state("pytorch/def_rpga_v1.pth")
print(f"Gaussians: {state.gaussian_count}, SH0 shape: {state.sh0.shape}")

# The SH0 values encode the adversarial color pattern
adversarial_colors = torch.sigmoid(state.sh0)  # [N, 3] RGB

Training

• Hardware: NVIDIA L4 (23 GB VRAM)
• Framework: PyTorch 2.11 + CUDA 12.8
• Rasterizer: diff-gaussian-rasterization (CUDA kernel)
• Optimizer: Adam (fused), lr=0.01, warmup + cosine decay
• Batch size: 80 views per step (67.1% VRAM utilization)
• Scene: 1M Gaussians
• Resolution: 480×640
• Steps: 20,000
• Config: See configs/train_cuda.toml

Evaluation Targets (Paper §5)

Benchmark	Metric	Paper Value	Our Target
Digital white-box angle sweep	AP@0.5 on Faster R-CNN	6.73	≤ 8.0
Hardest angle (60°)	AP@0.5	0.00	≤ 2.0
1:1 real car	AP@0.5	25.67	≤ 30.0

ANIMA Module

• Codename: DEF-RPGA
• Domain: Defense
• Wave: 7
• API: FastAPI at port 8407
• ROS2: Capture + bridge nodes for robotics integration
• Docker: docker compose -f docker-compose.serve.yml --profile serve up -d

License

Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED