RichardErkhov/mikewang_-_PVD-160k-Mistral-7b-awq

Quantization made by Richard Erkhov.

Github

Discord

Request more models

This repository contains a quantized version of PVD-160k-Mistral-7b, as presented in the paper Visually Descriptive Language Model for Vector Graphics Reasoning.

Project page: https://mikewangwzhl.github.io/VDLM/

Code: https://github.com/MikeWangWZHL/VDLM

PVD-160k-Mistral-7b - AWQ

• Model creator: https://huggingface.co/mikewang/
• Original model: https://huggingface.co/mikewang/PVD-160k-Mistral-7b/

Original model description:

license: apache-2.0 datasets: - mikewang/PVD-160K

Text-Based Reasoning About Vector Graphics

🌐 Homepage • 📃 Paper • 🤗 Data (PVD-160k) • 🤗 Model (PVD-160k-Mistral-7b) • 💻 Code

We observe that current large multimodal models (LMMs) still struggle with seemingly straightforward reasoning tasks that require precise perception of low-level visual details, such as identifying spatial relations or solving simple mazes. In particular, this failure mode persists in question-answering tasks about vector graphics—images composed purely of 2D objects and shapes.

To solve this challenge, we propose Visually Descriptive Language Model (VDLM), a visual reasoning framework that operates with intermediate text-based visual descriptions—SVG representations and learned Primal Visual Description, which can be directly integrated into existing LLMs and LMMs. We demonstrate that VDLM outperforms state-of-the-art large multimodal models, such as GPT-4V, across various multimodal reasoning tasks involving vector graphics. See our paper for more details.