Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
Inverse-LLaVA projects text embeddings into continuous visual space and fuses the modalities within intermediate transformer layers, enabling multimodal reasoning without explicit alignment pre-training.
Take a look