Shop Talk

I built ShopTalk, a multimodal shopping assistant that lets users search for products using text, images, or a combination of both, delivering more human-like and relevant results.
In the demo, users can upload an image, type a query, or do both, and the system uses BLIP to generate contextual captions and CLIP to embed both text and images into a shared vector space, retrieving and ranking the most relevant products through a Redis-backed similarity search with late fusion scoring.