Multimedia, social media content, and interaction are common means to attract customers in shopping. However these features are not always fully available for customers when they go shopping in physical shopping centers. The authors propose Smart Shopping Assistant, a multimedia and social media augmented system on mobile devices to enhance users’ experience and interaction in shopping. Smart Shopping turns a regular mobile device into a special prism so that a customer can enjoy multimedia, get useful social media related to a product, give feedbacks or make actions on a product during shopping. The system is specified as a flexible framework to take advantages of different visual descriptors and web information extraction modules. Experimental results show that Smart Shopping can process and provide augmented data in a realtime-manner. Smart Shopping can be used to attract more customers and to build an online social community of customers to share their interests in shopping.