Research Direction: Intelligent Technology and Design Research
Mentor group: Fei Jun (responsible mentor), Chen Xiaowen, Jin Rilong
Title: Synesthesia
Material of work: multimedia comprehensive material
Introduction: Synesthesia Device is a work that explores how artificial intelligence perceives human facial expressions. This is an artificial intelligence with "synesthesia" function, which can emit a corresponding unique smell while perceiving human expressions. Based on the on-board expression recognition algorithm, it can analyze the audience's facial expressions and output values of different emotions, and extract different proportions of odor liquid atomization and mixing through the operation of the program driver, converting the audience's expression into a unique smell

Question and answer analysis
How the works interact?
The Synesthesia Device consists of an image acquisition device on the left and an odor generation device on the right. When the audience walks into the image acquisition area and stays for 3 seconds, the camera will take a facial photo of the audience and enter the artificial intelligence program to analyze the expression of the audience in the photo. The test tube in the odor generating device on the right side is filled with wood flavor, chemical flavor, fruit flavor and odor liquid respectively. The program will drive the device to run and extract different proportions of odor liquid to the upper test tube based on the results of expression analysis, spray the taste through the atomization device and mix it in the hemisphere. Finally, the fan is turned on to blow out the smell mixed based on the audience's expression data.

The techniques used in the work?
The expression recognition code in the work is completed by combining TensorFlow with OpenCV. It can analyze the facial expressions in the image and output the specific gravity values of different emotions. These values are linked to the odor generation part of the device, and the Bluetooth module allows the Arduino to receive different emotional data in the audience's facial expression and use it to drive the operation of the entire main device.

Tutor's Comments
Synesthesia (synesthesia) is also translated as synesthesia, synesthesia or synesthesia. It is composed of Greek roots syn (common) and aisthesis (sensation). This is a perceptual state with neural basis, which means a sensory stimulus or cognitive pathway will spontaneously and involuntarily cause another perception or cognition. Synesthesia is the arbitrary connection of seemingly irrelevant sensory entities. Hu Shuai's work "Synesthesia Device" associates the AI expression recognition algorithm with the smell synthesis device, forming a connection between expression and smell, and interpreting the expressions and emotions contained in the expressions of the participating audience through the dynamically synthesized smell, forming the "smell" Synesthetic behavior. This work full of participation and sense of humor attempts to awaken the dusty synesthesia genes in the audience's body and construct new synesthesia reactions with the intervention of technology.