Projects
Debiasing from Source to Target: Building an Integrated Debiasing System for Hate Speech Detection
This is a course project of 11-830: Computational Ethics for NLP, where we integrated and examined debiasing methods on dataset, word embeddings, and model levels. This project provided insights into building effective debiasing systems for underrepresented social groups and controlling countereffects for hate speech detection models.
Fraud Detection on Imbalanced Dataset with GAN
This is a course project of 10-708: Probabilistic Graphical Models, where we proposed a conditional GAN-based Graphical Convolution network to solve the challenge of highly imbalanced fraud detection datasets. The proposed model generates the distribution of normal data and learns a classifier to spot the abnormal data precisely, improving the F1-macro scores by at least 2% on three datasets.
Multi-hop Visual Question Answering with WebQA Dataset
This is a course project of 11-777: Multimodal Machine Learning, where we aimed at solving the challenge of a multi-hop VQA benchmark called WebQA that requires extracting knowledge indistriminately from both text and images. We proposed a multimodal retriever-reader pipeline that maps text and image sources to unified representations, selecting useful information pieces via OFA, and generates natural language answers using a modified BLIP model. Our proposed model outperforms the VLP baseline model on retrieval scores and question answering scores, exhibiting the potential of unified multimodal approaches for multi-hop visual question answering tasks.
BrushLens: A Smart Phone Case for Assistive Touch
This is a course project of EECS 498: Engineering Interactive Systems, where we designed a smart phone case that helps visually-impaired people to identify, locate, and press buttons on the touch screen through users’ verbal commands. The hardware component is composed of an Arduino board and solenoids and the software system is mainly based on traditional computer vision algorithms and off-the-shelf speech-to-text plugins. The quantitative and qualitative user study suggests high functionality and usability of our proposed device, and this prototype has been selected by the HAIL lab at the University of Michigan.
Improving the Emoji Recommendation System in Messaging Apps
This is a course project of EECS598: Human Computer Interaction, where we designed a user-centered emoji recommendation system, providing insights into how breakdowns occur when communicating emotions through text. We iterated our keyboard prototype via initial surveys, contextual interviews, heuristic evaluations, and simplified user testing. The qualitative results indicate that our proposed keyboard allows users to spend 48% less time than the standard keyboard on locating desired emojis per dialogue turn.
Research on Servo Controller Data Interactive Modules and Human-machine Interface under Domestic Linux Operating Systems
This is the capstone requirement of Shanghai Jiao Tong University, where our group designed and implemented a human-machine interface and a client program that communicates with servo controllers and programmable logic controllers via different protocols. The back-end, written in C#, integrates OPC UA, Modbus, and ADS communication protocols and limits transmission delay to less than 500ms. Our designed human-machine interface has been installed as beta test on tobacco machines in Shanghai Tobacco Machinery Company.
Generating Diverse and High-Quality Abstractive Summaries with Variational Transformers
This is an unpublished working paper of my work on abstractive text summarization. In order to generate abstractive summaries with a certain amount of diversity while preserving the quality, I integrated the conditional VAE framework into the Transformer and fused the latent variables to the SOS token of the Transformer decoder inputs. The proposed model outperforms the baseline models in diversity scores and maintains comparable accuracy.
News2Poem: Stylistic Headline Generation for News Articles
This is a course project of EECS498: Natural Language Processing, where we proposed a ProphetNet-DAE structure to generate poetic headlines for news articles without supervised stylistic news article-poetic headline paired data. The model was applied on both English and Chinese News datasets and achieved improvement in BLEU and ROUGE-L scores.