publications

2023

1. OSAN: A One-Stage Alignment Network To Unify Multimodal Alignment and Unsupervised Domain Adaptation

   Ye Liu, Lingfeng Qiao, Changchong Lu, Di Yin, Chen Lin, Haoyuan Peng, and Bo Ren

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2023
2. VKIE: The Application of Key Information Extraction on Video Text

   Siyu An, Ye Liu, Haoyuan Peng, and Di Yin

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, Jun 2023

2022

1. Grafting Pre-trained Models for Multimodal Headline Generation

   Lingfeng Qiao, Chen Wu, Ye Liu, Haoyuan Peng, Di Yin, and Bo Ren

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, Dec 2022

   Abs

   Multimodal headline utilizes both video frames and transcripts to generate the natural language title of the videos. Due to a lack of large-scale, manually annotated data, the task of annotating grounded headlines for video is labor intensive and impractical. Previous researches on pre-trained language models and video-language models have achieved significant progress in related downstream tasks. However, none of them can be directly applied to multimodal headline architecture where we need both multimodal encoder and sentence decoder. A major challenge in simply gluing language model and video-language model is the modality balance, which is aimed at combining visual-language complementary abilities. In this paper, we propose a novel approach to graft the video encoder from the pre-trained video-language model on the generative pre-trained language model. We also present a consensus fusion mechanism for the integration of different components, via inter/intra modality relation. Empirically, experiments show that the grafted model achieves strong results on a brand-new dataset collected from real-world applications.

2019

1. Detecting Abnormal Start-Ups, Unusual Resource Consumptions of the Smart Phone: A Deep Learning Approach

   Xiaoqing ZHENG, Yaping LU, Haoyuan PENG, Jiangtao FENG, Yi ZHOU, Min JIANG, Li MA, Ji ZHANG, and Jie JI

   ZTE Communications, Dec 2019

2018

1. RNN-based sequence-preserved attention for dependency parsing

   Yi Zhou, Junying Zhou, Lu Liu, Jiangtao Feng, Haoyuan Peng, and Xiaoqing Zheng

   In Proceedings of the AAAI Conference on Artificial Intelligence, Dec 2018
2. Attention-based belief or disbelief feature extraction for dependency parsing

   Haoyuan Peng, Lu Liu, Yi Zhou, Junying Zhou, and Xiaoqing Zheng

   In Proceedings of the AAAI Conference on Artificial Intelligence, Dec 2018

2017

1. Learning context-specific word/character embeddings

   Xiaoqing Zheng, Jiangtao Feng, Yi Chen, Haoyuan Peng, and Wenqing Zhang

   In Proceedings of the AAAI conference on artificial intelligence, Dec 2017

2015

1. Character-based parsing with convolutional neural network

   Xiaoqing Zheng, Haoyuan Peng, Yi Chen, Pengjing Zhang, and Wenqiang Zhang

   In Twenty-Fourth International Joint Conference on Artificial Intelligence, Dec 2015