Yu Rong (荣钰)

2024

1. KDD

   Relaxing Continuous Constraints of Equivariant Graph Neural Networks for Broad Physical Dynamics Learning

   Zinan Zheng, Yang Liu, Jia Li, Jianhua Yao, and Yu Rong

   In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024

   Abs Bib PDF Code

   Incorporating Euclidean symmetries (e.g. rotation equivariance) as inductive biases into graph neural networks has improved their generalization ability and data efficiency in unbounded physical dynamics modeling. However, in various scientific and engineering applications, the symmetries of dynamics are frequently discrete due to the boundary conditions. Thus, existing GNNs either over-look necessary symmetry, resulting in suboptimal representation ability, or impose excessive equivariance, which fails to generalize to unobserved symmetric dynamics. In this work, we propose a general Discrete Equivariant Graph Neural Network (DEGNN) that guarantees equivariance to a given discrete point group. Specifically, we show that such discrete equivariant message passing could be constructed by transforming geometric features into permutation-invariant embeddings. Through relaxing continuous equivariant constraints, DEGNN can employ more geometric feature combinations to approximate unobserved physical object interaction functions. Two implementation approaches of DEGNN are proposed based on ranking or pooling permutation-invariant functions. We apply DEGNN to various physical dynamics, ranging from particle, molecular, crowd to vehicle dynamics. In twenty scenarios, DEGNN significantly outperforms existing state-of-the-art approaches. Moreover, we show that DEGNN is data efficient, learning with less data, and can generalize across scenarios such as unobserved orientation.

   @inproceedings{10.1145/3637528.3671957,
     title = {Relaxing Continuous Constraints of Equivariant Graph Neural Networks for Broad Physical Dynamics Learning},
     author = {Zheng, Zinan and Liu, Yang and Li, Jia and Yao, Jianhua and Rong, Yu},
     year = {2024},
     booktitle = {Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
     location = {Barcelona, Spain},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     series = {KDD '24},
     pages = {4548–4558},
     doi = {10.1145/3637528.3671957},
     isbn = {9798400704901},
     url = {https://doi.org/10.1145/3637528.3671957},
     numpages = {11},
     keywords = {equivariant graph neural network, physical dynamics},
   }

2. ICLR

   SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases

   Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, and 2 more authors

   In The Twelfth International Conference on Learning Representations, 2024

   Bib PDF Code

   @inproceedings{liu2024segno,
     title = {{SEGNO}: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases},
     author = {Liu, Yang and Cheng, Jiashun and Zhao, Haihong and Xu, Tingyang and Zhao, Peilin and Tsung, Fugee and Li, Jia and Rong, Yu},
     year = {2024},
     booktitle = {The Twelfth International Conference on Learning Representations},
     url = {https://openreview.net/forum?id=3oTPsORaDH},
   }

3. VLDB

   Inductive Attributed Community Search: To Learn Communities Across Graphs

   Shuheng Fang, Kangfei Zhao, Yu Rong, Zhixun Li, and Jeffrey Xu Yu

   Proc. VLDB Endow., Aug 2024

   Abs Bib Code Slides

   Attributed community search (ACS) aims to identify subgraphs satisfying both structure cohesiveness and attribute homogeneity in attributed graphs, for a given query that contains query nodes and query attributes. Previously, algorithmic approaches deal with ACS in a two-stage paradigm, which suffer from structural inflexibility and attribute irrelevance. To overcome this problem, recently, learning-based approaches have been proposed to learn both structures and attributes simultaneously as a one-stage paradigm. However, these approaches train a transductive model which assumes the graph to infer unseen queries is as same as the graph used for training. That limits the generalization and adaptation of these approaches to different heterogeneous graphs.In this paper, we propose a new framework, Inductive Attributed Community Search, IACS, by inductive learning, which can be used to infer new queries for different communities/graphs. Specifically, IACS employs an encoder-decoder neural architecture to handle an ACS task at a time, where a task consists of a graph with only a few queries and corresponding ground-truth. We design a three-phase workflow, "training-adaptation-inference", which learns a shared model to absorb and induce prior effective common knowledge about ACS across different tasks. And the shared model can swiftly adapt to a new task with small number of ground-truth. We conduct substantial experiments in 7 real-world datasets to verify the effectiveness of IACS for CS/ACS. Our approach IACS achieves 28.97% and 25.60% improvements in F1-score on average in CS and ACS, respectively.

   @article{10.14778/3675034.3675048,
     title = {Inductive Attributed Community Search: To Learn Communities Across Graphs},
     author = {Fang, Shuheng and Zhao, Kangfei and Rong, Yu and Li, Zhixun and Yu, Jeffrey Xu},
     year = {2024},
     month = aug,
     journal = {Proc. VLDB Endow.},
     publisher = {VLDB Endowment},
     volume = {17},
     number = {10},
     pages = {2576–2589},
     doi = {10.14778/3675034.3675048},
     issn = {2150-8097},
     url = {https://doi.org/10.14778/3675034.3675048},
     issue_date = {June 2024},
     numpages = {14},
   }

4. ICLR

   Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channel

   Xuan Li, Zhanke Zhou, Jiangchao Yao, Yu Rong, Lu Zhang, and Bo Han

   In The Twelfth International Conference on Learning Representations, 2024

   Bib PDF Code

   @inproceedings{li2024neural,
     title = {Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channel},
     author = {Li, Xuan and Zhou, Zhanke and Yao, Jiangchao and Rong, Yu and Zhang, Lu and Han, Bo},
     year = {2024},
     booktitle = {The Twelfth International Conference on Learning Representations},
     url = {https://openreview.net/forum?id=CUfSCwcgqm},
   }

5. Nature Methods

   scPROTEIN: a versatile deep graph contrastive learning framework for single-cell proteomics embedding

   Wei Li, Fan Yang, Fang Wang, Yu Rong, Linjing Liu, Bingzhe Wu, and 2 more authors

   Nature Methods, Apr 2024

   Abs Bib Code Slides

   Single-cell proteomics sequencing technology sheds light on protein–protein interactions, posttranslational modifications and proteoform dynamics in the cell. However, the uncertainty estimation for peptide quantification, data missingness, batch effects and high noise hinder the analysis of single-cell proteomic data. It is important to solve this set of tangled problems together, but the existing methods tailored for single-cell transcriptomes cannot fully address this task. Here we propose a versatile framework designed for single-cell proteomics data analysis called scPROTEIN, which consists of peptide uncertainty estimation based on a multitask heteroscedastic regression model and cell embedding generation based on graph contrastive learning. scPROTEIN can estimate the uncertainty of peptide quantification, denoise protein data, remove batch effects and encode single-cell proteomic-specific embeddings in a unified framework. We demonstrate that scPROTEIN is efficient for cell clustering, batch correction, cell type annotation, clinical analysis and spatially resolved proteomic data exploration.

   @article{Li2024,
     title = {scPROTEIN: a versatile deep graph contrastive learning framework for single-cell proteomics embedding},
     author = {Li, Wei and Yang, Fan and Wang, Fang and Rong, Yu and Liu, Linjing and Wu, Bingzhe and Zhang, Han and Yao, Jianhua},
     year = {2024},
     month = apr,
     day = {01},
     journal = {Nature Methods},
     volume = {21},
     number = {4},
     pages = {623--634},
     doi = {10.1038/s41592-024-02214-9},
     issn = {1548-7105},
     url = {https://doi.org/10.1038/s41592-024-02214-9},
   }

2023

1. Nature Comm.

   Collaborative and privacy-preserving retired battery sorting for profitable direct recycling via federated machine learning

   Shengyu Tao, Haizhou Liu, Chongbo Sun, Haocheng Ji, Guanjun Ji, Zhiyuan Han, and 11 more authors

   Nature Communications, Dec 2023

   Abs Bib PDF

   Unsorted retired batteries with varied cathode materials hinder the adoption of direct recycling due to their cathode-specific nature. The surge in retired batteries necessitates precise sorting for effective direct recycling, but challenges arise from varying operational histories, diverse manufacturers, and data privacy concerns of recycling collaborators (data owners). Here we show, from a unique dataset of 130 lithium-ion batteries spanning 5 cathode materials and 7 manufacturers, a federated machine learning approach can classify these retired batteries without relying on past operational data, safeguarding the data privacy of recycling collaborators. By utilizing the features extracted from the end-of-life charge-discharge cycle, our model exhibits 1% and 3% cathode sorting errors under homogeneous and heterogeneous battery recycling settings respectively, attributed to our innovative Wasserstein-distance voting strategy. Economically, the proposed method underscores the value of precise battery sorting for a prosperous and sustainable recycling industry. This study heralds a new paradigm of using privacy-sensitive data from diverse sources, facilitating collaborative and privacy-respecting decision-making for distributed systems.

   @article{Tao2023,
     title = {Collaborative and privacy-preserving retired battery sorting for profitable direct recycling via federated machine learning},
     author = {Tao, Shengyu and Liu, Haizhou and Sun, Chongbo and Ji, Haocheng and Ji, Guanjun and Han, Zhiyuan and Gao, Runhua and Ma, Jun and Ma, Ruifei and Chen, Yuou and Fu, Shiyi and Wang, Yu and Sun, Yaojie and Rong, Yu and Zhang, Xuan and Zhou, Guangmin and Sun, Hongbin},
     year = {2023},
     month = dec,
     day = {05},
     journal = {Nature Communications},
     volume = {14},
     number = {1},
     pages = {8032},
     doi = {10.1038/s41467-023-43883-y},
     issn = {2041-1723},
     url = {https://doi.org/10.1038/s41467-023-43883-y},
   }

2. KDD

   Privacy Matters: Vertical Federated Linear Contextual Bandits for Privacy Protected Recommendation

   Zeyu Cao, Zhipeng Liang, Bingzhe Wu, Shu Zhang, Hangyu Li, Ouyang Wen, and 2 more authors

   In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023

   Abs Bib

   Recent awareness of privacy protection and compliance requirement resulted in a controversial view of recommendation system due to personal data usage. Therefore, privacy-protected recommendation emerges as a novel research direction. In this paper, we first formulate this problem as a vertical federated learning problem, i.e., features are vertically distributed over different departments. We study a contextual bandit learning problem for recommendation in the vertical federated setting. To this end, we carefully design a customized encryption scheme named orthogonal matrix-based mask mechanism (O3M). O3M mechanism, a tailored component for contextual bandits by carefully exploiting their shared structure, can ensure privacy protection while avoiding expensive conventional cryptographic techniques. We further apply the mechanism to two commonly-used bandit algorithms, LinUCB and LinTS, and instantiate two practical protocols for online recommendation. The proposed protocols can perfectly recover the service quality of centralized bandit algorithms while achieving a satisfactory runtime efficiency, which is theoretically proved and analysed in this paper. By conducting extensive experiments on both synthetic and real-world datasets, we show the superiority of the proposed method in terms of privacy protection and recommendation performance.

   @inproceedings{10.1145/3580305.3599475,
     title = {Privacy Matters: Vertical Federated Linear Contextual Bandits for Privacy Protected Recommendation},
     author = {Cao, Zeyu and Liang, Zhipeng and Wu, Bingzhe and Zhang, Shu and Li, Hangyu and Wen, Ouyang and Rong, Yu and Zhao, Peilin},
     year = {2023},
     booktitle = {Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
     location = {Long Beach, CA, USA},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     series = {KDD '23},
     pages = {154–166},
     doi = {10.1145/3580305.3599475},
     isbn = {9798400701030},
     url = {https://doi.org/10.1145/3580305.3599475},
     numpages = {13},
     keywords = {vertical federated learning, privacy- preserving protocols, linear contextual bandits},
   }

3. CIKM

   Geometric Graph Learning for Protein Mutation Effect Prediction

   Kangfei Zhao, Yu Rong, Biaobin Jiang, Jianheng Tang, Hengtong Zhang, Jeffrey Xu Yu, and 1 more author

   In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023

   Abs Bib PDF Code Slides

   Proteins govern a wide range of biological systems. Evaluating the changes in protein properties upon protein mutation is a fundamental application of protein design, where modeling the 3D protein structure is a principal task for AI-driven computational approaches. Existing deep learning (DL) approaches represent the protein structure as a 3D geometric graph and simplify the graph modeling to different degrees, thereby failing to capture the low-level atom patterns and high-level amino acid patterns simultaneously. In addition, limited training samples with ground truth labels and protein structures further restrict the effectiveness of DL approaches. In this paper, we propose a new graph learning framework, Hierarchical Graph Invariant Network (HGIN), a fine-grained and data-efficient graph neural encoder for encoding protein structures and predicting the mutation effect on protein properties. For fine-grained modeling, HGIN hierarchically models the low-level interactions of atoms and the high-level interactions of amino acid residues by Graph Neural Networks. For data efficiency, HGIN preserves the invariant encoding for atom permutation and coordinate transformation, which is an intrinsic inductive bias of property prediction that bypasses data augmentations. We integrate HGIN into a Siamese network to predict the quantitative effect on protein properties upon mutations. Our approach outperforms 9 state-of-the-art approaches on 3 protein datasets. More inspiringly, when predicting the neutralizing ability of human antibodies against COVID-19 mutant viruses, HGIN achieves an absolute improvement of 0.23 regarding the Spearman coefficient.

   @inproceedings{10.1145/3583780.3614893,
     title = {Geometric Graph Learning for Protein Mutation Effect Prediction},
     author = {Zhao, Kangfei and Rong, Yu and Jiang, Biaobin and Tang, Jianheng and Zhang, Hengtong and Yu, Jeffrey Xu and Zhao, Peilin},
     year = {2023},
     booktitle = {Proceedings of the 32nd ACM International Conference on Information and Knowledge Management},
     location = {Birmingham, United Kingdom},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     series = {CIKM '23},
     pages = {3412–3422},
     doi = {10.1145/3583780.3614893},
     isbn = {9798400701245},
     url = {https://doi.org/10.1145/3583780.3614893},
     numpages = {11},
     keywords = {graph neural network, geometric graph learning},
   }

4. NeurIPS

   Equivariant Spatio-Temporal Attentive Graph Networks to Simulate Physical Dynamics

   Liming Wu, Zhichao Hou, Jirui Yuan, Yu Rong, and Wenbing Huang

   In Advances in Neural Information Processing Systems, 2023

   Bib PDF Code Slides

   @inproceedings{NEURIPS2023_8e2a75e0,
     title = {Equivariant Spatio-Temporal Attentive Graph Networks to Simulate Physical Dynamics},
     author = {Wu, Liming and Hou, Zhichao and Yuan, Jirui and Rong, Yu and Huang, Wenbing},
     year = {2023},
     booktitle = {Advances in Neural Information Processing Systems},
     publisher = {Curran Associates, Inc.},
     volume = {36},
     pages = {45360--45380},
     url = {https://proceedings.neurips.cc/paper_files/paper/2023/file/8e2a75e0c7b579a6cf176dc0858cde55-Paper-Conference.pdf},
     editor = {Oh, A. and Naumann, T. and Globerson, A. and Saenko, K. and Hardt, M. and Levine, S.},
   }