Tiantian Feng

This amazing picture was taken by my wife at Skyline Trail, Mount Rainier, 2023.

PS: Prepare enough water for this trail but make sure you are not the one carrying it.

My name is Tiantian Feng, and I grew up in Leshan, China, which is famously for Leshan Giant Buddha. I studied both in Chengdu (both middle school and high school at Chengdu Foreign Language School) and Nanjing (Nanjing University of Posts and Telecommunications, undergraduate). I obtained my master degree at the University of Southern California afterwards. I recently completed my Ph.D. in the Thomas Lord Department of Computer Science at University of Southern California in 2023. I am fortunate to be advised by Professor Shrikanth Narayanan, a globally recognized scientist in speech modeling, linguistics, affective computing, and human understanding.

My research focuses on leveraging sensors and computational methods for understanding natural human behaviors, also with a particular emphasis on inclusiveness and privacy. My research invovles applications such as speech understanding, multimodal understanding, and bio-signal processing, etc. Additionally, I have hands-on experience in industrial sensor design and deploying sensors in research studies. I have interned at both Meta and Amazon as the research scientist.

I am currently a postdoc researcher in SAIL lab at USC, and I am actively looking for tenure-track or similar positions in academia. Don’t hesitate to contact me if you would like to collaborate.

News

Apr 22, 2024	Receive IEEE EMBC NextGen Scholar Award!
Apr 15, 2024	1 conference paper got accepted to 2024 EMBC! Congrats to all co-authors!
Apr 12, 2024	Give 2 oral and 1 poster presentation at 2024 ICASSP for 2 conference papers and 1 workshop paper (all first author)! My colleagues presented 2 conference papers that I collabrated with!
Apr 05, 2024	I will be hosting Trustworthy Speech Processing Satellite Workshop at ICASSP 2024, April 15th, 2024 from 2:00 PM to 5:30 PM KST (Room AB203)! Please join us if you are interested in making speech processing safer, more private, and more inclusive!
Jan 29, 2024	Received 2024 ICASSP travel grant!
Dec 15, 2023	4 conference papers and 1 workshop paper got accepted to 2024 ICASSP! Congrats to all co-authors!
Dec 11, 2023	Successfully defended my Ph.D. thesis: Generative Foundation Model Assisted Privacy-Enhancing Computing in Human-Centered Machine Intelligence!
Oct 27, 2023	Received Outstanding Poster Presentation - 2023 Electrical Engineering Research Festival - Ming Hsieh Department of Electrical and Computer Engineering.
Sep 11, 2023	Oral presentation of my work funded by Amazon research at ACII 2023 in MIT Media Lab, Boston - PEFT-SER: On the use of parameter efficient transfer learning approaches for speech emotion recognition using pre-trained speech models!
Aug 08, 2023	Oral presentation of my work collabrated with Amazon Alexa and OSU at SIGKDD 2023 - FedMultimodal: A Benchmark for Multimodal Federated Learning!

Selected Publications

Scientific Data Dataset

TILES-2018, a longitudinal physiologic and behavioral data set of hospital workers

Karel Mundnich , Brandon M Booth , Michelle l’Hommedieu , Tiantian Feng , Benjamin Girault , Justin L’hommedieu , Mackenzie Wildman , Sophia Skaaden , Amrutha Nadarajan , Jennifer L Villatte , and 1 more author

Scientific Data, 2020

Bib Website

@article{mundnich2020tiles,
  title = {TILES-2018, a longitudinal physiologic and behavioral data set of hospital workers},
  author = {Mundnich, Karel and Booth, Brandon M and l'Hommedieu, Michelle and Feng, Tiantian and Girault, Benjamin and L'hommedieu, Justin and Wildman, Mackenzie and Skaaden, Sophia and Nadarajan, Amrutha and Villatte, Jennifer L and others},
  journal = {Scientific Data},
  volume = {7},
  number = {1},
  pages = {354},
  year = {2020},
  publisher = {Nature Publishing Group UK London},
  abbr2 = {Dataset},
}

Scientific Data Dataset

TILES-2019: A longitudinal physiologic and behavioral data set of medical residents in an intensive care unit

Joanna C Yau , Benjamin Girault , Tiantian Feng , Karel Mundnich , Amrutha Nadarajan , Brandon M Booth , Emilio Ferrara , Kristina Lerman , Eric Hsieh , and Shrikanth Narayanan

Scientific Data, 2022

Bib Website

@article{yau2022tiles,
  title = {TILES-2019: A longitudinal physiologic and behavioral data set of medical residents in an intensive care unit},
  author = {Yau, Joanna C and Girault, Benjamin and Feng, Tiantian and Mundnich, Karel and Nadarajan, Amrutha and Booth, Brandon M and Ferrara, Emilio and Lerman, Kristina and Hsieh, Eric and Narayanan, Shrikanth},
  journal = {Scientific Data},
  volume = {9},
  number = {1},
  pages = {536},
  year = {2022},
  publisher = {Nature Publishing Group UK London},
  abbr2 = {Dataset},
}

ACM Multimedia Dataset

MM-AU: Towards Multimodal Understanding of Advertisement Videos

Digbalay Bose , Rajat Hebbar , Tiantian Feng , Krishna Somandepalli , Anfeng Xu , and Shrikanth Narayanan

In Proceedings of the 31st ACM International Conference on Multimedia , 2023

Bib

@inproceedings{bose2023mm,
  title = {MM-AU: Towards Multimodal Understanding of Advertisement Videos},
  author = {Bose, Digbalay and Hebbar, Rajat and Feng, Tiantian and Somandepalli, Krishna and Xu, Anfeng and Narayanan, Shrikanth},
  booktitle = {Proceedings of the 31st ACM International Conference on Multimedia},
  pages = {86--95},
  year = {2023},
  abbr2 = {Dataset},
}

Preprint Generative AI

Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing?

Tiantian Feng , Daniel Yang , Digbalay Bose , and Shrikanth Narayanan

arXiv preprint arXiv:2402.09036, 2024

Bib

@article{feng2024can,
  title = {Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing?},
  author = {Feng, Tiantian and Yang, Daniel and Bose, Digbalay and Narayanan, Shrikanth},
  journal = {arXiv preprint arXiv:2402.09036},
  year = {2024},
  abbr2 = {Generative AI},
}

Voice Privacy Generative AI

Unlocking Foundation Models for Privacy-Enhancing Speech Understanding: An Early Study on Low Resource Speech Training Leveraging Label-guided Synthetic Speech Content

Tiantian Feng , Digbalay Bose , Xuan Shi , and Shrikanth Narayanan

arXiv preprint arXiv:2306.07791, 2023

Bib

@article{feng2023unlocking,
  title = {Unlocking Foundation Models for Privacy-Enhancing Speech Understanding: An Early Study on Low Resource Speech Training Leveraging Label-guided Synthetic Speech Content},
  author = {Feng, Tiantian and Bose, Digbalay and Shi, Xuan and Narayanan, Shrikanth},
  journal = {arXiv preprint arXiv:2306.07791},
  year = {2023},
  abbr2 = {Generative AI},
}

Preprint Generative AI

GPT-FL: Generative pre-trained model-assisted federated learning

Tuo Zhang* , Tiantian Feng* , Samiul Alam , Mi Zhang , Shrikanth S Narayanan , and Salman Avestimehr

arXiv preprint arXiv:2306.02210, 2023

Bib

@article{zhang2023gpt,
  title = {GPT-FL: Generative pre-trained model-assisted federated learning},
  author = {Zhang*, Tuo and Feng*, Tiantian and Alam, Samiul and Zhang, Mi and Narayanan, Shrikanth S and Avestimehr, Salman},
  journal = {arXiv preprint arXiv:2306.02210},
  year = {2023},
  abbr2 = {Generative AI},
}

Journal Trustworthiness

A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness

Tiantian Feng , Rajat Hebbar , Nicholas Mehlman , Xuan Shi , Aditya Kommineni , and Shrikanth Narayanan

APSIPA Transactions on Signal and Information Processing, 2023

Bib

@article{feng2023review,
  title = {A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness},
  author = {Feng, Tiantian and Hebbar, Rajat and Mehlman, Nicholas and Shi, Xuan and Kommineni, Aditya and Narayanan, Shrikanth},
  journal = {APSIPA Transactions on Signal and Information Processing},
  volume = {12},
  number = {3},
  year = {2023},
  publisher = {Now Publishers, Inc.},
  abbr2 = {Trustworthiness},
}

ICASSP 2022 Trustworthiness

Enhancing privacy through domain adaptive noise injection for speech emotion recognition

Tiantian Feng , Hanieh Hashemi , Murali Annavaram , and Shrikanth S Narayanan

In ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP) , 2022

Bib Code

@inproceedings{feng2022enhancing,
  title = {Enhancing privacy through domain adaptive noise injection for speech emotion recognition},
  author = {Feng, Tiantian and Hashemi, Hanieh and Annavaram, Murali and Narayanan, Shrikanth S},
  booktitle = {ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP)},
  pages = {7702--7706},
  year = {2022},
  organization = {IEEE},
  abbr2 = {Trustworthiness},
}

ICASSP 2024 Trustworthiness

TRUST-SER: On The Trustworthiness Of Fine-Tuning Pre-Trained Speech Embeddings For Speech Emotion Recognition

Tiantian Feng , Rajat Hebbar , and Shrikanth Narayanan

In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024

Bib Code

@inproceedings{feng2024trust,
  title = {TRUST-SER: On The Trustworthiness Of Fine-Tuning Pre-Trained Speech Embeddings For Speech Emotion Recognition},
  author = {Feng, Tiantian and Hebbar, Rajat and Narayanan, Shrikanth},
  booktitle = {ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages = {11201--11205},
  year = {2024},
  organization = {IEEE},
  abbr2 = {Trustworthiness},
}

SIGKDD 2023 Federated Learning

FedMultimodal: A Benchmark for Multimodal Federated Learning

Tiantian Feng , Digbalay Bose , Tuo Zhang , Rajat Hebbar , Anil Ramakrishna , Rahul Gupta , Mi Zhang , Salman Avestimehr , and Shrikanth Narayanan

In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2023

Bib Code

@inproceedings{feng2023fedmultimodal,
  title = {FedMultimodal: A Benchmark for Multimodal Federated Learning},
  author = {Feng, Tiantian and Bose, Digbalay and Zhang, Tuo and Hebbar, Rajat and Ramakrishna, Anil and Gupta, Rahul and Zhang, Mi and Avestimehr, Salman and Narayanan, Shrikanth},
  booktitle = {Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  pages = {4035--4045},
  year = {2023},
  abbr2 = {Federated Learning},
}

ICASSP 2023 Federated Learning

FedAudio: A federated learning benchmark for audio tasks

Tuo Zhang , Tiantian Feng , Samiul Alam , Sunwoo Lee , Mi Zhang , Shrikanth S Narayanan , and Salman Avestimehr

In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023

Bib Code

@inproceedings{zhang2023fedaudio,
  title = {FedAudio: A federated learning benchmark for audio tasks},
  author = {Zhang, Tuo and Feng, Tiantian and Alam, Samiul and Lee, Sunwoo and Zhang, Mi and Narayanan, Shrikanth S and Avestimehr, Salman},
  booktitle = {ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages = {1--5},
  year = {2023},
  organization = {IEEE},
  abbr2 = {Federated Learning},
}

ICASSP TSP 2024 Federated Learning

Partial federated learning: Unlocking non-biometric text information sharing for federated learning

Tiantian Feng , Anil Ramakrishna , Jimit Majmudar , Charith Peris , Jixuan Wang , Clement Chung , Richard Zemel , Morteza Ziyadi , and Rahul Gupta

2024

Bib

@article{feng2024partial,
  title = {Partial federated learning: Unlocking non-biometric text information sharing for federated learning},
  author = {Feng, Tiantian and Ramakrishna, Anil and Majmudar, Jimit and Peris, Charith and Wang, Jixuan and Chung, Clement and Zemel, Richard and Ziyadi, Morteza and Gupta, Rahul},
  booktitle = {ICASSP Workshops 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year = {2024},
  abbr2 = {Federated Learning}
}

INTERSPEECH 2022 Federated Learning

User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition on Federated Learning

Tiantian Feng , Raghuveer Peri , and Shrikanth Narayanan

In Proc. Interspeech 2022 , 2022

Bib Code

@inproceedings{feng22b_interspeech,
  author = {Feng, Tiantian and Peri, Raghuveer and Narayanan, Shrikanth},
  title = {{User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition on Federated Learning}},
  year = {2022},
  booktitle = {Proc. Interspeech 2022},
  pages = {5055--5059},
  doi = {10.21437/Interspeech.2022-10060},
  abbr2 = {Federated Learning},
}

ICASSP 2024 Foundation Model

Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, and Augmenting

Tiantian Feng , and Shrikanth Narayanan

In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024

Bib Code

@inproceedings{feng2024foundation,
  title = {Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, and Augmenting},
  author = {Feng, Tiantian and Narayanan, Shrikanth},
  booktitle = {ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages = {12116--12120},
  year = {2024},
  organization = {IEEE},
  abbr2 = {Foundation Model},
}

ACII 2023 Foundation Model

PEFT-SER: On the use of parameter efficient transfer learning approaches for speech emotion recognition using pre-trained speech models

Tiantian Feng , and Shrikanth Narayanan

In 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII) , 2023

Bib Code

@inproceedings{feng2023peft,
  title = {PEFT-SER: On the use of parameter efficient transfer learning approaches for speech emotion recognition using pre-trained speech models},
  author = {Feng, Tiantian and Narayanan, Shrikanth},
  booktitle = {2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII)},
  pages = {1--8},
  year = {2023},
  organization = {IEEE},
  abbr2 = {Foundation Model},
}

Preprint Wearables

Understanding Stress, Burnout, and Behavioral Patterns in Medical Residents Using Large-scale Longitudinal Wearable Recordings

Tiantian Feng , and Shrikanth Narayanan

arXiv preprint arXiv:2402.09028, 2024

Bib

@article{feng2024understanding,
  title = {Understanding Stress, Burnout, and Behavioral Patterns in Medical Residents Using Large-scale Longitudinal Wearable Recordings},
  author = {Feng, Tiantian and Narayanan, Shrikanth},
  journal = {arXiv preprint arXiv:2402.09028},
  year = {2024},
  abbr2 = {Wearables},
}

Scientific Reports Wearables

A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data

Tiantian Feng , Brandon M Booth , Brooke Baldwin-Rodrı́guez , Felipe Osorno , and Shrikanth Narayanan

Scientific reports, 2021

Bib Code

@article{feng2021multimodal,
  title = {A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data},
  author = {Feng, Tiantian and Booth, Brandon M and Baldwin-Rodr{\'\i}guez, Brooke and Osorno, Felipe and Narayanan, Shrikanth},
  journal = {Scientific reports},
  volume = {11},
  number = {1},
  pages = {8693},
  year = {2021},
  publisher = {Nature Publishing Group UK London},
  abbr2 = {Wearables},
}

ACM Transactions Wearables

Temporal dynamics of workplace acoustic scenes: Egocentric analysis and prediction

Arindam Jati , Amrutha Nadarajan , Raghuveer Peri , Karel Mundnich , Tiantian Feng , Benjamin Girault , and Shrikanth Narayanan

IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021

Bib

@article{jati2021temporal,
  title = {Temporal dynamics of workplace acoustic scenes: Egocentric analysis and prediction},
  author = {Jati, Arindam and Nadarajan, Amrutha and Peri, Raghuveer and Mundnich, Karel and Feng, Tiantian and Girault, Benjamin and Narayanan, Shrikanth},
  journal = {IEEE/ACM Transactions on Audio, Speech, and Language Processing},
  volume = {29},
  pages = {756--769},
  year = {2021},
  publisher = {IEEE},
  abbr2 = {Wearables},
}

Mobisys Workshop Wearables

Tiles Audio Recorder: An Unobtrusive Wearable Solution to Track Audio Activity

Tiantian Feng , Amrutha Nadarajan , Colin Vaz , Brandon Booth , and Shrikanth Narayanan

In Proceedings of the 4th ACM Workshop on Wearable Systems and Applications , 2018

Bib Code

@inproceedings{feng2018tiles,
  title = {Tiles Audio Recorder: An Unobtrusive Wearable Solution to Track Audio Activity},
  author = {Feng, Tiantian and Nadarajan, Amrutha and Vaz, Colin and Booth, Brandon and Narayanan, Shrikanth},
  booktitle = {Proceedings of the 4th ACM Workshop on Wearable Systems and Applications},
  pages = {33--38},
  year = {2018},
  abbr2 = {Wearables},
}

ICASSP 2019 Wearables

Discovering optimal variable-length time series motifs in large-scale wearable recordings of human bio-behavioral signals

Tiantian Feng , and Shrikanth S Narayanan

In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019

Bib

@inproceedings{feng2019discovering,
  title = {Discovering optimal variable-length time series motifs in large-scale wearable recordings of human bio-behavioral signals},
  author = {Feng, Tiantian and Narayanan, Shrikanth S},
  booktitle = {ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages = {7615--7619},
  year = {2019},
  organization = {IEEE},
  abbr2 = {Wearables}
}