Prof. Urbashi Mitra
University of Southern California
Date/Time: Sept. 5, 2025 – 15:00
Place: EE517
Digital Cousins: Generative Multi-Environment Mixed Q-Learning
We propose a strategy for generating trajectories for policy optimization on graphs. While many systems are well modeled by Markov Decision Processes (MDPs), this approach induces a large state space which challenges policy optimization. We propose a novel on-line/in-real-time Q-learning methodology based on the creation of synthetic MDPs (digital cousins) that run in parallel with the true system. These cousins are statistically related, but distinct from the true system. The approach enables the ensemble learning of multiple policies which can be efficiently fused. The new learning method is distinctly different from more classical learning strategies that mix off-line collected data with real-time trajectory tracking. The proposed mixed strategy offers significantly improved convergence rates and performance. We provide theoretical results on convergence as well as the ranking of the informativeness of the synthetic environments via learning analyses based on coverage. We provide preliminary results on a multi-agent version of the learning approach. Numerical examples from wireless communication networks and multi-agent UAV networks are provided.
Bio: Urbashi Mitra received the B.S. and the M.S. degrees from the University of California at Berkeley and her Ph.D. from Princeton University. Dr. Mitra is currently the Gordon S. Marshall Professor in Engineering at the University of Southern California with appointments in Electrical Engineering and Computer Science. Dr. Mitra is a Fellow of the IEEE and a foreign member of the Academia Europaea. She was the inaugural Editor-in-Chief for the IEEE Transactions on Molecular, Biological and Multi-scale Communications. Dr. Mitra has served as an Associate Editor for multiple IEEE publications. Dr. Mitra was a member of the IEEE Information Theory Society’s Board of Governors (2002-2007, 2012-2017), the IEEE Signal Processing Society’s Technical Committee on Signal Processing for Communications and Networks (2012-2017), and the Chair/Vice Chair of the IEEE Communications Society, Communication Theory Technical Committee (2017-2020). She is the recipient of: the 2024 Information Theory Society’s Aaron Wyner Distinguished Service Award, a 2024 IEEE Signal Processing Society Distinguished Lecturer, the 2021 USC Viterbi School of Engineering Senior Research Award, a 2016 UK Royal Academy of Engineering Distinguished Visiting Professorship, a 2016 US Fulbright Scholar Award, a 2016-2017 UK Leverhulme Trust Visiting Professorship, 2015-2016 IEEE Communications Society Distinguished Lecturer, 2012 Globecom Signal Processing for Communications Symposium Best Paper Award, Student Best Paper Award, as co-advisor, at the International Conference on Signal Processing and Communications, Bangalore India 2012, a 2009 DCOSS Applications & Systems Best Paper Award, Texas Instruments Visiting Professor (Fall 2002, Rice University), 2001 Okawa Foundation Award, 2000 OSU College of Engineering Lumley Award for Research, 1997 OSU College of Engineering MacQuigg Award for Teaching, and a 1996 National Science Foundation CAREER Award. Her research interests are in: model-based machine learning, wireless communications, structured statistics, communication and sensor networks, biological communication systems, detection and estimation and the interface of communication, sensing and control.