PhD Student @ MIT
I am a PhD candidate and an engineer working on the next generation of Artificial Intelligence Systems.
ktiwary [at] mit [dot] edu
Bio
Kushagra (Kush) is a Ph.D. student in Camera Culture group at the MIT Media Lab advised by Ramesh Raskar.
Kush’s overarching research interest is to create new types of Artificial Visual Intelligence through search. His topics of interest are Computer Vision, Imaging, Reinforcement Learning, and Physics-based Simulation, including the co-design of cameras.
His work has led to multiple patents, has been published in major computer vision and computational imaging conferences such as ECCV, ICCP, CVPR, and ICCV, and has received press coverage.
In 2023, Kush received the Qualcomm Innovation Fellowship and completed his Master's Degree (S.M 2021) at MIT. His thesis, “Discovering, Learning, and Exploiting Visual Cues", is at the intersection of Sensing, Imaging, Vision, and their downstream applications and explores visual cues in the modern context of data-driven imaging techniques.
Before MIT, he worked at Optimus Ride (acquired in 2022) where he led the design and deployment of a fleet-wide MultiTasking Computer Vision System for next-generation autonomous vehicles.
[CV] [Media Lab Overview]
Recent News
Oct. 2023: Co-organizer and speaker for Workshop on AI For Accelerating Scientific Discovery: Using AI to Accelerate Science, RD, and Augment Engineering & Design (website, slides upcoming!)
Sept. 2023: Talk in CSAIL Graphics Seminar Neural Rendering and Secondary Cues: Learning Hidden Neural Radiance Fields using Reflections and Shadows. (slides)
Aug. 2023: Work on 3D reconstruction with Visual Cues was presented at the Hyundai Vision Conference in South Korea!
July 2023: North America Winner of the Qualcomm Innovation Fellowship. (Press, QIF List)
July 2023: Paper Accepted at ICCV’23 DiSer: Designing Imaging Systems through Reinforcement Learning, is accepted at ICCV. (website)
May 2023: Graduation from my Masters at the Media Lab from the Camera Culture Group (thesis)
May 2023: Our Project on using household objects as cameras has been featured on the front page of web.mit.edu! (Project Page, Screenshot, MIT News)
Computer Vision & Imaging Research
Nikhil Behari, Akshat Dave, Kushagra Tiwary, William Yang, Ramesh Raskar
(under submission)
3D modeling from satellite imagery is essential in areas of environmental science, urban planning, agriculture, and disaster response. In this work, we introduce SUNDIAL, a comprehensive approach to 3D reconstruction of satellite imagery using neural radiance fields.
[arXiv]
Kushagra Tiwary*, Akshat Dave*, Nikhil Behari, Tzofi Klinghoffer, Ashok Veeraghavan, Ramesh Raskar, CVPR 2023
We convert objects with unknown geometry into radiance-field cameras to image the world from the object's perspective. Our key insight is to convert the object surface into a virtual sensor that captures cast reflections as a 2D projection of the 5D environment radiance field visible to the object.
[Project Page] [arXiv] [Video] [MIT News] [Featured on Front Page of MIT News]
Towards Neural Representations through Shadows
Kushagra Tiwary*, Tzofi Klinghoffer*, Ramesh Raskar,
ECCV 2022
Neural Representations cannot use shadows to learn about hidden information in the scene. We present a method that learns neural shadow fields which are neural scene representations that are only learnt from the shadows present in the scene.
[Project Page] [arXiv] [Video]
Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging
Kushagra Tiwary*, Tzofi Klinghoffer*, Siddharth Somasundaram*, Ramesh Raskar, ICCP 2022
Cameras were originally designed using physics-based heuristics to capture aesthetic images, but now with the advent of physics-based machine learning, we can design cameras to be task-specific i.e. directly for their intended application!
AI for Search & Discovery
I am actively working on this area. Shoot me an email if you’re interested in collaborating & stay tuned for some upcoming work in this area!
Accelerating Discovery: Using AI to accelerate Science, R&G, and Augment Engineering & Design.
Co-Organizer and Speaker at Workshop in Media Lab Member’s week 2023
Talk on: Accelerating R&D and AI-based Vision Stack Design
This workshop explores how AI can accelerate science, R&D processes, and augment engineering with a special emphasis on Research and Development (R&D). We will introduce the "Scientist AI" that can vastly accelerate time-consuming scientific and R&D design processes through case studies from the fields of Autonomous Vehicles, Computational Imaging, Drug Discovery, and Theraputics.
DISeR: Designing Imaging Systems with Reinforcement Learning
Tzofi Klinghoffer*, Kushagra Tiwary*, Nikhil Behari, Bhavya Agarwalla, Ashok Veeraghavan, Ramesh Raskar, ICCV 2023
We propose a new way to co-design imaging systems and task-specific perception models. The camera designer selects imaging hardware candidates, which are used to capture observations in simulation. The perception model is then updated and computes the reward for the camera designer using the captured observations. In our work, we implement the camera designer with reinforcement learning and the perception model with a neural network.
[Project Page] [arXiv] [Video]
Credit: Ricky Vasan (check out his art here!)
Personal
As a kid, I moved around a lot- from Jamshedpur & Dehradun (India), Sana’a (Yemen), Madrid (Spain), Oklahoma City, and Houston all by age 18.
I picked up bits of language and culture as I grew up in different multicultural environments around the world. So I’m currently fluent in Spanish and Hindi but have lost touch with my French & Arabic.
Although I am a curious soul trying to learn about everything, I love spending time with my family & friends and reading about Eastern philosophy. I grew up playing in USTA Tennis leagues around the Boston area in addition to golf and soccer (although soccer has been minimal since my ACL injury).
Tennis in the Boston Summer is beautiful!
Reach out if you’d like to talk about AI, RL, Imaging, or Vision!
