Ramya Hebbalaguppe

I'm a researcher in visual computing. My work spans: computer vision, reliable and robust machine learning, and computer graphics. I'm a Principal Scientist in Deep Learning and Artificial Intelligence Group (DLAI), TCS Research, IIT Delhi, India. My doctoral research advised by Prof. Chetan Arora at IIT Delhi is centered on proposing novel methods to enhance reliability in deep neural networks (DNNs)

Prior to this, I was fortunate to be working with Prof. Ramakrishna Kakarala at Nanyang Technological University on High Dynamic Range Imaging algorithms which formed a part of the image processing pipeline aimed at smartphone cameras. Our research was recognized with the Best Student Paper award at the 2012 SPIE conference in Burlingame, California. I completed my master's degree at DCUs School of Electronic Engineering and Computing in 2014, advised by Prof. Noel O'Connor and Prof. Alan Smeaton. I focused on reducing false alarms in surveillance camera networks. The result of this work, a portion of our research was licensed to Netwatch Systems.

In June 2015, I started working as a research scientist at TCS Research. Since the, I have been involved in various projects related to augmented reality. Specifically, I have focused on optimizing the layout of labels for immersive experiences and developing gestural interfaces for head-mounted devices and smartphones. As a team leader, I have overseen the development of a cost-effective industrial inspection framework. Recently, my team has been working on creative content generation (images, videos, 3D/4D data).

Outside work, I enjoy painting, traveling, cooking and baking, composting, planting tree saplings, and music.

> Email  /  Google Scholar  /  Twitter  /  Github

Representative papers spanning the following themes reliable machine learning (out-of-distribution detection, uncertainty quantification, continual learning), 3D/4D/2D computer vision are highlighted.

Recent developments in the field of diffusion models have demonstrated an exceptional capacity to generate high-quality prompt-conditioned image edits. Nevertheless, previous approaches have primarily relied on textual prompts for image editing, which tend to be less effective when making precise edits to specific objects or fine-grained regions within a scene containing single/multiple objects. We introduce a novel framework for zero-shot localized multi-object editing through a multi-diffusion process to overcome this challenge. This framework empowers users to perform various operations on objects within an image, such as adding, replacing, or editing many objects in a complex scene in one pass. Our approach leverages foreground masks and corresponding simple text prompts that exert localized influences on the target regions resulting in high-fidelity image editing. A combination of cross-attention and background preservation losses within the latent space ensures that the characteristics of the object being edited are preserved while simultaneously achieving a high-quality, seamless reconstruction of the background with fewer artifacts compared to the current methods. We also curate and release a dataset dedicated to multi-object editing, named LoMOE-Bench. Our experiments against existing state-of-the-art methods demonstrate the improved effectiveness of our approach in terms of both image editing quality and inference speed.

We introduce ReMOVE, a novel reference-free metric for assessing object erasure efficacy in diffusion-based image editing models post-generation. Unlike existing measures such as LPIPS and CLIPScore, ReMOVE addresses the challenge of evaluating inpainting without a reference image, common in practical scenarios. ReMOVE effectively distinguishes between object removal and replacement, a key issue in diffusion models due to stochastic nature of image generation.

Animating a virtual character based on a real performance of an actor is a challenging task that currently requires expensive motion capture setups and additional effort by expert animators, rendering it accessible only to large production houses. The goal of our work is to democratize this task by developing a frugal alternative termed Transfer4D that uses only commodity depth sensors and further reduces animators' effort by automating the rigging and animation transfer process. Our approach can transfer motion from an incomplete, single-view depth video to a semantically similar target mesh, unlike prior works that make a stricter assumption on the source to be noise-free and watertight.

We demonstrate state-of-the-art Deep Neural Network calibration performance via proposing a differentiable loss term that can be used effectively in gradient descent optimisation and dynamic data pruning strategy not only enhances legitimate high confidence samples to enhance trust in DNN classifiers but also reduce the training time for calibration.

We propose a novel Compounded Corruption(CnC) technique for the Out-of-Distribution data augmentation. One of the major advantages of CnC is that it does not require any hold-out data apart from the training set. Our extensive comparison with 20 methods from the major conferences in last 4 years show that a model trained using CnC based data augmentation, significantly outperforms SOTA, both in terms of OOD detection accuracy as well as inference time.

We propose a novel auxiliary loss function: Multi-class Difference in Confidence and Accuracy (MDCA) for Deep Neural Network calibration. The loss can be combined with any application specific classification losses for image, NLP, Speech domains. We also demonstrate the utility of the loss in semantic segmentation tasks.