Understanding
People
How can robots understand what people are doing and what they intend?
Robotics Engineer · Robot Learning and Deformable Manipulation
Optimal, assigned to General Motors R&D
I study how robots can understand what people want and carry what they learn from one interaction into the next. I use physical robots and simulation to test how policies learn from demonstrations and corrections, and what information they actually use when they act.
How can robots understand what people are doing and what they intend?
How can robots learn from demonstrations and corrections while retaining earlier skills?
What has a policy learned, and how does that shape its behavior?
How can predicting what happens next help a robot learn how to act?
Current work
The bimanual robot and its digital twin share a platform I built to study how policies learn from human corrections and what information they use to act.
Implemented workflow
When a policy fails, a person can take control of the arm. The workflow records the correction so it can be reviewed and added to the demonstrations used to train the next policy.
People and policies use the same arm commands, so their actions can be compared step by step. Policy evaluation is currently performed manually. Whether the corrections improve later policies without eroding earlier skills remains an open question.
Simulation and physical robot studies
In simulation, representation probes showed that the tested policies encoded links between words and objects. Activation patching showed that their action heads did not use those links.
Bimanual robot trials measured object choice separately from pick completion. In those trials, higher task success did not necessarily mean the policy was better at choosing the object named in the instruction.
A portable interface is designed to capture human manipulation demonstrations using the same recording and review tools as teleoperation.
Simulation tests compared teacher-student training with learned auxiliary heads to see what policies retain when privileged inputs are removed. Attention maps help diagnose behavior, but they do not by themselves establish a causal explanation.
The goal is to compare policies on partial progress, failure modes, recovery and repeatability as well as task success.
Offline tests evaluated a video-action model adapted to teleoperation data. Simulation tests compared actions inferred from video with recorded actions and commands sent to the robot. They assessed future-feature prediction, instruction grounding, and failure signals separately from task success. Physical validation is pending.
Experiments use a digital twin to study how changes in sensing, timing, contact and appearance affect transfer between simulation and hardware.
I designed and implemented the workflow from teleoperation and synchronized recording to dataset conversion, policy training, evaluation, and deployment. The physical robot and digital twin share the operator, data review, and policy interfaces, with separate hardware and physics backends.
The research platform
Implemented Six shared layers · physical and simulated backends · staged evaluation
Policy analysis instrument
Select an input source, then compare the nominal and controlled-perturbation views.
Vision Which regions redirect the policy response when visual evidence changes?
Research portfolio
Case studies with methods, results, and links to papers or code.
A person can use a click, box, image, or text to tell a drone what to follow. I co-developed the system, implemented the high-level controller, and designed the trajectory evaluation against motion capture.
Physical flight trials compared target and drone trajectories. Obstruction increased the distance between the aligned trajectories in the reported trials.
U-ASTROD · GameSec 2024
For the ASTROD research program, I architected the fusion pipeline that encoded LiDAR, odometry, and network traffic independently before combining their learned representations in a shared anomaly detector.
The published system paired modality-specific spatiotemporal encoders with feature-wise reconstruction scoring. It achieved up to 98% accuracy with vector-based reconstruction loss versus 72% with scalar-based loss, and 97% with half the training data.

Multimodal human instruction
Using the NatSGD dataset provided by Snehesh Shrestha (UMD), I implemented a kitchen-object detector, a language-conditioned recurrent policy, and two experimental gesture pathways. The notebooks establish working model and training prototypes, while task success and the benefit of gesture remain unvalidated.
Contribution boundary: I did the notebook implementation; the NatSGD dataset and paper belong to Shrestha and his co-authors. The model builds on Stepputtis et al.'s language-conditioned imitation-learning method.
Related research contextNatSGD later formalized speech, gesture, and synchronized robot demonstrations at UMD. Image: Shrestha et al., CC BY 4.0.
A filterable index of research systems, prototypes, course projects, and independent software.
Bimanual manipulation through multimodal perception, compliance, learned policies, simulation, and real-world deployment.
My language-conditioned policy and gesture-grounding experiments on the NatSGD dataset.

Weed perception, adaptive control, plant-safe hybrid-IK planning, and a co-designed end effector.
Bio-inspired leader–follower coordination using software latching.
Click, image, text, and box-conditioned tracking with foundation models and visual servoing.

Vision-based localization for aerial asset inspection and management.
YOLO-based visual inspection for detecting structural surface defects.
Superpixel segmentation for compact image representation.
Image outpainting through coordinate-conditioned neural representations.
Transfer learning for estimating scene depth from a single camera.
Visual apparel synthesis and personalized style evaluation.
Action recognition and immersive feedback for batting practice.
An AI-assisted shopping and personal-style application.
A bowler agent that adapts to batting styles and field configurations.
A lightweight soft-robotic prototype for pipe inspection.

Modality-specific LiDAR, odometry, and network encoders fused for abnormal-behavior detection.

A self-hosted workspace for daily capture, research, projects, and human–agent collaboration.

An exploratory outdoor-safety concept using asynchronous event streams.
A LAN-hosted team hub for conversation, project knowledge, files, and equipment records.

Independent products · 2025–present
Tools I build outside my robotics work to explore local-first software, technical knowledge management, and human–agent workflows.
01 / 02Use arrows, tabs, or swipe
“The notebook I wanted
but couldn’t find.”
A single-user workspace for daily notes, tasks, projects, journaling, and research capture. It occupies the space between heavyweight cloud tools and bare Markdown: fast enough for every thought, structured enough for long-running work, and owned entirely by the person using it.



“The operating layer
for a physical team.”
An actively developed, self-hosted team hub for labs, workshops, and robotics crews. It keeps conversation, technical knowledge, files, equipment, and agent-generated context on the local network—without external accounts, cloud dependency, or telemetry.



Research record
Selected peer-reviewed publications and preprints. Also published as V. P. Tharun. Earlier forecasting and data-mining work remains available below.
IEEE ROSE 2026 · DOI: 10.1109/ROSE69181.2026.11570974
How can domain structure and learned evidence work together to secure autonomous systems?
Connects first-principles system knowledge with data-driven detection for more efficient cyber-physical security.
International Conference on Social Robotics + AI (ICSR + AI 2025), LNAI 16132, pp. 253–267, Springer, 2026 · DOI: 10.1007/978-981-95-2382-5_18
How can vision and learned task planning make plant–robot interaction more collaborative?
Combines vision, hybrid inverse kinematics, and a feedforward network for precision-farming task execution.
GameSec 2024, LNCS 14908, pp. 306–325 · DOI: 10.1007/978-3-031-74835-6_15
How should autonomous systems be evaluated when abnormal behavior appears across multiple modalities?
Architected the multimodal fusion pipeline within a team framework for anomaly detection and evaluation beyond controlled laboratory conditions.
How can autonomous followers coordinate reliably with a leader in dynamic environments?
Introduces software latching and a layered perception–decision–control architecture for real-time multi-agent coordination.
Can a UAV track what a person means, regardless of how they specify it?
Combines DINO, CLIP, SAM, multimodal queries, visual servoing, and motion control on a PX4-enabled aerial platform.
Can compliant mechanisms produce a lightweight inspection robot for difficult pipe surfaces?
Develops a soft-robotic wall-climbing prototype for inspection inside and outside pipes.
How can lightweight time-series analysis provide useful early warning of unexpected rainfall?
Develops an efficient and implementable approach for short-term rainfall forecasting.
How do regression and statistical models compare when predicting regional rainfall intensity?
Evaluates multiple regression approaches using relative error on rainfall data from Coonoor.
Which classical learning method best characterizes groundwater quality from measured samples?
Compares decision trees, K-nearest neighbours, and support-vector machines on groundwater data.
How can data-mining methods turn raw water measurements into an actionable quality assessment?
Applies classical learning techniques to environmental water-quality analysis.
What temporal structure in historical rainfall can be converted into a practical forecast?
Uses time-series analysis to model and forecast rainfall patterns.
Experience
Present
Optimal – assigned to General Motors R&D
Bimanual deformable-object manipulation, multimodal perception, force-aware compliance, demonstration-driven policies, digital twins, sim-to-real evaluation, and ROS 2 deployment.
University of Maryland
Mobile-robot navigation security, abnormal-behavior detection, action recognition (graduate course project), and multimodal human–robot interaction.
Indian Institute of Science
Collaborative agricultural robotics, manipulation, and perception-driven precision weeding.
CTO Office, Wipro Digital
Computer vision, mobile AI, egocentric interaction, mixed reality, and rapid prototyping across emerging technologies.
Technical storytelling
I write about the assumptions behind models and demonstrations, the engineering between a benchmark and deployment, and the human consequences of technical choices.
Human in the Loom is an independent publication for longer essays on these themes.
Research direction
Robots that can understand people and learn from them.
My long-term aim is to develop algorithms that allow robots to interpret human intent in ways that support action and learning.
My work approaches this problem through policy understanding, representation learning, world models, embodied experience, human interaction, continual learning, and rigorous evaluation.
I am particularly interested in methods that remain interpretable and testable when they move from offline benchmarks to physical systems.
I hold an M.Eng. in Robotics from the University of Maryland and bring experience across academic research, industrial R&D, and real-world robotics deployment.
Collaborate