Plant–robot interaction · ICSR+AI 2025

Plan around
living geometry.

How can a manipulator reach weeds while treating the crop—not the robot—as the center of the planning problem?

This work combines image-based plant geometry recovery, a feedforward model, a human-inspired workspace construction strategy, hybrid inverse kinematics, adaptive control, and end-effector co-design. The method was evaluated in Gazebo and on a physical mobile manipulator in a greenhouse-style setup.

Inspect the method ↓Published paper ↗Controller code ↗

Authors

V. P. TharunAbhra Roy Chowdhury

VenueICSR+AI 2025 · LNAI 16132 · pp. 253–267

PublishedSpringer · 2 January 2026

Abstract

A plant is not a rigid obstacle with a stable collision box. Its root, stem, leaves, and reachable free space must be inferred—and the task plan must change with that geometry before the robot begins to move.

PublicationSpringer chapter ↗FiguresPreprint and figures ↗Open codeAdaptive end-effector controller ↗

ValidationGazebo + physical mobile manipulator

Physical precision-weeding rolloutVision · task planning · manipulation · end-effector action

Problem formulation

The target and the obstacle are intertwined.

In plant-centered work, the robot must enter a constrained canopy to reach a weed while avoiding contact that can damage nearby crop leaves or stems. Standard IK answers whether a pose is reachable; it does not decide whether the path is sensible for the plant.

Geometry

No clean collision primitive

Leaves overlap, bend, and change appearance. A conservative box blocks useful work; an optimistic box permits harmful contact.

Interaction

Not every contact is equivalent

Touching the weed is required. Touching the crop is undesirable. The task needs semantic geometry, not occupancy alone.

Planning

Reachability is only one objective

A candidate must balance target access, crop-contact risk, motion duration, and the end-effector’s approach needs.

Method

Recover geometry, reshape the workspace, then solve.

The planner constructs a plant-aware action region before selecting the manipulator pose. This reverses the usual order in which IK finds a solution first and safety logic rejects it afterward.

01SegmentRGB threshold · morphology

02Estimateroot · stem · leaf spread

03Modelfeedforward network · geometry

04Shapeplant-centered workspace

05Plan + acthybrid IK · end effector

Architecture and planning evidence

The figures below are extracted directly from the paper at their native proportions. Select any figure to inspect the full-resolution version.

Published architecture connecting plant-image perception, root and spread estimation, workspace extraction, hybrid inverse kinematics and the flower-carpet planner — **System architecture**Vision and manipulation branches meet at the Hybrid IK controller. Paper Fig. 2

Published flowchart of the iterative hybrid inverse-kinematics controller — **Hybrid IK controller**Iterative target comparison, neural prediction and standard-IK fallback. Paper Fig. 3

Original manipulator workspace shown as a three-dimensional point distribution — **Original workspace**Uncurated reachable samples before task-specific warping. Paper Fig. 4a

Warped manipulator workspace concentrated around the task-relevant region — **Warped workspace**Task-relevant samples reduce ambiguity and singular regions. Paper Fig. 4b

OmxNet feedforward neural-network architecture mapping Cartesian coordinates to four joint angles — **OmxNet**Feedforward mapping from x, y, z to four joint-angle outputs. Paper Fig. 5

Flower-carpet motif divided into four planning zones with store points — **Human-inspired planning prior**The flower-carpet arrangement motivates zones and reusable store points. Paper Fig. 6a

Plant-bed planning layout divided into four zones with intermediate halt points and crop and weed positions — **Flower-carpet planner**Zone centres become intermediate halt points between the bed centre and individual targets. Paper Fig. 6b

Method stages

01
Extract a plant mask
Channel-based segmentation and elliptical morphological closing recover a contiguous plant region from RGB observations.
02
Estimate semantic geometry
The system infers root/stem location, leaf-area spread, plant-bed layout, and the region in which a manipulation approach should be avoided.
03
Approximate geometry with a feedforward model
A learned component maps visual plant features to quantities required by planning, connecting perception to a compact geometric task representation.
04
Construct a human-inspired task workspace
The planning region is organized around the crop and target, inspired by the structured placement used when making flower carpets, rather than sampling the robot’s entire reachable space indiscriminately.
05
Apply hybrid inverse kinematics
Analytical reachability and learned/task-specific selection work together to identify a pose that is both executable and better aligned with plant-contact constraints.
06
Execute with the task-specific tool
The motion terminates in a co-designed end effector able to mechanically remove the weed and support targeted spray or watering operations.

Perception evidence

From pixels to plant-centred geometry.

The paper does not treat detection as the endpoint. Each visual stage produces a geometric quantity used by planning: crop/weed identity, leaf spread, stem centre, root estimate, or zone centre.

YoloR detections identifying crops and weeds in a plant bed — **Crop and weed detection**Detected classes establish which plants are protected and which are manipulation targets. Paper Fig. 7

Leaf-area estimation sequence from plant image through thresholding and connected-component analysis — **Leaf-spread estimation**Otsu thresholding and connected components approximate relative leaf area. Paper Fig. 8

Stem-centre detection sequence using RGB channels, excess green index, thresholding, elliptical closing and convexity defects — **Stem-centre pipeline**RGB-derived vegetation evidence is cleaned before convexity-based centre estimation. Paper Fig. 9

Root estimation geometry relating stem centre to the soil surface — **Root approximation**Stem centre and soil distance define a contact-minimizing approach arc. Paper Fig. 10

Original top-down image of the experimental plant bed — **Plant-bed input**Top-down observation before vegetation-index processing. Paper Fig. 11a

Thresholded binary mask of plants in the bed — **Threshold mask**Vegetation separated from the bed. Fig. 11b

Dilated plant mask joining fragmented plant regions — **Dilation**Fragmented regions are consolidated. Fig. 11c

Plant-bed image with detected contour centroids — **Contour centroids**Candidate plant positions are localized. Fig. 11d

Plant-bed image divided into planning zones with detected zone centres — **Zone-centre extraction**Centroids are grouped into the four planning zones used by the flower-carpet planner. Paper Fig. 11e

Hardware–software co-design

The tool changes what the planner should optimize.

Mechanical plucking, position control, contact limitation, and targeted fluid delivery were treated as one system rather than independent modules.

AGGRIP end effector

The gripper was co-designed with a mechanical engineer for plant-centered operations. It combines mechanical removal with targeted spray/water capability so the same tool can perform different local treatments after reaching the plant.

Mechanical weed engagement
Targeted spray or water delivery
Geometry compatible with constrained plant beds
Planner and mechanism designed together

Adaptive position–torque control

A separate open implementation uses a Dynamixel actuator in current-based position mode. PID position error determines commanded current subject to saturation and minimum actuation thresholds, while a near-target state changes the current command.

Current-based position operation
Position feedback from the actuator encoder
PID terms with bounded current command
10 ms update delays in the prototype loop

Physical implementation trace

The rollout video contains more than a final demonstration. These timestamped frames expose the cell geometry, task staging, target approach, local alignment, tool action, retraction, and transition to the next plant.

AGGRIP mechanism developmentCAD motion study of the task-specific gripping and treatment mechanism. Design video · 00:00–00:08

Wide view of the mobile manipulator and three physical plant beds before task execution — **Cell and task context**Manipulator, sensing mast, tool and three plant beds are visible together. Rollout · 00:00.5

Manipulator moving from its staging region toward the centre plant bed — **Task approach**Arm leaves the staging region and enters the selected bed. Rollout · 00:05.5

Manipulator aligning its end effector above a target plant in the centre bed — **Target alignment**The wrist and tool are oriented locally above the selected plant. Rollout · 00:09.5

End effector positioned at the target during the local plant operation — **Tool action**The task-specific end effector executes at plant level. Rollout · 00:13.5

Manipulator retracting from the plant bed after the local operation — **Retraction**The controller clears the canopy before the next transfer. Rollout · 00:19.5

Manipulator returned toward its staging region before the next plant task — **Task transition**The arm returns toward a safe intermediate configuration. Rollout · 00:27.0

Development history: the controller repository is a prototype artifact from the broader autonomous-weeding program. Its code establishes the control logic; it does not by itself establish crop-safety performance.

Published results

Planning changed success, contact, and time.

The paper compares the proposed hybrid IK task planner with a standard IK solver in the reported simulation and physical evaluation conditions.

+32.27%reported success-rate increase over standard IK

−0.45average unwanted plant interactions · Contact Factor

−58 saverage execution-time reduction

2 domainsGazebo simulation + physical greenhouse setup

Published comparison of hybrid inverse kinematics against standard inverse kinematics for success, unwanted plant interaction, and execution time — Reported change relative to the standard-IK baseline under the paper’s evaluation setup. Metrics retain their original units and should not be compared by bar length across panels.

Model and planner behaviour

Aggregate results are paired with the underlying training curves, trajectory comparisons, simulation frame, and Cartesian-error traces so the result can be inspected rather than reduced to one percentage.

Comparison of manipulator trajectories for three planners across low, medium and high plant-density layouts — **Planner trajectories**Home-to-pose, centre-to-pose, and flower-carpet traversal across three bed layouts. Paper Fig. 12

Gazebo simulation of the agricultural mobile manipulator working across three plant beds — **Simulation validation**The software path was first exercised in Gazebo before transfer to the physical robot. Paper Fig. 13

Four training and validation loss plots for two OmxNet variants on original and warped workspaces — **OmxNet learning curves**Training and validation loss for two models under original and warped workspace sampling. Paper Fig. 14

Cartesian x, y, z and distance errors between predicted and actual poses for three layouts — **Pose-error structure**Directionally consistent waypoint errors across three layouts suggest a data-coverage limitation. Paper Fig. 15

Evaluation dimension	Reported change	Why it matters	Boundary
Task success	32.27% increase relative to standard IK	Plant-aware candidate selection found useful task poses more often.	Relative change under the paper’s test setup; not a universal success rate.
Plant interaction	0.45 lower average Contact Factor	The plan reduced undesirable crop interactions while reaching the target.	Metric definition and scene distribution follow the paper.
Execution time	58 s lower on average	Constraining the task-relevant search space reduced time spent reaching workable poses.	Hardware, planner, and task complexity are setup-specific.

Reported values

These tables reproduce the paper’s comparison values directly, including the trade-off the summary statistic hides: iterative Hybrid IK improves success but takes longer than the one-shot standard solver.

Task planner	Execution time (s)			Safety quotient
Task planner	Low	Medium	High	Low	Medium	High
Home-to-pose	261	217	432	12	7	3.6
Centre-to-pose	210	169	382	∞	∞	5.5
Flower Carpet (ours)	185	148	336	∞	∞	22

IK solver	Success rate (%)			Execution time (s)
IK solver	Low	Medium	High	Low	Medium	High
Standard IK	46.3	43.1	36.36	131	105	213
Hybrid IK (ours)	78.57	66.6	59.2	193	172	336

OmxNet	Original-workspace RMSE (m)	Warped-workspace RMSE (m)
Model 1	0.0085	0.0030
Model 2 (ours)	0.0079	0.0025

Interpretation: the results support plant-aware planning within the reported greenhouse tasks. They do not establish robustness across crop species, growth stages, outdoor illumination, dense foliage, soil variation, or commercial operating durations.

Contribution and citation

Perception, planning, control, and hardware in one study.

The project evolved through physical autonomous-weeding development, controller prototyping, end-effector co-design, simulation experiments, and the published hybrid-IK evaluation.

I developed the end-to-end plant-aware manipulation system and led the published method.

My work covered the vision pipeline, plant-geometry representation, hybrid IK/task-planning formulation, robot integration, controller development, simulation and physical evaluation, and co-design of the task-specific end effector. Abhra Roy Chowdhury is credited as co-author and research collaborator.

Open evidence gap. A modern follow-up should publish trial-level outcomes, contact events, planning times, plant-species metadata, segmentation masks, and failure taxonomy—not only aggregate averages.

Citation

Tharun, V. P., and A. R. Chowdhury. “Vision Based Hybrid IK Task Planning with Feedforward Neural Network for Collaborative Plant-Robot Interaction in Precision Farming.” In Social Robotics + AI, LNAI 16132, pp. 253–267. Springer, 2026. DOI: 10.1007/978-981-95-2382-5_18.

Related project

How can independent sensing channels make an autonomous robot harder to deceive?

Open anomaly detection →