SE(3)-based representation of object deformations for robotic shape servoing: Formulation and assessment

[en] The robotic shape servoing of deformable objects presents significant challenges due to their theoretically infinite configuration space, making the choice of suitable shape and deformation representations crucial. Our study proposes an SE(3)-based servoing (SE3BS) framework that combines: (1) a representation of configurations and errors that capture both the object’s spatial pose and its intrinsic shape, (2) a representation of the deformations in local frames, by leveraging the se(3) Lie algebra, thereby inducing useful invariance properties of the deformation model, and (3) integration of local frames orientations into the feedback loop. While we demonstrate the approach within a standard Jacobian-based control law and analyze its stability via Lyapunov theory, the representation is compatible with more advanced model-free or model-based frameworks. To analyze its impact, we compare SE3BS to classical position-based methods in an extensive way on a 2D setup. In addition, we introduce a systematic methodology to analyze how controller performance varies with the choice of shape features. Simulation results show improved performance and reduced sensitivity to features selection, and we validate these findings on a real robotic platform. The present work details single-arm manipulation of linear objects, and our simulations illustrate its extension to dual-arm manipulation of cables and shells in 3D.

Disciplines :

Mechanical engineering

Author, co-author :

Dehaybe, Louis ; Université de Liège - ULiège > Aérospatiale et Mécanique (A&M)

Bruls, Olivier ; Université de Liège - ULiège > Département d'aérospatiale et mécanique > Laboratoire des Systèmes Multicorps et Mécatroniques

Language :

English

Title :

SE(3)-based representation of object deformations for robotic shape servoing: Formulation and assessment

Publication date :

February 2026

Journal title :

Robotics and Autonomous Systems

ISSN :

0921-8890

Publisher :

Elsevier BV

Pages :

105403

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://api.elsevier.com/content/article/PII:S092188902600076X?httpAccept=text/xml

Available on ORBi :

since 25 February 2026

Statistics

Number of views

41 (9 by ULiège)

Number of downloads

1 (1 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

Arriola-Rios, V.E., Guler, P., Ficuciello, F., Kragic, D., Siciliano, B., Wyatt, J.L., Modeling of deformable objects for robotic manipulation: A tutorial and review. Front. Robot. AI, 7, 2020, 82, 10.3389/frobt.2020.00082.
Navarro-Alarcon, D., Liu, Y.-H., Fourier-based shape servoing: A new feedback method to actively deform soft objects into desired 2-D image contours. IEEE Trans. Robot. 34:1 (2018), 272–279, 10.1109/TRO.2017.2765333.
Yu, M., Lv, K., Zhong, H., Song, S., Li, X., Global model learning for large deformation control of elastic deformable linear objects: An efficient and adaptive approach. IEEE Trans. Robot. 39:1 (2023), 417–436, 10.1109/TRO.2022.3200546.
Navarro-Alarcón, D., Liu, Y.-H., Romero, J.G., Li, P., Model-free visually servoed deformation control of elastic objects by robot manipulators. IEEE Trans. Robot. 29:6 (2013), 1457–1468, 10.1109/TRO.2013.2275651.
Cuiral-Zueco, I., Karayiannidis, Y., López-Nicolás, G., Contour based object-compliant shape control. IEEE Robot. Autom. Lett. 8:8 (2023), 5164–5171, 10.1109/LRA.2023.3292617.
Selig, J.M., Geometric Fundamentals of Robotics, vol. 128, 2005, Springer.
Lynch, K.M., Park, F.C., Modern Robotics. 2017, Cambridge University Press.
Sonneville, V., Brüls, O., A formulation on the special euclidean group for dynamic analysis of multibody systems. J. Comput. Nonlinear Dyn., 9(4), 2014, 041002.
Sonneville, V., Cardona, A., Brüls, O., Geometrically exact beam finite element formulated on the special euclidean group SE (3). Comput. Methods Appl. Mech. Engrg. 268 (2014), 451–474.
Lv, N., Liu, J., Jia, Y., Dynamic modeling and control of deformable linear objects for single-arm and dual-arm robot manipulations. IEEE Trans. Robot. 38:4 (2022), 2341–2353.
Artinian, A., Amar, F.B., Perdereau, V., Closed-loop shape control of deformable linear objects based on cosserat model. IEEE Robot. Autom. Lett., 2024.
L. Dehaybe, O. Brüls, An SE (3)-based formulation of the shape servoing problem, in: 2nd Workshop on Representing and Manipulating Deformable Objects At the IEEE International Conference on Robotics and Automation 2022, 2022.
Yu, M., Lv, K., Wang, C., Jiang, Y., Tomizuka, M., Li, X., Generalizable whole-body global manipulation of deformable linear objects by dual-arm robot in 3-d constrained environments. Int. J. Robot. Res. 44:4 (2025), 607–639.
Gu, F., Zhou, Y., Wang, Z., Jiang, S., He, B., A survey on robotic manipulation of deformable objects: Recent advances, open challenges and new frontiers. 2023 arXiv preprint arXiv:2312.10419.
Yin, H., Varava, A., Kragic, D., Modeling, learning, perception, and control methods for deformable object manipulation. Sci. Robot., 6(54), 2021, eabd8803, 10.1126/scirobotics.abd8803.
Deng, Y., Xia, C., Wang, X., Chen, L., Deep reinforcement learning based on local GNN for goal-conditioned deformable object rearranging. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2022, 1131–1138, 10.1109/IROS47612.2022.9981669.
Jangir, R., Alenyà, G., Torras, C., Dynamic cloth manipulation with deep reinforcement learning. 2020 IEEE International Conference on Robotics and Automation, ICRA, 2020, 4630–4636, 10.1109/ICRA40945.2020.9196659.
Wang, Y., Sun, Z., Erickson, Z., Held, D., One policy to dress them all: Learning to dress people with diverse poses and garments. Robotics: Science and Systems, RSS, 2023.
Wu, Y., Yan, W., Kurutach, T., Pinto, L., Abbeel, P., Learning to manipulate deformable objects without demonstrations. 2019 arXiv preprint arXiv:1910.13439.
Scheikl, P.M., Tagliabue, E., Gyenes, B., Wagner, M., Dall'Alba, D., Fiorini, P., Mathis-Ullrich, F., Sim-to-real transfer for visual reinforcement learning of deformable object manipulation for robot-assisted surgery. IEEE Robot. Autom. Lett. 8:2 (2023), 560–567, 10.1109/LRA.2022.3227873.
Pore, A., Corsi, D., Marchesini, E., Dall'Alba, D., Casals, A., Farinelli, A., Fiorini, P., Safe reinforcement learning using formal verification for tissue retraction in autonomous robotic-assisted surgery. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2021, 4025–4031, 10.1109/IROS51168.2021.9636175.
Salhotra, G., Liu, I.-C.A., Dominguez-Kuhne, M., Sukhatme, G.S., Learning deformable object manipulation from expert demonstrations. IEEE Robot. Autom. Lett. 7:4 (2022), 8775–8782, 10.1109/LRA.2022.3187843.
Seita, D., Jamali, N., Laskey, M., Tanwani, A.K., Berenstein, R., Baskaran, P., Iba, S., Canny, J., Goldberg, K., Deep transfer learning of pick points on fabric for robot bed-making. The International Symposium of Robotics Research, 2019, Springer, 275–290.
Seita, D., Florence, P., Tompson, J., Coumans, E., Sindhwani, V., Goldberg, K., Zeng, A., Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks. 2021 IEEE International Conference on Robotics and Automation, ICRA, 2021, 4568–4575, 10.1109/ICRA48506.2021.9561391.
Zhu, J., Cherubini, A., Dune, C., Navarro-Alarcon, D., Alambeigi, F., Berenson, D., Ficuciello, F., Harada, K., Kober, J., Li, X., Pan, J., Yuan, W., Gienger, M., Challenges and outlook in robotic manipulation of deformable objects. IEEE Robot. Autom. Mag. 29:3 (2022), 67–77, 10.1109/MRA.2022.3147415.
Mahoney, A., Bross, J., Johnson, D., Deformable robot motion planning in a reduced-dimension configuration space. 2010 IEEE International Conference on Robotics and Automation, 2010, 5133–5138, 10.1109/ROBOT.2010.5509649.
Yoshida, E., Ayusawa, K., Ramirez-Alpizar, I.G., Harada, K., Duriez, C., Kheddar, A., Simulation-based optimal motion planning for deformable object. 2015 IEEE International Workshop on Advanced Robotics and Its Social Impacts, ARSO, 2015, 1–6, 10.1109/ARSO.2015.7428219.
Duenser, S., Bern, J.M., Poranne, R., Coros, S., Interactive robotic manipulation of elastic objects. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2018, 3476–3481, 10.1109/IROS.2018.8594291.
Koessler, A., Filella, N.R., Bouzgarrou, B., Lequièvre, L., Ramon, J.-A.C., An efficient approach to closed-loop shape control of deformable objects using finite element models. 2021 IEEE International Conference on Robotics and Automation, ICRA, 2021, 1637–1643, 10.1109/ICRA48506.2021.9560919.
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A.K., Jamali, N., Yamane, K., Iba, S., Goldberg, K., Visuospatial foresight for multi-step, multi-task fabric manipulation. 2020 arXiv preprint arXiv:2003.09044.
Yan, W., Vangipuram, A., Abbeel, P., Pinto, L., Learning predictive representations for deformable objects using contrastive estimation. Conference on Robot Learning, 2021, PMLR, 564–574.
Almaghout, K., Cherubini, A., Klimchik, A., Robotic co-manipulation of deformable linear objects for large deformation tasks. Robot. Auton. Syst., 2024, 104652.
Lagneau, R., Krupa, A., Marchal, M., Automatic shape control of deformable wires based on model-free visual servoing. IEEE Robot. Autom. Lett. 5:4 (2020), 5252–5259, 10.1109/LRA.2020.3007114.
Navarro-Alarcon, D., Yip, H.M., Wang, Z., Liu, Y.-H., Zhong, F., Zhang, T., Li, P., Automatic 3-D manipulation of soft objects by robotic arms with an adaptive deformation model. IEEE Trans. Robot. 32:2 (2016), 429–441, 10.1109/TRO.2016.2533639.
McConachie, D., Berenson, D., Bandit-based model selection for deformable object manipulation. Algorithmic Foundations of Robotics XII: Proceedings of the Twelfth Workshop on the Algorithmic Foundations of Robotics, 2020, Springer, 704–719.
Qi, J., Ran, G., Wang, B., Liu, J., Ma, W., Zhou, P., Navarro-Alarcon, D., Adaptive shape servoing of elastic rods using parameterized regression features and auto-tuning motion controls. IEEE Robot. Autom. Lett. 9:2 (2024), 1428–1435, 10.1109/LRA.2023.3346758.
Berenson, D., Manipulation of deformable objects without modeling and simulating deformation. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, 4525–4532, 10.1109/IROS.2013.6697007.
Ruan, M., McConachie, D., Berenson, D., Accounting for directional rigidity and constraints in control for manipulation of deformable objects without physical simulation. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2018, 512–519, 10.1109/IROS.2018.8594520.
Navarro-Alarcon, D., Liu, Y.-h., Romero, J.G., Li, P., On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments. Int. J. Robot. Res. 33:11 (2014), 1462–1480.
Zhou, P., Zhu, J., Huo, S., Navarro-Alarcon, D., LaSeSOM: A latent and semantic representation framework for soft object manipulation. IEEE Robot. Autom. Lett. 6:3 (2021), 5381–5388, 10.1109/LRA.2021.3074872.
Zhu, J., Navarro-Alarcon, D., Passama, R., Cherubini, A., Vision-based manipulation of deformable and rigid objects using subspace projections of 2D contours. Robot. Auton. Syst., 142, 2021, 103798.
Zhu, J., Navarro, B., Fraisse, P., Crosnier, A., Cherubini, A., Dual-arm robotic manipulation of flexible cables. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2018, 479–484, 10.1109/IROS.2018.8593780.
Bretl, T., McCarthy, Z., Quasi-static manipulation of a Kirchhoff elastic rod based on a geometric analysis of equilibrium configurations. Int. J. Robot. Res. 33:1 (2014), 48–68, 10.1177/0278364912473169.
Ying, C., Yamazaki, K., Motion generation for shaping deformable linear objects with contact avoidance using differentiable simulation. 2023 IEEE International Conference on Robotics and Biomimetics, ROBIO, 2023, IEEE, 1–8.
Lv, K., Yu, M., Pu, Y., Jiang, X., Huang, G., Li, X., Learning to estimate 3-D states of deformable linear objects from single-frame occluded point clouds. 2023 IEEE International Conference on Robotics and Automation, ICRA, 2023, 7119–7125, 10.1109/ICRA48891.2023.10160784.
Smolen, J., Patriciu, A., Deformation planning for robotic soft tissue manipulation. 2009 Second International Conferences on Advances in Computer-Human Interactions, 2009, 199–204, 10.1109/ACHI.2009.31.
Jin, S., Wang, C., Tomizuka, M., Robust deformation model approximation for robotic cable manipulation. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2019, 6586–6593, 10.1109/IROS40897.2019.8968157.
Shetab-Bushehri, M., Aranda, M., Mezouar, Y., Özgür, E., As-rigid-as-possible shape servoing. IEEE Robot. Autom. Lett. 7:2 (2022), 3898–3905, 10.1109/LRA.2022.3145960.
Aranda, M., Antonio Corrales Ramon, J., Mezouar, Y., Bartoli, A., Özgür, E., Monocular visual shape tracking and servoing for isometrically deforming objects. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2020, 7542–7549, 10.1109/IROS45743.2020.9341646.
McConachie, D., Dobson, A., Ruan, M., Berenson, D., Manipulating deformable objects by interleaving prediction, planning, and control. Int. J. Robot. Res. 39:8 (2020), 957–982.
Yu, M., Lv, K., Wang, C., Tomizuka, M., Li, X., A coarse-to-fine framework for dual-arm manipulation of deformable linear objects with whole-body obstacle avoidance. 2023 IEEE International Conference on Robotics and Automation, ICRA, 2023, 10153–10159, 10.1109/ICRA48891.2023.10160264.
Zhaole, S., Zhu, J., Fisher, R.B., Dexdlo: Learning goal-conditioned dexterous policy for dynamic manipulation of deformable linear objects. 2024 IEEE International Conference on Robotics and Automation, ICRA, 2024, IEEE, 16009–16015.
Tang, Y., Chu, X., Huang, J., Au, K.S., Learning-based MPC with safety filter for constrained deformable linear object manipulation. IEEE Robot. Autom. Lett., 2024.
Saghour, C., Célérier, M., Fraisse, P., Cherubini, A., Visual servoing for dual arm shaping of soft objects in 3D. 2023 IEEE-RAS 22nd International Conference on Humanoid Robots, Humanoids, 2023, IEEE, 1–7.
Guthikonda, V.R., Dani, A.P., Shape servoing of deformable objects using model estimation and barrier Lyapunov function. IEEE/ASME Trans. Mechatronics, 2024.
Fonkoua, M.O., Chaumette, F., Krupa, A., Deformation control of a 3D soft object using RGB-D visual servoing and FEM-based dynamic model. IEEE Robot. Autom. Lett. 9:8 (2024), 6943–6950.
Saghour, C., Navarro-Alarcón, D., Fraisse, P., Cherubini, A., Dual-arm shaping of soft objects in 3D based on visual servoing and online FEM simulations. Int. J. Robot. Res., 2024, 02783649241301076.
Gu, F., Sang, H., Zhou, Y., Ma, J., Jiang, R., Wang, Z., He, B., Learning graph dynamics with interaction effects propagation for deformable linear objects shape control. IEEE Trans. Autom. Sci. Eng., 2025.
Caporali, A., Kicki, P., Galassi, K., Zanella, R., Walas, K., Palli, G., Deformable linear objects manipulation with online model parameters estimation. IEEE Robot. Autom. Lett. 9:3 (2024), 2598–2605.
Huang, Z., Lin, Y., Yang, F., Berenson, D., Subgoal diffuser: Coarse-to-fine subgoal generation to guide model predictive control for robot manipulation. 2024 IEEE International Conference on Robotics and Automation, ICRA, 2024, IEEE, 16489–16495.
Zhang, Y., Liu, F., Liang, X., Yip, M., Achieving autonomous cloth manipulation with optimal control via differentiable physics-aware regularization and safety constraints. 2024 IEEE International Conference on Robotics and Automation, ICRA, 2024, IEEE, 9931–9938.
Cao, B., Zang, X., Zhang, X., Chen, Z., Li, S., Zhao, J., Shape control of elastic deformable linear objects for robotic cable assembly. Adv. Intell. Syst., 6(7), 2024, 2300835.
Daniel, M., Magassouba, A., Aranda, M., Lequièvre, L., Corrales Ramón, J.A., Iglesias Rodriguez, R., Mezouar, Y., Multi actor-critic DDPG for robot action space decomposition: A framework to control large 3D deformation of soft linear objects. IEEE Robot. Autom. Lett. 9:2 (2024), 1318–1325, 10.1109/LRA.2023.3342672.
Cuiral-Zueco, I., López-Nicolás, G., Multiscale procrustes-based 3-D shape control. IEEE/ASME Trans. Mechatronics 29:3 (2023), 1738–1748.
Mcconachie, D., Berenson, D., Estimating model utility for deformable object manipulation using multiarmed bandit methods. IEEE Trans. Autom. Sci. Eng. 15:3 (2018), 967–979, 10.1109/TASE.2018.2822669.
O'neill, B., Elementary Differential Geometry. 2006, Elsevier.
Bishop, R.L., There is more than one way to frame a curve. Am. Math. Mon. 82:3 (1975), 246–251.
Mishani, I., Sintov, A., Real-time non-visual shape estimation and robotic dual-arm manipulation control of an elastic wire. IEEE Robot. Autom. Lett. 7:1 (2022), 422–429, 10.1109/LRA.2021.3128707.
J. Spillmann, M. Teschner, CoRdE: Cosserat rod elements for the dynamic simulation of one-dimensional elastic objects, in: Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2007, pp. 63–72.
M. Grégoire, E. Schömer, Interactive simulation of one-dimensional flexible parts, in: Proceedings of the 2006 ACM Symposium on Solid and Physical Modeling, 2006, pp. 95–103.
Yang, Y., Stork, J.A., Stoyanov, T., Online model learning for shape control of deformable linear objects. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2022, 4056–4062, 10.1109/IROS47612.2022.9981080.
Sanchez, J., Mohy El Dine, K., Corrales, J.A., Bouzgarrou, B.-C., Mezouar, Y., Blind manipulation of deformable objects based on force sensing and finite element modeling. Front. Robot. AI, 7, 2020, 73.
Barfoot, T.D., State Estimation for Robotics. 2024, Cambridge University Press.
Hutchinson, S., Chaumette, F., Visual servo control, part i: Basic approaches. IEEE Robot. Autom. Mag. 13:4 (2006), 82–90.
Chu, X., Lo, C.H., Proietti, T., Walsh, C.J., Au, K.W.S., Opposite treatment on null space: a unified control framework for a class of underactuated robotic systems with null space avoidance. IEEE Trans. Control Syst. Technol. 31:1 (2022), 193–207.
Yu, M., Zhong, H., Li, X., Shape control of deformable linear objects with offline and online learning of local linear deformation models. 2022 International Conference on Robotics and Automation, ICRA, 2022, IEEE, 1337–1343.
Blanes, S., Casas, F., On the convergence and optimization of the Baker–Campbell–Hausdorff formula. Linear Algebra Appl. 378 (2004), 135–158.
Casas, F., Murua, A., An efficient algorithm for computing the Baker–Campbell–Hausdorff series and some of its applications. J. Math. Phys., 50(3), 2009.
Qi, J., Ma, G., Zhu, J., Zhou, P., Lyu, Y., Zhang, H., Navarro-Alarcon, D., Contour moments based manipulation of composite rigid-deformable objects with finite time model estimation and shape/position control. IEEE/ASME Trans. Mechatronics 27:5 (2022), 2985–2996, 10.1109/TMECH.2021.3126383.
Wakamatsu, H., Hirai, S., Iwata, K., Modeling of linear objects considering bend, twist, and extensional deformations. Proceedings of 1995 IEEE International Conference on Robotics and Automation, vol. 1, 1995, 433–438, 10.1109/ROBOT.1995.525322.
Coumans, E., Bai, Y., PyBullet, a Python module for physics simulation for games, robotics and machine learning. 2016 http://pybullet.org.
Todorov, E., Erez, T., Tassa, Y., MuJoCo: A physics engine for model-based control. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, 5026–5033, 10.1109/IROS.2012.6386109.
Ebrahimi, M., Butscher, A., Cheong, H., A low order, torsion deformable spatial beam element based on the absolute nodal coordinate formulation and bishop frame. Multibody Syst. Dyn. 51:3 (2021), 247–278.
Bergou, M., Wardetzky, M., Robinson, S., Audoly, B., Grinspun, E., Discrete elastic rods. ACM SIGGRAPH 2008 Papers, 2008, 1–12.
Petit, A., Lippiello, V., Siciliano, B., Real-time tracking of 3D elastic objects with an RGB-D sensor. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2015, 3914–3921, 10.1109/IROS.2015.7353928.
Liu, F., Li, Z., Han, Y., Lu, J., Richter, F., Yip, M.C., Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy. 2021 IEEE International Conference on Robotics and Automation, ICRA, 2021, 12328–12334, 10.1109/ICRA48506.2021.9561177.
Sengupta, A., Krupa, A., Marchand, E., Visual tracking of deforming objects using physics-based models. 2021 IEEE International Conference on Robotics and Automation, ICRA, 2021, 14178–14184, 10.1109/ICRA48506.2021.9561401.
Shetab-Bushehri, M., Aranda, M., Mezouar, Y., Özgür, E., Lattice-based shape tracking and servoing of elastic objects. IEEE Trans. Robot. 40 (2024), 364–381, 10.1109/TRO.2023.3331596.
Yang, Y., Stork, J.A., Stoyanov, T., Learning differentiable dynamics models for shape control of deformable linear objects. Robot. Auton. Syst., 158, 2022, 104258.