[Question] Simulation calibration for Digital Twin

[Simoneutili]

Hi there!
I am currently developing a Digital Twin application for the TiAGO manipulator robot, aiming to align the behavior of the robot simulated in Isaac Sim with that of the physical robot for a simple velocity control task involving the arm DOFs.

To achieve this, I have implemented a reinforcement learning direct environment in Isaac Lab. The goal is to estimate simulation parameters that best replicate the real robot’s joint responses when applying identical reference velocities to each of the 7 arm DOFs. (my reward function is primarily designed to minimize the error between the simulated and real robot trajectories).
Currently, I am using implicit actuators, and my RL policy directly estimates damping and friction parameters for each DOF, while keeping stiffness set to zero. However, even considering the best outcomes I got, the policy deployment yields imprecise and erratic results (especially when changing the reference amplitude), particularly for DOFs whose responses are poorly aligned with the velocity reference.

In the above plots joint_configuration are Isaac Lab velocities, joint_states are the Gazebo velocities (which substitutes the real robot for these tests, also because i can emphasize the mismatch between the joint responses and the velocity reference by handling the URDF params.)
The problem is that I am not sure about which parameters are most critical to optimize for achieving accurate velocity matching. Should I configure a custom actuator model or switch to an alternative actuator type instead of the implicit actuator? I also observed during testing that variations in the physics simulation time-step affect the joint velocity responses significantly, even when the parameters remain unchanged. I have conducted several ablation studies, including modifications to the PhysX configuration using the PhysxCfg class. Despite this, the black-box nature of the joint dynamics model makes it challenging to identify specific areas of focus for improvement. Thanks in advance for the insights ;)