Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model visualizer should know about mimic tags (coupled joints) #18917

Open
jwnimmer-tri opened this issue Mar 1, 2023 · 5 comments · May be fixed by #22496
Open

Model visualizer should know about mimic tags (coupled joints) #18917

jwnimmer-tri opened this issue Mar 1, 2023 · 5 comments · May be fixed by #22496
Assignees
Labels
component: geometry illustration What and how geometry gets communicated to external visualizers priority: low type: feature request

Comments

@jwnimmer-tri
Copy link
Collaborator

See #17663 and #18728 for background.

When we are displaying a model with coupled joints, we should be enforcing the constraint during visualization.

@jwnimmer-tri jwnimmer-tri added type: feature request priority: low component: geometry illustration What and how geometry gets communicated to external visualizers labels Mar 1, 2023
@jwnimmer-tri jwnimmer-tri self-assigned this Mar 1, 2023
@RussTedrake
Copy link
Contributor

The limitation is more general than model visualizer. For instance, the inverse dynamics controller currently also falls down if a model is specified with constraints (e.g. mimic elements in urdf). This limitation popped up on our MIT project with Movo, which uses mimic tags.

I think the right solution is to use a general machinery like SystemConstraints rather than some narrow solution for model visualizer. That would suggest we might want to generalize the title of this issue and keep them in one place? I'm also ok if we want a separate issue to track.

Also, if 2 of our university friends are both running into this at the very beginning of their projects, perhaps it deserves a higher priority?

@jwnimmer-tri
Copy link
Collaborator Author

I'd like to have a separate issue for the inverse dynamics controller. Could you please file one explaining what's happening, with a high-level outline of how to reproduce the problem, and what you'd prefer to happen vs what actually happens?

I think one thing we lack is any robot model anywhere in Drake that uses mimic joints. We should work to have at least one in-tree, so that the all of us (including the dynamics team) can more easily understand and reproduce problems like this. We can collaborate across the two (or more) mimic-related issues to get the model added -- whichever one gets there first can add the file(s). (Possibly "no example of mimic joints" is documentation-like defect all on its own, and we should open a new issue for it, and coordinate there.)

I suspect when we throw it at other subsystems within Drake it will likewise fail to perform as desired.

My reasoning for using separate issues is that different people will probably work on them, with different priorities. To my understanding at least, the lack of effective mimic joints in model_visualizer is embarrassing (and maybe should have a warning?) but is not a major roadblock to collaboration. I expect that things like IDC failures are much more of a blocker, so we should prioritize them more urgently.

@rpoyner-tri
Copy link
Contributor

FWIW, I don't think this ticket has any strong connection to "mimic tags" -- won't the visualizers be reading constraints directly from the model(s) in Mbp?

@jwnimmer-tri
Copy link
Collaborator Author

My view of the victory condition is that a model loaded from XML with mimic tags visualizes correctly (i.e., by obeying the mimic constraint so that the robot always displays a valid posture).

Yes, the visualizer code would get that information from MbP. However, the victory condition is not that every possible kind of SAP constraint visualizes correctly -- only that this one specific thing does so. In that light, if we were to rename this issue to something like "visualize MbP's SAP constraints", that would would blur the goal.

@MengMengMengH
Copy link

The limitation is more general than model visualizer. For instance, the inverse dynamics controller currently also falls down if a model is specified with constraints (e.g. elements in urdf). This limitation popped up on our MIT project with Movo, which uses tags.mimic``mimic

I think the right solution is to use a general machinery like rather than some narrow solution for model visualizer. That would suggest we might want to generalize the title of this issue and keep them in one place? I'm also ok if we want a separate issue to track.SystemConstraints

Also, if 2 of our university friends are both running into this at the very beginning of their projects, perhaps it deserves a higher priority?

I met the same problem here.When I want to use in my simulation,I was told that it was a continous system and would ignore the drake:mimic joint.
I wonder if there are any solutions. I would be glad to receive your reply.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 18, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 18, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 18, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 19, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 19, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 19, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards RobotLocomotion#18917.
RussTedrake added a commit that referenced this issue Dec 19, 2024
This method creates a mapping from the constraints used in
MultibodyPlant/SAP to the constraints used in MathematicalProgram.

Towards #18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 31, 2024
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Dec 31, 2024
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 6, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 7, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 8, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 10, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 10, 2025
and by implication also to ModelVisualizer

Resolves RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 10, 2025
and by implication also to ModelVisualizer

Resolves RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 12, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards RobotLocomotion#18917.
sammy-tri pushed a commit that referenced this issue Jan 13, 2025
Also moves the joint limits / joint locking logic from the IK
constructor to AddMultibodyPlantConstraints.

Towards #18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Deprecates JointSliders::SetPositions(q), which has been replaced with
JointSliders::SetPositions(context, q).

Resolves RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Deprecates JointSliders::SetPositions(q), which has been replaced with
JointSliders::SetPositions(context, q).

Resolves RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Deprecates JointSliders::SetPositions(q), which has been replaced with
JointSliders::SetPositions(context, q).

Resolves RobotLocomotion#18917.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Since ModelVisualizer uses JointSliders, now ModelVisualizer also
satisfies the multibody constraints.

Resolves RobotLocomotion#18917.

Deprecates `JointSliders::SetPositions(q)`, which has been replaced with
`JointSliders::SetPositions(context, q)`.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Since ModelVisualizer uses JointSliders, now ModelVisualizer also
satisfies the multibody constraints.

Resolves RobotLocomotion#18917.

Deprecates `JointSliders::SetPositions(q)`, which has been replaced with
`JointSliders::SetPositions(context, q)`.
RussTedrake added a commit to RussTedrake/drake that referenced this issue Jan 19, 2025
Updates JointSliders to have discrete state (previously it was
stateless). Additionally, if the new ResolveConstraints logic is
active, we publish the (rounded) resolved constraints back to Meshcat,
so that the other sliders are updated.

Since ModelVisualizer uses JointSliders, now ModelVisualizer also
satisfies the multibody constraints.

Resolves RobotLocomotion#18917.

Deprecates `JointSliders::SetPositions(q)`, which has been replaced with
`JointSliders::SetPositions(context, q)`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: geometry illustration What and how geometry gets communicated to external visualizers priority: low type: feature request
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

4 participants