Unified annotation versioning #7917

fm3 · 2024-07-08T10:55:54Z

Summary

Annotation Versioning no longer works per layer, now a single version number is used for the whole annotation
This makes EditableMappings/Proofreading annotations revertable
EditableMappings are now stored by the tracingId (same as their volume layer)
TokenContext was introduced, an implicit context variable to pass around user tokens in tracingstore + datastore more prettily
When restoring older versions of annotations, the layer set may change now (if the layer had been added since then)

Steps to test:

WK Testing

Note: Since the FossilDB is opened with a new set of column families, switching to and from this branch requires a fresh FossilDB data dir. I use rm -rf fossildb/data; git co fossildb/data/.gitignore

Migration Testing

create annotations on master, switch branch, run migration, load again, annotate again
should work for different layer combinations
should work with pending updates (no page refresh after skeleton changes on master)
should work with proofreading annotations

TODOs

Backend

Frontend

{
    "myTracingId": {
        "branchpoints:" 3
        "nodeCount": 5
    },
    "myVolumeTracingId1": {
        "segmentCount": 5
    },
    "myVolumeTracingId2": {
        "segmentCount": 9
    }
}

Migration

Issues:

fixes Unified Annotation Versioning #7839
fixes Pass UserToken implicitly around tracingstore + datastore #8029
fixes History for Annotation Description #7703
fixes Fix or Remove Morton Order in Volume Data Fossil Keys #3546
fixes Merge volume/skeleton restore version view #5833
fixes Annotation Download: specify tracing version per layer #5925
contributes to Avoid double Update when editing Hybrid Annotations #5169
contributes to Task system for editable mapping/proofreading annotations #6854
contributes to Collaboration by forking/merging annotations #5819

Updated changelog
Updated migration guide if applicable
Removed dev-only changes like prints and application.conf edits
Considered common edge cases
Needs datastore update after deployment

fm3 · 2024-08-28T11:58:43Z

@philippotto A first rudimentary version seems to work: I can create an annotation on l4_sample, (note: volume buckets won’t load), take the annotation id and skeleton tracing id, and insert them in here (both in uri and in request body):

await fetch("http://localhost:9000/tracings/annotation/66cf0fc1d10000f6ae2f30bd/update?token=secretSampleUserToken", {
    "credentials": "include",
    "headers": {
        "Accept": "application/json",
        "Accept-Language": "en-US,en;q=0.5",
        "content-type": "application/json",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "same-origin",
        "Priority": "u=0",
        "Pragma": "no-cache",
        "Cache-Control": "no-cache"
    },
    "referrer": "http://localhost:9000/annotations/66cedcb2e000008402396d82",
    "body": "[{\"version\":1,\"transactionId\":\"lfh7mxycvn\",\"transactionGroupCount\":1,\"transactionGroupIndex\":0,\"timestamp\":1724832956301,\"authorId\":\"66c46740d10000d1005405b6\",\"actions\":[{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"createTree\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"id\":1,\"color\":[0.6784313725490196,0.1411764705882353,0.050980392156862744],\"name\":\"explorative_2024-08-28_Sample_User_001\",\"timestamp\":1724832956293,\"comments\":[],\"branchPoints\":[],\"groupId\":null,\"isVisible\":true,\"type\":\"DEFAULT\",\"edgesAreVisible\":true}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"createNode\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"additionalCoordinates\":[],\"radius\":1,\"rotation\":[0,0,0],\"viewport\":0,\"resolution\":0,\"id\":1,\"timestamp\":1724832956293,\"bitDepth\":8,\"interpolation\":false,\"position\":[3530,3603,1024],\"treeId\":1}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateSkeletonTracing\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"activeNode\":1,\"editPosition\":[3584,3584,1024],\"editPositionAdditionalCoordinates\":[],\"editRotation\":[0,0,0],\"zoomLevel\":1}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateTdCameraSkeleton\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\"}}],\"stats\":{\"treeCount\":1,\"nodeCount\":1,\"edgeCount\":0,\"branchPointCount\":0},\"info\":\"[\\\"UPDATE_LAYER_SETTING\\\",\\\"SET_HISTOGRAM_DATA_FOR_LAYER\\\",\\\"UPDATE_MESH_FILE_LIST\\\",\\\"UPDATE_CURRENT_MESH_FILE\\\",\\\"SET_INPUT_CATCHER_RECTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"SET_STORED_LAYOUTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING * 2\\\",\\\"SET_TOOL\\\",\\\"CREATE_NODE\\\"]\"},{\"version\":2,\"transactionId\":\"bmwafojnac\",\"transactionGroupCount\":1,\"transactionGroupIndex\":0,\"timestamp\":1724832956421,\"authorId\":\"66c46740d10000d1005405b6\",\"actions\":[{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateSkeletonTracing\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"activeNode\":1,\"editPosition\":[3552,3595,1024],\"editPositionAdditionalCoordinates\":[],\"editRotation\":[0,0,0],\"zoomLevel\":1}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateTdCameraSkeleton\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\"}}],\"stats\":{\"treeCount\":1,\"nodeCount\":1,\"edgeCount\":0,\"branchPointCount\":0},\"info\":\"[\\\"UPDATE_MESH_FILE_LIST\\\",\\\"UPDATE_CURRENT_MESH_FILE\\\",\\\"SET_INPUT_CATCHER_RECTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"SET_STORED_LAYOUTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING * 2\\\",\\\"SET_TOOL\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\"]\"},{\"version\":3,\"transactionId\":\"anjed8c2bb\",\"transactionGroupCount\":1,\"transactionGroupIndex\":0,\"timestamp\":1724832956939,\"authorId\":\"66c46740d10000d1005405b6\",\"actions\":[{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"createNode\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"additionalCoordinates\":[],\"radius\":1,\"rotation\":[0,0,0],\"viewport\":0,\"resolution\":0,\"id\":2,\"timestamp\":1724832956939,\"bitDepth\":8,\"interpolation\":false,\"position\":[3514,3571,1024],\"treeId\":1}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"createEdge\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"treeId\":1,\"source\":1,\"target\":2}},{\"name\":\"updateSkeletonTracing\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"activeNode\":2,\"editPosition\":[3530,3603,1024],\"editPositionAdditionalCoordinates\":[],\"editRotation\":[0,0,0],\"zoomLevel\":1}}],\"stats\":{\"treeCount\":1,\"nodeCount\":2,\"edgeCount\":1,\"branchPointCount\":0},\"info\":\"[\\\"UPDATE_CURRENT_MESH_FILE\\\",\\\"SET_INPUT_CATCHER_RECTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"SET_STORED_LAYOUTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING * 2\\\",\\\"SET_TOOL\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\",\\\"CREATE_NODE\\\"]\"},{\"version\":4,\"transactionId\":\"mgb0fqyu0c\",\"transactionGroupCount\":1,\"transactionGroupIndex\":0,\"timestamp\":1724832957039,\"authorId\":\"66c46740d10000d1005405b6\",\"actions\":[{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateSkeletonTracing\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"activeNode\":2,\"editPosition\":[3522,3587,1024],\"editPositionAdditionalCoordinates\":[],\"editRotation\":[0,0,0],\"zoomLevel\":1}},{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateTdCameraSkeleton\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\"}}],\"stats\":{\"treeCount\":1,\"nodeCount\":2,\"edgeCount\":1,\"branchPointCount\":0},\"info\":\"[\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"SET_STORED_LAYOUTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING * 2\\\",\\\"SET_TOOL\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\"]\"},{\"version\":5,\"transactionId\":\"9uu9b6j1qf\",\"transactionGroupCount\":1,\"transactionGroupIndex\":0,\"timestamp\":1724832957149,\"authorId\":\"66c46740d10000d1005405b6\",\"actions\":[{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"name\":\"updateSkeletonTracing\",\"value\":{\"actionTracingId\":\"3c2710c5-3b68-4bfa-ba7c-86553df9d8bf\",\"activeNode\":2,\"editPosition\":[3514,3571,1024],\"editPositionAdditionalCoordinates\":[],\"editRotation\":[0,0,0],\"zoomLevel\":1}}],\"stats\":null,\"info\":\"[\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"SET_STORED_LAYOUTS\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING * 2\\\",\\\"SET_TOOL\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\",\\\"CREATE_NODE\\\",\\\"SET_TD_CAMERA_WITHOUT_TIME_TRACKING\\\",\\\"MOVE_TD_VIEW_BY_VECTOR_WITHOUT_TIME_TRACKING\\\"]\"}]",
    "method": "POST",
    "mode": "cors"
});

Then reload and see two skeleton nodes. Adding/Removing layer, volume annotation, proofreading is all still broken. So are the version assertions. But this can be used as a starting point to adapt the frontend:

include actionTracingIds in update actions
send updates to new endpoint
some update action names have changed (need to distinguish btw skeleton + volume). This may change again, though…
~~adapt tracingstore js client to include annotation id in everything (note: some backend-internal routes still have to be adapted too)~~ I ended up looking up the annotation id by the tracingid instead.

Note: when switching to and from this branch, I clear the fossildb content with rm -rf fossildb/data; git checkout fossildb/data/.gitignore, otherwise the column family change is rejected

fm3 · 2024-10-15T10:19:46Z

The next part of the frontend could be added now: The Restore Older Version flow. The version list view should no longer feature tabs per layer, but instead only one global GET tracings/annotation/:annotationId/updateActionLog route exists with a single list. (I currently adapted the frotnend so that each tab uses this new route)

Also, the revertToVersion update action is no longer layer specific (I guess this may already be covered by philipps changes).

It is required that the revertToVersion update action is the only action in its update group (so that it gets its own version number). Not sure if the frontend already does that.

Since Philipp is out of office, maybe @MichaelBuessemeyer could have a look at this after #6613 is done?

Note that this branch opens fossildb with different column families. I run rm -rf fossildb/data; git co fossildb/data/.gitignore when switching to and from this branch to avoid errors. If you need your fossildb data, you can of course create a backup first.

MichaelBuessemeyer · 2024-10-15T13:47:22Z

Since Philipp is out of office, maybe @MichaelBuessemeyer could have a look at this after #6613 is done?

Sure :)

fm3 · 2024-10-23T09:14:46Z

@MichaelBuessemeyer the next batch of frontend todos has arrived 😇 I have not turned all of them into checkboxes in the description, feel free to do that if you prefer.

Some update actions are now duplicated, because I have to distinguish between SkeletonUpdateActions and VolumeUpdateActions, even if they are basically identical:
- updateUserBoundingBoxes → updateUserBoundingBoxesInSkeletonTracing, updateUserBoundingBoxesInVolumeTracing
- updateUserBoundingBoxVisibility → updateUserBoundingBoxVisibilityInSkeletonTracing, updateUserBoundingBoxVisibilityInVolumeTracing
- updateTracing → updateVolumeTracing, updateSkeletonTracing
no renaming, but note: these is an AnnotationUpdateAction (no actionTracingId)
- updateTdCamera
- revertToVersion
New update actions that were previously separate routes, either to wk or to tracingstore:
- addLayerToAnnotation
  - layerParameters: AnnotationLayerParameters (same as before: typ, name, magRestrictions, etc)
- deleteLayerFromAnnotation
  - tracingId: String
  - layerName: String
  - type: AnnotationLayerType (enum, "Skeleton", "Volume")
- updateLayerMetadata
  - tracingId: String
  - layerName: String
- updateMetadataOfAnnotation
  - name: Option[String]
  - description: Option[String]
The frontend should fetch the new annotation proto object from GET {tracingstore_uri}/tracings/annotation/:annotationId to get correct layer set, name, description for requested version (postgres only knows latest for the dashboard). This is especially relevant during version restore view (or if we eventually add permalinks to open previous versions).

MichaelBuessemeyer · 2024-10-25T12:23:59Z

Regarding the first point:

Some update actions are now duplicated, because I have to distinguish between SkeletonUpdateActions and VolumeUpdateActions, even if they are basically identical:

It seems like @philippotto already implemented this 🥳

But regarding:

updateUserBoundingBoxVisibility → updateUserBoundingBoxVisibilityInSkeletonTracing, updateUserBoundingBoxVisibilityInVolumeTracing

The frontend does not know this update action. Maybe it was deprecated and is no longer sent by the frontend? At least cannot find it in the frontend code of this branch and the master as well 🥴. Maybe this is an old update action that is still supported by the backend for compatibility reasons? But on the other side, the version restore view does not have such an entry for legacy purposes.
A quick check of the backend master code shows, that updateUserBoundingBoxVisibility indeed exists but I don't see any place in the frontend that would send such an update action.
Moreover, I just checked what happens if the visibility of one of the user bboxes is changed. The frontend sends an updateAction in form of a "updateUserBoundingBoxes".

=> Maybe the updateUserBoundingBoxVisibility updateAction can be removed from the backend code 🤔?

coderabbitai · 2024-10-25T12:45:58Z

Important

Review skipped

More than 25% of the files skipped due to max files limit. The review is being skipped to prevent a low-quality review.

116 files out of 230 files are above the max files limit of 75. Please upgrade to Pro plan to get higher limits.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

MichaelBuessemeyer · 2024-10-28T10:20:37Z

@fm3 While implementing the new update actions which replaced some routes I came across the following questions:

deleteLayerFromAnnotation
tracingId: String
layerName: String
type: AnnotationLayerType (enum, "Skeleton", "Volume")

Why is the layerName and type needed for this annotation action? The tracingId should be unique enough to identify the layer, isn't it?

Answer: This is only used for more detailed information in the version restore view.

For update action

updateMetadataOfAnnotation
name: Option[String]
description: Option[String]

the previous equivalent route was able to update the following annotation properties:

export type EditableAnnotation = {
  name: string;
  description: string;
  visibility: APIAnnotationVisibility;
  tags: Array<string>;
  viewConfiguration?: AnnotationViewConfiguration;
};

export function editAnnotation(
  annotationId: string,
  annotationType: APIAnnotationType,
  data: Partial<EditableAnnotation>,
): Promise<void> {
  return Request.sendJSONReceiveJSON(`/api/annotations/${annotationType}/${annotationId}/edit`, {
    data,
    method: "PATCH",
  });
}

Meaning the route was not only able to update the description and name but also the tags, visibility and viewConfig. Could this also be added to the updateMetadataOfAnnotation update action? If not, another update action is needed to update the missing three properties. At least that's how I see / understand it right now :)

Answer: The name and description will be versioned as update actions but the editAnnotation route will still exist and take updates regarding tags, visibility, and viewConfig.

MichaelBuessemeyer · 2024-10-28T10:47:33Z

Some more questions :D

The frontend should fetch the new annotation proto object from GET {tracingstore_uri}/tracings/annotation/:annotationId to get correct layer set, name, description for requested version (postgres only knows latest for the dashboard). This is especially relevant during version restore view (or if we eventually add permalinks to open previous versions).

What about sandbox annotations? The current route for this is: /api/datasets/${datasetId.owningOrganization}/${datasetId.name}/sandbox/${tracingType}

Answer: Keep it. & Sandbox is only supported for skeleton annotations.

Moreover, the old annotation info fetching route (/api/annotations/${annotationId}/info?timestamp=${Date.now()};`) is still required as the frontend needs to identify the tracing store URL for the annotation.

TODO [ ]: Always only use the info from the TS route. Maybe the core backend has layers that were deleted already and so on.

TODO: micha [ ] Test me with version restore

Moreover, is it possible to split up an annotation (its layers) across multiple tracing stores? If that's the case, the frontend needs to call all tracing stores available in the maybe outdated annotation info object from the backend and then merge this information?

There can be only one TS at a time for a single WK instance. => So this is not possible

fm3 · 2024-10-29T10:23:46Z

note to self:
note to self, open questions:

remove downsample feature? → yes
duplicate history?
- duplicate update actions (needed for merging editable mappings)
- duplicate v0 in addition to current version?
- Also in task assignment case? How to mark earliest accessible version? We don’t want users to revert too far, right?
- Should we also copy intermediate materialized versions? Or just 0 and current?
- What about intermediate bucket versions? They are not in the updates
- perf: duplicate api for fossilDB?
resetToBase as update action?
during apply, what to load in memory? is it all or nothing? → load all, as usually we request everything
during apply, what to flush to fossil? is it all or nothing? → flush all layers that were changed (updates for the layer exist)
avoid duplicate update applying (async cache?)

webknossos-datastore/proto/Annotation.proto

MichaelBuessemeyer · 2024-11-11T17:47:45Z

Making an annotation's mapping editable (starting proofreading actions should work again.

Although, I'd like to ask why the updateMappingName actions now makes a mapping editable. To me this sounds kinda like magic because I would not have expected an action called updateMappingName to make a mapping editable. Can we maybe make this more obvious? E.g. an extra action or so? or at least a parameter / property of the updateMappingName like makeMappingEditable or isMappingEditable?

fm3 · 2024-11-11T18:50:16Z

or at least a parameter / property of the updateMappingName like makeMappingEditable or isMappingEditable?

Yes, such an extra property is required! It’s called isEditable. For this particular context, both isEditable and isLocked should be set to true. https://github.com/scalableminds/webknossos/blob/master/webknossos-tracingstore/app/com/scalableminds/webknossos/tracingstore/tracings/volume/VolumeUpdateActions.scala#L359

Thanks for having a look at this!

MichaelBuessemeyer · 2024-11-12T08:24:44Z

Yes, such an extra property is required! It’s called isEditable. For this particular context, both isEditable and isLocked should be set to true. https://github.com/scalableminds/webknossos/blob/master/webknossos-tracingstore/app/com/scalableminds/webknossos/tracingstore/tracings/volume/VolumeUpdateActions.scala#L359

Thanks for having a look at this!

Oh right, thanks. I got confused between updateActions and dispatchActions here, because the dispatch action does not have fields like isLocked and isEditable. But it should work now ™️

…Tracings

fm3 · 2024-11-18T14:43:34Z

app/controllers/AnnotationController.scala

@@ -91,7 +66,7 @@ class AnnotationController @Inject()(
           // For Task and Explorational annotations, id is an annotation id. For CompoundTask, id is a task id. For CompoundProject, id is a project id. For CompoundTaskType, id is a task type id
           id: String,
           // Timestamp in milliseconds (time at which the request is sent)
-           timestamp: Long): Action[AnyContent] = sil.UserAwareAction.async { implicit request =>
+           timestamp: Option[Long]): Action[AnyContent] = sil.UserAwareAction.async { implicit request =>


unrelated to unified annotation versioning, just wanted to do manual get requests. I think this is a fair change. This is used only for user time tracking.

fm3 · 2024-11-18T15:34:52Z

@MichaelBuessemeyer the backend should be ready for review. I’ll continue by building the FossilDB migration, this will occupy me for some time. I’d suggest to first finish the frontend here as far as possible (maybe @philippotto can help with that once he’s back) and then you could start the backend review here. I’ll happily give an introduction in person.

…nting

MichaelBuessemeyer

@fm3 I think the TimeController also needs to be updated a little to serve the new format of the AnnotationStats for each time tracking entry (instead of an array of stats).
I did not test this so it might be working already, but to me the code in the Controller reads like it needs to be updated. Moreover, the pr does not have any changes for TimeController.scala also indicating a potential open TODO

fm3 · 2024-11-19T10:52:22Z

You’re right, I’ll add that! I didn’t even remember that the time tracking api includes the stats 😅

…ion numbers

MichaelBuessemeyer · 2025-01-20T16:07:24Z

The newest changes look also good to me :) Thanks for incorporating them. I also did some slight unstructured testing (tried merging annotations, compound tasks & compound projects)

MichaelBuessemeyer · 2025-01-21T09:24:35Z

I did some more testing (went trough WK Testing steps and did want I did not yesterday):

While testing I noticed that a single proofreading action results in 3 version entries. Is it maybe possible to reduce this to one version entry?

Sorry but I found another little frontend bug:

https://github.com/user-attachments/assets/2e4a65fa-ce8b-4784-8720-90bc82a427e6 Maybe the reason for this bug is that I selected the newest version and then closed the view 🤔?

…rsion)

…lowUpdate value in useWillUnmount

philippotto · 2025-01-21T15:56:49Z

While testing I noticed that a single proofreading action results in 3 version entries. Is it maybe possible to reduce this to one version entry?

we can probably slim this down, but let's do it in a follow up. I created #8345

https://github.com/user-attachments/assets/2e4a65fa-ce8b-4784-8720-90bc82a427e6 Maybe the reason for this bug is that I selected the newest version and then closed the view 🤔?

thanks for spotting this! I fixed this now. please have a look at my latest commit. I tried to clean up how the allowUpdate property is restored. the original problem was that useWillUnmount didn't have a dependency on the initialAllowUpdate property (and that could be set to false because of the dummy annotation in the default state).

…nds/webknossos into unified-annotation-versioning

MichaelBuessemeyer · 2025-01-22T06:25:14Z

thanks for spotting this! I fixed this now. please have a look at my latest commit. I tried to clean up how the allowUpdate property is restored. the original problem was that useWillUnmount didn't have a dependency on the initialAllowUpdate property (and that could be set to false because of the dummy annotation in the default state).

Awesome. Thanks for fixing this. I can no longer trigger the buggy behaviour 🎉

MichaelBuessemeyer · 2025-01-22T10:42:53Z

From my side this PR is ready to go ✔️

MichaelBuessemeyer

Thanks a lot for all the work you put into this @fm3 @philippotto 🙏

I could not find anything anymore and the code looks good to me.

Maybe a little final quick check from @normanrz regarding the migration guide changes would be useful before merging

fm3 · 2025-01-23T10:27:00Z

@philippotto could you have nother look at the new merge conflicts? For VolumeTracingLayer.scala it should be safe to just use “ours”.

normanrz

Migration looks good 👍

tools/hosting/docker-compose.yml

…fied-annotation-versioning

philippotto · 2025-01-23T12:18:51Z

@philippotto could you have nother look at the new merge conflicts? For VolumeTracingLayer.scala it should be safe to just use “ours”.

done

fm3 self-assigned this Jul 8, 2024

fm3 changed the title ~~Unified annotation versioning~~ WIP: Unified annotation versioning Jul 8, 2024

MichaelBuessemeyer self-assigned this Oct 15, 2024

MichaelBuessemeyer mentioned this pull request Oct 21, 2024

Unified annotation restore #8136

Merged

MichaelBuessemeyer reviewed Oct 29, 2024

View reviewed changes

webknossos-datastore/proto/Annotation.proto Outdated Show resolved Hide resolved

fm3 added 3 commits November 18, 2024 15:10

Merge branch 'master' into unified-annotation-versioning

42b9bf2

Fix unit tests, add two-level caching for materialized annotationWith…

8549b21

…Tracings

unify logging

ac5bd22

fm3 commented Nov 18, 2024

View reviewed changes

cleanup

e7a464f

fm3 and others added 2 commits November 18, 2024 19:58

init fossildb migration

d6a8e8e

update frontend to newest annotation stats schema and fix typing & li…

14d44c8

…nting

MichaelBuessemeyer reviewed Nov 19, 2024

View reviewed changes

fm3 added 3 commits November 19, 2024 12:06

include annotation stats per layer in time tracking api response json

cc8382b

continue if a layer’s stats couldn’t be updated

aa02568

add some function stubs to migration

03de53e

fm3 added 6 commits January 20, 2025 13:53

fix ordering of extra updates

3c594f5

append rather than prepend, pendingAnnotationupdates have higher vers…

11093d5

…ion numbers

add explaining comment

7004c8e

put editable mapping updates to correct id when merging annotations

9621b3e

Merge branch 'master' into unified-annotation-versioning

78a8ecf

Merge branch 'master' into unified-annotation-versioning

5524288

fm3 and others added 6 commits January 21, 2025 10:55

rephrase migration guide

9090d09

Merge branch 'master' into unified-annotation-versioning

4cfcf8d

fix duplicate updates if addLayer is in the history

b9c73db

don't access dataSource.id.name (fixes toast when previewing older ve…

90d87c2

…rsion)

clear potential get variables after closing version restore view

670f775

refactor how allowUpdate is reset; fix incorrectly captured initialAl…

dec00a1

…lowUpdate value in useWillUnmount

philippotto mentioned this pull request Jan 21, 2025

Reduce amount of update actions stored #8345

Open

Merge branch 'unified-annotation-versioning' of github.com:scalablemi…

a7f5f44

…nds/webknossos into unified-annotation-versioning

fm3 added 2 commits January 22, 2025 13:32

Merge branch 'master' into unified-annotation-versioning

cd75d49

restore modified condition

937c69b

MichaelBuessemeyer approved these changes Jan 23, 2025

View reviewed changes

use UnsupportedOperationException in VolumeTracingLayer

9dba180

normanrz approved these changes Jan 23, 2025

View reviewed changes

tools/hosting/docker-compose.yml Show resolved Hide resolved

philippotto added 2 commits January 23, 2025 13:03

Merge branch 'master' of github.com:scalableminds/webknossos into uni…

892a3ef

…fied-annotation-versioning

format

4de7e69

fm3 added 2 commits January 23, 2025 15:29

Merge branch 'master' into unified-annotation-versioning

aa7f47d

Merge branch 'master' into unified-annotation-versioning

42ca63f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unified annotation versioning #7917

Unified annotation versioning #7917

fm3 commented Jul 8, 2024 •

edited

Loading

fm3 commented Aug 28, 2024 •

edited

Loading

fm3 commented Oct 15, 2024 •

edited

Loading

MichaelBuessemeyer commented Oct 15, 2024

fm3 commented Oct 23, 2024 •

edited

Loading

MichaelBuessemeyer commented Oct 25, 2024

coderabbitai bot commented Oct 25, 2024 •

edited

Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

MichaelBuessemeyer commented Oct 28, 2024 •

edited

Loading

MichaelBuessemeyer commented Oct 28, 2024 •

edited

Loading

fm3 commented Oct 29, 2024 •

edited

Loading

MichaelBuessemeyer commented Nov 11, 2024

fm3 commented Nov 11, 2024

MichaelBuessemeyer commented Nov 12, 2024

fm3 Nov 18, 2024

fm3 commented Nov 18, 2024

MichaelBuessemeyer left a comment

fm3 commented Nov 19, 2024 •

edited

Loading

MichaelBuessemeyer commented Jan 20, 2025

MichaelBuessemeyer commented Jan 21, 2025 •

edited by philippotto

Loading

philippotto commented Jan 21, 2025

MichaelBuessemeyer commented Jan 22, 2025

MichaelBuessemeyer commented Jan 22, 2025

MichaelBuessemeyer left a comment

fm3 commented Jan 23, 2025

normanrz left a comment

philippotto commented Jan 23, 2025

Unified annotation versioning #7917

Are you sure you want to change the base?

Unified annotation versioning #7917

Conversation

fm3 commented Jul 8, 2024 • edited Loading

Summary

Steps to test:

TODOs

Issues:

fm3 commented Aug 28, 2024 • edited Loading

fm3 commented Oct 15, 2024 • edited Loading

MichaelBuessemeyer commented Oct 15, 2024

fm3 commented Oct 23, 2024 • edited Loading

MichaelBuessemeyer commented Oct 25, 2024

coderabbitai bot commented Oct 25, 2024 • edited Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

MichaelBuessemeyer commented Oct 28, 2024 • edited Loading

MichaelBuessemeyer commented Oct 28, 2024 • edited Loading

fm3 commented Oct 29, 2024 • edited Loading

MichaelBuessemeyer commented Nov 11, 2024

fm3 commented Nov 11, 2024

MichaelBuessemeyer commented Nov 12, 2024

fm3 Nov 18, 2024

Choose a reason for hiding this comment

fm3 commented Nov 18, 2024

MichaelBuessemeyer left a comment

Choose a reason for hiding this comment

fm3 commented Nov 19, 2024 • edited Loading

MichaelBuessemeyer commented Jan 20, 2025

MichaelBuessemeyer commented Jan 21, 2025 • edited by philippotto Loading

philippotto commented Jan 21, 2025

MichaelBuessemeyer commented Jan 22, 2025

MichaelBuessemeyer commented Jan 22, 2025

MichaelBuessemeyer left a comment

Choose a reason for hiding this comment

fm3 commented Jan 23, 2025

normanrz left a comment

Choose a reason for hiding this comment

philippotto commented Jan 23, 2025

fm3 commented Jul 8, 2024 •

edited

Loading

fm3 commented Aug 28, 2024 •

edited

Loading

fm3 commented Oct 15, 2024 •

edited

Loading

fm3 commented Oct 23, 2024 •

edited

Loading

coderabbitai bot commented Oct 25, 2024 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

MichaelBuessemeyer commented Oct 28, 2024 •

edited

Loading

MichaelBuessemeyer commented Oct 28, 2024 •

edited

Loading

fm3 commented Oct 29, 2024 •

edited

Loading

fm3 commented Nov 19, 2024 •

edited

Loading

MichaelBuessemeyer commented Jan 21, 2025 •

edited by philippotto

Loading