Dynamic Highlighting

[Naman-Bhrgv]

Hi,
I am using Potato for my research, however, I haven't been ale to setup Dynamic Highlighting task.

Reproduction

I tried to include dynamic highlighting for dialogue_analysis task. Added dynamic_key.tsv file path to dialogue-analysis.yaml . Both keyword.tsv and dialogue-analysis.yaml files are present here.

I have attached the snippet of annotation_schemes part of dialogue-analysis.yaml where I have added the path of dynamic_key.tsv file.

"annotation_schemes": [
        {
          "annotation_type": "highlight",
          "name": "certainty",
          "description": "Highlight which phrases make the sentence more or less certain",
          "labels": [
            "certain", "uncertain"
          ],

          # If true, numbers [1-len(labels)] will be bound to each
          # label. Highlight selection annotations with more than 10 are not supported
          # with this simple keybinding and will need to use the full item
          # specification to bind all labels to keys.
          "sequential_key_binding": True,
        },
        {
            "annotation_type": "radio",
            "name": "sentiment",
            "description": "What kind of sentiment does the given text hold?",
            "labels": [
               "positive", "neutral", "negative",
            ],

            # If true, numbers [1-len(labels)] will be bound to each
            # label. Aannotations with more than 10 are not supported with this
            # simple keybinding and will need to use the full item specification
            # to bind all labels to keys.
            "sequential_key_binding": True,     
           "keyword_highlights_file":"dynamic_key.tsv"                      
        },
        
           
    ]

Error Description

From what I understood from documentation, the keywords provided in tsv file will be highlighted automatically. However, even after adding the tsv file, the annotation task doesn't change. Can you tell me how to setup dynamic highlighting.
Thanks in advance!

[Jiaxin-Pei]

Hi @Naman-Bhrgv,

There are two issues in your current setup.
(1) "keyword_highlights_file":"dynamic_key.tsv" should be added outside "annotation_schemes", for example:

"user_config": {
      "allow_all_users": True,
      "users": [  ],
    },

 "keyword_highlights_file": "dynamic_key.tsv",

(2) In the .tsv file, the column name should be Label instead of labels.

Please fix these two issues and try again.
You can also check our this example project to see how dynamic highlight is configured.

[Naman-Bhrgv]

Thanks @Jiaxin-Pei it worked for me, btw I am also studying at University of Michigan and this tool has helped me a lot in my research.

[Naman-Bhrgv]

Hi @Jiaxin-Pei ,
Is there a way to edit the annotation for keywords specified in dynamic highlighting?

[davidjurgens]

What do you mean by "edit the annotation" here? Like to record which words were highlighted at the time of annotation?

…

On Mon, Jun 24, 2024 at 8:30 PM Naman-Bhrgv ***@***.***> wrote: Hi @Jiaxin-Pei <https://github.com/Jiaxin-Pei> , Is there a way to edit the annotation for keywords specified in dynamic highlighting? — Reply to this email directly, view it on GitHub <#83 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHO4LMRYLOVGKI3QPU6B53ZJC23PAVCNFSM6AAAAABJL3IUTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGY4DGNJYGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Naman-Bhrgv]

No, I meant to edit the label pre-specified for words in the tsv file (which was input of dynamic highlighting).

[Jiaxin-Pei]

@Naman-Bhrgv Do you mean that you want to do span annotation for the dynamically highlighted words?

[Naman-Bhrgv]

Yes exactly

[Jiaxin-Pei]

@Naman-Bhrgv We don't currently support doing span annotations for the dynamically highlighted words. However, if you could describe your task in more detail, I can take a look and see if there are other ways to make it work! We do have some other relevant features that can potentially help!

[davidjurgens]

@Jiaxin-Pei I think it's possible to support those kinds of annotations with some extensions to the tool though. It might be worth a look

…

On Tue, Jun 25, 2024 at 9:04 AM Jiaxin Pei ***@***.***> wrote: @Naman-Bhrgv <https://github.com/Naman-Bhrgv> We don't currently support doing span annotations for the dynamically highlighted words. However, if you could describe your task in more detail, I can take a look and see if there are other ways to make it work! We do have some other relevant features that can potentially help! — Reply to this email directly, view it on GitHub <#83 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHO4LNWEDH7RPRISFX4RFTZJFTG7AVCNFSM6AAAAABJL3IUTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBYHA4TANBZGI> . You are receiving this because you commented.Message ID: ***@***.***>

[Jiaxin-Pei]

@davidjurgens Yes! I think we can either extend the label suggestions or the Dynamic highlighting to support this. Happy to take a look after @Naman-Bhrgv describes his exact use case.

[Naman-Bhrgv]

@Jiaxin-Pei I am following a two step approach for data annotation. These steps are-

1. We use spacy to label all the entities in text.
2. Then we load the text in potato and edit the entities labelled by spacy (incase they are incorrect) and label additional entities in the text.
   Please provide some suggestions for it.

[Jiaxin-Pei]

Thanks a lot for sharing the details!
I think this is aligned with our development plan on label suggestions. I can prioritize this feature and try to get this up early next week. Would this work for you? @Naman-Bhrgv

[Naman-Bhrgv]

Yeah that would be great. Thanks Jiaxin!

…

On Wed, Jun 26, 2024 at 10:05 AM Jiaxin Pei ***@***.***> wrote: Thanks a lot for sharing the details! I think this is aligned with our development plan on label suggestions <https://potato-annotation.readthedocs.io/en/latest/productivity/#label-suggestions>. I can prioritize this feature and try to get this up early next week. Would this work for you? @Naman-Bhrgv <https://github.com/Naman-Bhrgv> — Reply to this email directly, view it on GitHub <#83 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO7US5F36MCD5JCPLDCUCK3ZJLKD7AVCNFSM6AAAAABJL3IUTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJRHE2DIOJUGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>