Support a transparent Lookup function

[EnnoMeijers]

Mark Lindeman wrote a VS Code plugin to search for a term through the NoT, see https://marketplace.visualstudio.com/items?itemName=MarkLindeman.vsc-extension-termennetwerk. This plugin also offers to reuse output of the NoT as Linked Data. This means that "our" on the fly generated SKOS version of the orginal data starts to lead a life on its own. It would be better if there was an option to retrieve the source data as it is, without the transformation into SKOS. In this case we would offer a transparent option for the Lookup service. Would this be possible?

[ddeboer]

Yes, but there are complications/objections.

1. We need to maintain an extra set of queries besides the SKOS ones.
2. We have to answer the question: which set of triples should we return for any given term? Only the ?term ?s ?o may not be sufficient if you want to embed links to other resources. SPARQL DESCRIBE is notoriously underdefined.
3. Fundamentally, mapping to SKOS is part of the Network of Terms’s functionality. However, if we’re just returning original source data, it’s really the source that should properly dereference the URIs and serve the data.
4. The transparent lookup cannot be part of the GraphQL API because there’s no consistent (SKOS) typing anymore between sources, so we have to expose it in a separate API.
5. If it’s in a separate API, and developers really want the ease of SKOS, how likely is this new API going to be used?

Perhaps @mightymax can elaborate on that last point: which use cases is your plugin trying to solve? Would a transparent lookup be relevant for those use cases?

[mightymax]

Basically answers 1-3 were the reasons to use the SKOS representation in my plugin. There is no way of knowing which outgoing triples are important for any given subject the NoT returns.

From a user perspective: it is probably not very useful to repeat a data structure when using a found term (because "data bij de bron"@nl). You just want the iri, maybe augmented with the prefLabel as comment. The plugin provides this functionality in its settings BTW.

I can always send a simple construct query to the endpoint of the source to get all outgoing triples (with the iri pre-bound to ?s):

construct { [] ?p ?o } where { ?s ?p ?o }

[EnnoMeijers]

It feels uncomfortable to use the NoT API to augment a URI with the SKOS data the NoT provides. This data is primarily intended to support harmonized search over multiple sources. The data the API gives back is provided to make an informed decision on which term to use or to give the opportunity to walk through the hierarchy (if any).

With the NoT Construct Queries we do not intend to represent the full knowledge of the source, just facilitating easy harmonized search. For sources originally described in SKOS we use a straightforward 1:1 mapping but sources described in different vocabularies (schema, wikidata, etc) the mapping to SKOS is done quite rough and is not intended to represent a semantic equivalent of the original knowledge in the source expressed in SKOS. Reusing this in new publications of Linked Data introduces semantic noise to my opinion and should be avoided.

I would suggest to not offer the option to augment the URI with more than just the prefLabel unless you are able to get hold of the original published data either through content negotiation (which should be supported), a sparql construct query directly on the endpoint of the source (you could import the catalog to find them) or the real source data provided through the NoT if we decide to support this.

This said, I am really enthusiastic about using the NoT API in a VS Code extension. It is another nice demonstration of the flexibility and ease of use of the NoT API.

[mightymax]

The pasting of a snippet of the SKOS representation in the extension is not something I would use myself. I just want the Iri, maybe augmented with the skos:prefLabel as comment, e.g.:

<http://data.beeldengeluid.nl/gtaa/55777> # Heineken

Taking into account this issue and my own findings, I have removed support for pasting full skos representation and an option to augment the result like in my example. I have released a new version 1.0.3. This also adds support for SPARQL documents using a bind declaration, e.g.:

bind(<http://data.beeldengeluid.nl/gtaa/55777> as ?term1)

Thank you both for the feedback!

[EnnoMeijers]

Thank you for the update Mark, this is reassuring and the SPARQL option is interesting!

At the same time I realize that anybody can use the NoT API output in anyway they want and it is not our role to try to convince parties not to reuse the data. For this reason I will keep this issue open to have an internal discussing about the possibilities for supporting the passing through or proxying of the source data through the NoT API. In that case we could offer (and promote) the reuse of the real source data in stead of our internal SKOS mapped data.