- From: Carlos Bobed <cbobed@unizar.es>
- Date: Mon, 10 Oct 2022 17:05:03 +0200
- To: Chris Harding <chris@lacibus.net>, adamsobieski@hotmail.com
- Cc: semantic-web@w3.org
Hi all, El 10/10/2022 a las 12:55, Chris Harding escribió: > Hi, Adam and Carlos - > > Here's my 2c > > Carlos - this is a good point. However, it may be that some specific > embedding sets (e.g. the set produced by T5 using Wikipedia as of Jan > 1 2020) could be selected as standards and be represented by objects > in the ontology. Using such an ontology would be the AI equivalent of > consulting a particular human expert. Human experts give different > answers according to their individual knowledge and experience. The > standard embeddings would give different answers depending on the > models used to generate them.rds, > I was proposing going that way, but so far only with static embeddings. The main problem using any contextual language model (T5, BERT-*) is that they will give you different vectors for the same item depending on the context. As Mike has sketched out, using some semantic tagging to ground the context (similar to positional encoding) could be a way to fix the representation in the particular context that it is appearing (this would happen as well with graph convolutional networks if used to learn a dynamic/contextual model). But ... it's really interesting, if you give me a concept / item, a defined (and shared/accessible) embedding space, and the context of the item for that meaning; that would be a very good anchor to grasp the definition of the item. Best, Carlos
Received on Monday, 10 October 2022 15:05:20 UTC