- From: Owen Ambur <Owen.Ambur@verizon.net>
- Date: Thu, 19 May 2022 17:08:15 -0400
- To: W3C AIKR CG <public-aikr@w3.org>
- Message-ID: <961aeb8e-8f11-38e2-402c-ad2499c23682@verizon.net>
Carl, your question prompted me to discover and convert to StratML Part
2 format NIST's Four Principles of Explainable Artificial Intelligence,
at https://stratml.us/drybridge/index.htm#4PEAI
StratML is particularly relevant to these objectives:
Objective 2.2.2: Roles & Relationships
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_84b329d4-d7ae-11ec-8473-5e9a1883ea00>
- Define groups according to their role or relationship to the systems.
Objective 3.1: Performance Metrics
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_84b32d30-d7ae-11ec-8473-5e9a1883ea00>
- Develop performance metrics for explanation accuracy.
With respect to the latter, they say:
Researchers in AI have developed standard measures of algorithm and
system accuracy ... While these established decision accuracy
metrics exist, researchers are in the process of developing
performance metrics for explanation accuracy.
As such metrics become available, I'll look forward to documenting them
in StratML format.
In the meantime, the human baseline upon which to improve is pretty low
for each of the four principles (documented as goals and objectives in
my StratML rendition): Evidence/Reasoning
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_5c96d488-d7b7-11ec-aa31-bf6087babdf6>,
Comprehensibility
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_171c856c-d7b5-11ec-aa96-28ee86babdf6>,
Accuracy
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_171c8da0-d7b5-11ec-aa96-28ee86babdf6>
& Knowledge Limits
<https://stratml.us/carmel/iso/part2/4PEAIwStyle.xml#_171c91c4-d7b5-11ec-aa96-28ee86babdf6>.
Owen
On 5/19/2022 10:30 AM, carl mattocks wrote:
> Paola et al
>
> Should AIKRCG take on the a task of determining whether 'making
> assertions in a *tbd *structured way' can contribute to reliability,
> verifiability and explainability ?
>
> Carl
> It was a pleasure to clarify
>
>
> On Wed, May 18, 2022 at 9:07 PM Paola Di Maio <paola.dimaio@gmail.com>
> wrote:
>
> Thank you Owen
>
> Kotseruba should be pleased with your rendition, I hope you reach out
> and send them your diagram-
> maybe get feedback
>
> I ll be happy to include the new image and the source in the report-
> does it mean that AI KR ID diagram without an explanation or use
> case is not to be mentioned?
> Let me know otherwise Carl-
>
> I think I mentioned before, the problem is that anyone can make
> any statement
> with StratML, or any other language, and that can be useful
>
> but that does not mean that the assertion is true,
> or whether it holds true under changing conditions (such as time)
>
> So the usefulness is limited to making assertions in a structured way
> (which is a good starting point, but the AI KR problems in certain
> domains
> are beyond the assertability o statements)
>
> AI KR are trying to address algorithmic reliability, verifiability
> and explainability
> among other things- given real word complexities
>
> In the meantime, we have to work with what is available, I guess
>
> On Tue, May 17, 2022 at 10:43 PM Owen Ambur
> <Owen.Ambur@verizon.net> wrote:
>
> Thanks, Paola. I see in the original source that Springer
> provides links enabling expansion of the graphics so that they
> are easier to read, e.g., this one
> <https://link.springer.com/article/10.1007/s10462-018-9646-y/figures/3>
> for the graphic you are including in your report.
>
> The graphic still doesn't mean much to me. However, if
> textual descriptions are available for the
> initiatives/projects whose names/acronyms are listed at the
> bottom of the graphic, I'll be happy to render them in StratML
> format -- for potential inclusion in a catalog of AI/ML
> initiatives developed by the AIKR CG.
>
> In the meantime, the goals and objectives embedded in the
> conclusion of Kotseruba & Tsotsos' wall of text
> <https://en.wikipedia.org/wiki/Wikipedia:Wall_of_text> are now
> available in StratML format at
> https://stratml.us/drybridge/index.htm#CCAPA
>
> With reference to their Goal 8: Reproducibility
> <https://stratml.us/carmel/iso/CCAPAwStyle.xml#_0f32842a-d592-11ec-b45d-08b00083ea00>
> - Provide full technical detail to support reproducibility of
> research - it would be good if such detail were provided in an
> open, standard, machine-readable format, like StratML.
>
> Doing so would support their Goal 7: Evaluation
> <https://stratml.us/carmel/iso/CCAPAwStyle.xml#_0f327d72-d592-11ec-b45d-08b00083ea00>
> - Develop objective and extensive evaluation procedures.
>
> Conversely, failing to do so is a form of artificial ignorance
> <https://www.linkedin.com/pulse/artificial-ignorance-owen-ambur>.
>
> Kranthi, I'd love to see what ThoughtFlow might be able to do
> with my StratML rendition of Kotseruba & Tsotsos' plan... and
> perhaps eventually a catalog of AI/ML initiatives applying
> their goals and objectives as a taxonomy of such efforts.
>
> BTW, w/re their Objective 4.1: Emotions & Intentions
> <https://stratml.us/carmel/iso/CCAPAwStyle.xml#_0f3272a0-d592-11ec-b45d-08b00083ea00>
> - Detect the emotional states and intentions of interlocutors
> - we human beings could help machines help us if we made both
> our intentions (goals/objectives) and our values (undergirding
> our emotions) explicit in a format they can "read".
>
> Owen
>
>
> -------- Forwarded Message --------
> Subject: Re: Publishing the report
> Date: Tue, 17 May 2022 08:59:58 +0800
> From: Paola Di Maio <paoladimaio10@gmail.com>
> <mailto:paoladimaio10@gmail.com>
> To: Owen Ambur <Owen.Ambur@verizon.net>
> <mailto:Owen.Ambur@verizon.net>
> CC: Kranthi Kiran <kranthi@thoughtflow.io>
> <mailto:kranthi@thoughtflow.io>
>
>
>
> Thank you Owen
> You need to read the paper to understand the image
> https://link.springer.com/article/10.1007/s10462-018-9646-y
> I realized I mis-spelled the author's name
>
>
> On Tue, May 17, 2022 at 3:50 AM Owen Ambur
> <Owen.Ambur@verizon.net> wrote:
>
> Paola, here are some of the issues I have with the
> visualization copied below:
>
> 1. I don't know what any of the terms mean, particularly
> those at the bottom of the hierarchy.
> 2. It is somewhat uncomfortable having to tilt my head to
> read them and they are pretty densely packed together.
> 3. Since it is in image format, the URL to the source is
> un-clickable.
>
> As I said, the image reminds me of what I've seen of
> Kranthi's ThoughtFlow application. My thought is that
> perhaps he might be able to improve upon its usability and
> utility. However, I don't know how much value that would
> provide either to those who may already understand the
> diagram or those of us who do not.
>
> Owen
>
>
> On 5/15/2022 8:21 PM, Paola Di Maio wrote:
>> Thank you
>> we can also add a mention in the report to Chris'app
>> since he participated in meetings and made available his app
>>
>> Here is the diagram Carl Provided
>> https://lists.w3.org/Archives/Public/public-aikr/2020Jun/0005.html
>> I ll add the URL so that the original size is accessible
>>
>> i we include it in the report, it should be described and
>> ideally, have a short test case/use case/example
>> to demonstrate itse use and benefits - I do not remember
>> the modelling being shared nor discussed
>> but I did not attend the meetings-
>>
>> regarding the AI KR vs COGSCI diagram, is just pointing
>> to a relation between the two knowledge domains
>> I you do not understand it, it means I have to provide a
>> better description/explanation
>> in the report
>>
>> anything else not clear in the report please feel free
>> to highlight
>> thanks-
>>
>> On Mon, May 16, 2022 at 12:31 AM Owen Ambur
>> <Owen.Ambur@verizon.net> wrote:
>>
>> Paola, I defer to Carl to answer your question. The
>> text of his diagram is difficult to read. I see that
>> it references StratML but it does not accurately
>> reflect the logic of the schema, in which:
>>
>> a) Objectives are children of Goals, and
>>
>> b) Performance Indicators are children of
>> Objectives. Nor do I have a StratML rendition of
>> Carl's plan at
>> https://stratml.us/drybridge/index.htm#AIKRCG
>>
>> Presumably, if he were to use my XForm (or Chris'
>> StratNavApp) to flesh out his plan, he'd have better
>> understanding of the structure and semantics of the
>> StratML schema.
>>
>> On the other hand, if and hopefully when Carl
>> provides a description of his plan in free flowing
>> text, I'll be happy to render it in StratML format.
>> Ideally, it would add value to the intelligence
>> already built into the StratML schema. The
>> opportunity to do so is unlimited.
>>
>> In the meantime, I left Kranthi on the cc list
>> because your visualization of the "convergence
>> between AI/KR and COGAI" (which doesn't mean much to
>> me) reminds me of the output of his ThoughtFlow
>> application. Perhaps he might be able to make it
>> more comprehensible to folks like me.
>>
>> Owen
>>
>>
>> On 5/15/2022 3:21 AM, Paola Di Maio wrote:
>>> Thank you Owen
>>> I just added the AI KR ID ontology diagram shared by
>>> Carl via the list
>>> , waiting or a description and use case to be added
>>> to the drat
>>> is the AI KR ID ontology diagram based on stratml?
>>>
>>>
>>> good news about the performance, and editable
>>> record? could be a rather good feature
>>>
>>>
>>>
>>> On Sat, May 14, 2022 at 11:10 PM Owen Ambur
>>> <Owen.Ambur@verizon.net> wrote:
>>>
>>> Looks good to me, Paola. Thanks a bunch.
>>>
>>> When it's published, I may wish to reference it
>>> in my Wikipedia article
>>> <https://en.wikipedia.org/wiki/Strategy_Markup_Language>
>>> on StratML and/or a posting on LinkedIn.
>>>
>>> I'll also be on the lookout for opportunities to
>>> render a plan(s) in StratML format to help me
>>> and other less AI/ML expert folks to comprehend
>>> what System Level Knowledge Representation and
>>> Neuro Symbolic Integration might be able to do
>>> for us.
>>>
>>> If AIKR CG's plan is updated, that might be a
>>> good place to start creating such
>>> documentation. An edit link is available at
>>> https://stratml.us/drybridge/index.htm#AIKRCG.
>>> Please note, however, that the user must save
>>> the file locally. I have no desire to maintain
>>> UIDs & passwords. I also want to encourage
>>> users to host their own plans and reports on
>>> their own websites -- for aggregation, indexing,
>>> and other value-adds by intermediary service
>>> providers.
>>>
>>> BTW, legislation has been introduced
>>> <https://www.linkedin.com/posts/owenambur_congressgov-library-of-congress-activity-6930927767131758592-38yC?utm_source=linkedin_share&utm_medium=member_desktop_web>
>>> in the U.S. Senate that would require agencies
>>> to publish quarterly performance reports on the
>>> Performance.gov site. When such reports are
>>> published in an open, standard, machine-readable
>>> format, it should be pretty easy for AI agents
>>> to "learn" and help human beings understand the
>>> requirements (value chains) required to achieve
>>> their objectives. What else might knowledge be
>>> about?
>>>
>>> Owen
>>>
>>>
>>> On 5/14/2022 1:53 AM, Paola Di Maio wrote:
>>>> Owen
>>>> thank or your suggestions, I have incorporated
>>>> them in the report which I plan to make live soon
>>>>
>>>> take a look again please
>>>> https://docs.google.com/document/d/15Y1qICrKpPcilwIwQdA5lmTwW1iO4cywL2NESdek70E/edit?usp=sharing
>>>>
>>>> apologies it took me so long
>>>>
>>>> Carl and Richard, let me know i you spot
>>>> something obviously wrong
>>>>
>>>>
>>>> PDM
>>>
Received on Thursday, 19 May 2022 21:08:35 UTC