- From: Paola Di Maio <paola.dimaio@gmail.com>
- Date: Fri, 12 Jun 2020 08:46:03 +0800
- To: carl mattocks <carlmattocks@gmail.com>
- Cc: W3C AIKR CG <public-aikr@w3.org>
- Message-ID: <CAMXe=So8XU-nZqP7zQc0ShyN4pzZfQ8p8FCz3Yxzx=JcjJTP+g@mail.gmail.com>
However, we can develop KR say ontologies, which are implementation independent, Perhaps look at the role of the ontologist, to get a sense of the role for the AI KR strategist? I stress that I dont think of this as a strategy but adopting this construct as it is part of the stratml vocab and can be useful way of of looking at it When it comes to ML, as per your table (ML Evaluation) it is definitely not the role of the AI KR strategist to evaluate machine learning imho Track (monitor) and evaluate are part of the same task imho It could be the role of the AI strategist to decide whether ML is needed in the AI, and to evaluate the alternatives The role of the AI KR strategies would be to devise and implement the KR according to the type of AI design/methods being used by the system. The evaluation of a ML techniques irequires different approaches from the evaluation of NL systems, for example On Fri, Jun 12, 2020 at 8:23 AM Paola Di Maio <paola.dimaio@gmail.com> wrote: > Thank you for clarifying. The table is neat and a good starting point for > elaboration. It is a bit scattered but that can be sorted. > > I dont see KR though > > 1. Can you please point out which of these items fall under KR? (say, > ontology?) > > my experience, the AI KR development work starts after the case for the AI > system has already been made, , ie, when the need for the AI system has > already been identified, ( the first two points). The system design is in > place schedules and resources to be in place, > KR activities are not AI general system development activities. What I > mean is that KR activities can start only after the goal and scope of the > AI system have been defined. (not sure whose role is to define them, > perhaps the system developer or the AI strategist) > > On Thu, Jun 11, 2020 at 10:44 PM carl mattocks <carlmattocks@gmail.com> > wrote: > >> >> Please peruse this GOAL Dependency approach to sequencing >> >> *Goal Activities* >> >> *Title AI Strategist Goal* >> >> *Goal Dependency* >> >> *AIKR Strategist Protocol* >> >> Give an understanding of the possible applications of AI to >> conversations/decisions about business strategy >> >> >> >> Business Strategy >> >> >> >> Document, Requirements, Quality, Robust, Ontological Statements, Ethics, >> Lawful, Machine Learning Evaluation, Track >> >> yes >> >> Identify which areas of the requirements warrant AI solutions versus >> which can be achieved with other types of solutions >> >> >> >> Requirements >> >> >> >> Business Strategy, Document, Quality, Robust, Ontological Statements, >> Ethics, Lawful, Machine Learning Evaluation, Track >> >> yes >> >> Document the vision, values, goals, objectives for one or more AIKR >> objects >> >> >> >> Document >> >> >> >> Business Strategy, Requirements, Quality, Robust, Ontological Statements, >> Ethics, Lawful, Machine Learning Evaluation, Track >> >> yes >> >> Define the limits of quality. If a product has limits of quality/action, >> then these should be stated >> >> Quality >> >> >> >> Business Strategy, Document, Requirements, Robust, Ontological >> Statements, Ethics, Lawful, Machine Learning Evaluation, Track >> >> yes >> >> Evaluate machine learning models >> >> >> >> Machine Learning Evaluation >> >> >> >> Business Strategy, Document, Requirements, Quality, Robust, Ontological >> Statements, Ethics, Lawful, Track >> >> yes >> >> Track AIKR object performance outcome via KPI (Key Performance Indicator) >> based on supervised learning models measurements >> >> >> >> Track >> >> >> >> Document, Requirements, Quality, Robust, Ontological Statements, Machine >> Learning Evaluation >> >> yes >> >> Ensure AI Systems are designed to handle uncertainty and tolerate >> perturbation from a likely threat perspective, such as, design >> considerations incorporate human, social and technology risk factors >> >> >> >> Robust >> >> >> >> Document, Requirements, Quality, Ontological Statements, Ethics, Lawful, >> Machine Learning Evaluation, Track >> >> yes >> >> Employ ontological statements when explaining AIKR object audit data, >> veracity facts and (human, social and technology) risk mitigation >> >> Ontological Statements >> >> >> >> Business Strategy, Document, Requirements, Quality, Robust, Ethics, >> Lawful, Machine Learning Evaluation, Track >> >> yes >> >> Ensure AI Systems adhere to principles of ethics >> >> >> >> Ethics >> >> >> >> Document, Requirements, Quality, Robust, Ontological Statements, Lawful, >> Machine Learning Evaluation, Track >> >> yes >> >> Ensure AI Systems comply with all applicable laws and regulations, such >> as, provision audit data defined by a governance operating model >> >> >> >> Lawful >> >> >> >> Document, Requirements, Quality, Robust, Ontological Statements, Ethics, >> Machine Learning Evaluation, Track >> >> yes >> >> Carl Mattocks >> >> It was a pleasure to clarify >> >> >> On Wed, Jun 10, 2020 at 8:46 AM carl mattocks <carlmattocks@gmail.com> >> wrote: >> >>> >>> Please find attached (summary extract below) Strategic Plan - The role >>> of the AI KR Strategist (A strategist is a person with responsibility >>> for the formulation and implementation of a strategy). >>> >>> To help validate the list of goals we invite proposals for a sequencing >>> (of the goals) that could be the foundation of an AI Strategy Protocol. >>> >>> Please provide your proposals as a reply to this post.. intent is to >>> apply them to the StratMl version at 9:00 am (NY time) June 23 meeting. >>> >>> >>> >>> thanks >>> >>> Carl Mattocks >>> AIKRCG co-Chair >>> >>> >>> Strategic Plan >>> >>> *The role of the AI KR Strategist* >>> >>> For: *Artificial Intelligence Knowledge Representation Community Group >>> (AIKR CG)* >>> >>> Submitted By: *Carl Mattocks* >>> >>> y *CarlMattocks@WellnessIntelligence.Institute* >>> >>> *Overview* >>> >>> This plan defines the role of the AI KR Strategist. >>> >>> Contents >>> >>> *No table of contents entries found.* >>> 1. Articulation 1.1. Vision >>> >>> For all AI systems to have clearly and transparently documented goals >>> and performance data showing that they are being achieved. >>> 1.2. Mission >>> >>> The mission of an AI Strategist is to define the purpose and goals of AI >>> systems, as well as the KPIs by which we can determine if the system is >>> meeting its goals. >>> 1.3. Scorecard >>> >>> >>> >>> *Goals* >>> >>> *Objectives* >>> >>> *Performance Indicators* >>> >>> *Commentary* >>> >>> *Goals with no perspective* >>> >>> Quality (see pp1) >>> >>> Adherence to Environmental Impacts (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Efficiency (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Ethics (see pp1) >>> >>> Accountability (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Autonomy (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Confidentiality (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Veracity (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Machine Learning Evaluation (see pp1) >>> >>> Track (see pp1) >>> >>> Precision Recall >>> >>> >>> >>> >>> >>> >>> >>> Accuracy >>> >>> >>> >>> >>> >>> >>> >>> Confusion Matrix >>> >>> >>> >>> >>> >>> >>> >>> Per-class accuracy >>> >>> >>> >>> >>> >>> >>> >>> Log-Loss >>> >>> >>> >>> >>> >>> >>> >>> AUC-ROC Curve >>> >>> >>> >>> >>> >>> >>> >>> F-measure >>> >>> >>> >>> >>> >>> >>> >>> NDCG >>> >>> >>> >>> >>> >>> >>> >>> Regression Analysis >>> >>> >>> >>> >>> >>> >>> >>> Quantiles of Errors >>> >>> >>> >>> >>> >>> >>> >>> "Almost correct" predictions >>> >>> >>> >>> >>> >>> >>> >>> Trustworthy (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Lawful (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Ontological Statements (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Track (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Business Strategy (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Requirements (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Document (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Robust (see pp1) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> 1.4. Goals.... >>> >>> It was a pleasure to clarify >>> >>> >>> On Tue, Jun 9, 2020 at 11:56 PM Paola Di Maio <paola.dimaio@gmail.com> >>> wrote: >>> >>>> oops- >>>> Thanks Carl Owen and Chris >>>> for capturing and sharing. >>>> I repeat here my comment shared with participants after the session - >>>> also pasted below >>>> >>>> if the plan we worked on yesterday is to describe the role of an AI >>>> Strategist >>>> then we need to make explicit what the goal of the role and associated >>>> activities is/are >>>> >>>> AI systems /software development can be very sophisticated and well >>>> understood practice >>>> in systems/software engineering . It is not a strategy role to do >>>> requirements analysis and write >>>> technical systems specifications, especially not technically advanced >>>> ones >>>> So we should review and devise an AI strategist/strategy goals >>>> accordingly, taking into account the technical system specifications. >>>> or viceversa? >>>> A strategist could identify the need for a technical system based on a >>>> business problem/need, but then pass the development >>>> to a specialist (a systems developer/engineer) who has the skills to do >>>> so >>>> So upon further reflection, I dont think requirements belong to an ai >>>> strategy. unless better specified >>>> >>>> pdm >>>> paola wrote: >>>> Thanks for starting this plan >>>> As I read what had been drafted I commented with my software/system >>>> developer hat on >>>> The system lifecycle is considered a process, rather than a strategy, >>>> and consists of >>>> identifying requirements, specifications and documentation and all >>>> Its well understood and documented and there are several models, but >>>> the lifecycle itself >>>> is fairly standard, it applies to AI systems as to any system >>>> So before I have a go at reorganising and perhaps streamlining a bit >>>> those goals >>>> I think we should define better what is the role of the AI strategist >>>> does it overlap the role of a ai system analyst and designer (which is >>>> what I have in mind) >>>> ie identify what needs to be done >>>> or is it more a role to decide HOW these goals need to be achieved in >>>> terms of resources, schedules, policies, coordination with management >>>> etc >>>> Also, as I mentioned, this may depend on what other roles the AI >>>> strategist works with/depends on- >>>> p >>>> >>>> >>>> On Wed, Jun 10, 2020 at 11:45 AM Paola Di Maio <paola.dimaio@gmail.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Wed, Jun 10, 2020 at 12:38 AM carl mattocks <carlmattocks@gmail.com> >>>>> wrote: >>>>> >>>>>> Role of Data Scientist has a process that is described in the links >>>>>> below : >>>>>> >>>>>> *Cross-industry standard process for data mining*, known as >>>>>> *CRISP-DM*,[1] >>>>>> <https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining#cite_note-Shearer00-1> is >>>>>> an open standard <https://en.wikipedia.org/wiki/Open_standard> process >>>>>> model that describes common approaches used by data mining >>>>>> <https://en.wikipedia.org/wiki/Data_mining> experts. It is the most >>>>>> widely-used analytics <https://en.wikipedia.org/wiki/Analytics> model >>>>>> >>>>>> >>>>>> https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining >>>>>> >>>>>> >>>>>> these lecture notes provide good background notes >>>>>> >>>>>> >>>>>> https://wiwi.hs-duesseldorf.de/personen/thomas.zeutschler/Documents/HSD_W_ITAiBA_Zeutschler_SS2016_Lecture2_CRSIP_DM.pdf >>>>>> >>>>>> >>>>>> this is an IBM presentation on the The Data Science Process >>>>>> >>>>>> https://www-01.ibm.com/events/wwe/grp/grp304.nsf/vLookupPDFs/Polong%20Lin%20Presentation/$file/Polong%20Lin%20Presentation.pdf >>>>>> >>>>>> >>>>>> enjoy >>>>>> Carl >>>>>> >>>>>> It was a pleasure to clarify >>>>>> >>>>>> >>>>>> On Tue, Jun 9, 2020 at 12:13 PM Owen Ambur <Owen.Ambur@verizon.net> >>>>>> wrote: >>>>>> >>>>>>> Carl, thanks for leading another productive editing session on the >>>>>>> AIKR CG televideo conference today. I found myself agreeing with much of >>>>>>> what Paola was saying, and I continue to be impressed by Chris' capability >>>>>>> to comprehend and capture the essence of our dialog in his StratNavApp. >>>>>>> For the benefit of those who were unable to join us, I am providing a link >>>>>>> here >>>>>>> <https://www.stratnavapp.com/StratML/Part1/861566c8-e9be-4642-b52f-f673fa499f4e/Styled> >>>>>>> so that they can view our draft plan as it currently exists. >>>>>>> >>>>>>> I'm sorry that Justin was not able to participate to ensure that we >>>>>>> are capturing the knowledge he has gained in considering the roles of AI >>>>>>> strategists. However, it would be great if he and others could contribute >>>>>>> their comments and edits on our draft in Chris' app >>>>>>> <https://www.stratnavapp.com/> and perhaps participate in our text >>>>>>> televideo conference on June 23. >>>>>>> While the StratML collection does not yet contain a model >>>>>>> performance plan for Data Scientists, it does include one for Chief Data >>>>>>> Officers (CDOs), at https://stratml.us/drybridge/index.htm#PP4CDO >>>>>>> >>>>>>> Unless someone has a better source, I will soon convert the contents >>>>>>> of this one to StratML format: >>>>>>> https://www.sas.com/en_us/insights/analytics/what-is-a-data-scientist.html >>>>>>> >>>>>>> BTW, Carl, your reference to OASIS's Business Centric Methodology >>>>>>> (BCM) prompted me to convert its content to StratML format at >>>>>>> https://stratml.us/drybridge/index.htm#BCM >>>>>>> >>>>>>> These elements of the BCM are closely related to the purposes of the >>>>>>> StratML standard: >>>>>>> >>>>>>> Objective 1.1: Goals >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c0b98-aa65-11ea-9885-42f42983ea00> >>>>>>> - Determine the business goals. >>>>>>> Objective 1.3: Participants >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c0d32-aa65-11ea-9885-42f42983ea00> >>>>>>> - Identify the project participants. >>>>>>> Objective 1.4: COIs >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c0e40-aa65-11ea-9885-42f42983ea00> >>>>>>> - Identify the Community of Interest. >>>>>>> Goal 2: Goals >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c1106-aa65-11ea-9885-42f42983ea00> >>>>>>> - Understand the business goals. >>>>>>> Objective 3.3: Standards Bodies >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c1750-aa65-11ea-9885-42f42983ea00> >>>>>>> - Map interoperability requirements to standard bodies. >>>>>>> Objective 3.4: Formats >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c182c-aa65-11ea-9885-42f42983ea00> >>>>>>> - Map interoperability requirements to internal legacy system formats. >>>>>>> Objective 3.5: Templates >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c191c-aa65-11ea-9885-42f42983ea00> >>>>>>> - Capture information in a way that can be reused over time and among >>>>>>> participating organizations. >>>>>>> Objective 3.5.1: Strategies >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c19f8-aa65-11ea-9885-42f42983ea00> >>>>>>> - Implement strategies for interoperability. >>>>>>> Objective 3.5.5: Standards >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c1da4-aa65-11ea-9885-42f42983ea00> >>>>>>> - Apply standards for interoperability and contracts. >>>>>>> Objective 3.5.7: Performance >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c206a-aa65-11ea-9885-42f42983ea00> >>>>>>> - Monitor and manage performance. >>>>>>> Objective 4.1: Standards >>>>>>> <https://stratml.us/carmel/iso/BCMwStyle.xml#_6b2c2b82-aa65-11ea-9885-42f42983ea00> >>>>>>> - Use existing standards. >>>>>>> >>>>>>> It would be great if BCM practitioners were to use the StratML >>>>>>> standard (ISO 17469-1). Anyone who may wish to apply the BCM template as a >>>>>>> performance plan for their organization in StratML Part 2 format could do >>>>>>> so by clicking on the link provided here. >>>>>>> >>>>>>> Owen >>>>>>> >>>>>>> -------- Forwarded Message -------- >>>>>>> Subject: Re: ai strategist role >>>>>>> Date: Tue, 9 Jun 2020 11:22:53 -0400 >>>>>>> From: carl mattocks <carlmattocks@gmail.com> >>>>>>> <carlmattocks@gmail.com> >>>>>>> To: Paola Di Maio <paoladimaio10@googlemail.com> >>>>>>> <paoladimaio10@googlemail.com> >>>>>>> CC: Owen Ambur <Owen.Ambur@verizon.net> <Owen.Ambur@verizon.net>, >>>>>>> Chris Fox <chris@chriscfox.com> <chris@chriscfox.com>, Paul Alagna >>>>>>> <pjalagna@gmail.com> <pjalagna@gmail.com> >>>>>>> >>>>>>> Data Scientist is a complimentary role.. which is supported by an >>>>>>> open source method that has strong emphasis on machine learning and is Plan >>>>>>> Do Check Act oriented. >>>>>>> >>>>>>> Carl >>>>>>> >>>>>>> On Tue, Jun 9, 2020, 11:12 AM Paola Di Maio <paola.dimaio@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks for starting this plan >>>>>>>> As I read what had been drafted I commented with my software/system >>>>>>>> developer hat on >>>>>>>> The system lifecycle is considered a process, rather than a >>>>>>>> strategy, and consists of >>>>>>>> identifying requirements, specifications and documentation and all >>>>>>>> Its well understood and documented and there are several models, >>>>>>>> but the lifecycle itself >>>>>>>> is fairly standard, it applies to AI systems as to any system >>>>>>>> So before I have a go at reorganising and perhaps streamlining a >>>>>>>> bit those goals >>>>>>>> I think we should define better what is the role of the AI >>>>>>>> strategist >>>>>>>> does it overlap the role of a ai system analyst and designer (which >>>>>>>> is what I have in mind) >>>>>>>> ie identify what needs to be done >>>>>>>> or is it more a role to decide HOW these goals need to be achieved >>>>>>>> in >>>>>>>> terms of resources, schedules, policies, coordination with >>>>>>>> management etc >>>>>>>> Also, as I mentioned, this may depend on what other roles the AI >>>>>>>> strategist works with/depends on- >>>>>>>> p >>>>>>>> >>>>>>>
Received on Friday, 12 June 2020 00:47:00 UTC