ABOUT ML & Verification of AI Claims

Here are the two most recent additions to the StratML collection:

    ABOUT ML Project - https://stratml.us/drybridge/index.htm#AML

    Toward Trustworthy AI Development: Mechanisms for Supporting
    Verifiable Claims - https://stratml.us/drybridge/index.htm#MSVC

in the latter, the very first objective is directly related to the 
StratML standard:

    Objective 1.1: Goals & Values
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_1589903e-aebb-11ea-863f-edd70a83ea00>
    - Clarify the goals and values.

Others include:

    Objective 6.1: Criteria, Objectives & Frameworks
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_937934a4-af42-11ea-867a-27142883ea00>
    - Establish consensus on the criteria, objectives, and frameworks
    for interpretability research.

    Objective 7.3: Benchmarks
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_9379482c-af42-11ea-867a-27142883ea00>
    - Develop benchmarks for PPML to unify goals and measure progress.
    [PPML stands for Privacy-Preserving Machine Learning.]

    Objective 7.6: Standardization & Interoperability
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_93795542-af42-11ea-867a-27142883ea00>
    - Work on standardization/interoperability.

    Objective 9.1: Standardization
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_93796956-af42-11ea-867a-27142883ea00>
    - Assess the feasibility of standardizing compute accounting.

    Objective 9.2: Reporting
    <https://stratml.us/carmel/iso/MSVCwStyle.xml#_93796d16-af42-11ea-867a-27142883ea00>
    - Automate or simplify compute reporting.

With reference to ABOUT ML's plan, it would be good if the guide 
contemplated in this objective were published as a model performance 
plan in StratML Part 2 format:

    Objective 2.1: Guidance
    <https://stratml.us/carmel/iso/AMLwStyle.xml#_41c51098-af19-11ea-bd1a-b48e1e83ea00>
    - Develop a guide to initiating and scaling a documentation pilot.

With reference to Objective 1.2: Questions 
<https://stratml.us/carmel/iso/AMLwStyle.xml#_41c50cba-af19-11ea-bd1a-b48e1e83ea00> 
- Gather questions in a database, there are currently 183 in their 
database 
<https://docs.google.com/spreadsheets/d/1HYlWBos7AXxV4TRvWkK_EF6iFFWNjk8NEevlh1JC9Bk/edit#gid=0>.  
 From my perspective, it would be good if those questions were restated 
as objectives in an open, standard, machine-readable format like 
StratML.  That's more than I am motivated to do right now, but for 
anyone who may be interested in checking them out, the database includes 
the URLs for the research papers from which the questions were gleaned.

Owen

Received on Monday, 15 June 2020 20:36:55 UTC