- From: carl mattocks <carlmattocks@gmail.com>
- Date: Thu, 28 May 2020 09:16:52 -0400
- To: W3C AIKR CG <public-aikr@w3.org>
- Message-ID: <CAHtonumU34ckAhSJ77CiEoF4EtZ6OZK-HbYjZcuw-qnw=J5xog@mail.gmail.com>
Regarding the Role of an AI Strategist I suggest we adopt as-is the GOALS from https://www.stratnavapp.com/StratML/Part1/d95875f4-04b2-46d3-a953-c441e16f428a/Styled Goal: Ethics Navigate through the potential ethical and legal issues of AI technology, while driving forward the execution of smarter, more intelligent products and services and processes ------------------------------ Goal: Strategy develop and drive corporate strategy to lead assessments of new product ideas, develop business cases and lead prototype creation to reach formal business recommendation on new opportunities geared at enhancing service and member experience ------------------------------ Goal: Problems Articulate in solving a business problem how much will be done by an AI system and how much by the human system AND add them to Strategy (previously discussed) https://www.stratnavapp.com/StratML/Part2/861566c8-e9be-4642-b52f-f673fa499f4e/Styled <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/rEsV5jT4sswXrPTTNfeC6nDjtm-nY1ZIrbl-LZSSKUkUYZgjnbTv8Udz3_YAbqwqUVZN_Mh-3G1zCay-QQbmAsIiYP6DYRPQ-gHFsm5Onb8Xu6c7fTMAyYa1jfbH_uQG3qOf0p7m2kbD7BomBZ8yw2ziX7qxU7mHkQZpdgBzER8fLV30znD2wfdD9K_Y1ANHX7NkMFXFwAQKoHCD7YHDD6uoVfsVF69ud0Ng7AUmt8FAnsV3Q5sx6TZ74Ta_vL9klYIGl8iJD0V01kG3Pf-6AQ> VisionFor all AI systems to have clearly and transparently documented goals and performance data showing that they are being achieved.MissionThe mission of an AI Strategist is to define the purpose and goals of AI systems, as well as the KPIs by which we can determine if the system is meeting its goals. ------------------------------ Goal: Ethical Ensure AI Systems adhere to pivotal principles, such as, confidentiality, autonomy, accountability and veracity ------------------------------ Goal: Machine Learning Evaluation Evaluate machine learning modelsStakeholders: Artificial Intelligence Knowledge Representation Community Group (AIKR CG) Role: Community of Interest Objectives: Objective: Trustworthy Provide the foundation for a trustworthy AIKR Other Information: Evaluation metrics are tied to machine learning tasks. Perhaps the easiest metric to interpret is the percent of estimates that differ from the true value by no more than X%. Objective: Track Track Classification Performance Indicators Other Information: Ontological Statement: Classification Accuracy is the ratio of number of correct class label predictions to the total number of input samples data. Ontological Statement: F1 Score measure the Harmonic Mean between precision and recall. The range for F1 Score is [0, 1]. It tells you how precise your classifier is (how many instances it classifies correctly), as well as how robust it is (it does not miss a significant number of instances). Performance Indicator: Precision Recall Quantitative in Outcome Other Information: Ontological Statement: Precision is the number of correct positive results divided by the number of positive results predicted by the classifier. Ontological Statement: Recall is the number of correct positive results divided by the number of all relevant samples (all samples that should have been identified as positive). Performance Indicator: Accuracy Quantitative in Outcome Other Information: Ontological Statement: Classification Rate or Accuracy is given by the relation: True Positives + True Negatives / All Instances (True & False Positives + True & False Negatives) Performance Indicator: Confusion Matrix Quantitative in Outcome Other Information: Ontological Statement: A confusion matrix is a summary of prediction results on a classification problem. The number of correct and incorrect predictions are summarized with count values and broken down by each class (the types of errors being made) Types : * True Positives : The cases in which we predicted YES and the actual output was also YES. * True Negatives : The cases in which we predicted NO and the actual output was NO. * False Positives : The cases in which we predicted YES and the actual output was NO. * False Negatives : The cases in which we predicted NO and the actual output was YES. Accuracy for the matrix can be calculated by taking average of the values lying across the “main diagonal” Type StartDate EndDate Description Target Number of True Positives Target Number of False Positives Target Number of True Negatives Target Number of False Negatives Actual [To be determined] Performance Indicator: Per-class accuracy Quantitative in Outcome Performance Indicator: Log-Loss Quantitative in Outcome Other Information: Ontological Statement: Logarithmic loss (related to cross-entropy) measures the performance of a classification model where the prediction input is a probability value between 0 and 1 - Log loss increases as the predicted probability diverges from the actual label Logarithmic Loss or Log Loss, works by penalising the false classifications. It works well for multi-class classification. When working with Log Loss, the classifier must assign probability to each class for all the samples. where, y_ij, indicates whether sample i belongs to class j or not p_ij, indicates the probability of sample i belonging to class j Log Loss has no upper bound and it exists on the range [0, ∞). Log Loss nearer to 0 indicates higher accuracy, whereas if the Log Loss is away from 0 then it indicates lower accuracy. In general, minimising Log Loss gives greater accuracy for the classifier. Performance Indicator: AUC-ROC Curve Quantitative in Outcome Other Information: Ontological Statement: check performance of multi - class classification AUROC (Area Under the Receiver Operating Characteristics) curve.Ontological Statement: Area Under Curve(AUC) is one of the most widely used metrics for evaluation. It is used for binary classification problem. AUC of a classifier is equal to the probability that the classifier will rank a randomly chosen positive example higher than a randomly chosen negative example. True Positive Rate (Sensitivity) : True Positive Rate is defined as TP/ (FN+TP). True Positive Rate corresponds to the proportion of positive data points that are correctly considered as positive, with respect to all positive data points. False Positive Rate (Specificity) : False Positive Rate is defined as FP / (FP+TN). False Positive Rate corresponds to the proportion of negative data points that are mistakenly considered as positive, with respect to all negative data points. Performance Indicator: F-measure Quantitative in Outcome Other Information: F1 Score is the Harmonic Mean between precision and recall. Ontological Statement: F-measure represents both Precision and Recall it helps to have a measurement that represents both of them. F-measure is calculated using Harmonic Mean (in place of Arithmetic Mean). Ontological Statement: Mean Absolute Error is the average of the difference between the Original Values and the Predicted Values. It gives us the measure of how far the predictions were from the actual output. Ontological Statement: Mean Squared Error(MSE) takes the average of the square of the difference between the original values and the predicted values. Performance Indicator: NDCG Quantitative in Outcome Other Information: Ontological Statement: Normalized discounted cumulative gain (DCG) is a measure of ranking quality. In information retrieval, DCG measures the usefulness, or gain, of a document based on its position in the result list. Performance Indicator: Regression Analysis Quantitative in Outcome Other Information: Root Mean Square Error (RMSE) Ontological Statement: Root Mean Square Error (RMSE) is the standard deviation of the residuals (prediction errors). Residuals are a measure of how far from the regression line data points are; RMSE is a measure of how spread out these residuals are. Performance Indicator: Quantiles of Errors Quantitative in Outcome Other Information: Quantiles (or percentiles), which is the element of a set that is larger than half of the set, and smaller than the other half. Performance Indicator: "Almost correct" predictions Quantitative in Outcome ------------------------------ Goal: Lawful Ensure AI Systems comply with all applicable laws and regulations, such as, provision audit data defined by a governance operating model ------------------------------ Goal: Ontological Statements Employ ontological statements when explaining AIKR object audit data, veracity facts and (human, social and technology) risk mitigation factors ------------------------------ Goal: Track Track AIKR object performance outcome via KPI (Key Performance Indicator) based on supervised learning models measurements ------------------------------ Goal: Document Document the vision, values, goals, objectives for one or more AIKR objects ------------------------------ Goal: Robust Ensure AI Systems are designed to handle uncertainty and tolerate perturbation from a likely threat perspective, such as, design considerations incorporate human, social and technology risk factors Carl It was a pleasure to clarify On Thu, May 28, 2020 at 1:22 AM StratNavApp.com <mail@stratnavapp.com> wrote: > Here is an update on your projects on StratNavApp > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/mTDzGfBLX2fVW9_RJaXU23JLd64bcuSrKcq7SlYzh-ZWy_yjKDgI6uv3y4ZNml3aSdmdP0PGZOiV3y2cFub_oVHoiUWiYl7oNeWX3biAklgL9hDL8i3InMw-aXlNtwMwHCIaeyP42T7DMJ_Q4JgtltyObIgTLFCkiTN3ygga0px55idzmntCffrSIxyCrQ> > : > > To view, update, comment on or respond to any of these, please click on > the *View* link next to it, and edit the item or add a note of your own - > not by replying to this email. > > For stakeholder: Paul Alagna view/contribute > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/UTmoLahXS6sWrh9M10hPfRHA78eJu3YTXsXL3MWMMGHrJNgdYBnhTBQVw6beShnXpP80ExcH7HreHrxMSZY2xcIO5pHybo6E6KZNDcAJjUr5I-9aGL6LB8bOiZ8B0kW232rRgzzaJaaHm_732wZPWX2CdazGYHzcSFl-weEHfGG0Lkau_ChMp_1ZGYbpViFFDE7hViOZQjMbzLBOeCRfGSXkxOCw1nO4baTeWIcUPCgqQ9hMRN9Lb8V8OznxZHHEE6XMY5WcDf-4TEUERfJ5M4UyhJpdBbet0cVJQck23_QLOaJWXLTZIw4bQnI13IpJf1ay42mK> > > *In project: StratML for AIKR* > On 27/05/2020 at 20:25, *Paul Alagna* > > - added the phone number: > 7323225641 > - changed the email address: > pjalagna@gmailPJAlagna@Gmail.com > - added the mobile number: > 7323225641 > > For goal: AI Strategists view/contribute > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/tNE2YzBlu_5_eS3O-PMBZz-8oj7VfvELilZ3NutUSp_3V8K5hZ--ptAOa6-cFv4sgBcSfs_xqeK73OOz1NmnlcGHd8DKJc52NXWrfbAyCzSfJM3pQiIEcaX9d0GKs8rYXltWkS-H6nEPUZ5QiAqXU_E3aylNQUl5VoqN3ZFwWhznwU46wFsYDJ7LV1QX5qvCObC14AzCrC5odE0x0IuBO3IsNDuYEC_iY3aPjGauSIwBGUrH8kICyru2ZgCu13ElYxYfe1VEuus0UEh7f30vL5mNd_M3XDIIYOYH7M1ueQroCqLoSkXYAGHZW6X8RxQ> > > *In project: StratML for AIKR* > On 27/05/2020 at 06:47, *Chris Fox* wrote: > > We now have two attempts to define the role of the AI Strategist in > StratML format. > > - > https://www.stratnavapp.com/StratML/Part1/d95875f4-04b2-46d3-a953-c441e16f428a/Styled > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/2TmjD4g955eAY6vZvU3g9IpqXPnqj96RjJx6NDp_sembIPLKi8RCpcFvGGFQlvYAV-jIw4Ew6VGm5bY5_iBtH_6T078jxgxW4SDGtRm6k4We4Ie631jbm-0xmbJmYCtjRIn92sKK0X_aRB8lHWb2-QuvGNge2RifceIEBlINT_w1_hMGHdWS7U4fuSTJJpN9-f8VLhF080WLG9KunTfK6ginqD-rrJreBGtUAISDXD8uj53jdhYBYB_obIz6sqccs8y8eCSrk5xa0pHGcGgB8w> > and > - this one > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/taFcSAQX8zFh1qLhKNr0swPkg0skuWqbAIB3zURLv5ze7eu6730cqT_jvC51DIs6jz4UTRYi8D6q4PSSvyBEpYF9YrjF93dIGVtE1hPUXZCf8depw-sL9wnjb9yCO_yL42yPyGFxRbs2NX_TGl6Zp6EDCFG87lCUei8UgFiS2e9EK4c4i7MyW7HRiMSGKrdatf2m2znyUQNBOZCsPTkHZ1gJLQqcGY3NloAcfduNGF_yExpki70vkRSQCNT5CcP3> > . > > I think if we are to say we have achieved this goal, we need to work > towards a single one. > > For project: Roles of AI Strategists view/contribute > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/keuPPmZdSKiP4rrR2CRClMimi7kpXjg8byT1ax0w6K3wHfYiVovZR9T2ShMX5s6WBJaDd2vSSyQlxWxMqRZVgQwLxtzXXrqRPOulZBlDm0EbIRzK5IzebC8r1aCbyWA3kDmKaX4sOcSEClO9kTaRx4PV2XlRNuq-nnfBSKf-pp0OticPrpz53w4WQ0kUzLIYpxqOZNsC8RO6TZeGEIjASwX1-n5M99vISrX9W1sx91iQE0Qm2GgGlgRCAyIKZJRowhRQsCNG> > > *In project: Roles of AI Strategists* > On 27/05/2020 at 06:38, *chris@chriscfox.com <chris@chriscfox.com>* > wrote: > > We now have two StratML plans for the role of the AI Strategist. This one, > and the one we previously discussed at > https://www.stratnavapp.com/StratML/Part2/861566c8-e9be-4642-b52f-f673fa499f4e/Styled > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/rEsV5jT4sswXrPTTNfeC6nDjtm-nY1ZIrbl-LZSSKUkUYZgjnbTv8Udz3_YAbqwqUVZN_Mh-3G1zCay-QQbmAsIiYP6DYRPQ-gHFsm5Onb8Xu6c7fTMAyYa1jfbH_uQG3qOf0p7m2kbD7BomBZ8yw2ziX7qxU7mHkQZpdgBzER8fLV30znD2wfdD9K_Y1ANHX7NkMFXFwAQKoHCD7YHDD6uoVfsVF69ud0Ng7AUmt8FAnsV3Q5sx6TZ74Ta_vL9klYIGl8iJD0V01kG3Pf-6AQ> > > I think that the object should be to merge the two so that we only have > one. > > You are receiving this message because you are subscribed to > StratNavApp.com. To stop receive messages from this service, simply click > unsubscribe > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/7VznTxnjcaFTe-OFuiUnRpOe0RzF7lHBNHzChR4pAn6SdNsdPibwOWp4NHH9SbZGz55AxBYv3dZyh7RcUij1l3DOmoNMhE8MjSxetoCyIvOcaK863XiG53FsBmhTRL7nlAqr6wzZVfERLkgUiOXfwEdh56-LS-4E_Pm8FOQn3I0Fyo8Cdq1hTCSmMCAnyH0mqxSxgcUOjzXd5qz9bclP3Xq8wvgxEFy4lmfxlQkNV5cK_pXUFdbYDdJ01z2YAPrMMUzNFFpW2xC93yz7Oe8dm4pIqzBmYGgKw_egYbTQvL-YvVpnftjmrx9L3Dpg50dRyjQ0ejai0n9v8Z2A5L2ML72u1fAZ2PtqDQ> > or change your subscription options > <https://chfbdgg.r.af.d.sendibt2.com/tr/cl/0FKqn1-Wo0Z3HlNcGtcfMAsCazzAR17kiFAvzkx6zJVzhO1glbWgapcDW6fAi2YDIelrN1PYlfPYx2uySISI_5AAvvAJTN3s8hlQjI_ipPZl6noPx8HBK-IWbIMug2FaVmXZqeE44xohiwRCByynHH--oRbrtivj1phfr3kaKD_ragVLuVFb1rdQwtz73iiAEF2Mvzdck5aASyOosDokSmsxFJNg7DisniR4Kv4> > . > > > Copyright Chris C Fox Consulting Limited. Chris C Fox Consulting Limited > is registered in England and Wales as a Private Limited Company: Company > Number 6939359. Registered Office: Unit 4 Vista Place, Coy Pond Business > Park, Ingworth Road, Poole BH12 1JY >
Received on Thursday, 28 May 2020 13:17:51 UTC