Re: Connectionists vs Reductionists, Hard vs. Soft or Hard + Soft KR, Re: neural networks being purported as KR?

Stephen and others on this thread

thanks for the in depth exposition and insights, apologies for my brief
answer but
I am on multiple deadlines

I know what is a concept <g>  we all do, although it can be helpful to
review-
but a non concept? Have you figured that one out?

I recently heard of 'negative knowledge' Inot something I had heard of
before- must be that?

 have also seen that quite a lot of new research in AI is completely mind
wrecking (for lack of better words), from the cognitive point of view, and
on top of that, it does not need to be peer reviewed or published in
journals to catch like wildfire in the AI community * I promise myself I
must go to one of these conferences to look at these people in the eye
*shivers peer reviews are becoming ''irrelevant" in some domains like AI

So, while it is great to see rapid evolution of ' new knowledge''  and who
cares whether its peer reviewed, it is a bit worrying
if not only (some of) it does not make any sense, it is also completely
false, not validated
and ridiculous and as I see it, a threat to the future of humanity and to
sanity
I may be wrong.

Today I learn what looks like a relatively novel concept called
Differentiable Logic, (look it up if interested) this is also entirely new
to me and tickling my imagination, simoultaneosly  triggering the red
button

Milton: I post to these lists (apologies for cross posting) as a form of
message in the bottle

Through the SW and Ontology open web lists like these I have met and
exchanged with folks who have been
in this space for decades, who may be able to contribute constructive
insights into the state of the
art in AI, logic etc etc.  I wonder if anyone else sees what I see and gets
worried,

Thanks for the views, will send more rational emails when the anxiety
subsides

PDM


On Tue, Jul 30, 2019 at 1:25 AM Stephen D. Williams <sdw@lig.net> wrote:

> Hans,
>
> Yes: I think for most people most of the time, defining a concept always
> results in paragraphs of natural language.  This implies that, beyond
> has-a, is-a, etc., it is difficult to express the ideas of a concept
> without relying on the complex web of understanding of other concepts.  And
> that, beyond ontologies like foaf, we don't have enough shared context of
> our webs of understanding to express many concepts in a definitive or
> directly machine understandable way.
>
> If anyone has a better definition and grounding for the concept of
> concepts, please enlighten me.  It is difficult to research completely
> based on such a common and widely used word.
>
> Part of my thinking with the concept morphology thinking is that it is
> helpful to factor out as much concrete data as possible from a concept,
> clarifying what part is simple data relative to the understanding & web of
> understanding references.  I was looking at this from a human training
> point of view, although i think it is instructive from the KR / KU
> (knowledge understanding) points of view: We are using 3D gaming to enhance
> training, making concepts more memorable. [1] My question: Can an adaptive,
> templatized, polymorphic game chassis adapt, in one or more senses, to
> different concepts.  For concepts where the concept itself is easily
> understood and remembered and it is the data that is difficult to remember,
> and where that data has a morphology similar to other concepts, the answer
> is yes.  My go-to example is concept morphologies such as 'hierarchy":
> Maslow's Hierarchy of needs has a similar form as recommended food groups
> or levels of architecture.  Although they are completely different
> concepts, the hard to remember parts are data that can be captured in the
> same simple way.  I a sense, what is left is a concentration of the
> concept, the extract of knowledge.  For example, understanding the concepts
> of a movie, actors plots, directors, plot, etc. is separate from a
> knowledgebase of actual movies, actors, directors, and plots.  Although the
> knowledge that the concept extract relies on may seem infinite, the concept
> extract itself (What is a movie?) is very finite and usually static while
> the concept data (What are all movies?) may be large and always growing.
>
> The tricky bit is representing or even referring to the essence of a
> concept, this concept extract, separate from its data.  I can see several
> ways this is and can be done, but nothing seems satisfyingly complete:
> Simpler math & logic concepts are directly representable and sometimes even
> derivable although more complex concepts seem less so, but are far removed
> from most concepts.  Relational operators, including things like Allen's
> Interval Algebra can be built into a system as fundamental and highly
> reusable features.  But nothing explicit can represent more than a small
> range of concept building blocks, except in the most elementary and shallow
> way as in semantic web triples, perhaps used by first order logic or
> similar implication systems: Implication logic systems, even with the
> addition of probabilistic graph reasoning, only implement a subset of the
> types of knowing and understanding.  Efforts to build large, complete
> triplestores, ontologies, and encyclopedia knowledge graphs are useful in
> certain ways now and may become very useful when we find ways to build more
> understanding depth, but uses now are usually just database queries.
>
> There are a lot of ways to think about AI / ML et al.  One of the most
> useful to me is that an AI / ML system exists and is effective if it can
> predict accurate results based on inputs and its internal KR.  Next levels
> include predicting sequences and time based events, learning and
> remembering on the fly, and various types of deeper reasoning, but not just
> apparent reasoning that turns out to be less-useful a shallow correlation.
>
> But, basically, an AI/ML system is a prediction machine for something more
> complex, probabilistic, and/or noise tolerant than a solution built from
> Algorithms book algorithms.
>
> As humans, we find it satisfying to capture explicit knowledge in more
> broadly applicable ways, such as knowledge graphs.  There are many ways we
> can make that useful, especially with a human in the loop doing searches
> etc.  But we continue to have a gap between that kind of KR and a useful
> and automatic prediction machine.
>
> Symbolic / logical / computationalist / reductionist / bottom-up (where
> the bottom is usually symbolic statements) / mostly deterministic
> vs. connectionist / analogical / holistic (global, sort of top-down) /
> emergent (bottom up from a neuron-like element) / mostly probabilistic /
> learning:
>
> There is a lot to be said about these alternatives.  I think there is
> value in both / all.  Any real system is going to have elements of both,
> perhaps in various senses, just like we ourselves do.
>
> Some interesting related links, although a couple of them seem very
> obsolete:
> https://www.simplypsychology.org/reductionism-holism.html
> https://psychology.wikia.org/wiki/Connectionism
> https://www.jstor.org/stable/27759324?seq=1#page_scan_tab_contents
> Connectionism, Reduction, and Multiple Realizability
> https://web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html
> https://journals.sagepub.com/doi/abs/10.1177/092137408800100102
> Reductionism, Connectionism, and the Plasticity of Human Consciousness
> https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2395673  Who's Afraid
> of Multiple Realizability?: Functionalism, Reductionism, and Connectionism
>
>
> The exciting thing about ML related techniques is that they can already
> perform an interesting range of actual, useful prediction, in many cases
> from highly noisy, messy raw data all the way to final decisions and
> actions.  We see the limitations, so far, of incorporating, leveraging, or
> an alternative for the KR in a number of interesting KBs.  But some ML
> successes are encroaching on that territory in impressive ways.  There is
> nothing definitive that prevents us from expecting that to continue to
> grow.  I expect that soon there will be an interesting meeting of
>
>
> [1] Yebo (formerly Change My Path) https://cped.co/d
>
>
> Stephen
>
> On 7/29/19 4:15 AM, hans.teijgeler@quicknet.nl wrote:
>
> Stephen,
>
>
>
> Isn’t it so that, no matter how elementary you go, you always end up with
> an explanation in a natural language of those information elements?
>
>
>
> Regards,
>
> Hans
>
> 15926.org <http://data.15926.org/>
>
> ______________________________________________--
>
> *From:* Stephen D. Williams <sdw@lig.net> <sdw@lig.net>
> *Sent:* maandag 29 juli 2019 12:04
> *To:* paoladimaio10@googlemail.com
> *Cc:* SW-forum <semantic-web@w3.org> <semantic-web@w3.org>
> *Subject:* Re: Hard vs. Soft or Hard + Soft KR, Re: neural networks being
> purported as KR?
>
>
>
> Paola,
>
>
>
> As far as I've been able to determine, there is no universal definition of
> concept so I find the concept of non-concept to be weakly defined at best.
>
>
>
> I think 'concept' has a granularity of meaning relative to a particular
> context, usually more than a triple and less than a large knowledge graph.
> I intuitively and practically define a concept as a set of elementary
> statements, facts, or reactions that can be used for prediction or action
> in a usually cohesive way somewhat separate from other cohesive facts.
> Good examples would be each mathematical operator, rule, or complete
> technique.  The actual size, in whatever measure, is secondary to the
> cohesiveness and usual granularity of use.
>
>
>
> Somewhat orthogonal to this, my research a few years ago led me to an area
> that doesn't seem to have been explored:
>
> The morphology of concepts, or rather the morphology of capturing most or
> all of the essence of concepts: There is a somewhat finite set of
> morphologies that can capture the essence of most concepts.  This is
> distinctly a different idea than the seeming (pre-ML boom) universality of
> triple graphs.
>
>
>
> Paola, I think you can see a set of triples or logic clauses as being able
> to represent a concept but are having trouble seeing an equivalence with a
> trained connectionist network.  While, without visualization or similar
> extensions, it may be difficult to identify the cohesive locality of
> concepts in a ML network, I don't have so much difficulty in imagining that
> they are there, interlocking and perhaps haphazardly related.  They can
> still be active in just the same ways, just only indirectly visible.
> (There are some good examples of building in visualization that sometimes
> largely indicates what is going on.)
>
>
>
> I think examples could be useful.  Imagine we are trying to capture the
> logic of traffic and walk lights, sensors, and weekly schedules in a
> system.  We could represent all of the states, events, and rules in first
> order logic, some equivalent set of triples, direct coding in software.  Or
> we could hand design a logic circuit, timer chips (the 555 TTL etc.),
> relays, flipflops, etc.  Or instead of designing such a circuit, we could
> assemble more than enough sensors, logic circuits, connection wires, signal
> lights, etc., just trying every combination in brute force but with no
> logic or intelligence until we produced a circuit that embodied the logic
> needed.
>
>
>
> Or, we could connect all of the sensors, switches, and add some test
> signals that indicate correct operation (also needed for the brute force
> solution and useful for testing the logic-based systems) and allow a ML
> system to converge on a working system through training on the error
> signal.  Both the brute force circuit search and the ML result embody the
> same logic, the concepts in the explicitly logical approaches, even if they
> seem somewhat opaque.  In a more complex situation, requiring internal or
> intermediate results that apply in different ways to many other cases,
> finding a solution with brute force may take a very, very long time to
> find.  ML methods have been evolving to handle more depth and nuance in
> various ways.
>
>
>
> Stephen
>
>
>
> On 7/29/19 1:52 AM, Paola Di Maio wrote:
>
> Stephen
>
>
>
> thank you, I am thinking along those lines and we could have a discussion
> about
>
> hard vs soft KR
>
>
>
> My challenge today is to find a reference for ''non concept'
>
> (as used here  Concept Recognition with Convolutional Neural Networks to
> Optimize Keyphrase Extraction Andreas Waldis , Luca Mazzola(B) , and
> Michael Kaufmann School of Information Technology, Lucerne University of
> Applied Sciences, 6343 Rotkreuz)
>
>
>
> The notion of non concept seems to be increasingly used in ANN to contrast
>
> 'the notion of concept in traditional KR,
>
>
>
> But I cannot find out if this has been formally defined *I doubt it?
>
>
>
> So no only a lot of peer reviewed scholarly articles make false statements
>
> (say things which are not true) but also reference notions which have never
>
> been formally defined (most of this vague research seems to be happening
> in Switzerland)
>
>
>
> This is not science, is it?
>
>
>
> Together with other things I am not figuring out, I tind this unsettling
> and adding
>
> to uncertainties
>
>
>
>
>
> PDM
>
>
>
> On Mon, Jul 29, 2019 at 11:51 AM Stephen D. Williams <sdw@lig.net> wrote:
>
> Thanks, great info!
>
>
>
> Knowledge graphs, semantic web, and other explicit knowledge
> representations are important.  But, as you point out, they currently don't
> capture everything, especially not probability and complex rational but not
> simple logic relationships.  ML and related enabled training, capture, and
> usage of this aspect of information, but most or all structure is opaquely
> inferred and usually shallow.  Somehow we need all of the capabilities of
> both in a merged or married structure.  I totally believe in the value of
> both.
>
>
>
> My observation is that we almost certainly reason with structure, logic,
> math in general, language, etc. on top of ML-like mechanisms (albeit far
> more varied, numerous, and expansive), and we fill in the gaps all the time
> with ML-like sub-rational or at least sub-logical reasoning.  Most of us
> most of the time reason largely in this sub-logical reasoning mode, often
> interpreting, viewing, or fitting those thoughts and conclusions to formal
> structures afterwards or in a trailing overlapping fashion.
>
>
>
> https://www.ted.com/playlists/384/how_your_brain_constructs_real
>
>
>
> As a software & security architect and now robotic hardware designer, I
> never think in formal equations or systems, yet I automatically explore
> creative solution spaces with many, many constraints and patterns in mind.
> I can often construct equations of various types to summarize and
> communicate.  I now find this to be true of business and interpersonal
> situations (various concerns, constraints, interests, conflicting aspects)
> and mechanical systems.  For instance, this is a summary of part of the
> robotic actuator I've been working on:
>
>
>
> I1 * r/R * CVT - I2 * R/2 = RIVT
>
>
>
> I never could have found that solution through explicit logical deduction,
> yet the process of working through and mentally testing possibilities is
> very concrete and not mysterious at all.  How do you experience your
> knowledge, thinking, problem solving, and creativity?  Do you feel strongly
> and deeply aware of it or is it mysterious and opaque?  Do you visualize a
> little, a lot, or not at all?  (We're recently realized that something like
> 10% of people cannot visualize anything, ever, yet a number of animators
> are among those!  They seem to use drawing as external working memory /
> thought space.)
>
>
>
> We already have ML solutions that are able to learn far more than simple
> table lookups.  Creating a program to play a video game using strictly
> explicit logic, KG methods would be challenging while we have straight
> foward ML solutions already:
>
>
>
>
> https://thenextweb.com/artificial-intelligence/2018/08/23/researchers-gave-ai-curiosity-and-it-played-video-games-all-day/
>
>
> https://towardsdatascience.com/an-exploration-of-neural-networks-playing-video-games-3910dcee8e4a
>
>
>
> It seems promising to extrapolate from these for some distance.  Our
> challenge is to find a intersection method that allows us to combine these
> two mostly independent paths to create really interesting solutions.  My
> sense is that there are two possibilities:
>
>
>
> We get an interface method working that allows an ML system to access
> computational algorithms / KG / semantic / logical knowledge through a kind
> of port, portal, map, or similar.
>
>
>
> Or that we evolve the ML systems to the point where they can embody, in a
> reasonably efficient and accurate way, this computational / KG / semantic /
> logical knowledge along with the more probabilistic, analog networked
> knowledge.
>
>
>
> Or both.
>
>
>
>
>
> Rather than connectionist vs. reductionist etc., perhaps these are useful
> labels:
>
>
>
> Hard knowledge: computational algorithms / KG / semantic / logical
> knowledge
>
>
>
> Soft Knowledge: ML, vision, everything connectionist or probabilistic
>
>
>
> Or sharp vs. round or curved or smooth.
>
>
>
> Stephen
>
>
>
> On 7/26/19 9:02 AM, ProjectParadigm-ICT-Program wrote:
>
> Thank you Paola for pointing this out.
>
>
>
> Again I must beat the drum.
>
>
>
> Knowledge is much more than extracting structure from facts and data. If I
> just recall that the collection of facts is subject to the uncertainty
> principle, any structure deduced cannot be complete, and the application of
> free will, and/or axiom of choice create a dichotomy, knowledge is much
> more.
>
>
>
> We are limited by our sensory apparatus, our hard wiring in our human
> brain, including the shortcuts made when processing visual data, and the
> limitations of natural language.
>
>
>
> I agree that knowledge reasoning should be fairly straightforward, but
> making the jump from KR to knowledge itself implies we come up with some
> consistent many worlds modeling scheme in which the virtual, mathematical
> and (many interpretations of) the physical world coexist, reconciling
> incompleteness, uncertainty principle, sensory limitations and application
> of free will and choice.
>
>
>
> A convergence of efforts by string theorists, researchers in human brain
> cognitive and biological structure fields, theoretical physicists and
> mathematicians working on finite groups, category theory, algebraic
> topology and logical structures for consistent super theories, and an odd
> mix of linguists and philosophers (including Buddhists) is doing just that.
>
>
>
> But they are far from a consensus.
>
>
>
> The point I am trying to make is that KR is more than semantics and
> ontologies and knowledge graphs, graphs, category theory diagrams and
> Feynmann diagrams and any other visualization tools we use.
>
>
>
> The implicate order David Bohm theoreticized underlying quantum reality
> and the reality of our physical world, cannot be captured by some mix of
> formal logic, semantic structures, ontologies or computable frameworks.
>
>
>
> And we we want someday A(G)I to be able to grasp human knowledge in
> general, we must create a growth path towards formal structures which have
> meta-layers above (knowledge) graphs, formal logic and ontologies.
>
>
>
> Mathematically speaking, using formal logic, ontologies and generalized
> graphs is necessary but insufficient for this general formal structure.
>
>
>
> And now I must add that deep learning and machine learning also fall short
> in terms of KR'
>
>
>
> If we let computer scientists, logicians, mathematicians and software
> engineers try to come up with KR which is fit for the AI we envision we
> will need for future applications we will fail miserably.
>
>
>
> We need neuroscientist, and specialists in the field of cognitive
> sciences, biologists and even psychologists, and philosophers and
> physicists to help us complete the general framework for knowledge, and to
> establish which parts can be effectively captured in a formal fashion,
> which provide suitable technologies and tools for KR.
>
>
>
> Mike Bergman did a nice expose on knowledge graphs at:
>
> A Common Sense View of Knowledge Graphs
> <http://www.mkbergman.com/2244/a-common-sense-view-of-knowledge-graphs/>
>
>
>
>
>
> A Common Sense View of Knowledge Graphs
>
> This article, based on a comprehensive history and definitions of the
> concept, provides a common-sense view of h...
>
>
>
> But historically even mandalas qualify as knowledge graphs, in a very
> stylized way. And they can be used to visualize very complex mathematical
> structures without the use of edges or arrows, thus removing the time
> component associated with the transition the edge or arrow represents,
> making knowledge representation in a time-independent fashion possible.
>
>
>
>
>
>
>
> Milton Ponson
> GSM: +297 747 8280
> PO Box 1154, Oranjestad
> Aruba, Dutch Caribbean
> *Project Paradigm*: Bringing the ICT tools for sustainable development to
> all stakeholders worldwide through collaborative research on applied
> mathematics, advanced modeling, software and standards development
>
>
>
>
>
> On Thursday, July 25, 2019, 11:52:23 PM ADT, Paola Di Maio
> <paola.dimaio@gmail.com> <paola.dimaio@gmail.com> wrote:
>
>
>
>
>
> Sorry to bang on this topic, but its the task at hand at the moment
>
>
>
> I just found an article, which is good scientific survey then  purports NN
> as a type of KR
>
> (casually sneaks in NN as the latest KR)
>
>
>
> This is published in a Springer peer reviewed publication and my makes all
> of my hairs stand up on my head
>
>
>
> This is the kind of rubbish that without further qualification is being
> passed down
>
> as the latest research, and  which the future generations of AI scientists
> are being fed-
>
>
> wonder if anyone else has a problem with this proposition
>
> (sign of the times?)
>
> I am doing my best within my means to identify and contain this peril
>
>
>
> Article
> https://link-springer-com.nls.idm.oclc.org/article/10.1007/s00170-018-2433-8
>
>
> A survey of knowledge representation methods and applications in machining
> process planning
>
> The machining process is the act of preparing the detailed operating
> instructions for changing an engineering design into an end product, which
> involves the removal of material from the part. Today, machining ...
>
> Xiuling Li, Shusheng Zhang, Rui Huang… in The International Journal of
> Advanced Manu… (2018)
>
>
>
>
>
>
>
>
>
> --
>
>
> *Stephen D. Williams*
> Founder, Yebo, VolksDroid, Blue Scholar
>
> 650-450-8649 | fax:703-995-0407 | sdw@lg.net <sdw@lig.net> |
> https://HelloYebo.com | https://VolksDroid.org | https://BlueScholar.com |
> https://sdw.st/in
>
>
>
> --
>
>
> *Stephen D. Williams*
> Founder, Yebo, VolksDroid, Blue Scholar
>
> 650-450-8649 | fax:703-995-0407 | sdw@lg.net <sdw@lig.net> |
> https://HelloYebo.com | https://VolksDroid.org | https://BlueScholar.com |
> https://sdw.st/in
>
>
>
>
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virusvrij.
> www.avg.com
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> <#m_1039863845649598750_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
>
> --
>
> *Stephen D. Williams*
> Founder, Yebo, VolksDroid, Blue Scholar
> 650-450-8649 | fax:703-995-0407 | sdw@lg.net <sdw@lig.net> |
> https://HelloYebo.com | https://VolksDroid.org | https://BlueScholar.com |
> https://sdw.st/in
>