Re: Solomon''s curse and search Bias

Here is my  take based on the same problem I encounter every day when searching for information that deals with sustainable development AND other issues.
Western search engines like Google and Bing and Yahoo Search use algorithms like PageRank, which use a one size fits all approach as a starting point and then use user search history and location to adapt both search results and advertising to more or less suit the user's needs.
There are several approaches possible to search.
(1) the approach used by Google and its western competitors(2) the approach used by Chinese search engines which shamelessly and without any concern for privacy and digital rights uses deep learning and massive amounts of data and data gleaned thru webcam cameras from the eye movements of users to create user profiles that center on geographical location and demographics to suit user needs, mainly for online shopping and other established prioritized user activity when browsing and searching(3) a brute force approach which somehow has to organize all the results into categories, e.g. the librarian category folders, regardless of user location and demographics, e.g. as promoted by the General Data Protection Regulation of the European Union(4) a brute force approach with category folders but with user location and demographics, which we are able to input to an input form similar to those for acceptance of cookies(5) an AI approach which uses a form to let the user allow for user location and demographics use and creates a popup dashboard which we can continuously tweak and modify. This has several advantages. The first and foremost to let the user allow the AI to heuristically prune the research results and also offers the user to see both the results with or without pruning. This approach will by default use category folders.
The last three approaches will substantially reduce the need for deep learning and massive amounts of data gathered without consensus or worse, without  permission, are in compliance with the GDPR and create a consensus based interactive AI approach in online search.
In my humble opinion options 4 and 5 are the ways to go. Milton Ponson
GSM: +297 747 8280
PO Box 1154, Oranjestad
Aruba, Dutch Caribbean
Project Paradigm: Bringing the ICT tools for sustainable development to all stakeholders worldwide through collaborative research on applied mathematics, advanced modeling, software and standards development 

    On Monday, March 4, 2019 10:01 AM, Thomas Passin <tpassin@tompassin.net> wrote:
 

 On 3/4/2019 6:54 AM, Paola Di Maio wrote:
> Sherman and Thomas P
> thanks a lot for sharing your search journey with friendly narrative and
> about this project. Looks really neat!
> 
> I am still thinking about the need to carry ot some structured data 
> search on open web via  general search engines tho.  I hope Google may 
> consider adopting your architecture or at least some of these ideas 
> which in principe work at least within a closed data set,

The kinds of things you are trying to explore can also be approached 
using a framework of library science, and specifically the concepts of 
"navigation" and "collocation".  Collocation means that items that are 
somehow similar can be found "near" each other in some sense of the word 
"near".

For a look an an attempt to provide good navigation and collocation for 
a small private data set, you could look at my paper "Browser bookmark 
management with Topic Maps" for the Extreme Markup Languages conferences 
from 2003 -

http://conferences.idealliance.org/extreme/html/2003/Passin01/EML2003Passin01.html

This work was my effort to get the most out of very limited amount of 
information, namely, titles of web pages. The viewpoint behind the work 
derived from the library science concepts of collocation, navigation, 
and "subject language".  To quote from my paper: "A subject language is 
a vocabulary for describing the subject of a work in such a way that it 
can be recognized and thereby found — that is, to provide navigation 
capability for a collection."

Another point to keep in mind is that a good user interface must be very 
different for a small project as contrasted with one that encompasses a 
large collection of data.  Just for a tiny example, a pick list for ten 
items can be usable whereas one for 10,000 items is not.  A UI for a 
large data set is hard to design even if your system has good 
collocation and navigation properties.

TomP