Re: Carbon Efficiency of Semantic Web and Linked Data Queries

Zach, thanks for your moderating input.

Marcos, here’s another relevant industry resource on physical/digital
infrastructure/capacity/resource consumption relevant to deducing carbon
impacts of digital processing/networking/edge-end user utilization
(US-only):

https://www.osti.gov/servlets/purl/1372902/

Presumably, if you frame the problem on a macro-level, combined with
available carbon consumption calculators for the micro-level, you’ll be
somewhere in the playing field to begin to deduce/ballpark SemWeb as a
fraction of total digital processing/display/etc in terms of energy /
resource consumption and carbon footprint.

Hope this helps.

Cheers,
—David

On Thu, Jun 20, 2019 at 11:04 AM Marco Neumann <marco.neumann@gmail.com>
wrote:

> I'd agree Zachary, but thank you for chipping in David these are still
> interesting links in general and will come in handy elsewhere.
>
>
> On Thu, Jun 20, 2019 at 3:20 PM Zachary Whitley <zachary.whitley@gmail.com>
> wrote:
>
>> This is way outside of any discussion that can reasonably be related to
>> semantic web technologies. Marco's original question was about carbon
>> efficiency and semantic web and linked data queries.
>>
>>
>>
>>
>> On Thu, Jun 20, 2019 at 8:42 AM David McDonell <david@iconicloud.com>
>> wrote:
>>
>>> Maybe add a carbon tax to bitcoin (and Libra) to correct currency value?
>>>
>>>
>>> https://www.economist.com/the-economist-explains/2018/07/09/why-bitcoin-uses-so-much-energy
>>>
>>>
>>> On Thu, Jun 20, 2019 at 8:16 AM David McDonell <david@iconicloud.com>
>>> wrote:
>>>
>>>> Don’t know about that.
>>>>
>>>> Here’s a report that adds more detail:
>>>>
>>>> https://eta.lbl.gov/publications/united-states-data-center-energy
>>>>
>>>>
>>>> On Thu, Jun 20, 2019 at 5:29 AM Marco Neumann <marco.neumann@gmail.com>
>>>> wrote:
>>>>
>>>>> now double that with libra. Is there a power estimation tool (like
>>>>> PowerTop for linux) that converts generalized compute cycles into power
>>>>> consumption or vice versa for bitcoin or are these numbers just based on
>>>>> estimated up-time and power usage of data centers?
>>>>>
>>>>>
>>>>> On Thu, Jun 20, 2019 at 12:54 AM Henry Story <henry.story@bblfish.net>
>>>>> wrote:
>>>>>
>>>>>> Bitcoin itself uses up 67.3 TWh, more than Switzerland and less than
>>>>>> the Czech Republic
>>>>>> according to this page
>>>>>>    https://digiconomist.net/bitcoin-energy-consumption
>>>>>>
>>>>>> Trying to reach global consensus is expensive.
>>>>>> Linked Data allows local consensus, which is much cheaper.
>>>>>>
>>>>>>
>>>>>> On 19 Jun 2019, at 23:09, David McDonell <david@iconicloud.com>
>>>>>> wrote:
>>>>>>
>>>>>> And here’s a frothy commercial sector industry report on data center
>>>>>> concentration (including AWS) in N.Virginia (DC metro area) citing MWattage
>>>>>> consumption numbers, drawn from standard grid sources:
>>>>>>
>>>>>>
>>>>>> https://www.datacenterknowledge.com/amazon/why-northern-virginia-data-center-market-bigger-most-realize
>>>>>>
>>>>>> Point is, carbon efficiency has to address the backbone
>>>>>> infrastructure dimension; edge/end-user profiles are feel-good but dwarfed
>>>>>> in comparison.
>>>>>>
>>>>>> On Wed, Jun 19, 2019 at 4:57 PM David McDonell <david@iconicloud.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I think those latter three G-locations have abundant nuke power from
>>>>>>> the ‘local’ grid; whole different set of issues there;-)
>>>>>>>
>>>>>>> On Wed, Jun 19, 2019 at 12:06 PM Marco Neumann <
>>>>>>> marco.neumann@gmail.com> wrote:
>>>>>>>
>>>>>>>> I like the way Google is going almost carbon neutral here in Hamina
>>>>>>>> Finland by way of using cold seawater to cool systems. I hope they will
>>>>>>>> also hook up the onsite sauna* to use excess HPC heat soon ;)
>>>>>>>>
>>>>>>>> I am still surprised they continue to run supercomputer clusters in
>>>>>>>> places like Texas (Frontera), Tennessee (Summit) and Livermore, CA (Sierra)
>>>>>>>>
>>>>>>>>
>>>>>>>> https://medium.com/arcticstartup-news/saunas-to-use-data-centres-excess-heat-c552e70946b
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 19, 2019 at 2:17 PM David McDonell <
>>>>>>>> david@iconicloud.com> wrote:
>>>>>>>>
>>>>>>>>> Thought this might be of relevance to the discussion, re global
>>>>>>>>> data infrastructures (from my LinkedIn feed):
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://www.digitalinformationworld.com/2019/06/the-world-s-most-creative-data-centers-infographic.html
>>>>>>>>>
>>>>>>>>> On Tue, Jun 18, 2019 at 6:34 AM Marco Neumann <
>>>>>>>>> marco.neumann@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> While we in the Semantic Web / Linked Data community don't seem
>>>>>>>>>> to fall into the category of worst offenders in energy consumption, (I am
>>>>>>>>>> just looking at the forecast and data traffic breakdown on the internet[1]
>>>>>>>>>> and the remarks made by the data-centre expert in Cheltenham[2] that
>>>>>>>>>> digital mobile camera phone sobriety could reduce data traffic in Europe by
>>>>>>>>>> 40%  immediately) current federated SPARQL queries seem to be less
>>>>>>>>>> efficient than one would have hoped for 20 years ago.[3] You are probably
>>>>>>>>>> doing more for your carbon footprint by turning off your monitor completely
>>>>>>>>>> rather than leaving it in stand-by mode [4] than by optimizing your
>>>>>>>>>> federated SPARQL queries or going way of Solid Pods. It seems to be still
>>>>>>>>>> difficult to estimate the number of deployed SPARQL solutions in industry
>>>>>>>>>> and their footprint in terms of resource allocation. One of the best known
>>>>>>>>>> projects but still heavily centralized SPARQL services the wikidata WDQS
>>>>>>>>>> has a rather modest footprint if you go by the numbers published recently
>>>>>>>>>> [5].
>>>>>>>>>>
>>>>>>>>>> Still and since this is my subject interest here the support and
>>>>>>>>>> implementation for federated SPARQL query solutions is surprisingly
>>>>>>>>>> underdeveloped [3] . Looking forward to learn more about updates here from
>>>>>>>>>> QuWeDa 2019 [6]
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/white-paper-c11-741490.html
>>>>>>>>>> [2] https://www.nature.com/articles/d41586-018-06610-y
>>>>>>>>>> [3] https://svn.aksw.org/papers/2017/FedEval-summary/public.pdf
>>>>>>>>>> [4]
>>>>>>>>>> https://www.energuide.be/en/questions-answers/how-much-power-does-a-computer-use-and-how-much-co2-does-that-represent/54/
>>>>>>>>>> [5]
>>>>>>>>>> https://wikitech.wikimedia.org/wiki/Wikidata_query_service/ScalingStrategy
>>>>>>>>>> [6] https://sites.google.com/site/quweda2019/home
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 17, 2019 at 8:31 PM Zachary Whitley <
>>>>>>>>>> zachary.whitley@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I wanted to add some perspective. The principal components of
>>>>>>>>>>> aluminum refining are electricity and carbon and takes a significant amount
>>>>>>>>>>> of electricity and produces large amounts of greenhouse gasses. Most of the
>>>>>>>>>>> electricity consumed is produced by coal. Yes, we should be concerned about
>>>>>>>>>>> energy consumption for computing but I wouldn't be surprised if you would
>>>>>>>>>>> save more electricity and produce fewer greenhouse gasses by *expending*
>>>>>>>>>>> computing resources on making aluminum production and recycling more
>>>>>>>>>>> efficient.
>>>>>>>>>>>
>>>>>>>>>>> [1] https://en.wikipedia.org/wiki/Aluminium_smelting
>>>>>>>>>>> [2]
>>>>>>>>>>> http://www.world-aluminium.org/statistics/primary-aluminium-smelting-power-consumption/#histogram
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 17, 2019 at 3:09 PM Steffen Staab <
>>>>>>>>>>> staab@uni-koblenz..de <staab@uni-koblenz.de>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I don’t believe that a case can be made for physically
>>>>>>>>>>>> decentrallized p2p being more energy efficient.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Compute centers can be placed where energy is cheap and
>>>>>>>>>>>> cooling inexpensive.
>>>>>>>>>>>> Indeed this has been done a lot.
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Cooling reduces energy needs. Generated warmth could even be
>>>>>>>>>>>> re-used. Not thinkable for a DSL-box.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. Modern CPUs use less energy when unused. There is less need
>>>>>>>>>>>> to re-use unnecessary compute cycles
>>>>>>>>>>>> in DSL boxes (well, I guess these modern CPUs are only in
>>>>>>>>>>>> laptops so far - still).
>>>>>>>>>>>>
>>>>>>>>>>>> 4. decentralized energy production is good. Globally, however,
>>>>>>>>>>>> people increasingly live in cities. This is not where most
>>>>>>>>>>>> energy is or will be produced (though it can become more than
>>>>>>>>>>>> today).
>>>>>>>>>>>>
>>>>>>>>>>>> For sure, there is a lot of fruitful, middle ground between
>>>>>>>>>>>> going for DSL boxes vs all using the same centralized compute center.
>>>>>>>>>>>> I don’t believe in the extremely decentralized scenarios very
>>>>>>>>>>>> much.
>>>>>>>>>>>>
>>>>>>>>>>>> Steffen
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 17.06.2019 um 17:38 schrieb Henry Story <
>>>>>>>>>>>> henry.story@bblfish.net>:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 17 Jun 2019, at 01:14, Marco Neumann <
>>>>>>>>>>>> marco.neumann@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I would agree Henry. I think p2p networks are provably more
>>>>>>>>>>>> cost efficient than centralized services in particular for small data
>>>>>>>>>>>> providers. I think there now could be made a case with regards to energy
>>>>>>>>>>>> efficiency. Taking your example of underused resources I would not be
>>>>>>>>>>>> surprised to finding big tech already taking advantage of this network
>>>>>>>>>>>> infrastructure of the underutilized nodes (aka your browser) rather than
>>>>>>>>>>>> benefiting the individual end-users directly.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> also good point with regards to using local resources,  similar
>>>>>>>>>>>> to modern energy networks where most of the budget is not consumed by its
>>>>>>>>>>>> production but its transportation, storage and infrastructure.
>>>>>>>>>>>>
>>>>>>>>>>>> Is there work on p2p search for solid pods underway? I need to
>>>>>>>>>>>> look at HTTP/2 and solid pods more closely I guess. my pod on
>>>>>>>>>>>> solid.community is currently not in a good shape and I am not really having
>>>>>>>>>>>> the feeling of being in control of my own data. Is it more advisable to run
>>>>>>>>>>>> my own solid pod?
>>>>>>>>>>>>
>>>>>>>>>>>> https://neumann.solid.community/public/
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> It depends on how much you want to involve yourself in these
>>>>>>>>>>>> early stages.
>>>>>>>>>>>>
>>>>>>>>>>>> In 1993 I installed Linux on my father’s 40Mhz Laptop to see
>>>>>>>>>>>> how well it fared,
>>>>>>>>>>>> but it required quite a lot of knowledge to do that. Now
>>>>>>>>>>>> everybody runs Linux
>>>>>>>>>>>> on their phone and calls it Android.
>>>>>>>>>>>>
>>>>>>>>>>>> At this point the cloud version would be less work to get going
>>>>>>>>>>>> I guess :-)
>>>>>>>>>>>>
>>>>>>>>>>>> I think of the web when deployed on individual instances as
>>>>>>>>>>>> peer to peer,
>>>>>>>>>>>> and with Solid it really is so, since for example you
>>>>>>>>>>>> authenticating to a server,
>>>>>>>>>>>> requires the Guard to become a client to fetch data from
>>>>>>>>>>>> another server.
>>>>>>>>>>>> Each node can be in one and the other role at different times -
>>>>>>>>>>>> which is not
>>>>>>>>>>>> to say that some nodes like browsers won’t specialize.
>>>>>>>>>>>>
>>>>>>>>>>>> P2P file sharing with duplication of content across nodes
>>>>>>>>>>>> should really be
>>>>>>>>>>>> named something else, more like distributed content sharing.
>>>>>>>>>>>> Adding such features
>>>>>>>>>>>> on Solid pods would be possible, but I think they are trying to
>>>>>>>>>>>> restrict to keep focus.
>>>>>>>>>>>> Adding it the right way - with RDF data to link to other copies
>>>>>>>>>>>> on other pods - would
>>>>>>>>>>>> be a nice research project. Perhaps the most important place to
>>>>>>>>>>>> add that for
>>>>>>>>>>>> Solid servers would be as distributed (encrypted) backups of
>>>>>>>>>>>> one's pod on friends pods.
>>>>>>>>>>>>
>>>>>>>>>>>> Henry
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 16, 2019 at 5:25 PM Henry Story <
>>>>>>>>>>>> henry.story@bblfish..net <henry.story@bblfish.net>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> My guess is that such studies have not been done, mostly
>>>>>>>>>>>>> because widespread
>>>>>>>>>>>>> deployment as would happen if Solid became widespread has not
>>>>>>>>>>>>> happened
>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But there are some reasons one could be optimistic.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. everyone has a DSL box at home currently that is on and not
>>>>>>>>>>>>> doing much
>>>>>>>>>>>>> a lot of the day, so consuming energy for nothing. Instead
>>>>>>>>>>>>> with Solid Pods
>>>>>>>>>>>>> those would be doing something useful, and could use
>>>>>>>>>>>>> electricity from solar
>>>>>>>>>>>>> energy produced locally. So you don’t increase local
>>>>>>>>>>>>> electricity costs
>>>>>>>>>>>>> that much, you can use locally produced electricity, but you
>>>>>>>>>>>>> increase some
>>>>>>>>>>>>> consumption of data.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. It is likely that most people communicate with local
>>>>>>>>>>>>> friends, and in
>>>>>>>>>>>>> most case don’t cross frontiers due to language barriers. This
>>>>>>>>>>>>> may not be
>>>>>>>>>>>>> the case for the W3C community, but for the wider populations
>>>>>>>>>>>>> this is a
>>>>>>>>>>>>> lot more likely.  So in a way Solid pods communicating with
>>>>>>>>>>>>> local friends
>>>>>>>>>>>>> would use less energy, since packets would not need to be sent
>>>>>>>>>>>>> around the
>>>>>>>>>>>>> world.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. There are a lot of optimization strategies that can be made
>>>>>>>>>>>>> by having
>>>>>>>>>>>>> widely deployed pods. For example used in p2p networks, by
>>>>>>>>>>>>> fetching copies
>>>>>>>>>>>>> of data heavy media in the nearest cache.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 4. With the internet of things growing, having the packets
>>>>>>>>>>>>> stay as far as
>>>>>>>>>>>>> required in the home rather than go to large service
>>>>>>>>>>>>> providers, should
>>>>>>>>>>>>> also improve data costs as well as privacy. That is the role
>>>>>>>>>>>>> of a local DSL
>>>>>>>>>>>>> box turned into a data pod is in any case going to grow in
>>>>>>>>>>>>> importance, so
>>>>>>>>>>>>> one may as well use this growing infrastructure.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Since producing energy locally is more efficient, and
>>>>>>>>>>>>> communicating locally
>>>>>>>>>>>>> when that is needed is better, there are reasons to think that
>>>>>>>>>>>>> some of
>>>>>>>>>>>>> the advantages of large providers may be offset in other ways.
>>>>>>>>>>>>> That is
>>>>>>>>>>>>> without counting the huge improvements in efficiency in
>>>>>>>>>>>>> communication
>>>>>>>>>>>>> that come with HTTP2, reactive frameworks, and cpu
>>>>>>>>>>>>> efficiencies.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Henry
>>>>>>>>>>>>>
>>>>>>>>>>>>> > On 16 Jun 2019, at 12:41, Marco Neumann <
>>>>>>>>>>>>> marco.neumann@gmail.com> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Has anybody done work on Carbon Efficiency of Semantic Web
>>>>>>>>>>>>> and Linked Data Queries?
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > The very nature of distributed data sets has to come with a
>>>>>>>>>>>>> substantial computational footprint every time a query is issued to a
>>>>>>>>>>>>> single node or a cluster of nodes for a federated query. On the other hand
>>>>>>>>>>>>> decentralization might actually outperform more centralized services in the
>>>>>>>>>>>>> future.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > I can find a number of papers and articles related to carbon
>>>>>>>>>>>>> efficiency in general computing and cloud computing environments and data
>>>>>>>>>>>>> centers but nothing specifically related to the improvement of operational
>>>>>>>>>>>>> efficiency introduced by Semantic Web and Linked Data infrastructures.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > There is CO2GLE which attempts to estimate the CO2 emissions
>>>>>>>>>>>>> per second released by web search engines like Google as a reference here:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> https://qz.com/1267709/every-google-search-results-in-co2-emissions-this-real-time-dataviz-shows-how-much/
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Regards,
>>>>>>>>>>>>> > Marco
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > --
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > ---
>>>>>>>>>>>>> > Marco Neumann
>>>>>>>>>>>>> > KONA
>>>>>>>>>>>>> >
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---
>>>>>>>>>> Marco Neumann
>>>>>>>>>> KONA
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the
>>>>>>>>> cloud" M: 703-864-1203 EM: david@iconicloud.com URL:
>>>>>>>>> http://iconicloud.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>>
>>>>>>>> ---
>>>>>>>> Marco Neumann
>>>>>>>> KONA
>>>>>>>>
>>>>>>>> --
>>>>>>> David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the
>>>>>>> cloud" M: 703-864-1203 EM: david@iconicloud.com URL:
>>>>>>> http://iconicloud.com
>>>>>>>
>>>>>> --
>>>>>> David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the
>>>>>> cloud" M: 703-864-1203 EM: david@iconicloud.com URL:
>>>>>> http://iconicloud.com
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>> ---
>>>>> Marco Neumann
>>>>> KONA
>>>>>
>>>>> --
>>>> David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the
>>>> cloud" M: 703-864-1203 EM: david@iconicloud.com URL:
>>>> http://iconicloud.com
>>>>
>>> --
>>> David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the
>>> cloud" M: 703-864-1203 EM: david@iconicloud.com URL:
>>> http://iconicloud.com
>>>
>>
>
> --
>
>
> ---
> Marco Neumann
> KONA
>
> --
David McDonell Co-founder & CEO ICONICLOUD, Inc. "Illuminating the cloud"
M: 703-864-1203 EM: david@iconicloud.com URL: http://iconicloud.com