Re: Eliminate duplicates in expansion?

On 05/23/2012 11:47 PM, Markus Lanthaler wrote:
>> I think all of the algorithms should clean sets... we could also
>> take the position that no cleaning should be done for performance
>> reasons. That's really the strongest counter-point I can see now -
>> performance... because multi-hundred-thousand-member sets are not
>> going to be performant for this algorithm.
>
> Yes, exactly.. the problem is that expansion is used as the base of
> every other algorithm and sometimes even multiple times. In framing
> e.g. it gets called 2 (+1) times and then we check again for
> duplicates in the subject map generation. If you have large sets
> this adds up (3 x n²).
>
> If you feel strongly about this I'm not opposed to change my
> implementation but I think it a) does not bring any advantages and
> b) has a potential huge performance cost.

Yes, good point... we should make this an issue and discuss it on the
telecon.

-- manu

-- 
Manu Sporny (skype: msporny, twitter: manusporny)
Founder/CEO - Digital Bazaar, Inc.
blog: PaySwarm Website for Developers Launched
http://digitalbazaar.com/2012/02/22/new-payswarm-alpha/

Received on Thursday, 24 May 2012 03:51:07 UTC