Re: Remixing and Aggregating Task Force

A doubly-linked list solution has the potential to run into concurrency
issues if multiple actors are trying to add items to a feed at the same
time and they're generally a little brittle to maintain.

The CRDT approach would be to add property `position` to `Add` that is
between 0 and 1. To insert an item between two other numbers you randomly
sample a position from the two surrounding items rather than specifically
referencing them as `before` and `after`. Then to get the ordering you just
sort the positions. This runs into precision issues if you keep inserting
items between the same two elements, but you can use something like logoot
<https://inria.hal.science/inria-00432368/document/> rather than floats to
represent the numbers which will keep adding precision as necessary.

- Theia

On Fri, Jun 6, 2025 at 11:05 AM Evan Prodromou <evan@prodromou.name> wrote:

> `Remove` is fine if the objects in the Collection are unique.
>
> `Add` could use some additional (!) properties like:
>
> `before`: an ID of the item the object will be immediately before
> `after`: an ID of the item the object will be immediately after
> `index`: a number, an exact index in the collection
>
> If the objects in the collection are not unique, `Remove` could also use
> these additional properties, as well as:
>
> `relativeIndex`: the index of the item to remove from the collection
> relative to other copies of the same object; "remove the third instance of
> this note from the collection".
>
> Evan
> On 2025-06-05 8:52 p.m., emelia wrote:
>
> Hi all,
>
> I think we'd probably need at minimum a way to signal that the
> OrderedCollection has had its ordering updated — I don't think current
> Add/Remove activities would be enough here. Move doesn't really offer the
> correct semantics for reordering a collection.
>
> – Emelia
>
> On 6 Jun 2025, at 02:29, Ryan Barrett <public@ryanb.org>
> <public@ryanb.org> wrote:
>
> That's a lot! Definitely a maximalist approach. To keep scope manageable,
> and have a prayer of concluding in any reasonable time frame, with
> something we have a chance of getting implementors to prioritize, I'd argue
> that we *not* try to address all of that.
>
> Specifically, I don't think we need to or should specify how feed
> publishers choose the objects in their feeds, especially not in a first
> pass. There's successful prior art in the wild from Bluesky and others that
> define feeds only as ordered, pageable lists of post ids. That allows
> maximum flexibility for implementations to construct feeds however they
> want, and doesn't tie their hands, but still allows interoperability
> between feed publishers, consumers, and other services.
>
> Filtering, aggregation, moderation, algorithms, etc are all great! But I
> don't know that we should bake them into a feed standard. AP already has
> ordered collections; I can see a very manageable first pass here at just
> standardizing "here are my feeds, with some metadata, as ordered
> collections." That, we might actually have a chance of getting eg Mastodon
> to implement by sometime in 2026. 😁
>
> On Thu, Jun 5, 2025 at 2:22 PM Aaron Gray <aaronngray@gmail.com> wrote:
>
>> OOh, also this pipeline design also covers moderation of streams, both
>> automated and manual.
>>
>> And also the algorithmic processing of streams including handling streams
>> for younger users who may have their content filtered or limited in some
>> way, either timewise or content wise.
>>
>> Aaron
>>
>> On Thu, 5 Jun 2025 at 22:19, Aaron Gray <aaronngray@gmail.com> wrote:
>>
>>> Evan,
>>>
>>> Thank you again for the massive triage process I set in sway by my
>>> observation of unclosed issues and you embarked on handling we are now
>>> probably more than two years down the line.
>>>
>>> On Thu, 5 Jun 2025 at 19:43, Evan Prodromou <evan@prodromou.name> wrote:
>>>
>>>> We have a few projects in the Fediverse that are doing remixing and
>>>> aggregating content and other activities from other Actors. Two good
>>>> examples are Surf and Channel.org.
>>>>
>>> This is a very interesting area that I have thought about a lot over the
>>> years. And think it really needs to be more generalized to handle and
>>> encompass all possible use cases.
>>>
>>> Basically theres a pipeline of four processes, that of selecting posts
>>> from a number of feeds to a number of feeds, that of optionally filtering
>>> those posts based on criteria, that of ordering or sorting of collected
>>> posts in some form of automated or manual process in those feeds, and that
>>> of posting the resulting posts to one or a number of output feeds.
>>>
>>>
>>>> I think this is an interesting enough pattern that we should open up a
>>>> Task Force at the SocialCG to investigate standardizing some of these
>>>> features. Important questions to ask are:
>>>>
>>>>    - How should an aggregated feed appear on the Fediverse? Is it just
>>>>    another Actor that Announces activities and content that meet its
>>>>    requirements? Or is there another structure?
>>>>
>>>> Similar to, in and out boxes for continuity with the existing
>>> ActivityPub model.
>>>
>>>
>>>>
>>>>    - How do we handle ordering of feeds? Some feeds might not be
>>>>    reverse chronologically ordered, but use some other algorithmic ordering.
>>>>
>>>> Ideally there is full control over this. And should allow for both
>>> automated and manual ordering.
>>>
>>>>
>>>>    - How can feeds be transparent about what the sources are? Actors,
>>>>    search terms, hashtags, etc. What is coming into this feed? Can consumers
>>>>    inspect the sources?
>>>>
>>>> There should be full control over visibility.
>>>
>>>>
>>>>    - Is the feed replicable elsewhere?
>>>>
>>>> This brings up the idea of relays too, yes we should allow for
>>> duplication for both redundancy and handling workloads.
>>>
>>>>
>>>>
>>>> Chairs, I'd like to propose an agenda item to discuss this possibility
>>>> at the next CG meeting.
>>>>
>>> A meeting I would like to attend. Sorry I am not one for dealing with
>>> the pragmatic after mouth of prematurely set in stone standards, hence my
>>> absence from the process once I realized the reality of the protocol and
>>> its implications.
>>>
>>>> Evan
>>>>
>>> Aaron
>>>
>>>
>>> --
>>> Aaron Gray - @AaronNGray@fosstodon.org | @aaronngray@threads.net |
>>> @AaronNGray@Twitter.com
>>>
>>> Meta-Mathematician, Independent Open Source Software Engineer, Computer
>>> Language Researcher and Designer, Type Theorist, Computer Scientist,
>>> Environmentalist and Climate Science Researcher and Disseminator.
>>>
>>>
>>
>> --
>> Aaron Gray - @AaronNGray@fosstodon.org | @aaronngray@threads.net |
>> @AaronNGray@Twitter.com
>>
>> Meta-Mathematician, Independent Open Source Software Engineer, Computer
>> Language Researcher and Designer, Type Theorist, Computer Scientist,
>> Environmentalist and Climate Science Researcher and Disseminator.
>>
>>
>
> --
> https://snarfed.org/
>
>
>