cross-posted from: https://hachyderm.io/users/maegul/statuses/111820598712013429
Is decentralised federated social media over engineered?
Can’t get this brain fart out of my head.
What would the simplest, FOSS, alternative look like and would it be worth it?
Quick thoughts:
* FOSS platforms intended to be big single servers, but dedicated to …
* Shared/Single Sign On
* Easy cross posting
* Enabling and building universal Multi-platform clients.
* Unlike email, supporting small serversNo duplication/federation/protocol required, just software.
You must log in or register to comment.
E-mail. E-mail does support small servers.
Btw, I think you are mixing up a few topics here, so let’s see what you actually want.
- Protocols are what computers use to communicate with each other. No protocols means no interaction between different computers/servers. Without protocols, none of the things you ask for can be possible.
- Federated services don’t have single sign on. On the contrary, single sign on is a centralized service not a distributed one. To clarify that: I cannot log into lemmy.world with my feddit.de accout, same as I cannot log into hotmail with my gmail account. In both cases I log into my instance/provider and this allows me to communicate with people on other instances/providers. Federation is the process of sharing content between instances. SSO on the other hand is a centralized service that then communicates with other services to let you log into these other services. For example, I can log into my Google account and then use this to login to other sites. This only works because people trust Google. This would not work as a decentralized service with untrusted servers.
- Duplication is used on federated services for a few reasons. First, it’s a kind of caching mechanism distributing the load. If someone posts something on one instance, it’s transferred only once to the other instances which then serve it to all their users. Without duplication, each individual view would have to be requested again from the original instance. The other advantage is that the admins of all the instances retain control over the content. If the other instance goes offline, users can still see “their” copy of the content. And if the other instance doesn’t moderate their content, the mods/admins of your instance can do that themselves.
So as you see, these concepts aren’t there just for fun, but for a purpose.
Protocols are what computers use to communicate with each other. No protocols means no interaction between different computers/servers. Without protocols, none of the things you ask for can be possible.
By “Protocol” I was referring specifically to ActivityPub and the difficulties of engineering against that standard and then the compute resources required to perform the federation tasks. And in addition, the work of inventing a new protocol and the systems around it, as oppose to relying on the protocols and systems that already exist, which I presume would be mostly sufficient to run what I suggest. Obviously at a general level a “protocol” is required for any interoperation between platforms or servers over the internet, so I’m honestly not sure where you’re coming from with this feedback.
Federated services don’t have single sign on. On the contrary, single sign on is a centralized service …
I’m not talking about the fediverse currently having SSO.
I was brief in the top post, in part to start a conversation but because it was originating from mastodon. But the “platforms … dedicated to …” part of my post was referring to the idea that a system of trust would be committed to amongst the suggested platforms where they’d be mutually trusted sources of authentication.
Duplication is used on federated services for a few reasons …
I said “over engineered” not “just for fun”.
The premise of my post is to question and ponder the tradeoffs involved. Keeping in mind that social media has for a long time and continues to operate through centralised servers with many many people being very happy with it, the UX and development issues created by the underlying design of the fediverse may not be worth the technical features.
Implicit in my post is the idea that decentralised/federated social media might be a bit of technological idea that technologists like without thinking through the practicalities and usability issues. Many have bounced off of the fediverse for essentially this reason … because in the end the point of social media is the social part not the independent decentralised data resilience part.
I like the fediverse as much as anyone … I was here before the reddit migration promoting lemmy and kbin on mastodon. But I’d prefer an open and free social media space that works for people rather than one that’s futilely married to a tech idea and particular implementation (ActivityPub).
In the end, I’m wondering if there’s a middle way and have simply put up my first attempt at a set of ideas.
Otherwise, beyond the substance of your reply, I have to say that your reply was unnecessarily condescending. My post was rather short, so thinking it was without any meaning or coming from a completely ignorant place makes some sense, but without really having much to offer to the actual issues or discussion in your reply (you seem to have missed the essential thrust of simply having a few large servers committed to being open and working together as an easier alternative to big-corp social media), you managed to also make it clear that you thought my ideas were silly or juvenile. Perhaps I am being silly, but I feel your tone and presumptions were unnecessary.
Sorry, no condescension intended.
Your post read like one written by someone with very minimal knowledge about the subject, which might have been a misunderstanding on my part. So I tried to cover the basics before talking about the rest.
There is really no shame in asking questions about something where you don’t have experience. There are far more topics I have no idea about than there are topics where I do have a deep understanding.
So to get on the same page, I’ll summarize what I understood, please correct me if you mean something different.
- You don’t like ActivityPub, you want a new protocol
- The system should make it easy to create new, small instances
- The instances should share sessions with the other instances (=single sign on) based on trusting them
- You prefer a centralized system?
- You want the system to not use a single protocol (ActivityPub), but use multiple protocols?
- ActivityPub based services have bad UX due to the complexity of the protocol
Is this correct?
We have a few contradictions here.
You cannot have a system where anyone can easily create servers and at the same time have shared sessions based on trust. These two requirements conflict with each other.
Either servers only work with servers they trust, and then you can’t just create a new small server and interact with the network.
Or anyone can easily create a new small server, but then you can’t do anything based on trust, since you never know if that server was created with malicious intent.
Regarding centralized/decentralized you have to differentiate between implementation and management.
All major social networks run distributed systems. If you want to serve billions of users, you need to run millions of servers. These servers are distributed around the globe to give fast access to users everywhere. Chances are pretty high that your ISP has a few racks of Facebook, Netflix, YouTube and Tiktok servers.
Their distributed system is orders of magnitude more complex than everything running ActivityPub combined.
But their system works, because they have tens of thousands of highly paid specialists to make them work.
ActivityPub based services on the other hand have almost no funding and manpower.
Mastodon is the best in this respect. They have 6 people who are actually working on the system.
Lemmy has two developers who earn close to minimum wages.
Kbin has a single guy developing it.
That’s the real reason why the UX is crap.
If anything, ActivityPub and the services running on them are extremely underengineered and underdeveloped.
Btw, there is something rather close to what you seem to want: online forums with Google single sign on.
The forums are not interacting at all with other forums. No federation or anything at all. There are enough commercial solutions that work really well. And with Google Single Sign On you also don’t have to register for each forum.
Sorry, no condescension intended. …
All good!!
And thanks for the reply! I don’t have time now to read through it, but will later. Just wanted to say I’m aware my post was very short and understand where you were coming from … and that, for me, if I address an issue I have with tone, it’s in the interests of the community, kinda “tone policing” as awful a term that is to say out loud … to just let someone know they may not come off as well as they or we would like. But I’m all good personally and look forward to reading your post!
as oppose to relying on the protocols and systems that already exist,
Which alternative to ActivityPub would you suggest?
Well, none really, because the work of ActivityPub wouldn’t be performed in my idea. There wouldn’t be mass copying of content to all servers. And the trade off would be that you wouldn’t be able to follow or see content from another server either (just like modern Twitter and Reddit).
The work around though is that SSO should enable easy cross posting from one server to another, so information/posts would be easily shared between servers. But more importantly would be the commitment to open APIs and building/supporting aggregating or unifying clients, so that from a single interface you can follow and read and reply to content from any server.
In terms of servers and users, the idea is essentially to do the content copying/unifying at the user side rather than the server side.
As far as what protocols are necessary here … basically the internet we have now … which is very much the point.
In simple terms the idea starts from a common praise of the fediverse … that it enables posting between Twitter and Reddit and that it is open. Well, how about we build Reddit and Twitter clones just with FOSS, open APIs and cross posting features?
In simple terms the idea starts from a common praise of the fediverse … that it enables posting between Twitter and Reddit and that it is open. Well, how about we build Reddit and Twitter clones just with FOSS, open APIs and cross posting features?
Isn’t that opening the door for future enshittification? Like as soon as Lemmy/Mastodon reach 100k users, a big corporation buys the only instances (as there is no federation in this scenario) to the admins, and turn it into a corporate product? What’s preventing that without federation?
Good point!
I’d say more or less the same thing at the core of the fediverse, which is an open and shared commitment to values, features and practices. If the platforms are FOSS, and you can always download your content/posts, than any shitification will be met with moving, no?
There’s the risk of losing all the threads attached to the posts, I suppose. But I’m not sure that that’s an absolute problem. There could be solutions around commitment to archiving and hosting archives etc. But also, the federation thing of copying everything everywhere creates problems around privacy and safety (like people have left the fediverse because they’ve found it to be worse in terms of racism etc than twitter). Having content on single servers with higher walls allows for better safety and privacy features. So it’s a trade off.
As for a big instance being bought … well like I said, people can just move. Shared sign on should make that a nicer experience than here. And some feature for hosting archives of posts from other servers (which the structure of the fediverse actively makes very difficult because content has to be uniquely addressable across the whole network) could probably go a long way to protecting good content. On the fediverse, people have to move whenever a server goes down, but can’t take their posts with them.
On top of all of that I’m not entirely clear on how much a federated protocol prevents corporate enshitification. When Threads federates, and should for example something like Tumblr and maybe some others do so such that the majoriyt of the fediverse is actually on corporate platforms funded by ads and data tracking (which can apply just as well to content that federates with such platfroms from otherwise FOSS platforms) … is that not some form of enshitification? Which isn’t to touch on the things that can happen to the ecosystem once big-corp wield such large user bases.
What protects the fediverse then? I’d guess it’s that the platforms we have are FOSS and those of us who want to break away and start our own instances can. But that’s not the protocol or federation … that’s FOSS platforms and commitment to values and working together. If I’ve got a point with this … the broader abstract idea is that many of these problems are not tech problems but people problems, and, very much to my broader point, confusing people problems with tech problems can lead you astray into making solutions that do not succeed because you’re blindsided by your tunnel vision love for the tech solution.
I’m genuinely curious to know if/how I’m wrong about this. Because a decent read of the fediverse in the wake of the “2023 migrations” is that, to many people, federation itself was form of social media shitification … in that it was seens as an overly complex and annoying technology that actively disrupted and sometimes worsened the core features and motivations of social media … socialising with people … all while acolytes could only tell complainants that their issues weren’t meaningful or serious enough and were likely due to them not using the fediverse correctly. Now I’m here, obviously, because I’m a fan. But I can’t shake the possibility that the fediverse might have fatal flaws and am always a fan of solutions that stick with what’s easy and tried and true (generally good engineering advice IMO).
The more simple approaches have already been tried and tend to die before they live.
Social media requires a network effect in order to be successful. Given the established players have had nearly 2 decades to accumulate vast networks, it would be a huge uphill struggle to start from zero content and users. Federated & decentralised social media is the answer to this—you get a network for free, giving the software a chance to stand on its own merits.
For this to all work correctly, they must all talk the same, ideally standard, language (the activitypub protocol) and for decentralised software to actually be decentralised, there can be no single point of failure (therefore caching). As someone mentioned, SSO is inherently centralised, even with something like OpenID, if your authority is down, your account is unusable, so it wouldn’t really add much to the experience as it stands (and possibly may risk complicating it more for new users).
Yea, buy in and network effects are certainly the tricky part. But that’s also true of the fediverse … it’s been going a long time and in many ways was really “gifted” with Musk’s twitter purchase (seriously, if someone else were to take charge there and reset it back to pre-musk, you’d see a bunch of people leave masto) … and Spez’s API pricing. Before these events, the fedi was pretty quiet compared to now. Lemmy, before the reddit migration was very quiet and may very well have failed by now or soon were it not for the migration.
Moreover, ActivityPub doesn’t get you seamless network effects. Lemmy and mastodon mostly don’t have cross-traffic, and that’s because their platforms basically lack any mutual support for each other. If they worked well with each other, Lemmy would be a much busier place (and masto would be better structured). Same probably goes to some extent for things like Peertube and bookwyrm. There’s also the lowest common denominator effect when it comes to features. One platform may support/provide “Quote posts”. But because Mastodon doesn’t, and they have the bigger user base, it doesn’t really matter, as no one else will see the quote posts and so the new platform doesn’t really have much to offer new users, which in turn basically turns the fediverse into the mastoverse (which is actually happening) and undermines the promise of enabling new platforms with built in network effects. Mastodon could just become one big single server or platform today and many probably wouldn’t mind.
Otherwise, RE SSO, I had in mind that trusted platforms would be mutual sources of authentication such that an account on one is effectively an account on all of them.
I wouldn’t be on Lemmy if it wasn’t federated.
Why? Because I don’t want to be trapped into another ecosystem that will do a Twitter/Reddit when it gets big enough. Reddit clones are plentiful, but they’re all fragmented in the user base. At least with Lemmy that doesn’t matter, I can use any community from my home instance.
ActivityPub in itself really isn’t that bad, there’s a lot more that goes into the platform as a whole. And it’s also a good choice because it’s a standard protocol that everyone agrees on. Is it perfect? Probably not. But it works good enough, and we now have Lemmy alternatives in the form of Sublinks and K/Mbin. There’s also a decent chunk of toy ActivityPub projects out there as well.
I can still be trapped in a dying ecosystem but that’s true of every site and at least I have the option of taking my data and converting it to another software if I want to.
And it’s also a good choice because it’s a standard protocol that everyone agrees on.
Not sure how true this is in practice. See eg: https://hachyderm.io/@hrefna/111812820133158591 And there’s of course BlueSky and ATProto that may alter that truth drastically (we’ll see)
Otherwise, I hear you. Thing is I’m aiming for something in the middle. If the platforms are all FOSS and commit to easy movement/migration features (which the fediverse sucks at to be fair) as well as good quality aggregators (which the fediverse also sucks at), then maybe your concerns aren’t a fatal possibility?
Also, and I do apoligise for making this a tad personal … it seems you’re on an instance that’s mostly for you or small group of friends (which is awesome! How’s it going?) … which is a level of enjoyment of what the fediverse offers beyond most people and even beyond what many capable of doing so would want to do for their social media. Meeting things in the middle is also about bringing better social media to more people, which could naturally deprive the technically capable of their abilities to be flexible with their needs (though in a way BlueSky’s promise is kinda to do both, if it succeeds).
I think the biggest issue is account management. Having all these different instances wouldn’t be as bad if it were easy to switch accounts or combine their subscriptions, making it truly user-driven instead of depending on the behavior of each individual instance.
Exactly. And that’s kinda part of my point. Federation is all about copying content from server to server, so that the central organising structure is the server or instance. Duplicate enough content and a user won’t need to move … is kinda the idea. The reality though is more complex than that, as platforms and their designs get in the way, as does defederation and unpredictable admins. Reality is that the fediverse isn’t user driven, and once you see that, the whole decentralised thing starts to become more questionable (IMO).
So given all of that, I’m wondering what happens if you take away the whole federated idea but still retain some of the aims and principles and try a more straightforward approach that uses the tools and technologies we already have.
There are some tools that can help with this but I don’t think completely automatically. I don’t use any of them.
See: https://github.com/dbeley/awesome-lemmy#tools - scroll up and down the page too might be something that suits you
The fediverse isn’t over engineered, it’s just not quite focused on the right aspects. A federated social network needs to be more like a block chain, where the content is centralized, and the instances (miners) are decentralized. The content is the important part, and with everything being tied to an instance, it makes the content harder to access. You have instances defederating, going down, closing, and version conflicts, all that makes it harder for a network to gain traction.
You are describing a different thing than what the idea of the fediverse is. Content is collected at an instance and these instances federate. That’s why its called Fediverse: people basically form groups, these group federate. It’s a social thing, there is trust involved. With blockchain, the idea is that you don’t need to trust a central entity.
I think you talk about something like nostr.
It doesn’t need to have the full trustless or buring energy for fun, but it does need to be resilient against instances going down, which currently isn’t the case.
What specific edgecase do you have in mind. The fediverse is coping data and is quite resiliant against data loss.
If an instance shuts down everything from it is just gone. ML already ran into dns issues once, if it goes, 20% of lemmy is just gone.
It is not gone. You still have an old copy of the post on your instance.
e.g. https://szmer.info/post/383045
Replaying, posting and making new content is pointless there but the old stuff still stands and it can be read.
I think thats just a thing that will get better over time
But who will moderate the content? Who is to say to what is legal and where? In USA and in EU different pieces of information can be shown. CSAM needs to be removed. Main lemmy devs removed only active mod on !anime[email protected] due differences in censorship.
The same people that do now?
Blockchain is immutable so you can’t remove the content.
The same people can’t moderate the content if content is centralised, there would need be an overlord that sets the rules.
Block chain was more on analogy than implementation. The key is that data isn’t bound to an instance, and ideally most people never need to know about instances.
Sounds like you’re describing BlueSky there. Have you looked into it?
Unless you’re talking about something more nostr/web3.0?
bluesky depends on one single entity. they promised a lot about their protocol, but they have yet to show that other instances other than the official one can operate in a fully independent manner.
I was under the impression that it’s clear that additional relays can work within their system? Have they not setup anything in the protocol for how that’d work?
I still think we need a simple social media protocol that gives me the power to curate my feed rather than hoping my admins don’t defederate with everybody else (followed by hordes of drooling goons telling me to start my own instance).
Well that’s kinda the point of my quick suggestion in the original post.
Instead of committing to federation, how about committing to aggregating clients that allow you to do exactly this. Right now, there’s no app that will work for both lemmy/kbin and mastodon/microblogging. No way to unify the notifications or even combine the feeds or just have a unified interface for the two platforms (that are, let’s face, both just full of text messages and feeds).
By allowing each platform to be distinct but remain open with their APIs and “play nice with each other” while leaning into the value of aggregators as a primary part of the value proposition of the system, users might be better served.
I have a hard time imagining what that looks like, which is just a failure of my ability to think about these technologies. But what I’m talking about is a little different, simply because I don’t think we can go from these diverse systems into something simple and elegantly connected.
I mean something like email but structured differently. Though email still has spam filters and blacklists, and a new social media protocol might still need those (inevitably infringing on my curatorial freedom similarly to defederation).
My point is that I’m still looking for something new, rather than to reform the defediverse.
Edit:
I might be wrong. It might be good to leverage what we started here and reform the tech to give users more freedom, and take pressure from admins.
Also… maybe email is not the example I should follow. Maybe it’s more like torrents. P2P social media.
Well I’m spitballing here, so I wouldn’t worry about not being able to imagine it! I’m struggling too!!
Is there a chance that BlueSky is more like what you’re after?
No, that’s fairly centralized too. I think I want a peer-to-peer social media protocol. Maybe more like torrents than email.
The fundamental problem is that all this data needs to be hosted somewhere. P2P systems have the issue of persistence: either posts only stick around as long as the people who posted them keep their server online, which is then a burden on anyone who wants to be active in the community, or everyone shares the responsibility for hosting, and then what happens if someone posts CP? Is it just mirrored across the entire P2P system, and each person has to individually root out the CP or just be okay with hosting CP?
Torrents work because you have to actively join a torrent. But discoverability is handled from the outside, through trackers. Trackers choose what they want to host.
Tor or really I2P are the closest equivalents, but they work because everything is encrypted going through them. It’s a privacy thing. With social media, everything is public by design.
Persistence could be traded off for decentralization. Just like torrents’ associated data are stored on people’s computers (and the data dies if nobody is seeding) this kind of social media doesn’t have to be permanently stored on a server.
Yea right. Me too I think. It’s out there and has been for a while. Just don’t think it’s ever taken off.
You might find this interesting: https://pfrazee.com/blog/why-not-p2p
I’m checking this out!
Yea I didn’t know before seeing that that one of the BlueSky devs (the author of the blog post) was heavily involved in p2p stuff (eg beaker browser).
You are on SJW, are you really concerned about defederation? You guys seem to be doing well over there
They’re not the worst, but I want literally nobody restricting my access to literally anybody (criminal behavior a grudging exception, and even that I’d prefer to take care of myself).
Also, I fled a couple other places first.
I just don’t want a Mommy and Daddy telling me which servers are Naughty or Nice. I don’t want technology that enables those restrictions at all.
Hi there! Looks like you linked to a Lemmy community using a URL instead of its name, which doesn’t work well for people on different instances. Try fixing it like this: [email protected]
Usenet?
I do not know enough about how Usenet worked apart from picking up the impression that much of what was done there is being reinvented in the fedi (however accurate that is).
For me, the commitment to having good aggregating and unifying clients, and the commitment to open APIs that would necessitate, is pretty central to my suggestion. Not sure how much of that was in the usenet system (though probably more than I’m aware!).
Usenet servers had no web UI since the web was not yet invented. They ONLY provided an API and the presentation layer was done completely by the client. There were dozens of client programs people used, maybe 100s. Clients handled all features like subscriptions, tracking read articles, boosting or blocking topics or users, etc. So in that regard it was much more private for users. The servers were computers 1000x slower than a modern mobile phone. So yes, Lemmy seems overbuilt.
The servers were computers 1000x slower than a modern mobile phone. So yes, Lemmy seems overbuilt.
Well yea, part of my thinking here is whether all the work of federation, from building the software, debugging and testing it (which AFAICT is a huge pain in the ass) and then actually running it as a job … is actually worth it … when users could very well be happy with something much simpler and the mission of creating a more open, safe and “billionaire proof” social web easier to achieve with something more straightforward.
Usenet had a counterpart to federation in that posts got automatically propagated between servers. I haven’t read the Wikipedia article about it but that might be a good place to start if you want to learn about it. Being a server admin did require some ongoing effort, just like with Lemmy.
HTTP is all you need 👍
