I gather that it had a use unrelated to Lemmy but for Lemmy posts I make a case here that it is more misleading than helpful. For a moment, please ignore the underlying reasons why things are the way they are and focus on how the issue presents to the end-users.
(1) By pulling in solely the post, but not any of the comments, it at best provides only partial information - which if all you wanted to read was the post, then why bother pulling it here at all? (as opposed to retrieving from its original location - I mean, to do it you already need the full URL…) While if instead you wanted all of the comments… - e.g. to be able to reply to - then too bad, b/c it won’t do that?
(2) It also does not pull in any of the old vote counts. So if hypothetically a post had 1000 upvotes, and then after pulling it here it received adjustments +2 from upvotes and -4 from downvotes, then its total would then be 998, right? Except PieFed would instead display “-2”, a qualitatively different score for a highly popular post that is a terrible misrepresentation of the actual facts about it.
(3) It conveys a distorted view of things to the end-users. e.g. see [email protected] where there are 6 posts from the last 2 months, right? Right?! No, there is actually only a single post there in its entirety, then a few more that I and what I assume was Blaze pulled in - note how those other 5 have zero comments, and total scores near zero, due to the aforementioned issues. Really the “earliest” post that PieFed.social reliably has from that community is from 4 days ago, and then beyond that is a scattered, partial mess. There are actually MANY more posts from the last two months, which are not represented here. Ergo, the initial impression that a quick glance at this community offers turns out to be false, due to these federation issues.
(4) showing only partial information is often called a “false positive” or type I style of error, whereas showing nothing at all for those posts that are not fully here avoids that pitfall. If certain content is not here then… well it is not here, and that’s that, but for only some of it to be here leads to much confusion, imho.
Almost entirely distinct from this issue, the ability to find an existing post given its URL should be added to the search menu, b/c that is where people will go to find it. But ofc all the more so if the retrieval button is removed or made less prominent, so that that find ability is not lost along with that.
I understand that there are hard limitations of the federated model itself. So if e.g. older comments and votes cannot retroactively be pulled in - or possibly even if so - then maybe this function should just be abolished? Or perhaps a couple more layers of “are you sure you want to do this?” added, or better yet moving it from its prominent place showing up to everyone on almost every page to a more subdued location where only those who know what it is and what will happen if it is used are likely to access it? I now feel that I actively made the situation in [email protected] worse by pulling in those posts, and wished now that I hadn’t done so, as it could lead people astray into thinking “this is all the posts that the community has to offer from this time-period” (NO, it actually has MANY MANY more than that, on the original server!?!!). Now that I know this I can refrain from using it, but it would be nice to help others who climb this ladder after me as well:-). So I am sharing my thoughts with you in case that helps.
PieFed is freaking awesome and you all who work on it are magnificently extraordinary to share your knowledge with the world:-).
No worries about the delay. Once a reply to me from someone didn’t federate and I didn’t see it by chance until a month later - see https://lemmy.world/comment/12146899
I think you are asking for something different actually. This feature that I like and use is different from what you’re asking - in some ways they’re the exact opposite. But at the same time they aren’t mutually exclusive - in fact I’d say they are complimentary.
But if I’m understanding correctly, it’s easy enough to add into piefed the "See original post at " thing. For a rough job, I just put this line below
[HTML_REMOVED]See on original instance at [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
above this bit, https://codeberg.org/rimu/pyfedi/src/branch/main/app/templates/post/_post_full.html#L2
And then every post on my instance will take me to the original author’s instance. (There’s probably a bit of tweaking to do here - for example I put it above the “Home > Communites…” line as a quick hack which probably isn’t the most user friendly UX.)
I put a small diff file at the end of this post as well.
But - back to the original feature - I think you’ve got exactly what the benefit is. Here’s a real life example of how it was useful:
For a while, I had issues with federation that was causing posts from lemmy.world not to federate to my instance for three whole days. But with this option, I could manually load in posts into my instance and add comments (and for some reason the replies to those comments tended to come in right away). Without this option I wouldn’t have been able to comment at all unless I waited, or used an account on a difference instance.
I’m fine with storing extra metadata so the post original (typical federation vs being manually pulled in - and perhaps a third for posts that get backfilled when a new community is first added?). In fact your suggestions on displaying this to the end user, like a special icon, are good ones.
But keep in mind that the vote discrepancy already exists on backfilled posts, and note that it could also happen with something that arrives through normal federation.
(E.g. Beehaw defederated from lemmy.world at some point but AFAIK piefed.social can still talk to Beehaw. Then up/down votes from users on Beehaw won’t make it into the vote count on the version of the post on lemmy.world but would make it to piefed.social - which might not be that big of a deal if it’s just a couple of users, but if the difference was massive then we’d have exactly the problem you state.)
We wouldn’t be the first to notice this either. See for example https://discuss.tchncs.de/post/1907501
diff --git a/app/templates/post/_post_full.html b/app/templates/post/_post_full.h
tml
index 3467929…df06190 100644
-– a/app/templates/post/_post_full.html
+++ b/app/templates/post/_post_full.html
@@ -1,4 +1,5 @@
[HTML_REMOVED]
+ [HTML_REMOVED]See original on [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
{% if post.type == POST_TYPE_IMAGE -%}
[HTML_REMOVED]
[HTML_REMOVED]
Part 2 of 2.
That said: I thought that we have a problem that Lemmy does not: we allow users to pull in older posts. But your linked example seems to disprove that - do you know how they got those posts then, even though they are “old”? Or perhaps I am misunderstanding something: we CAN get older posts, but we usually choose NOT to, UNLESS someone specifically requests the pulling in of such a post. Damn it does get complicated:-). In general though, I hope you see what I meant to convey: if we can only get new posts and cannot get old ones accurately, then perhaps it is better not to try to get those old ones (except as ghosts/shortcut links)?
As for existing backfilled posts - yeah, it is unfortunate, but it is what it is and I don’t think “backwards compatability” to existing entries is our primary concern right now? If it were, they could perhaps be deleted - if we could even figure out which ones they are? - or left alone. A year from now they will confuse the heck out of people… but that can be an adventure for them to puzzle over, so long as things work properly from here forward - do you agree? :-D
Anyway it’s just something fun to think about:-).
Hey, thank you for this engaging set of posts - and the entire post and thread, really. It really opened my eyes up and made me think hard - exploring a new mental space, in Star Trek speak ( see https://memory-alpha.fandom.com/wiki/All_Good_Things…_(episode)#Act_Ten for the reference - it happens near the end of the episode)
I think I prefer 1 or the ? version. Going with 2 means I can’t interact with a post that fails to federate (or is older than my instance) at all. With 1 or ? I can. Being able to backfill comments would be icing on the cake.
From the linked post of https://discuss.tchncs.de/post/1907501 it seems the standard bearer is just what the home instance of the community has for that post. So comments and votes made to a post after defederation do not count. This makes sense - if you can’t trust the other instance, probably you can’t trust their admins either - and it’s easy for an admin with database access to add fake comments and up/downvotes, etc. So we just pick the obvious source of truth and roll with that.
One small issue here - from screenscraping we can get the vote total count but maybe not the individual votes. We wouldn’t know who voted which way. That’s not a big deal - we can just make up a “[email protected]” ap id to own the vote in the backfilled cases, but I think it makes the case for ? rather than 1.
(As an aside I don’t think Lemmy does backfilling, only pyfedi does it - as the top comment says, “just… it does not backfill previous content and votes.” But the post itself I think must get picked up when a new comment or up/downvote gets seen.)
Damn I loved that Star Trek episode - it was so great! It’s so easy to just keep doing the same old thing - e.g. to explore the stars, but to join a card game - THAT would be a NEW adventure, surely.
But as for the possible, one idea would be to make a hard separation: let’s say a PieFed user joins a new community XYZ. As of that very moment, all newly made posts along with all votes come in, and everything that can make it there through any federation networking issues are reflected in that posts, BUT all old posts do not show up - even if new votes or comments are sent, those are simply discarded - and perhaps a message is displayed “there are no more posts available from this community”. So everything NEW is fully there (to the extent possible/feasible), while everything OLD is fully missing (it would have been only partial/misleading information anyway), and this way there is nothing in-between.
At least, it’s a thought:-). Surely it would not please everyone, but it might be a least worst option?
Err - so basically do what Lemmy does?
Like I said, that wouldn’t work for me. But then again, I’m an admin of my single user instance. And I can see the needs of “real” instances - ones trying to build a community of many users - being different from mine.
So maybe allow it as an admin toggle, default to Lemmy behaviour. That would seemingly cover everyone - and the admins on piefed.social can manually pull in an older post/comment if there’s a compelling reason to make an exception (and add the cautions about possibly inaccurate vote counts and such).
Part 1 of 2.
In the past I have used the rainbow-colored Fediverse icon for things like a quick short-cut to the original instance, to check things like the community or instance rules, but mostly ever since Lemmy started being able to read the static.wikia types of images, I haven’t needed it anymore, now that all instances display more or less identical content for a given post.
With the exception of that known Lemmy.World federation issue. What happened there, from what I hear, is that the Fediverse was never meant to have a singular server holding 80% of the entire userbase, and to have every community be joined by someone on every instance (which Blaze has been doing with his army of alts, to aid newcomers who couldn’t figure out how to join a community that nobody else on the instance had already joined yet). Read more here: https://aussie.zone/post/13429731 (github issue linked from there). TLDR: the issue is already solved (probably) but is in the 0.19.6 version, which Lemmy.ml is testing but until Lemmy.World upgrades to it this issue will continue. I’ve seen it happen on piefed.social, but also startrek.website, discuss.online, and other servers as well.
But in general, I was not asking to be able to view every single post from its original server (though again, that would be somewhat useful, occasionally), nor even being able to pull in a post when it is not there, but to make the pulling in of a post be either full or else none. Here is a simple mathematical equation that may perhaps help: 1 + 1 = 2 is true, and 1 + 1 = ? could also be true, but 1 + 1 = 1 is not. The first one is fully and precisely true, the second is vague and unsatisfying that someone doesn’t know the answer… but at least it is still technically correct. Whereas the third answer is just flat false.
It sounds like from what you and others are saying that it is literally impossible to be able to get the answer of “2” via the current methods of Federation. Okay then, well that seems to inform our answer: if we absolutely cannot get the proper answer of “2”, then we should put “?”, but in no case should we (imho at least) show “1”, just b/c after we pull in a post it receives +1 upvote. “x+1=1” is invalid for nearly all values of x, except the one that happens to be true, and yet ALL of the others are false, hence we should not shorten x+1 to merely say “1” as if 1 + 1 = 1 were a correct statement.
Alternatively, which depending on interrelation with other reasons might be the better way even, perhaps we should simply disallow the pulling in of posts altogether - if they cannot be done “properly” then perhaps to avoid misleading aka false statements, it should not be done at all?
Still further alternatives could be to (1) pull in “ghost” or “shortcut” versions of the post, which are displayed VERY noticably different than “real” ones, and which when clicked go to the original server, and which also have something like “?” or “n/a” rather than a vote count; or (2) similar to 1 but which when clicked show all of the comments, those having been pulled in via some other means (API?) but there too the vote count(s?) should not be displayed, unless they are known to be accurate.
Other matters also interrelate with the above though. e.g. I notice that piefed.social has defederated from hexbear.net (https://piefed.social/instance/hexbear.net does not resolve and https://piefed.social/instances?search=hex&submit=Search looks empty) - so if any votes can from that instance, those should not be counted. Web scraping would include them though, and others e.g. those from people who piefed.social (or any PieFed instance, just picking on that one:-) has banned. Perhaps this is a minor point, but it does touch on how those vote counts are interrelated to other matters, which if brought in via “alternative” means will make it harder to figure out what to display for them. And… oops, I see that you already got to the reverse of that point, where Beehaw would add vote counts, beyond what e.g. web scraping would be able to pick up. Well, nothing is fully perfect, but I did want to suggest that we avoid any KNOWN sources of misinformation. Some corner cases like that… can be saved for perhaps another day when further developing this ActivityPub implementation makes more sense, compared to other more urgent things that need to be worked on in the meantime. i.e., especially if something is also a problem for Lemmy, then it seems understandable to me that if it is likewise a problem for PieFed, then there is little “expectation” that we would do better than them, in every single possible way?
Damn, I’ve hit a space limit.