Growth narratives on the fediverse:

Well I appreciate the effort regardless. If you want any support in getting towards a more “proper” network analysis, I’ve dm’d you a link you can use to get started. If nothing else it might allow you to expand your scope or take your investigations into different directions. The script gets more into sentiment analysis for individual users, but since Lemmy lacks a basic API, the components could be retooled for anything.

Also, you might consider that all a scientific paper is, at the end of the day, is a series of things like what you’ve started here, with perhaps a little more narrative glue, and the repetitive critique of a scientific inquiry. All scientific investigations start with exactly the kind of work you are presenting here. Then you PI comes in and says “No you’ve done this wrong and that wrong and cant say this or that. But this bit or that bit is interesting”, and you revise and repeat.

So lets just cover a few things…

Hypothesis testing:

The phrase “if your post got less than 4 comments, that was statistically significant” can be misleading if we don’t clearly define what is being tested. When you perform a hypothesis test, you need to start by stating:

Null hypothesis (H₀): For example, “the average number of comments per post is λ = 8.2.”

Alternative hypothesis (H₁): For example, “the average number of comments per post is different from 8.2” (or you could have a directional alternative if you have prior reasoning).

Without a clearly defined H₀ and H₁, the statement about significance becomes ambiguous. The p-value (or “significance” level) tells you how unusual an observation is under the assumption that the null hypothesis is true. It doesn’t automatically imply that an external factor caused that observation. Plugging in numbers doesn’t supplant the interpretability issue.

“Statistical significance”

The interpretation that “there is a 95% probability that something else caused it not to get more comments” is a common misinterpretation of statistical significance. What the 5% significance level really means is that, under the null hypothesis, there is only a 5% chance of observing an outcome as extreme as (or more extreme than) the one you obtained. It is not a direct statement about the probability of an alternative cause. Saying “something else caused” can be confusing. It’s better to say, “if the observed comment count falls in the critical region, the observation would be very unlikely under the null hypothesis.”

Critical regions

Using critical regions based on the Poisson distribution can be useful to flag unusual observations. However, you need to be careful that the interpretation of those regions aligns with the hypothesis test framework. For instance, simply saying that fewer than 4 comments falls in the “critical region” implies that you reject the null when observing such counts, but it doesn’t explain what alternative hypothesis you’re leaning toward—high engagement versus low engagement isn’t inherently “good” or “bad” without further context. There are many, many reasons why a post might end up with a low count. Use the script I sent you previously and look at what happens after 5PM on a Friday in this place. A magnificent post at a wrong time versus a well timed adequate post? What is engagement actually telling us?

Model Parameters and Hypothesis Testing

It appears that you may have been focusing more on calculating the Poisson probabilities (i.e., the parameters of the Poisson distribution) rather than setting up and executing a complete hypothesis test. While the calculations help you understand the distribution, hypothesis testing requires you to formally test whether the data observed is consistent with the null hypothesis. Calculating “less than 4 comments” as a cutoff is a good start, but you might add a step that actually calculates the p-value for an observed comment count. This would give you a clearer measure of how “unusual” your observation is under your model.

So I modeled that with a Poisson distribution, and I learnt that to a 5% significance level, if your post got less than 4 comments, that was statistically significant. Or in other words – there is a 95% probability that something else caused it not to get more comments. Now that could be because it is an AMAZING post – it covered all the points and no one has anything left to say. Or it’s because it’s a crappy post and you should be ashamed in yourself. Similarly a “good post”, one that gets lots of comments, would be any post that gets more than 13 comments. Anything in-between 4 and 13 is just an average post.

So, like, I do have a background in stats and network analysis, and I’m not sure what you are trying to say here.

if your post got less than 4 comments, that was statistically significant.

Statistically significant what? What hypothesis are you testing? Like, how are you setting this question up? What is your null?

Because I don’t believe your interpretation of that conclusion. It sounds like mostly you calculated the parameters of a poisson and then are interpreting them? Because to be clear, thats not the same as doing hypothesis testing and isn’t interpretable in that manner. Its still fine, and interesting, and especially useful when you are doing network analysis, but on its on, its not interpretable in this manner. It needs context and we need to understand what test you are running, and how you are setting that test up.

I’m asking these questions not to dissuade you, but to give you the opportunity to bring rigor to your work.

Should you like, to further your work, I have set up this notebook you can maybe use parts of to continue your investigations or do different investigations.

I’m glad I’m not the only one who saw it.

Shit aint free.

I mean when we were over at (the bad place), people were literally donating monthly to pay for server costs. (the bad place) in the early days was basically one giant instance.

except it didnt have a working video player. for-fucking-ever.

so I donate a bit to my mortal enemy @desalines and a bit to lemmy.world to keep the project going. Maybe 5 bucks a month between the two?

If you see this go to the patreon rn and sign up for like, $1 a month.

Are yall not sponsoring this project on patreon or otherwise?

I pitch in something like 1-2 bucks to desalines and a few bucks to .world every month.

[goes on mastadon]

random opinions:

Not saying I don’t go in for it sometimes. But its a bit like Twitter. It feels like an entire auditorium talking all at once, to everyone, all at once.

Draw me a picture of a full glass of wine. Full to the brim. Practically spilling over.

I don’t get the karma hangup thing. Like… Lemmy does have Karma, but we just don’t culturally make it a priority.

I refuse to believe any year past 2020 happened.

Too many cooks: Handwringing. Whataboutism.

The authors misunderstand how to think of the (and even) elements of the fediverse. It’s still taking a competitive view/ worldview/ framing, and when that’s all you understand, sure. But the right way to understand the fediverse is as protocols, like email, and each branch as a flavor of email, or some other misguided metaphor. And it’s it’s only a problem when infinite growth or exp. scaling is your goal. However if neither of those things are your goal, it’s more of an annoyance.

Commercial capture: More handwringing. Misidentification.

Meta took a crack at capture. It didn’t seem to have worked. The fediverse is populated by the leavers, not the takers. The Internet happens at the edge and the normies are always just catching up a few years too late. The point of the fediverse is that it’s a extraordinarly easy to vote with your feet. If the fediverse can fall victim to a 51% attack, fine, well just leave and do it again.

Guilty by association: Again, more handwringing. Also, we should do that.

Federated p2p file sharing e2e file sharing for unsavory bits that governments and corporations don’t want you to have sounds like a great idea.

It’s in the CIA field manual, that when you want to destroy an organization from within, urge caution, and question every unfounded problem.

yeah I think it would be more clarifying if you kept the modes distinct.

I like the focus on accuracy and it’s citations. I’ve tried deep research (in contrast with chat got) few times and it’s generations have been basically worthless.

I definitely have a use for something like this, but similarly to most of the issues I have with the applied use of these products, it boils down to a few consistent issues around guard rails, a kind of cautious insistence on singular approaches, and a lack of agency.

For example, I would live to be able to drop it a guy hub link and have it dig through the repo, and write me an ipynb demonstrating the capabilities of the repo. Or where I could give it a script, sloppy with a bunch of my own garbage in it, and it cleans it up and makes it nice. Deep research is no where near capable of this and I attribute it to an overly cautious development approach on the part of OpenAI. As well, because of structural limits, these models lack the kind of nested or branched thinking that would be required to hold onto big picture goals and concepts.

I do however think we’ll see things change with the new gpts coming out which are much cheaper to run for inference. Basically, to do the kind of work that deep research claims to be doing, we need a more complex internal model structure with many gpts running in both series and parallel, perhaps in more of a graph model.

I also don’t think it will be OpenAI to do this. They’ve been too cautious with their development approach.

At the end of the day I want what deep research claims to be, but it’s clearly not it yet.

Hmm. Interestingly it barely covers anything from deep research outside of the first and last sections.

I don’t mean this in a pretentious “guy in a turtleneck sniffing his own farts” way.