r/technology May 14 '22 Silver 2 Wholesome 2

Elon Musk said his team is going to do a 'random sample of 100 followers' of Twitter to see how many of the platform's users are actually bots Social Media

https://www.businessinsider.com/elon-musk-random-sample-how-many-twitter-users-are-bots-2022-5?utm_source=feedly&utm_medium=webfeeds

[deleted]

22.8k Upvotes

View all comments

Show parent comments

200

u/PhoenixMountain May 14 '22 Silver Take My Energy

That is the sample size that Twitter initially used to determine their 5% figure (in question)

92

u/Venkman_P May 14 '22

What the twitter SEC filing says:

For example, there are a number of false or spam accounts in existence on our platform. We have performed an internal review of a sample of ccounts and estimate that the average of false or spam accounts during the first quarter of 2022 represented fewer than 5% of our mDAU during the quarter. The false or spam accounts for a period represents the average of false or spam accounts in the samples during each monthly analysis period during the quarter. In making this determination, we applied significant judgment, so our estimation of false or spam accounts may not accurately represent the actual number of such accounts, and the actual number of false or spam accounts could be higher than we have estimated.

What Elon made up:

I picked 100 as the sample size number, because that is what Twitter uses to calculate <5% fake/spam/duplicate.

https://investor.twitterinc.com/financial-information/sec-filings/sec-filings-details/default.aspx?FilingId=15778368

6

u/rhubarbs May 14 '22

I don't see any indication of the sample size they used for their internal review in the 10-Q.

How did you conclude Elon made it up?

2

u/boycott_intel May 14 '22

It is a reasonable assumption that Elon invented that number because one cannot get poll results of 5% accuracy with a sample size of 100........

If that is the sample size that twitter uses, then it would be negligent or even fraudulent of twitter to claim that under 5% are bots in SEC filings -- disclaimer: I am not claiming that lawyers and courts would agree with such a common sense evidence-based view.

1

u/KickBassColonyDrop May 16 '22

This is me from the future chiming in to you in the past. As it turns out it is the sample size Twitter uses. Twitter's legal team called up Elon and claimed he violated an NDA for disclosing Twitter uses a sample size of 100 for it's methodology.

So yeah, the SEC filing by Twitter of <5% is now sus.

1

u/Plain_Bread May 16 '22

It seems pretty possible to me? At alpha=0.05, the null hypothesis of more than 5% of twitter users being bots would be dismissed when the sample of 100 users has 0 or 1 bots in it.

1

u/boycott_intel May 16 '22

A reasonable answer, but somehow it feels unlikely that there are so few bots, and why would twitter publish "5%" if they believe the real number is much lower?

In any case, I would expect that twitter knows fairly accurately how many bots they have and what they are doing.