# Social media forensics: the hard part is not fooling yourself

> I built a skill that reads a scraped crowd for coordinated behaviour, and the hard engineering was not finding a signal but stopping the tool from calling every signal a conspiracy.

Social media forensics is extracting and analysing digital evidence from social platforms for an investigation. This skill is the coordination corner of it: I built it to read a scraped pile of comments or posts for brigading, sockpuppet accounts, and a set of channels quietly run as one operation. It is the reading half of a pair, one tool collects the rows and this one interprets them. Where [Social Engineering Detection](/garden/social-engineering-detection) is about one attacker working one target, this is about the manufactured many. Every check it runs is built around one problem: the patterns that look most like a conspiracy are the ones most likely to fool you.

![From a flag to a finding. Steps: 1) Collect to a file: The scrape stays on disk, never in the agent's context; 2) Free checks first: Text reuse, bursts, cadence, accounts, overlap; 3) Cohesion, not density: A burst counts only if distinct accounts say one thing; 4) Authors, not rows: One loud account is a repeat poster, not a crowd; 5) Bill only what is left: Paraphrase and image templates, ranked in the corpus; 6) Second channel plus control: One channel is a lead, agreement is a finding; 7) Report what you tested: A clean check clears that channel, not the case](/garden-assets/experience/atom_notes/social-forensics/social-forensics-flow.svg?v=2caf811208)

The discipline as a funnel. Each step throws out one way a signal fools you, and only a flag that survives all seven earns the word finding.

## It reads the crowd without the crowd entering my context

This is a separate skill for one reason: a thousand comment rows would land in the [agent's context window](/garden/agentic-engineering) and compact it mid-task. So the corpus is collected to a file and read into summaries. The rows stay on disk, only the finding comes back.

## The checks it runs

Each check answers one narrow question. The free ones (standard library) run first; the two billed checks embed through an API and only earn their cost once the free channels have been read.

| Check                 | The question it answers                                                               |
| --------------------- | ------------------------------------------------------------------------------------- |
| `summary`             | How much did I get, from how many distinct people, and what are the loudest comments? |
| `themes`              | What narratives are here, and how many independent authors carry each?                |
| `duplicates`          | Is matching text coming from different accounts, or one account repeating itself?     |
| `bursts`              | Did many distinct accounts arrive together and say the same thing?                    |
| `cadence`             | Does anyone post faster than a person can type? Does the crowd have a night?          |
| `accounts`            | Do the accounts look mass-created? Is anyone flooding links?                          |
| `overlap`             | Who shows up on more than one post, and is that more than chance?                     |
| `semantic` (billed)   | Is anyone making the same point in different words, or another language?              |
| `thumbnails` (billed) | Are two accounts serving the same image, or building to one shared template?          |

How each check actually works, and why none of the free ones use machine learning, is its own note: [the methods behind them](/garden/measuring-coordination).

## How it refuses to overclaim

**Density is not coordination.** A hundred near-identical comments in five minutes looks like a brigade and is usually a post going viral. A burst counts only when many distinct accounts arrive together and say the same thing, the cohesion gate. Density is printed as context, never as the test.

**Count authors, not rows.** Twenty comments from one account is one person, and a ratio near one author per comment is exactly what a competent sockpuppet operation aims at. Every count comes paired with its distinct-author count, so the output is what you tested, not what you want to conclude.

**Two channels, or it is only a lead.** Every check flags something on every corpus, so no single flag is a finding, and a clean result is about that one channel only. Six thumbnails from three suspect accounts, all built to one composition, sat a median 31 bits apart out of 64, so a hash calls them unrelated; only an embedding channel against a control sees the shared template. For a load-bearing claim, run the same checks on a thread nobody suspects and compare the flag rates.

## What a run looks like

`duplicates` on a small demo corpus does the one thing the verdict rests on: it flags the two different accounts posting the same line, and hides the account repeating itself, because that is a repeat-poster, not a network. The corpus is a toy, the numbers are the tool's.

```text
$ social_forensics.py duplicates launch-post.json
28 comparisons at threshold 0.7, min length 40
1 cross-author clusters, 1 same-author clusters

== cross-author: 2 rows, 2 authors, both 1.0-1.0 (verbatim)
   authors: Carol M, Dave R   |   posted 89s apart
   Carol M: This is exactly the kind of update we needed today...
   Dave R:  This is exactly the kind of update we needed today...

(1 same-author clusters hidden: one account repeating itself is a
 repeat-poster, not a network.)
```

## What it cannot see

It never names who made anything: a reused file, a shared template and a shared point are three findings, none of them an author. It misses paraphrase unless the paid semantic channel ran, misses replies it never collected, and cannot run co-occurrence network analysis, the strongest method in the literature, on a three-post sweep, since that needs roughly fifty distinct items. A null result licenses a narrow sentence, never a broad one.

## Where the method comes from

I did not invent these rules. The four-questions discipline, keeping coordination, authenticity, source and impact separate, is EU DisinfoLab's [CIB detection tree](https://www.disinfo.eu/publications/coordinated-inauthentic-behaviour-detection-tree/), and each signature carries a [DISARM](https://github.com/DISARMFoundation/DISARMframeworks) technique id so a finding drops into a standard writeup. The caution is empirical: co-occurrence networks are the strongest method (Pacheco et al., [ICWSM 2021](https://ojs.aaai.org/index.php/ICWSM/article/view/18075); null form, [Tumminello et al. 2011](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0017994)) but degenerate below roughly fifty items, shown directly on [TikTok](https://arxiv.org/abs/2505.10867) comment data; the two-channel and control rules come from claims that [dissolved once controls were applied](https://arxiv.org/abs/2502.17344); the cohesion gate follows [Magelinski, Ng and Carley](https://arxiv.org/abs/2105.07454); a [survey](https://arxiv.org/abs/2408.01257) maps the rest.

## The bottom line

The payoff is not any single detector. It is that the tool will not let me overclaim: it counts authors instead of rows, gates a burst on agreement instead of volume, and refuses to read a channel without a control. Most OSINT work is exactly this, telling a sound trail from a plausible-looking one, and most of that is saying what a check tested and then stopping.