Neuroscience assessment — technical notes · 1

We are starting a months-long assessment of what neuroimaging can and cannot support. The studies themselves will be published when they are finished and double-checked. This series records the technical problems met on the way, because they are instructive in their own right — and because a reader who wants to audit the result needs to know how each number was made.

The problem

Step one is a list of the most-cited papers in functional MRI, EEG, MEG, PET and near-infrared imaging. „Most cited“ sounds like a fact. It is a measurement, and like every measurement it has an instrument, a sampling frame and a drift.

Decisions taken

  • Open indices only. OpenAlex and Semantic Scholar, both with public APIs. Closed indices (Web of Science, Scopus) would give numbers nobody outside a subscribing institution can reproduce. Reproducibility outranks coverage here.
  • Two indices, kept apart. They disagree — different coverage, different reference matching, different treatment of preprints. We record both counts on every row and do not average them. A disagreement is data.
  • Every query stored with its date. Citation counts move daily. A count without a retrieval date is not a number. The raw response is written to a dated file that is never edited; re-running produces a new file.
  • Attention is not evidence. A review paper collects citations without adding a measurement. A paper can be highly cited because it is wrong. So the citation ranking is only the candidate list. The real work is the next layer: for each concept the field leans on, which papers introduced it, which supply data for it, whether those papers share a lab or a dataset, and which papers are cited as if they supported it. That is a dependency graph, and it will be published as one.

The testing analogy

Anyone who has audited a test suite will recognise the shape. A green dashboard tells you how many tests passed, not what they covered. We built a coverage-algebra check for that in our own work: which requirements are load-bearing yet covered by no independent test. The concept-support graph is the same check applied to a scientific literature. The question is identical — what is everything standing on, and is that thing itself tested?

Next note: what an index actually returns for „fMRI“, and why the first query was wrong.