Turning a PDF bibliography into linked notes

Why reference lists convert badly

Reference lists are set with a hanging indent: the first line of each entry starts at the left margin and every continuation line is indented. That is the opposite of a paragraph indent, and it is the shape that causes the trouble.

List nesting depth is inferred from how far a line's left edge sits from the document's usual left margin. In a hanging indent, the continuation lines sit further right than their own first line, which is exactly the geometry of a nested list item. Some entries therefore arrive as a top-level item with children under it, and which ones depends on the reference style and the width of the author names.

Numbered styles add a second problem. An entry beginning with a bracketed or plain number is recognised as an ordered list item, so a numbered bibliography converts into a numbered list, which is nearly right. An author-date style beginning with a surname is recognised as nothing at all, so it converts into paragraphs — which is also nearly right, and differently so.

What you actually get back

Expect one of three shapes, depending on the style of the original.

Normalise to one line per reference first

Every later step is easier if each reference occupies exactly one line, so do that before anything else. It is the only step that genuinely needs your eyes, because deciding whether two lines are one reference or two requires reading them.

The reliable signal is the year. Almost every citation style puts a four-digit year in a fixed position, so a line without one is nearly always a continuation of the line above it. Sorting that out is a single pass down the list, and once it is done the rest is mechanical.

The pass that does most of the work

With one reference per line, a single find-and-replace turns the list into note links. This is the author-date case; the numbered case is the same idea with a different pattern.

Find:    ^([A-Z][A-Za-z'-]+), ([A-Z]\.(?: [A-Z]\.)*) \((\d{4})\)\. (.+)$
Replace: - [[$1 $3]] — $4

Before:  Mehta, R. K. (2023). Distributed systems in practice.
After:   - [[Mehta 2023]] — Distributed systems in practice.

Why the link text should be short

It is tempting to make the note name the full title, which is more informative in the list. It is the wrong choice, and you find out several hundred notes later.

A note named for surname and year is the name you will type when you cite it, which means autocomplete finds it from four keystrokes. A note named for a full title is one you will search for rather than type, and the link you write by hand a year from now will not match it. The title belongs in the note, on the first line, where it is readable and searchable but not load-bearing.

The one case for a longer name is an author with two papers in one year, which every convention solves the same way, with a letter suffix.

What not to automate

• Digital object identifiers. They are frequently split across lines in the PDF and rejoined wrongly, and a wrong identifier resolves confidently to a different paper. Check each one against the source rather than trusting the conversion.
• Author disambiguation. Two different researchers sharing a surname and a publication year will collapse into one note, and nothing in the text can tell you that happened. Only you know the field.
• Page ranges in the reference itself. En dashes, hyphens and minus signs all appear in real bibliographies and they are visually near-identical, so a script that parses ranges will be subtly wrong often enough to be worse than no script.
• Anything in a reference that was set in small capitals. That styling carries no information in the file, so a name set in small caps arrives in whatever case the font actually used, which is sometimes all capitals.

Keep the original, and say where it came from

Each generated note should record the document its reference came from and the page it was on. It costs one line and it answers the question you will actually have later, which is not what the paper said but where you found the citation.

Keep the PDF too. A converted bibliography is a working index, not a replacement for the source, and the moment a reference looks wrong the first thing you will want is the page it was printed on.