Lec 2: reading matters

A. issues with trusting AI answers: correctness and difficulty

  1. ask the same pinned LLM a textbook question, then one just past the worked-out literature. Both answers sound fluent, only the first can be checked, and nothing in the response marks the difference
  2. first tempting fix, ask the LLM how confident it is. It gives you a number that means little on its own: calibrated confidence (2023) takes targeted prompting or post-hoc scaling, and detecting confabulation (2024) takes several samples clustered by meaning
  3. second tempting fix, reasoning models. The trace feels like it shows why the answer is right, but it may rationalize after the fact (1977) instead of reporting the computation. Think-aloud protocols (1980) are the ancestor, except there the person thinking did the talking; here one system writes both the steps and the answer, so reading it is not verifying it
  4. the disconnect is measured: meaningless filler tokens let transformers solve tasks they fail when answering head-on (2024), and models trained on corrupted traces whose steps bear no relation to the problem (2025) perform like models trained on correct ones. A persuasive trace is not evidence
  5. third tempting fix, ask a second provider. Disagreement is useful; agreement is weak, since shared training data, architecture, and fluency incentives correlate their errors (algorithmic monoculture, 2021), and measured on mistakes rather than answers, model errors overlap well beyond chance and more so as capability rises (ICML 2025). Independence is a property you argue for, not assume; auditing a model that judges a model is W10’s topic
  6. even assuming LLMs always give a technically correct answer, can we safely just delegate?
  7. how hard a question feels says little about how checkable the answer is; a hard-feeling textbook derivation can be checked step by step, while an easy-feeling question about one’s own project may have no source to check against
  8. nor is difficulty visible in the request itself. A team ran an LLM router for four months and withdrew it (2026), since what makes a task hard shows up later, in what the tool calls turn out to require
  9. so the delegation call stays yours even on optimistic assumptions; nothing along the tech stack (the mechanisms of objective, context, and sampling, covered in W8, the machinery beyond vanilla transformers) signals it

B. the same two questions at the frontier

  1. the systems from W1 return, and both leave the same work behind. Co-Scientist (Nature, 2026) is models checking models, generation agents feeding a ranking tournament with a reflection agent as reviewer, running lit review, generation, and ranking at scale, while validity still rests on evaluation and selection outside the loop. The field report (2026) saw it from inside: agents handled well-scoped requests with the difficulty sorting already done by humans, the validity call stayed with the researchers, and their time moved from writing code to defining how correctness would be measured, now more managers than engineers
  2. past the literature and past checkable are different boundaries. A counterexample to a flow conjecture open since 1999 came out of a four-prompt chat (2026), and the object is eight routings on seven nodes, so anyone can settle it in an afternoon; an independent verifier did that the next day while the trackers still list it unreviewed, no paper and no referee. Checking against credible sources is one way to verify, not the only one; here the check came from understanding the question well enough to run it yourself, a small case of W3’s claim that building grows what you can check
  3. what that case does not prove: the conjecture was a for-all claim, which one object can kill; most questions, including the ones a UROP or MEng project poses, ask how much or whether something works, and under what conditions; no single object decides those. One reported success also says nothing about the prompts that returned plausible non-counterexamples.
  4. indeed, the opposite failure exists too: the first claimed LLM solution to an Erdős problem turned out to have been solved by Erdős himself in 1977 (2026), nobody had checked the literature.
  5. both stories above start after the hard step, someone had already framed the question as a finite mathematical object, and the taste to frame it that way comes from having read deeply in the area
  6. a disagreement worth putting in front of the room, since the staff do not agree either. One view holds that a proof is not a cure for cancer, where the product matters and the producer does not; a proof is an exercise whose value was what the prover learned, including the side findings a person notices and a system aimed at one goal never looks for. The counter is that a proof is also an object, and objects move work: within weeks of the unit-distance counterexample (2026), four mathematicians used techniques from it to disprove a related sum-product conjecture. The side finding arrived, through people reading what the machine produced
  7. the human job, then, is framing, evaluation and selection

C. issues with AI-assisted reading

  1. parts A and B asked what you can tell about the model’s answers, correctness and difficulty; this part asks what reading builds in you (a working map of the field), and what outsourcing it costs.
  2. in a controlled CS1 experiment (2024), students with an assistant stopped looking things up elsewhere and learned no more than those without. Agreeing with a summary is the cheapest of the three acts (agree, verify, read), and two results say why that costs you: the generation effect (1978), material you produce is remembered better than material you receive, and desirable difficulties (1992), conditions that slow learning down while it happens improve what survives
  3. sometimes the summary is all you need; the judgment is knowing which papers you will build on
  4. what a summary misses even when every sentence is right: the delta lives between documents, in relations a single-document summary cannot carry. A famous paper gets recited from reputation, an obscure one parroted from its abstract, and neither places the work
  5. an engineering paper’s central claim is its delta, and the four-question recipe (motivation, solution, old idea, delta) unpacks that claim
  6. the four-question recipe has its own ancestor, the Heilmeier catechism (mid-1970s), which asked these questions of funding proposals; we ask the same questions of technical papers, and the answer is allowed to be “nothing, honestly”
  7. the mechanism under the missing delta: the R in RAG is the vector space model (1975) with learned coordinates, retrieval ranks by similarity and generation continues toward the likely, so an unprompted summary drifts to what is typical of its sources
  8. and similarity is always relative to an anchor. term specificity (1972): a term is specific only against a chosen collection. A reader who has read deeply carries a concept map, so who to compare against already encodes the prior; a cold question gets generic neighbors, many of them and none ranked by what your task needs

D. reading practice that works

  1. before retrieval and before generation, the dream ‘research’ machine was one that kept a reader’s trails: the memex (1945) would store what a reader connects, so a path once found could be followed again later
  2. one idea, have the model maintain a wiki of what you have read, folding each new source into pages that already exist, flagging where it contradicts something earlier, filing your own good answers back as pages (the LLM-wiki pattern, 2026). The claim is compile rather than retrieve, each source integrated once instead of the pile searched again per question. The page indexes what you read rather than replacing it, a one-line entry tells you which article to open
  3. Good class notes were never a transcript; the rewriting into your own structure is what made them worth returning to, similarly for cheatsheets; the wiki is that move applied to reading
  4. the same move at the single-paper grain: say your understanding to the model and let it correct you, then write paraphrase notes of the paper’s motivation and methods once the conversation has tested them; this is part C’s generation effect with the checking added, and the notes reread later without rereading the paper; an example from your peer (2026) alongside the conversations that produced it (architecture, guidance)
  5. skim first, delegate second: one pass over the whole paper for what it covers, then the model explaining the sections you care about, checked against your mental model. Ask when you get stuck, not when you start, and name the methodology step or the claim you cannot follow rather than the paper. Figures are what a summary-only reading never sees
  6. the wiki idea is similar to the Zettelkasten, which grew to about 90,000 cross-indexed cards from 1952 onward and stood behind roughly 50 books and 550 articles, and its owner did the filing by hand for four decades. The memex was never built for the same reason nobody else copied the practice. What changed is the price of the bookkeeping
  7. the memory layers now shipping in AI tools reach for the same thing; the difference is who owns the structure, here the reader does, and it hands the model hand-picked neighbors where a cold question gets only generic ones

Lab 2

  1. open with a live read: everyone gets the same unfamiliar arXiv paper and ten minutes to answer three staff-written questions about it, then a staff member works the same paper in front of the room, saying out loud what they skip, what they read twice, and where they went for each answer. The lesson is the order and the skipping, not the speed
  2. draft three candidate toy research questions: no existing or readily available answer key, answerable by the student this semester, and one the student wants answered
  3. the boundary test asks one thing about a question: does an answer key already exist somewhere? Run all four, your three plus one staff-provided control that looks open but has a published answer; the control shows what both checks look like on a question with a known answer, the reference to read your three against
  4. first check, sampling: ask the same pinned model the same question several times and read whether the answers agree in substance rather than in wording (detecting confabulation (2024), done by hand). Answers that agree suggest the model has seen an answer, so a key probably exists; answers that scatter suggest the question sits past the boundary
  5. second check, the literature: search for a confirming source and write down where you looked and what stays uncertain; a second model agreeing does not count as a source, since two models agree about twice as often as two people (2026)
  6. a question “survives” the test if its answers scatter and no source turns up; pick among the survivors with a partner’s critique and commit one as the semester’s question
  7. pick a would-be neighbor paper for the committed question and produce an AI summary of it, then skim the full paper, figures included, before auditing
  8. point out where that summary shows the common symptoms (dropped limitation, detail drift, etc.); a summary is supposed to leave things out, so omission alone is not a defect; a dropped detail counts as a finding only when putting it back would change what the reader concludes or does next, a dropped limitation usually does, a dropped implementation detail usually does not; for the factual ones record the source of truth in the paper and the correction, and for the rest cite the page that shows what the summary lost; mark where the delta went and which relations to prior work never appear
  9. start the committed question’s wiki page: the question, the audited neighbor plus two to four more papers as the starting anchor set, and one line each on how the neighbor relates; this is the map from part D, and the page grows as W7 sources the data, W9 evaluates, and W12 writes up
  10. looking ahead: finalize the question against the chosen build in W3

Page updated August 28, 2026.