flâneur

Carla Ostmann

6 followers · 3 following · 376 views

on the atlas — 44

highlights — 393

  • flagging papers that use unusual combinations of keywords or that cite papers in journals that aren’t often cited together
    How novel is that research paper? Competition to quantify concept crowns winner | Science | AAAS
  • 41,600 researchers to score 37,480 recently published papers on a 100-point scale
    How novel is that research paper? Competition to quantify concept crowns winner | Science | AAAS
  • At that scale, funders can see neglected bottlenecks across many fields at once, regulators can trace a claim through evidence, corrections, and dependencies, and models can propose discriminating tests against the current field instead of a private scrape of papers.
    Constellations of Borrowed Light
  • A failed trial could surface wherever the target hypothesis was being reused, with the affected scope marked explicitly. A failed experiment would enter the shared record before the result is written up, so the next team starts from the current field instead of rebuilding a private map.
    Constellations of Borrowed Light
  • A record gets written only when writing to it is the cheaper path, the way a commit is cheaper than mailing a patch around.
    Constellations of Borrowed Light
  • The missing layer sits beneath the archive, where what an archive holds can change. A writable substrate gives AI a place to deposit what it produces; the frontier is the object a scientist reads and writes.
    Constellations of Borrowed Light
  • Underneath all of those is the same missing thing: structured dependencies between scientific objects (claim to evidence, claim to contradicting trial, claim to retraction, claim to dependent claim). Whatever isn’t recorded as a dependency does not propagate when something changes. Anyone who has joined a mature project knows the feeling: before contributing, you first have to rebuild the missing map.
    Constellations of Borrowed Light
  • The incentive is a trap: each researcher is locally rewarded for publishing more, and cited more for it, while the field is globally worse off as its collective attention narrows onto the same dominant prose.
    Constellations of Borrowed Light
  • The error-correcting tradition only works when a correction can find the claim it corrects; a field producing more work than the paper system can carry loses that property by default.
    Constellations of Borrowed Light
  • A star can go dark and keep shining for years. Amyloid worked the same way: the simplest version had failed in trial after trial, and the field designed the next one around it anyway.
    Constellations of Borrowed Light
  • Amyloid-hypothesis papers and review narratives continued to circulate while failed trials lived in separate registries and therapeutic trackers. No canonical dependency layer forced the original claim and the contradictory trial record to travel together so a downstream reader saw both at once.
    Constellations of Borrowed Light
  • What it lacks is the first layer beneath all of those: a writable substrate where a frontier can be updated, inherited, and used by the next researcher, clinician, or agent.
    Constellations of Borrowed Light
  • The question stops being how to produce more candidates and becomes the harder one: which candidates deserve scarce experiment time and human trust.
    Constellations of Borrowed Light
  • A frontier is the current map of what can be acted on, with gaps inside it and unknowns beyond it.
    Constellations of Borrowed Light
  • Production was never the whole of science. A field advances by generating variation, many hypotheses, methods, and competing readings of the same data, and then doing the slower work that follows: selecting what survives scrutiny, retaining what failed so the next group does not repeat it, and carrying corrections back to the claims they overturn. AI has made the first half cheap and left the second half where it was. Cheap variation only compounds if selection and memory keep pace, and they have not.
    Constellations of Borrowed Light
  • what happens when a field produces more work than it can keep track of.
    Constellations of Borrowed Light
  • Lenat concluded that there really is no way to get a working automated discovery program without doing the hard work of hand-coding in a lot of common sense, and that there would be a point at which this system would finally achieve escape velocity, and would never be exhausted again.
    Cyc – Yuxi on the Wired
  • Lenat designed a new language called RLL (“Representation Language Language”)
    Cyc – Yuxi on the Wired
  • 2th lesson: Representation matters a lot. AM worked so well for mathematics, because AM used Lisp code as data. Lisp is the perfect tool if you want to search over the space of interesting mathematical functions. You can modify a Lisp expression, and get a different mathematical function that is possibly interesting. In contrast, if you were to modify assembly code, you’d most likely end up with nonsense. Indeed, Lenat found that he could not extend AM to “go meta” and discover new heuristics, because Lisp is good for math, not “heuretics” (the study of heuristics). Modifying a Lisp expression…
    Cyc – Yuxi on the Wired
  • discovered mathematical concepts autonomously by distilling concepts out of patterns and remixing and stirring together previous concepts
    Cyc – Yuxi on the Wired
  • If you had a big enough neural net then, yes, you might be able to do whatever humans can readily do. But you wouldn’t capture what the natural world in general can do—or that the tools that we’ve fashioned from the natural world can do. And it’s the use of those tools—both practical and conceptual—that have allowed us in recent centuries to transcend the boundaries of what’s accessible to “pure unaided human thought”, and capture for human purposes more of what’s out there in the physical and computational universe.
    What Is ChatGPT Doing … and Why Does It Work?—Stephen Wolfram Writings
  • the reason a neural net can be successful in writing an essay is because writing an essay turns out to be a “computationally shallower” problem than we thought.
    What Is ChatGPT Doing … and Why Does It Work?—Stephen Wolfram Writings
  • he regression (or any other machine learning tool) learns the patterns in the data, and then a human interprets these patterns as measures of causes. In that presentation, I showed that with this approach more data can actually produce worse inference.
    elevanth.org/2021_06_21_RFDT_2_of_3.html
  • People think more like graphs than like regressions.
    elevanth.org/2021_06_21_RFDT_2_of_3.html
  • An image recognition AI thinks like a regression. It’s job is the compress patterns into some algorithmic form that can categorize and repeat those patterns. The AI, like a regression, knows nothing about the causes of those patterns. So it is confused by superficial similarities—it has no way to judge them as superficial—and can be fooled by tiny changes that would not fool any person.
    elevanth.org/2021_06_21_RFDT_2_of_3.html
  • What I mean is research has sufficient documentation and justification to reduce error and empower others to make up their own minds about its value. Research should be intelligible.
    Which Kind of Science Reform | Elements of Evolutionary Anthropology
  • it’s foreseeable that nations will stop transfering public money to private publishers and start requiring that all research be deposited in central repositories.
    Which Kind of Science Reform | Elements of Evolutionary Anthropology
  • The biggest problem is the leadership of the institutes, the directors, people like me. Most of us got appointed by being successful in a system that lacks the reforms that are needed.
    Which Kind of Science Reform | Elements of Evolutionary Anthropology
  • In the future, we should have software assistance that helps us integrate theoretical models with research designs, statistical models, diagnostics, and result summaries.
    Which Kind of Science Reform | Elements of Evolutionary Anthropology
  • Workshops for grant writing are commonplace. Workshops for doing research are rare.
    Which Kind of Science Reform | Elements of Evolutionary Anthropology
  • By focusing relentlessly on the essential feature of a problem while ignoring all other aspects. The simplicity of his model of communication is a good illustration of this style. He also knew to focus on what is possible, rather than what is immediately practical.
    Quanta Magazine
  • So in a radio system, for example, even though both the initial sound and the electromagnetic signal sent over the air are analog wave forms, Shannon’s theorems imply that it is optimal to first digitize the sound wave into bits, and then map those bits into the electromagnetic wave.
    Quanta Magazine
  • a notion we might call “Limit Thinking.” This abstract approach forces one to focus solely on the features of a system essential for its performance, so that one can make predictions or evaluations, regardless of the specifics of how each individual system is built.
    Limit Thinking
  • If the total number of patients included in a systematic review is less than the number of patients generated by a conventional sample size calculation for a single adequately powered trial, consider rating down for imprecision.
    GRADE handbook
  • We are more confident in the results when we have direct evidence. Direct evidence consists of research that directly compares the interventions which we are interested in, delivered to the populations in which we are interested, and measures the outcomes important to patients.
    GRADE handbook
  • All statistical approaches have limitations, and their results should be seen in the context of a subjective examination of the variability in point estimates and the overlap in CIs.
    GRADE handbook
  • Therefore, when we refer to inconsistencies in effect size, we are referring we are referring to relative measures (risk ratios and hazard ratios, which are preferred, or odds ratios).
    GRADE handbook
  • Investigators should explore explanations for heterogeneity, and if they cannot identify a plausible explanation, the quality of evidence should be downgraded.
    GRADE handbook
  • Chapter 8 of the Cochrane Handbook provides a detailed discussion of study-level assessments of risk of bias
    GRADE handbook
  • make the reasons for their ultimate judgment apparent.
    GRADE handbook
  • one should be confident that there is substantial risk of bias across most of the body of available evidence before one rates down for risk of bias.
    GRADE handbook
  • The judicious consideration requires evaluating the extent to which each trial contributes toward the estimate of magnitude of effect. This contribution will usually reflect study sample size and number of outcome events – larger trials with many events will contribute more, much larger trials with many more events will contribute much more.
    GRADE handbook
  • In deciding on the overall quality of evidence, one does not average across studies
    GRADE handbook
  • Expert opinion is not a category of quality of evidence.
    GRADE handbook
  • Source of control group results is implicit or unclear, thus, they will usually warrant downgrading from low to very low quality evidence.
    GRADE handbook
  • quasi-RCT) without important limitations also provide high quality evidence, but will automatically be downgraded for limitations in design (risk of bias) – such as lack of concealment of allocation and tie with a provider (e.g. chart number).
    GRADE handbook
  • observational studies without special strengths or important limitations provide low quality evidence
    GRADE handbook
  • randomized trials without important limitations provide high quality evidence
    GRADE handbook
  • The GRADE approach to rating the quality of evidence begins with the study design (trials or observational studies) and then addresses five reasons to possibly rate down the quality of evidence and three to possibly rate up the quality.
    GRADE handbook
  • GRADE is “outcome centric”; rating is done for each outcome, and quality may differ - indeed, is likely to differ - from one outcome to another within a single study and across a body of evidence.
    GRADE handbook