Events

In this presentation, we argue that contemporary research is situated within a broader crisis of epistemology. A crisis where Large Language Models (LLMs) play a central, structuring role. Rather than treating LLMs as neutral assistants that merely automate parts of research, we examine how they reshape the cognitive and epistemic conditions under which knowledge is produced. To explore what it means to compute within this crisis, we present an experiment using two different LLM tools – a general-purpose model (GPT-4o) and a purpose-built research system (Elicit). The application of these tools is within the practice of the systematic literature review.

Our analysis moves beyond the question of whether LLMs can replicate human labour in screening or synthesizing studies. Rather, we focus on what their application does to the epistemological foundations of systematic review work. Drawing on conceptual concepts of bullshit  and nonconscious cognition, we show how LLMs introduce forms of reasoning that resist traditional criteria of transparency, stability, and reproducibility.

To expand on these findings, we incorporate insights from workshop-based engagements with researchers at a university and a university college. In these sessions participants used an LLM to assess study eligibility. Through these sessions it was apparent that LLMs operate  as both methodological obstacles and productive disruptions in the production of systematic literature reviews. Their non-transparent and non-reproducible outputs expose assumptions, epistemic commitments, and sociotechnical imaginaries that researchers bring to systematic review practices. Rather than replacing established methods, LLMs foreground the tensions between procedural systematicity and the lived, interpretive work of research.

Across both the experiment and workshops, we show that LLMs function less as tools of automation and more as epistemic provocations: instruments that reveal how researchers themselves imagine and enact knowledge practices under postdigital conditions.