Since the AoE 2 paper seems to have picked up steam (ayyy), and IT IS 13 dense af PAGES, I would like to clarify a few things:
1. I do not deal with consciousness bc it isn't well-defined nor measurable,
2. The AoE II bit is for substrate dependence, and
3. The core argument/proof of the paper is that claiming existence (or lack thereof) human-like attributes needs better experimental setups.
Using AoE II (and later Boston) is to emphasise the distinction between our interpretation of what AI does when we observe it, vs what it does.
E.g., an LLM outputting an explanation is a sequence of tokens. Calling it an 'explanation' is our observation of it. Assuming it means something (like self-awareness) is an assumption impacting your experimental setup and thus your conclusions.
The paper is dense because one needs to be careful when providing these types of arguments (otherwise, trust me, it'd been a lot shorter). So I wrote a less-formal, more-digestible thing here:
显示更多