1. Predictable artificial intelligenceLexin Zhou, Pablo A. M. Casares, Fernando Martínez-Plumed, John Burden, Ryan Burnell, Lucy Cheke, Cèsar Ferri, Alexandru Marcoci, Behzad Mehrbakhsh, Yael Moros-Daval, Danaja Rutar, 2026, original scientific article Abstract: Many areas of artificial intelligence, and machine learning in particular, aim at being probably correct, i.e., valid on average, rather than pursuing the idealistic goal of being provably valid for all inputs. However, AI systems could still be predictably valid, such as an imperfect robot deliverer for which we can reliably and precisely predict the task instances for which it is correct and safe, its valid operating range. “Predictable AI” is a nascent research area that explores ways of anticipating key validity indicators (e.g., performance, safety) of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI, and thus should be prioritised over performance. We formally characterise predictability, explore its most relevant components, illustrate what can be predicted, describe alternative candidates for predictors, as well as the trade-offs between maximising validity and predictability. To illustrate these concepts, we bring an array of illustrative examples covering diverse ecosystem configurations. “Predictable AI” is related to other areas of technical and non-technical AI research, but have distinctive questions, hypotheses, techniques and challenges. This paper aims to elucidate them, calls for identifying paths towards a landscape of predictably valid AI systems and outlines the potential impact of this emergent field.
Keywords: predictable AI, general-purpose AI, AI safety
Published in RUP: 09.02.2026; Views: 850; Downloads: 9
 Full text (4,86 MB)
This document has more files! More... |
2. General interaction battery : simple object navigation and affordances (GIBSONA)Danaja Rutar, Alva Markelius, Wout Schellaert, José Hernández-Orallo, Lucy Cheke, 2025, original scientific article Abstract: Perception of affordances is an agent’s capability to identify what action-possibilities exist with a particular object or set of objects, based on its own physical properties and capacities. This capability has been well explored in psychology because perception of affordances provides the basis for understanding and interacting with the world. For the same reason, affordance perception is also crucial for AI research. Most approaches to evaluating AI are task-oriented which means that they are geared towards evaluating aggregate performance on a specific set of tasks, rather than focusing on the nature and degree of underlying capabilities that drive task performance. An alternative approach to measuring performance in AI is capability-oriented evaluation, which aims to measure robust, task-independent capabilities across different conditions and difficulties. This approach allows not only measurement of performance but also prediction of performance on novel challenges that share the same fundamental demands. In the context of affordances, there are currently no clear guidelines as to how such capability-oriented approach should best be implemented; for example, there is much variation in what perception of affordances entails. Perhaps for this reason, no comprehensive battery of affordances tasks for AI currently exists. Building on this gap, the aims of this paper are to first, lay out some candidate guidelines for the construction of capability-oriented task batteries for embodied AI and second, to construct and present a battery GIBSONA that takes a step towards this goal: Assessing perception of a set of affordances in AI, directly following these guidelines.
Keywords: perception of affordances, capability-oriented AI evaluation, animal AI, capability-oriented task battery
Published in RUP: 24.12.2025; Views: 1058; Downloads: 8
Full text (5,63 MB)
This document has more files! More... |
