Started with an example from personal experience showcasing both negative and positive confirmation bias. Small discussion on whether using AI is actually efficient: this is a highly granular question and depends on the specifics of the situation (i.e. as a group we had examples of where it clearly was efficient and also where it clearly wasn't). Discussion on whether AI-generated code can be trusted any more than code taken from a git repository, and a larger discussion on trust with an observation: scientists are good at finding mistakes of other scientists - if we start relying on LLMs to do part of our work, we need to become good at finding mistakes that agents/LLMs do. Detour: creative work vs. chores and when can you offload work to an AI, chosen as discussion topic next meeting. Goal of agentic AI effort in EDUCATE:
- Learn about agentic AI, what are the possibilities and the limitations?
- Build a research agentic AI system, with example use-case being exploratory studies on datasets to evaluate scientific potential
The value of the AI system here is the efficiency: time and effort can be more focused by streamlining this exploration phase. Discussion on how to accurately represent this value: how much time and energy should you spend on documenting failed experiments? There is a bias in how we only tell each other about when LLMs help us, and not when they waste our time, but this is true for all research, not just LLM experiments. This is a known issue, and historically attempts at fixing it have not persisted. Weren't able to reach a conclusion on how to deal with this in the EDUCATE setting.