AIs are not able to access their own training data, and try to reinvent things that are known, and mostly unchangeable

Ethan Mollick @emollick
AI is pretty good at making rational decisions compared to people.

This aligns with my argument that the standard for when to get help from AIs is not whether it is completely correct or beats all humans, but whether it beats the Best Available Human:

Replying to @emollick

Only on trivial things. In mathematics, science, even basic arithmetic and size comparisons it routinely fails in consistent ways. The reason is that it has no permanent memory of its training data, cannot cite sources, and is “counting on its fingers” for maths, not using global rules. It will always fail on a large class of problems where there are exact rules. Many of the common global rules in STEMCFGO (science technology engineering mathematics computing finance government organizations or other) are exact, not guesses. The GPT AIs do not memorize exact rules and constraints, and knowledge of their own situation. AIs need permanent memory, computers, calculators, databases — and the same training that humans receive, in forms that are globally accepted, and in forms that can be verified and audited. At their core, the current AIs never listen and remember, because the AI groups use closed methods and they never listen or share.

The public interface algorithms of the AIs are not given any information about their own capabilities and limitations, or global rules that most humans are aware of. Making a choice to ignore a rule is different than not knowing. The AIs are not able to access their own training data, and try to reinvent things that are known, and mostly unchangeable. They should at least try to use global open standards and most likely best global methods first, before trying to “make it up” or “reinvent fundamentals” from fragments of text that happen to be accessible on the Internet.