Wikipedia:Reference desk/Archives/Computing/2024 August 8

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is a transcluded archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

August 8

A new test for comparing human intelligence with Artificial intelligence, after the Turing test has apparently been broken.

1. Will AI discover Newton's law of universal gravitation (along with Newton's laws of motion), if all that we allow AI to know, is only what all physisists (including Copernicus and Kepler) had already known before Newton found his laws?

2. Will AI discover the Einstein field equations, if all that we allow AI to know, is only what all physicists had already known before Einstein found his field equations?

3. Will AI discover Gödel's incompleteness theorems, if all that we allow AI to know, is only what all mathematicians had already known before Gödel found his incompleteness theorems?

4. Will AI discover Cantor's theorem (along with ZF axioms), if all that we allow AI to know, is only what all mathematicians had already known before Cantor found his theorem?

5. Will AI discover the Pythagorean theorem (along with Euclidean axioms), if all that we allow AI to know, is only what all mathematicians had already known before pythagoras found his theorem?

If the answer to these questions is negative (as I guess), then may re-dicovering those theorems by any given intelligent system - be suggested as a better sufficient condition for considering that system as having human intelligence, after the Turing test has apparently been broken?

HOTmag (talk) 18:08, 8 August 2024 (UTC)

Most humans alive could not solve those tests, yet we consider them intelligent. Aren't those tests reductive? Isn't it like testing intelligence by chess playing? We consider some chess machines very good at chess, but not "intelligent". --Error (talk) 18:31, 8 August 2024 (UTC)

According to my suggestion, the ability to solve the problems I've suggested, will not be considered as a necessary condition, but only as a sufficient condition. HOTmag (talk) 18:39, 8 August 2024 (UTC)

It is impossible to test whether something will happen if it may never happen. The only possible decisive outcome is that it does happen, and then we can say in retrospect that it was going to happen. It does not make sense to expect the answer to be negative.  --Lambiam 21:36, 8 August 2024 (UTC)

I propose that we ask the AI to find a testable theory that is consistent with both quantum mechanics and general relativity (in the sense that either emerges as a limit). This has two advantages. (1) We do not need to limit the AI's knowledge to some date in the past. (2) If it succeeds, we have something new, not something that was known already. Alternatively, ask it to solve one of the six remaining Millennium Prize Problems. Or all six + quantum gravity.  --Lambiam 21:49, 8 August 2024 (UTC)

I suspect that any results from the test proposed in the first post would be impossible to verify. AI needs data to train on: lots of it. Where exactly would one find data on "what all physisists (including Copernicus and Kepler) had already known before Newton found his laws" in the necessary quantity, while ensuring that it wasn't 'contaminated' by later knowledge? AndyTheGrump (talk) 21:56, 8 August 2024 (UTC)n

The same problem plagues the Pythagorean theorem, which most likely was dicovered independently multiple times before Pythagoras lived (see Pythagorean theorem § History), while it is not known with any degree of certainty that it was known to Pythagoras himself. Euclid does not ascribe the theorem to anyone in his Elements (Book I, Proposition 47).^[1].  --Lambiam 22:34, 8 August 2024 (UTC)

Where tests based on intellectual tasks fail, we might need to start relying on more physical tests.

Humans take less energy to do the same intellectual tasks (I've mainly got inference in mind) as AI. While in the future it might not prove that I am a biological being, measuring my energy consumption to perform the same tasks and comparing it with that of an AI trained to do general tasks could be a way forward.

For bio-brains, training to do inference tasks is based on millions of years of evolution, the energy for training might be more than for LLMs, I don't know - but it is already spent and optimised for day-to-day efficiency. I think natural selection is an inefficient and wasteful way to train a system, but it has resulted in some very efficient inference machines.... Komonzia (talk) 04:31, 9 August 2024 (UTC)

I think all of you are missing my point. You are claiming from a practical point of view, while I'm asking from a theoretical point of view, which may become practical in a thousand years, or will never become practical.

I'll try to be more clear now:

1. If we let our sophisticated software be aware of a given finite system of axioms, and then ask our software to prove a given theorem of that axiom, I guess our sophisticated software will probably do it (regardless of the time needed to do it).

2. Now let's assume, that X was the first (person in history) to discover and prove the Pythagorean theorem. As we know, it had happened long before Euclide phrased his well known 6 axioms of Euclidean Geometry, but X had discovered and proved the Pythagorean theorem, whether by implicitly relying on the Euclidean axioms, or in any other way. Let's also assume, theoretically speaking, that we could collect all of the works in mathematics that had been written before X discovered and proved the Pythagorean theorem. Let's also assume, theoretically speaking, that we could let our AI software be only aware - of this mathematical collection we are holding (i.e. not of any other mathematical info discovered later). Since it does not include the Euclidean axioms, then what will our AI software answer if we ask it whether the well-formed formula reflecting the Pythagorean theorem is necessarily true, for every "right triangle" - according to what the mathematicians who preceded X and who wrote those works meant by "right triangle"? Alternatively, I'm asking whether (under the conditions mentioned above about what the AI is allowed to know in advance), AI can discover the Pythagorean theorem, along with the Euclidean axioms.

3. Note that I'm asking this questions (and all of the other questions in my original post, about Newton and Einstein and Gödel and Cantor), from a theoretical point of view.

4. The task of turning this theoretical question into a practical question, is technical only. Maybe, in a hundred years (or a thousand years) we will have the historical collection I was talking about, so the theoretical question will become a practical one.

5. Anyways, if the answer to my question is negative (as I guess), then may this task of re-dicovering those theorems by any given intelligent system - be regarded as a better sufficient condition for considering that system as having human intelligence? Again, as of now I'm only asking this question from a theoretical viewpoint, bearing in mind that it may become practical in some years. HOTmag (talk) 08:49, 9 August 2024 (UTC)

We can give pointers to what scientists and philosophers have written about possible replacements or refinements of the Turing test, but this thread is turning unto a request for opinions and debate, which is not what the Wikipedia Reference desk is for.  --Lambiam 10:58, 9 August 2024 (UTC)

My question is a yes/no question. HOTmag (talk) 11:01, 9 August 2024 (UTC)

OK. Yes. At some point in time there will be a case where a computing device or system of some kind will discover some proof of something. No. That isn't going to happen today. What you are truly asking is for an opinion about when it will happen, but you haven't narrowed down your question to that point yet. It is an opinion request because nobody knows the future. The suggestion is to narrow your question to ask for references published on the topic, not for a request about what will happen in the future. 75.136.148.8 (talk) 16:06, 9 August 2024 (UTC)

Again, it's not a question "about when it will happen". It's a yes-no question: "may this task of re-dicovering those theorems by any given intelligent system - be regarded as a better sufficient condition for considering that system as having human intelligence?". As I said, It's a yes-no question. Now you answer "Yes" (at the beginning of your answer). Ok, so if I ignore the rest of your answer, then I thank you for the beginning of your answer. HOTmag (talk) 16:23, 9 August 2024 (UTC)

Your extreme verbosity is getting in the way of your question. Now that you've simplified it to a single question and not a diatribe about AI math proofs, the answer is more obvious. Turing test is a test of mimicry, not a test of intelligence. So, replacing it with a different test to see if it is "better" does not really make sense. Computer programs (that could be called AI) have already taken axioms and produced proofs. They are not tests of intelligence either. They are tests of pattern matching. 75.136.148.8 (talk) 16:37, 9 August 2024 (UTC)

People with brains discovered it. A computer with a simulated brain will be able to rediscover it. Not necessarily with Generic Pre-trained Transformer algorithms, because the current generation is only trained to deceive us into thinking it can do things involving language, conversation, etc. But if a computer sufficiently can simulate a brain, no, there is nothing stopping it from following the same process that a human has done, possibly better or more correctly. In my opinion, there is no deeper soul than that which our brains trick us into thinking we have.

Note: even if simulating a brain is not possible (with every chemical reaction and neuron, if that ends up being needed), then there is nothing theoretically stopping it from being capable of growing a brain and using that -- or utilizing an existing brain. See wetware computer. Komonzia (talk) 16:52, 9 August 2024 (UTC)

Ask an AI to devise some much better test than the pitiful bunch above. NadVolum (talk) 18:19, 9 August 2024 (UTC)

See automated theorem proving. Depending on the logic involved (in particular for propositional logic and first-order logic), we have programs that will, in the spherical cow abstraction, prove any valid theorem of a set of axioms. This also implies that we can, in theory (obvious pun not intended), enumerate all valid theorems of an axiomatisation, i.e. enumerate the theory (logic). This is just mechanical computation (though there is a lot of intelligence involved if we want to prove interesting theorems quickly). Finding an interesting set of axioms is a very different task, and, I would think, a much harder one. But see Automated Mathematician for a program that discovered mathematical concepts by experimentation. --Stephan Schulz (talk) 16:31, 18 August 2024 (UTC)

The list of 5 human discoveries that the OP proposes as new "litmus tests" of capability of a computer based AI have in common that they are all special cases of general work (e.g. Pythagoras' theorem is the case of Law of cosines reduced when an angle of a triangle is a right angle so its cosine is zero) that subsequent human commentators regard as notably elegant, insightful and instructive or have been shown later to be historically important. Each discovery statement can be elicited by posing the appropriate Leading question (such as "Is it true that...."). AI has not yet impressed us with any independant senses of elegance, insight, scholarship or historical contextualization, nor is AI encouraged to pose intelligent leading questions. AI therefore fails the OP's tests. If this question was an attempt to reduce investigative human thought to a sterile coded algorithm then that attempt also fails. Philvoids (talk) 22:54, 9 August 2024 (UTC)

There is one case I know of where AI was tasked with improving an algorithm heavily looked at by humans already, and did so: https://arstechnica.com/science/2023/06/googles-deepmind-develops-a-system-that-writes-efficient-algorithms/

That case is especially remarkable because it wasn't trained on code samples that would have led it to this solution. However as User:75.136.148.8 noted above, it's not necessarily a marker of intelligence - the method used to devise the optimization is the optimization equivalent of fuzz testing. Komonzia (talk) 08:55, 10 August 2024 (UTC)

That is an interesting example, but the popular reporting is quite bad. The original Nature (journal) article is here. In particular, the approach was only used for sorting networks of small fixed size (sequences of 3-8 elements) and "variable length" algorithms with a maximum size that essentially are built by selecting the right fixed-size sorting network. This is useful and interesting, but it's very different from finding a fundamentally new sorting algorithm. --Stephan Schulz (talk) 16:40, 18 August 2024 (UTC)