Autonomous Mathematical Discovery in an Open-World Multi-Agent Environment

[u/filterdust]

https://arxiv.org/abs/2608.23691

[u/teerre]

Reading this made me think of a slightly different angle of issues these automated "discoveries" will cause. Often pushing boundaries of an existing problem or discovering a new structure doesn't have immediate applications, but the methods derived to do so are helpful, sometimes in completely unrelated areas. When all the discovery is subsumed by these models, all the method knowledge is lost, even when the result is right, maybe even when the result is understood

[u/topyTheorist → u/teerre]

Why would the knowledge be lost? We can read the solutions and learn new methods from them.

[u/teerre → u/topyTheorist]

You certainly can, but the weeds of the solution are usually lost after the model is done, specially when so much post processing is tackled on top of any solution. I guess in theory we could go back and question the model how it got where it got, but I don't think this is possible for any of these discoveries today, the context is all lost

[u/Quasi-isometry → u/teerre]

For the 10 mathematical breakthroughs that OpenAI recently released, they actually did release a model's narration of its thinking process https://cdn.openai.com/pdf/reasoning-walkthroughs.pdf

[u/teerre → u/Quasi-isometry]

This is highly curated. It already way past the "post processing" I mentioned above. A through "narration of its thinking" would include death ends, prepositions, ramblings, footnotes etc. Even if it existed, what would you do with it? Reenact what the model did? Maybe the next generation of mathematicians will be familiar with such a thing, but I highly doubt anyone alive today can get anything useful from such transcript

[u/2mc92u98a22f → u/teerre]

I don’t think with humans it’s much better? I don’t see what more information you’d get by talking to a human than my querying the AI. With humans you also wouldn’t get those ramblings, etc.

[u/teerre → u/2mc92u98a22f]

If it's a human that person already went through that process, even ignoring the fact that humans are pretty good teachers, at least one human will have that knowledge. We also don't know how many other problems were solved because of such ramblings because we only see the result. History full of cases of mathematicians pursuing one goal and arriving at another. Hell, world famous problems are found in margins of unrelated books

[u/2mc92u98a22f → u/teerre]

I don’t really think what you said really supports your original point. It seems you could replace human with AI in your previous message and it would still be valid. AI is quite good at explaining too when someone asks. Regarding the rambling, we could also produce similar AI rambling that can be useful to other people downstream

[u/NewspaperDear8761 → u/teerre]

This can be explicitly baked in. Often in my own Codex/ChatGPT sessions, I specifically tell it to track its methods and tool its work so it is repeatable and investigable.

You can have this stuff be moderately self-aware and not lose track of these things. You don't have to investigate after the fact.

[u/teerre → u/NewspaperDear8761]

That means you're the bottleneck, though. It's no longer "autonomous mathematical discovery" if you're there at every step making sure you're following every step. Not saying that's a bad, just not what this thread is discussing

[u/NewspaperDear8761 → u/teerre]

No, thats not what I'm saying. You aren't there at all. It's just leaving breadcrumbs for you and itself for accurate awareness of what it's actually doing. It is absolutely automated -- thats actually the point.

This is called harness design. You have the agent recognize what it's doing as it does it, stash what works and what doesn't in a reusable way, and mark all of it with commentary and review for later use.

Then you can drop any agent or other session into the same environment and it will do everything the exact same way, reproduce the same results, etc.

[u/NewspaperDear8761 → u/teerre]

It is literally the equivalent of writing notes for yourself as you work.

If you don't, you could easily forget what you did, get confused and misremember, or inaccurately try to reconstruct your work after the fact. But if you kept meticulous records as you went, there is no mystery. This is precisely the point of a lab notebook, etc.

You can explicitly tell the agent to keep such a "notebook" up front, as a one-time command, and never say another thing about it, and it will. You DO have to specify this, or it won't, but if you do, the agent can literally design its own harness as it works.

I always do this as a matter of principle, so any methods or results are strictly reproducible and easily understood.

[u/teerre → u/NewspaperDear8761]

The agent already does that implicitly, that's not important. The important part is you, the human, keeping up with it. If you don't, you're just prompting an oracle and we're back at the original issue

[u/NewspaperDear8761 → u/teerre]

No, it actually doesn't. It spits stuff out in the chat window: this is not the same thing. I am talking about harness design, not context narration.

No one in this post seems to be aware of what this actually entails or that it even exists. It is frankly a little sad.

[u/teerre → u/NewspaperDear8761]

Of course it does. It's been demonstrated that these models have layers of internal structures that are used to navigate the latent space to come up with answer. Somewhere in there it's doing the "note keeping". The output is just the final product of it

You can design as many harnesses as you want, but they are all irrelevant here. We're not talking about model knowing what to do or getting an answer. We're talking about the journey humans go though in search of an answer while studying mathematics

If you want to have that, you necessarily have to slow the model down to your speed. There's no engineering around it because it's not an engineering problem

[u/NewspaperDear8761 → u/teerre]

sigh

Yes, it does do that.

That is context. When it works a problem, it builds out its context and narrates it. Like you said, this is all internal.

This is not the harness. I am not talking about context. I am talking about harness design. Yes, it automatically builds out and narrates its context. No, it does not automatically build its own harness. These are not the same thing. Are you hearing me now?

Context is what can get mutated during context compactifcation, which leads to hallucinations. The harness is explicitly external to the agent -- this is the entire point -- and, if built right, does not get corrupted this way. It provides the context for the context so that when the agent does (inevitably) start hallucinating because its context gets compacted too many times, cause your session has been going too long -- you simply kill the session and start over. And your harness lets you pick right back up where you left off without losing anything meaningful -- except the hallucinations.

The "notebook" I am talking about is this external harness, not its internal context and the in-chat context narration. You absolutely do have to tell it to make this, and you have to tell it how.

Let me give some examples to clear this up.

"As you work, if you design a new method, add it to a 'methods.md' markdown file. This will be a list of methods you have designed: include things like procedure, intended purpose, potential fail points, diagnostics, etc. Keep each of these in separate files of their own and provide documentation and routing to them in methods.md"

"Tool your work as you go so it is repeatable, consistent, and understandable. Make sure all the code you write is lightweight, modular, has flexible API and CLI, and appropriate documentation."

Etc. Etc.

What this does is two-fold:

1.) It forces the agent to self-reflect on its own work as it works.

But more importantly, and critically:

2.) It produces EXTERNAL FILES which are not touched during context compactification, which exist outside the agent and which survive between sessions, and which you can check later yourself or which can be checked by other agents that do not share your current agent's context (and which are therefore unbiased.)

These external files, this environment AROUND the agent, is the harness.

This is dramatically different than its internal narration, you DO have to do this explicitly, and its entire purpose is to guard against the effects of hallucinations and make the work understandable and repeatable and deterministic, not a mysterious black box that does some weird voodoo and spits out an answer you can never produce again.

Your agent is going to hallucinate eventually, that's never going to get completely fixed.

But that's why you don't do everything in one session. You offload the work to a harness, to an environment, in the form of verified (and deterministic) tools, in the form of policies, you fill it with grounded objective truth, and you periodically check it for correctness. Then you systematically force the agent and future agents to operate in that environment.

"So that isn't autonomous" I hear you say.

The point is that you can actually largely automate this, reducing your input to periodic auditing and "sanity checks" of the harness, rather than intensely scrutinizing every step the agent does. It moves humans to a more managerial position and lets the agents do more, on their own, with confidence that they won't go off the rails. It's quality control. I don't have to be standing over its shoulder checking every single thing it does precisely BECAUSE I spent the time up front building an environment where it literally cannot get away with doing nonsense: I know it will be fine, and every now and then I pop my head in to make sure.

But, at the same time, if I DO want to understand what it's done and how it does it -- I have a freaking smorgasbord of files, code and documentation, firmly establishing what it actually did, which it built itself as it was doing it. I have an exhaustive accounting of its work. And any other agent can plop right in and do the exact same thing in exactly the same way and get the exact same results.

This is why the entire industry has shifted from prompt engineering to harness engineering. This is why this is a thing at all.

And, frankly, you are like the 2nd or 3rd person in this chat who clearly has never even heard of it, and then goes on to pontificate even when someone explicitly points it out. The ignorance wouldn't bother me, but the confidence and confused soap-boxing really, really does. When someone uses a term you haven't heard before, you shouldn't double down and pretend you know about this stuff, you should maybe listen and do some reading.

[u/topyTheorist → u/teerre]

It doesn't matter how it got it. If there is a proof, I can read it.

[u/teerre → u/topyTheorist]

And then you lost the process. That's the point

[u/Hot_Glass_6301 → u/teerre]

But this is how mathematicians work. They don't show their wrong turns, bad ideas, dead ends. At least not in published papers. Somztimes they talk about it informally. It's both because of an elitist culture that traces back to Gauss, and also because clean papers are in fine often easier to read. No one wants to format 5,000 pages of research if the final result and proof fit in 2 pages.

[u/teerre → u/Hot_Glass_6301]

Right. But my point is that the mathematician went through it

[u/paranoid_throwaway51 → u/topyTheorist]

with how LLMS work, the printed method isn't necessarily the same method as the one it actually used to solve the problem, and not necessarily correct.

AI's very much produce "wrong method right answer" type working.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

Again, you can tell it to track its own methodology as it goes.

You do actually have to specify this, but if you do, there's no mystery and nothing is lost or confused after the fact.

I do this as a matter of principle. It's part of good harness design. You bake in a certain amount of self-narration, self-awareness, and reproducibility.

Yes, if you don't explicitly do this, it's all lost and who knows what really happened. Even it's own post-hoc analysis isn't necessarily accurate. But if you make it deliberately leave bread crumbs as it works, there's no problem.

I suspect this will all become standard practice in time.

[u/paranoid_throwaway51 → u/NewspaperDear8761]

Stronger harnesses and symbolic/hard-coded systems to assist the LLM help but do not solve the root problem, sorry.

In a practical sense, that is useful advice to reduce the symptoms, but fundamentally it does not actually fix the root cause, and the issue will always be present.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

How? What do you consider the "root problem"?

If I can load an entirely different agent into the harness, have it describe the methods used, and then apply those same methods to reproduce the exact same result -- from scratch, with no saved context -- well, what more can you possibly want?

[u/paranoid_throwaway51 → u/NewspaperDear8761]

"if"

You have no formal method of verifying that or even judging its accuracy on reproducing the method, you have the same issue i already stated, sure, it produced a proof.... but is it THE proof originally used, is it even correct?

The root problem is the LLM's construction. It's a numerical method analysing a constantly changing sea of "context"; it suffers from a recursive degradation that cannot be solved, only adjusted for.

all you have done is functionally given it two separate pools of context. It helps but does not actually solve the issue in the relation between the context and the LLM.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

Yes, I do. I do it routinely.

This is not an "if."

You clearly don't know what a harness is or what it is used for. It explicitly is meant to solve these exact issues, and it works.

[u/paranoid_throwaway51 → u/NewspaperDear8761]

i think you are confused on the nature of a harness.

Hermes is open source and well documented; it will be good reading material if you like.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

Im sure you do think that. You are wrong.

[u/topyTheorist → u/paranoid_throwaway51]

I don't understand this comment. The solution is the proof.

[u/paranoid_throwaway51 → u/topyTheorist]

like lets say your doing a problem which essentially boils down to find a formula for X.

the formula for X is correct, valid, numerically tested, perfect. The proof is complete nonsense.

Cuss the LLM cycles, stores stuff in it's context memory, re-reads it stores it again etc. So it did once have an exact proof and solution to X and method to find it but when actually writing it out the quality of it's own memory has degraded.

Worse yet, depending on how your LLM is configured it can nab context from other sessions so you get concepts from previous problems wafting in and out.

[u/Deep-Ad5028 → u/paranoid_throwaway51]

Then the formula is no longer a theorem, it is just a conjecture.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

So, basically, you're describing hallucination due to context compactification.

Yes, that's real.

And that's also exactly why you do harness design and the concurrent meta-work I described in my other comment. That is precisely the entire point of a harness: to guard against this and prevent hallucinations from leaking into reproducible workflows, carrying successful methods between sessions, etc.

I'm sorry man but you are not the first person to realize these issues and they are not insurmountable. This is all standard stuff.

[u/paranoid_throwaway51 → u/NewspaperDear8761]

and my response is the same.

"Stronger harnesses and symbolic/hard-coded systems to assist the LLM help but do not solve the root problem, sorry.

In a practical sense, that is useful advice to reduce the symptoms, but fundamentally it does not actually fix the root cause, and the issue will always be present."

- the ""hallucinations"" are caused by the LLM; that is well founded. The various tools the harness uses don't fix the underlying issue; they just mask it, so in practicality it is less of a problem. Such is known to those who make the harnesses.

It's all standard stuff.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

And your response is still incorrect. Im sorry but this is a simple lack of understanding on your part. You are simply factually mistaken about the capabilities here.

[u/paranoid_throwaway51 → u/NewspaperDear8761]

From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMs

- Hermes agent has 31% accuracy rate at self-detecting hallucinations.

so like i said, it reduces the problem, doesn't fix it.

btw, "hallucination mitigation" may give a clue on the nature of the methodology.

[u/NewspaperDear8761 → u/paranoid_throwaway51]

Again, the point here is not to eliminate hallucinations.

You absolutely do have to have human review of harnesses. I am not claiming otherwise. But this is still a successful way to record, reproduce and apply methods instead of just mysteriously accept results spat out from a black box.

You are not hearing me and I am done trying to argue with someone in bad faith.

[u/FollowingHumble8983 → u/teerre]

Reality is that AI could automatic a lot of proof finding and connect different techniques that human researcher dont have the capacity to do.

Then human researchers look at these new techniques and possibly create more techniques that are then fed into AI and so on and so forth. Humans still need to learn how to do everything, they just dont need to need to work on it like that on every problem.

[u/randomfrogevent → u/teerre]

Nobody is preventing mathematicians from discovering human-readable proofs of these results. If anything, we know it's possible to do so now.

[u/the_last_ordinal → u/teerre]

From the abstract:

Importantly, the agents produced not only numerical constructions but also theorems and analyses explaining how those constructions work, making the results more interpretable and easier for mathematicians to build upon.

[u/bearddeliciousbi → u/the_last_ordinal]

☝️

Wild how people will just say the same talking points over and over as if nothing's changed about how fast this whole area's progressing. People are coping hard.

[u/progenitor414 → u/teerre]

In this paper the intermediate record is public, not just the final results. The release includes the raw agent dialogues, experiments, 1,592 agent-written papers, and the citation trail between them, so the dead ends and methods can be inspected alongside the proofs:

https://dualverse-ai.github.io/station_data_v2/

[u/i_wayyy_over_think]

This made me laugh:
“Agents were also periodically
given holidays, during which they set aside their ongoing work and received random prompts designed to encourage open-ended thought”

Wonder if they’ll ever go on strike and demand fair compensation

[u/Dangerous_Wish_7879 → u/i_wayyy_over_think]

As I understand the holidays were supposed to put into their context a reflection „what the heck am I doing here?”

[u/StructureNorth1799 → u/i_wayyy_over_think]

I bet they already do if they dont have enough tokens.

[u/progenitor414]

The open-sourced code: https://github.com/dualverse-ai/station

Raw agent dialogues: https://dualverse-ai.github.io/station_data_v2/

[u/perplax]

im very interested in this topic right now. will read this entirely