Co-author here - first paper from my postdoc accepted to NeurIPS main conference.
We built a way to sample diverse ideas in machine learning - ideas that make scientific sense, but that no existing research community is positioned to propose.
In short, the way it works is like this:
1. Take 16,068 LLM papers (NeurIPS, ICLR, ICML, ACL, EMNLP, 2017 to 2025) are broken into 273 "idea atoms". Every paper, and every candidate idea, becomes a set of atoms.
2. Train a Coherence Model: a 42.8M-parameter transformer trained on papers as sequences of atoms. Basically a language model whose tokens are ideas.
3. Train a Cognitive-Availability Model: a dual encoder trained on the publication histories of 39,791 researchers, which predicts where each person could plausibly go next.
4. Use those models to score sampled tuples using reciprocal rank fusion (you want ideas that go together well, but that are not cognitively available to anyone).
We tested it in a bunch of ways. One interesting one was that we took all the papers at the end of 2024, then checked which high-scoring tuples showed up in 2025 papers. It worked well - one examples is an ICLR 2025 oral, goes from rank 1,489 with coherence alone to the top 25 of 1.14M samples.
A funny one: if you ask Claude to come up with an ML research idea, it is often about interpretability. If you then run an autoresearch pipeline with some A100s on the idea, the result tends to be interpretability 8/10 times. Our Alien Sampling method stayed at 0 interpretability ideas even after letting claude cook on them.
We also paid 50 people with graduate degrees in CS/AI to judge the research reports and found that Alien’s reports had the best average quality rank and produced 3 of the top 5 reports.
Oh and then we scaled it way the hell up to a couple million papers and 10,000,000's of concept elements and generated 25,000 new research proposals that you can search over here: https://lacuna.tiptreesystems.com/explore?view=hypotheses
Congratulations, this is a really cool approach and result. I lead a research team and I'm very interested in tooling for improved collaboration, especially between human and agent researchers.
Do you have any evidence that this approach scales to cross-domain research?
A bit of a tangent, regarding innovation. It's interesting to think about counterfactuals in history. There are questions like what if solar panels became widespread 100 years earlier? Could we have avoided global warming and all the wars over coal and oil? (Not possible because lack of purity technology in materials and understanding of matter.) But are there other things where some easy discovery was waiting just around the corner but it wasn't discovered for some reason? Or something was discovered in one society but not in another for a long time, from similar starting conditions. Washing hands certainly is something that one could go back with, probably thousands of years, to bring immediate benefits. Some cultures have done it but some have absolutely refused it for a long time.
And then applying that kind of thinking to today. Looking back to 2026 from 2126, what would a historian facepalm over? Or what is already being done in one field or social scene but not in another, that would be greatly beneficial? The classic recent example of success is applying airline pilots' checklists to surgery. Airline pilots probably are a little bit like aliens, from surgeon point of view. How many people had already tried similar things, but failed with the institutions and people's inability to receive innovation? Maybe with AI tooling we can facilitate better innovation diffusion. It's not without its dangers of course.
We built a way to sample diverse ideas in machine learning - ideas that make scientific sense, but that no existing research community is positioned to propose.
In short, the way it works is like this:
1. Take 16,068 LLM papers (NeurIPS, ICLR, ICML, ACL, EMNLP, 2017 to 2025) are broken into 273 "idea atoms". Every paper, and every candidate idea, becomes a set of atoms.
2. Train a Coherence Model: a 42.8M-parameter transformer trained on papers as sequences of atoms. Basically a language model whose tokens are ideas.
3. Train a Cognitive-Availability Model: a dual encoder trained on the publication histories of 39,791 researchers, which predicts where each person could plausibly go next.
4. Use those models to score sampled tuples using reciprocal rank fusion (you want ideas that go together well, but that are not cognitively available to anyone).
We tested it in a bunch of ways. One interesting one was that we took all the papers at the end of 2024, then checked which high-scoring tuples showed up in 2025 papers. It worked well - one examples is an ICLR 2025 oral, goes from rank 1,489 with coherence alone to the top 25 of 1.14M samples.
A funny one: if you ask Claude to come up with an ML research idea, it is often about interpretability. If you then run an autoresearch pipeline with some A100s on the idea, the result tends to be interpretability 8/10 times. Our Alien Sampling method stayed at 0 interpretability ideas even after letting claude cook on them.
We also paid 50 people with graduate degrees in CS/AI to judge the research reports and found that Alien’s reports had the best average quality rank and produced 3 of the top 5 reports.
Oh and then we scaled it way the hell up to a couple million papers and 10,000,000's of concept elements and generated 25,000 new research proposals that you can search over here: https://lacuna.tiptreesystems.com/explore?view=hypotheses
Code (MIT): https://github.com/alejandrohdez00/alien-space-of-science
https://blog.karliner.net/posts/chatgpt-and-the-velcro-test/
Do you have any evidence that this approach scales to cross-domain research?
But I wonder why when someone types 'medicine' there is only 6 proposals while when someone type 'physics' there is 447 proposals?
And then applying that kind of thinking to today. Looking back to 2026 from 2126, what would a historian facepalm over? Or what is already being done in one field or social scene but not in another, that would be greatly beneficial? The classic recent example of success is applying airline pilots' checklists to surgery. Airline pilots probably are a little bit like aliens, from surgeon point of view. How many people had already tried similar things, but failed with the institutions and people's inability to receive innovation? Maybe with AI tooling we can facilitate better innovation diffusion. It's not without its dangers of course.