The real "[Common] Lisp curse" is that once you taste it, there's no going back.
Other than that, the Common Lisp ecosystem is doing well thank you very much. Many contributors to SBCL, regular contributions to the important ecosystem libraries, vivid projects (Lem), new editors (mine) and plugins (OLIVE), some companies picking Lisp and somehow supporting the ecosystem (hosting conferences, contracting implementors). https://github.com/CodyReichert/awesome-cl/ · https://github.com/azzamsa/awesome-lisp-companies/ And yes, new vibe-coded projects too. Some are useful (ICL, cl-tuition…), some reinventing the wheel (badly) went nowhere. For some, we'll see.
Reading the first paragraph of the article, I agree with them that Lispers don't collect as many dependencies.
But then once I read the second paragraph, I agree with you. Lisp is doing fine in terms of important ecosystem libraries. The article sets that as the context when it asserts as fact that "established libraries are incredibly stable, tested by hundreds of thousands of developers, included in thousands of CI pipelines and maintained by experts on the topic."
The thing about lisp, it’s that it’s fairly expressive. You only need a few foundational libraries (mostly algorithms) and it’s very easy to build on the rest. In something like Python, or Go, once you define a conceptual model in a library, you need another library for the glue code to bind it to another ecosystem. In Lisp, almost everything is using the basic list, alist, and plist,…
So you design from the core logic and write shim to external dependencies. Instead of bringing deps from the get go.
Lisp missed its timing. When it was above and beyond everything else and a totally different world, it was way too expensive and way too inefficient.
Today, when mainstream languages have stolen much from Lisp, the benefit of using a language where you can generally just add a gem, egg, or crate is much higher than the benefit of the one thing that is still unique to Lisp: macros. Reflection is inelegant as hell when you compare, but it will get you almost all the way macros do.
At the same time, typed languages have become much more like dynamic languages. The types have disappeared into var and let statements; they are now inferred rather than repeated. Pattern matching has been added, and more—without compromising the benefits for larger teams and larger code bases.
Purists will disagree, but I haven't seen a real world problem solved in Lisp that I couldn't also solve in other languages.
And yes, I am interested in being proven wrong. I want to know what kind of problems still exist that are that hardcore. But I suspect a big part of it is that those who code in this still think the "hello world" for Java looks something like this:
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello, World!");
}
}
when in reality it now looks like this:
void main() {
println("Hello World");
}
and in C# it looks like this:
Console.WriteLine("Hello, World!");
And comes with full first class support for Linux and Mac.
Lisp is an intellectual tower and has given an incredible amount to the programming community in terms of ideas and a playground for people to try things. I'm not denying any of that — it should be celebrated. But my thesis is this: most of that has been moved, copied, stolen, and adapted to mainstream programming languages.
Lots of people have used lisp, scheme or scripting languages and never go back. Lisp was created in the 50s, if it converted everyone it would have done it a long time ago.
You probably have no idea how much Lisp is out there in real, production systems. Just because the majority is required to be on Python, doesn't mean it's the best stack for solving problems. Gathering language stats for Lisp is also challenging, because it's not a single PL - it's a whole family of dialects. And these dialects and communities are absolutely thriving. Modern Lisp landscape is very, very different from LISP you're talking about (invented in the 50s).
This is absolutely correct. I recently found myself asking my agent to modify a local copy of an open source project to fix bugs that I found. It fixed the bugs, and it rebases against the public repo on occasion.
Why haven't I submitted my patches? Because my agent wrote them and I didn't review them. I would feel bad submitting my patches without looking at them, and I don't have time to look at them.
Why not ask someone involved if they want it or tell someone the prompt that made it? Someone who does the work of checking it and sanitizing it can take the credit.
Depends on the project but some actually adopt an outright hostile response to this. Sometimes justifiably. Sometimes not.
It's honestly not worth it. I understand both sides here, but people are going to have to come up with new ways of collaborating. Pull requests are likely not it anymore.
KMyMoney stopped importing (some) OFX files for me. I asked an LLM to debug.
It was a wild journey. An obscure bug in some QT library that was triggered because of something completely unrelated that I had installed briefly in 2019 (and uninstalled years ago).
I had it fix the issue by cleaning the leftovers from 2019, but it's still a QT bug. I had it do a writeup, but I'll never submit it. It's fairly sophisticated and I don't know QT code at all.
One positive result is that you get a lot more variation in vulnerabilities. That probably sounds negative, but it isn't.
As an example, a black hat doesn't get 70% of web sites from one shared bug in PHP. They need to target specific sites, and the exploit isn't likely to work in very many other places.
That sounds like a very reasonable thing to do, yes. If you ask Opus 5.5 to write you a dynamically typed interpreted language that can query a local SQL database, implicitly parse HTML form submissions, and be embedded in HTML templates interpreted by an Apache module, you'll probably have a usable alternative to PHP in 37 minutes. And if you ask it to do it in Rust with no unsafe, it won't have any memory error vulnerabilities.
I was saying that you could just not have exploitable bugs, at least of the memory-safety variety that was on display in PHP2hell, not that you could mitigate the exploitability of the exploitable bugs you will inevitably have.
Which, probably, you will also have to do.
I suspect that a more effective mitigation to having exploitable bugs would be to have an LLM read all your HTTP requests and the actions the web server proposes to take in response and give a Jev-like approve/disapprove decision.
I've been playing around with LLM red teaming different server stacks with known or intentionally vulns in there and seeing how much damage they can do.
The one situation where they really can't get anywhere profitable is a unikernel webapp. With no source and no shell they find the vuln but the worst they can do with it is possibly poison or exfiltrate data as a one off, or crash the server. Often they can't even figure that out. Can't even leave themselves a reliable way back in.
Of course the traditional problem with unikernels is that you have to port EVERYTHING you need yourself. But the cheapness of this is exactly what we're talking about so.
Looks like security through obscurity's back on the menu.
What sorts of vulns are they finding in unikernels? Logic bugs, that sort of thing, or are there areas where they could get that tiny scope to do anything at all?
The advantage there shouldn't be obscurity, so much as very limited attack surface in the whole exposed stack.
Obscurity in the sense that the bad guy doesn't have your code doesn't hurt, but I think the real win could be that they need to specifically target your system to find your vulnerabilities. That could be a lot of compute and time to dedicate to a single target. So that kind of obscurity changes the economics of it for them.
These last few months, I have been building a capability-based language and runtime for fun based on the "port EVERYTHING" idea.
The language is restricted to permissions granted on startup and can only access the system via the runtime, which only supports a particular subset of linux syscalls. So, outside of the stdlib, whichever feature you need, you simply use an LLM to port an existing python/rust/JS/TS/Java library to the language and use it.
As the runtime has its own scheduler and GC, I have actually been wondering about the feasibility of skipping the linux kernel and making the runtime itself the "os." Given that so much modern hosting is actually done on virtual hardware, the drivers could be hosted within the runtime as well.
That's exciting! I'm interested to hear about the language design. I vibecoded a unikernel web server a few weeks ago, and I've been trying to extend it into a full capability-secure microkernel called KokalOS.
The language borrows syntax from Python (and a wee bit from Haskell) and is essentially syntactic sugar over facilities provided by the runtime:
- statically typed from end to end
- closed and open/extensible types
- closed types are immutable, open ones have two variants: immutable and mutable
- arrays have two variants: immutable and mutable
- no OOP/classes. inheritance is nominal and mainly exists to provide access to class membership (is X a Y?)
- functions operating on data
- UFCS exists to provide nice features like "apple".substr(...)
- first class functions and closures
- monomorphized generics
- tailcalls
- operator overloading
- modules and namespaces to control access
- erlang/beam style first class processes. So you `spawn` a function and communicate between processes using send/recv instead of async/await
Processes are completely isolated at the runtime level. Each has its own stack, heap, mailbox and GC.
The bytecode carries all type information. The VM will compile using AOT-on-load.
My own runtime-becomes-a-os idea is still just an idea. I haven't really pursued it with seriousness. Will probably do that once the language and runtime stabilize.
It sounds like you already have a pretty complete OS; you just don't have it running on bare metal (or on a hypervisor) yet. I hope I get a chance to see the language someday!
A few years ago, after ChatGPT but before agentic, I was asking AI for help with some projects. It kept recommending really good, well designed APIs, and gave me code snippets for them.
One small issue: they didn't exist! It had just hallucinated them!
Which was a real shame, since they were much nicer than the actual stuff I ended up finding.
I just realized, thinking about this now, that these days (and perhaps even back then), I could just say "ok now take this code and build the missing library so it actually works!"
Exactly. In almost all of those cases, the hallucinated functions absolutely should have been present in the API, and any reasonable developer would have expected them to be.
Perhaps. I just never found it useful to spend a day learning the failure modes and edge cases of a general-purpose library I could hack into my use case with enough wrapper code handling the impedence mismatch when I could spend a day building exactly what we need, with the added benefit of owning the code when requirements inevitably change. Like, I'll use numpy at least half the time for numeric code; I'm not a masochist; but most libraries just don't pull their weight.
When worldwideweb.app was first being developed an the RFCs and so forth for web servers and protocols were being discussed, one of the sticking point was what to call a URL --- the initial suggestion was "Universal Resource Locator", but the argument was made that users and writers were not librarians and that the distributed nature would preclude a given web page having a universally agreed-upon location and name, so instead the present "Uniform Resource Locator" was adopted.
What if there was a code sharing site where it was unambiguous what a given snippet of code would be named and where it would be stored, so that a user could just browse it and find a list of implementations for a given problem?
This of course runs up against one what John Ousterhout notes is one of the most difficult problems in computer science --- naming things --- in his book:
I've tried to do it on a small-scale with my own work, storing bits of (la).tex code in a single file, but on reviewing it, I'll often find that I have written a given solution a second time and given the macro a slightly different name....
The thing is, solutions related to computing techniques, are few and can be found in algorithm textbooks. Some are then put into standard libraries.
Almost everything else is domain specific where you have to understand the domain first before understanding the code. And modeling a domain is all about tradeoffs between accuracy, performance, specific vs generic,…
Instead of collecting such solutions, a better approach is to master basic data structures, and then improve your modeling capabilities. After that, it’s mostly learning domain as they come (dsp, physic modeling, operating system concepts, relational database, graph theory, 3D geometry, accounting,…)
Less external dependencies. I noticed Claude was vendoring (copy-pasting) Lucide icon svgs into my codebase (with proper attribution) because I was only using a small handful of them. That's awesome.
>Except when it's a license violation or in something that ends up having a security vulnerability.
What's more secure isn't so simple. If you vendor something with a vulnerability then it wont be patched. But vendoring may also mean you took the 2% of the library you need and not the 98% which reduces the surface area for vulnerabilities. And it can cut the other way - you may update a package that was previously good and is now vulnerable (regressions, supply chain attacks).
There is a balance that must be struck, and I think LLM's strike a better balance than people do. Dependencies are a convenience with a cost and LLM's can trivially copy or re-implement things that we cant.
> But vendoring may also mean you took the 2% of the library you need and not the 98% which reduces the surface area for vulnerabilities
If you aren't using a library function you're not vulnerable to exploits in that code: unused but imported code is inert (possible usable as a gadget if you're vulnerable elsewhere, but at that point you're screwed anyways) and most compilers/runtimes completely eliminate it.
> you may update a package that was previously good and is now vulnerable (regressions, supply chain attacks)
There are better mitigations to this than not using a library.
You still completely ignore the license violations.
Every time I try to get a npm based project running (or one of my forks of one), half the packages don't exist anymore or I need to spend hours fixing breaking changes.
It's usually more straightforward to just rebuild it from scratch — not to mention infinitely more satisfying — and that was true before AI!
Yes it does depend on the dependency, and a static asset is on one end of the extreme.
It depends on the "load bearing" code. Obviously something like crypto is not something we should implement ourselves but core business logic would be "load bearing" in another way and is probably not something you want to farm out to a dependency.
The real looking for libs often turns to be going down with potential security risks. In such cases is much preferable to not reuse unsigned and unclaimed code.
I find this concern rather misguided. Telling the LLM to integrate a "mature library" is often cheaper and faster than rebuilding (a fraction of) their features from scratch. If you really cannot be bothered with searching yourself, these days you can even tell the agent to scan pip/npm/whatever registry for you and give you options.
> I’d rather rely on proven libraries with well-defined abstractions than generate hundreds of lines of code which solve a local problem for me.
I 100% disagree with this. Supply chain attacks are already at an all-time high due to agentic hacking assistance, and that trend shows no sign of slowing. Every dependency you add is another potential attack vector, and just checking the dependency’s code doesn’t solve the issue - your dependencies also have dependencies!
The cost of rolling your own has gotten so low that it’s almost always a better security tradeoff to reimplement the thing yourself. Is that a good thing? I’m not sure.
If coding is anything like image generation, everybody is getting the same output anyway, which means we are back to nearly standard libraries just that are being dynamically produced. I see your claim as a new version of security through obscurity. It is way easier to patch all variants of a product when you can identify their lineage as variants in the first place.
And rolling your own introduces brand new security holes. You're arguing security through obscurity, which is a particularly dead concept in the age of AI.
Also, you clearly don't work with products that have any sort of serious corporate liability. Being able to blame a supplier/library is almost always preferable to assuming the liability yourself
What about the vulnerabilities you yourself introduce when using AI? or is the assumption that (1) there will be zero or (2) you won't ever open source so no one will know about them?
> The cost of rolling your own has gotten so low that it’s almost always a better security tradeoff to reimplement the thing yourself. Is that a good thing? I’m not sure.
Why stop there? We could eliminate all external dependencies and even build our own compiler and OS, since the cost of rolling own solution has apparently gotten so low, right? Sometimes I feel like I'm living in a completely different dimension from some of the commenters here.
I'm not a SE but I dabble with code. Lisp was really something else and I really liked it when I was reading SICP. Then I tried Clojure and things just clicked and it was fun to write.
Actually this is not a curse but a blessing! Being able to implement any functionality without the need to resort to dependencies is a godsend.
Yes, some dependencies are great. But unfortunately the vast majority are problematic: Not properly supported, badly tested, with unhelpful maintainers, funny licenses, undocumented and, definitely, not being able to cover your requirements 100% so you have to change your requirements to meet the dependencies instead of the other way around.
Decades ago a relative showed me his Amiga. Before putting in the disk (these were on 3.5 inch floppies iirc) with the game I wanted to play on it, he put in another disk.
I asked him with the first disk was, and I remember distinctly he said it was an operating system. I asked him what an operating system was, and he said it was like a program, it was a program technically, that makes it easier to run other programs, just it was a program that didn't do much on its own and you wouldn't use it without another program right afterward. I said it didn't seem easier if you had to load a different disk than the one you wanted to use, and you still had to load the one you wanted afterward. He looked at me disdainfully and patiently explained that all or almost all the software in the future would use operating systems and they whole process would get easier or even invisible but there would always be an operating system in there somewhere at some point in the very near future. I was skeptical, but I nodded along.
I imagined an operating system as a dressed up software library, which was more intuitive to me as a concept.
Your intuition was right. That is how AmigaOS was designed. Your relative was also right, after the first model (Amiga 1000), all Amigas had Kickstart (the core OS) preinstalled in ROM.
Looking back, AmigaOS was remarkably complete and modern for 1985: preemptive multitasking microkernel, shared libraries, multimedia support, everything was modular.
Seems a bit hyperbolic to call it a curse. Things like these are on a spectrum for all languages and languages like C and many others are afflicted by ecosystems that are more or less community driven and leverage libraries to a greater or lesser extent. Doesn't really have to do with Lisps per se.
I’m reminded of Chuck Moore’s practice of always reimplementing everything. There’s benefits and drawbacks to this; calling it a curse is certainly a strong opinion.
It is... a meme? I think it is an explanatory construct that was invented by lispers who were facing the declining Lisp popularity back then in 90s. It was looking very promising, and then it just lost its steam and now almost no one uses it. One can imagine "Lisp is a bad language" as an alternative explanation, but most lispers do not (and didn't) agree and they prefer the Lisp curse, because it paints Lisp as a language that it too good for its success.
I'm not taking sides here, I love Lisp, though I don't use it anymore (except elisp in emacs, but it doesn't count). My point is that debating the name itself distracts from the actual issue.
It really applies to all languages where the very process of developing extends the language. It is the modern day variant of the Babel curse: you end up with a perfect language for your problem understood by a single individual.
There was a variation of this with Scheme: the standard language was small, so each implementation extended it in different directions, and there was plenty of disagreement about which directions were good or bad.
This "curse" is not unique to Lisp. There's an old saying something of the sort that for the first week after you install a Forth you have the same Forth as everybody else. After that, you have your own Forth dialect.
Any language that's powerful for creating new syntax and doesn't have an equally powerful community repository gets this same effect to some extent.
well i can't say for the whole community but i remember someone blogging around 2010 about dropping mainstream languages for his project because all the graph libraries he found were complicated to assess and inadequate.. so he went to write his own tiny lib on commonlisp and was done in a short time.
the lisp dialect growth is also famous, it's less the case these days because people know about it, but it existed
After programming in multiple Lisp dialects over 20+ years, it’s only sorta true. The biggest problem is that “Lisp” is ambiguous. Common Lisp? Scheme? Emacs Lisp? Clojure? They are all legitimate Lisps, but they all have slightly different syntax, semantics, and standard libraries. Even among some Lisps, there are differences (notably Scheme, though Racket has helped by becoming somewhat dominant in some areas). Common Lisp helped reduce differences at the cost of supporting almost everything it was seeking to unify, and yet still has platform gaps that implementations have to fill in. But within a given Lisp, there is a lot of reuse. Racket has a large set of libs. Clojure runs on the JVM and it’s common to mix and match Clojure and Java libs.
note that i'm mostly pro-lisp, and wasn't criticizing, just reporting what i saw when reading about it.
i'd add one thing to your point, is that lispers know how to port concepts from lang a to lang b very easily, support libs, macro layer if need be (see goroutines in clojure) so fragmentation is not as hard it was in mainstream languages (ps: even though today everybody goes into ast / transpiler to experiment)
I've been reinventing the wheel since I started programming. Back then, we had no choice--there weren't any open-source libraries. But I also enjoy reinventing the wheel because (a) the wheel can be exactly how I want it, and (b) I get to learn and understand new things.
Coding agents have actually cut both ways for me. I can reinvent the wheel more easily and with higher quality, but I can also integrate an external library much more easily than before.
The other day I needed my system to download SFTP, so I had the agent build an SFTP client. It turned out that I actually needed FTPS, so I had the agent build that too.
Another time I wanted to integrate with Git repos, but instead of trying to reinvent that, I asked the agent to integrate libgit2. Of course, I asked it to add an API that matched the semantics of my system, so from my perspective, the wheel was exactly how I wanted it.
I think what's going to happen is that there will be more vertically integrated platforms that do it all. If you think about it, it's really annoying to cobble together a system from different libraries, each of which might have different semantics.
What we want, as developers, is to have a single, well-designed, integrated platform that exposes all the functionality we need in a consistent way. That's a lot easier now that LLMs can mold code to your exact specifications.
I'm biased, of course, because that's exactly what I'm building with GridWhale.
Same except I had open source libraries, I just somehow always had a worse time with them than just building it myself. (Just even getting them set up in the first place, or updated, and so on... most of them seem to be someone's "always have stuff to do at my job" insurance policy!)
But even if they were pleasant and did what I needed (this one time I installed a 5000 line lib that was missing 20 lines to do the thing it was actually designed to do.. like, they couldn't have added 20 more lines?!), an issue I have is spoilers.
Like, at university, they constantly said, ok here's a big problem in computer science..
(3 seconds later) .. aaand here's how we solved it! And I'm like agh, no! Spoilers! I want to figure it out, jeez!
That's also how I feel about "computer, just magick my entire program into existence." Spoils the fun of figuring it out!
I'm currently working on a project that uses Tcl as its primary programming language. One of the great features about the Tcl interpreter is that it lets you introspect things like variables and procedures at runtime. You also get the full source code of procedures at runtime, even those that were themselves defined at runtime. Instead of the normal work flow of having Claude make changes, starting up a web service, testing the web service with some Javascript driver code, I instead work through Claude in Chrome and expose a web-based interpreter from within the running web service itself. I can instruct Claude to make changes at runtime and immediately after making that change, the code can be tested by Claude. Those changes can then be dumped to disk and reincorporated back into the base source code I have. Not something you'd want to expose to the wider internet, but it's let me move _very_ quickly and build out apps within minutes. I've considered making use of Tcl's safe-interpreter mechanism so other Claude instances could possible collaborate within the same primary app without stepping on each other.
It's ridiculous how stubborn programmers could be. They'd cling to their beloved Blub paradigms and almost cultishly refuse to accept clearly pragmatic takes, sitting just in front of their eyes.
Of course, there's no tool without flaws. Nevertheless, oh boy, how amazingly practical Lisp-dialects can be in reality (even with all their imperfections), it's just astounding. Sometimes I think maybe it is some kind of ego-defeating stance or whatever, maybe smart programmers just can't believe how the stupidly simple idea of Lisp can ever satisfy their galaxy-sized brains? To me, Lisp is like a boundless sea of incredible idea generation on the spot, turning it into practical reality like some magic. Effortlessly, painlessly, and with joy.
I don't know, maybe I had to go through the pain of learning the intricacies of a dozen different stacks, using them for years, only to discover Lisp later in my life. Perhaps I wouldn't know the difference otherwise?
Just the other day, I was watching a YT video, which I of course fully control directly from Emacs - that is very useful for taking notes. I can mute, pause, speed-up, rewind, toggle subs, etc. All while typing a message, a note, or while looking at a PR diff. I thought, "hmm, floating window would be nice". Without even missing a beat, I opened eca-emacs buffer, I didn't even have to type the prompt - I used speech-recognition. Agent then (running in Elisp REPL), connected to my WMs REPL driven by Fennel, using tools in my Clojure-made harness with its own REPL, tapped into the lua-based mpv, analyzed things on the fly and just built the solution, right there. I didn't even have to compile, restart, reload, refresh anything. Now I can detach a video playing in the browser into mpv window and later put it back. With a press of a key.
It's almost as if Lisp has been quietly waiting for this moment. LLMs and Lisp REPLs make such a great duo - if you're a programmer and haven't seen it in practice yet, you should seriously be experiencing deep FOMO right now.
Which models have you found most take advantage of this kind of capability? Opus 5.5 and GPT-6 Sol and Astra seem to like to minimize their tool calls, blatting out a single edit with dozens of lines of code in it rather than editing a few at a time. And they virtually never use things like the Python REPL, even though they use Python all the time.
> And they virtually never use things like the Python REPL, even though they use Python all the time.
REPLs in non-homoiconic languages like Python, and homoiconic Lisp do differ. Every single step in Read-Eval-Print-Loop does. That's why Lispers sometimes refer to Lisp REPL as a "true REPL", because in practical comparison, Python's REPL is more like an "interactive shell". Homoiconic means that code-is-data, and the Read step in REPL doesn't need to do any acrobatics with parsing, building AST, etc, it's just directly reading the code and passing to Eval as is.
Here's a sense of what it looks like to be writing software with Lisp: https://youtu.be/tHgApFhevrQ?t=2272
I purposely moved the marker so you don't have to watch the whole talk. Jeaye here coded his entire presentation from inside a terminal, embedded into a game.
That's how typically Lispers write software - "from inside out". Lisp, uniquely fits for LLMs, because when you give them a Lisp REPL, they stop guessing and start empirically analyzing current state of things and produce working solution faster, costing fewer tokens.
The typical flow with non-homoiconic PLs is is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation technically stateless. It is a whole different story with Lisp.
This was referring to the LLMs' tendency to blat out hundreds of tokens into a single tool call, not to Clojure.
The REPL in the part of the presentation you link to only evaluates two forms, (range 10) and (bounce!), neither of which takes much advantage of homoiconicity.
cchase88 was talking about a degree of homoiconicity in Tcl that has been abandoned by most dialects of Lisp:
(That might look wrong, but it's actually correct. In Tcl the list {name} is exactly the same as the quasiquoted string "name" and the bare word name.)
You can't do that in Racket or SBCL, but it does exist in Clojure; it's clojure.repl/source. However, the part of the presentation you link to doesn't use anything similar, and it's not really a mainstream part of Lisp development anymore, the way it was 50 years ago.
I think there is some truth to this. But it may just be that the threshold for viable open-source collaborative projects increased by an order of magnitude or two. E.g., today if you make a cool datetime library, nobody will care, they can recreate the parts they need in 5 minutes. If you make something much harder to reproduce, then maybe they will. So I just file this under “you need to be more ambitious.”
I do think the current ecosystem of open-source libraries won't survive as it is, as AI screws up the incentives too much. But agents recreating the wheel for every project isn't practical either.
What I think we'll see happen is that AI vendors will create private ecosystems for sharing code. I.e. you subscribe to Codex and you get access to a suite of libraries that speed up your development process. Behind the scenes there'll be a mirror of the how the current ecosystem works, with agents submitting bugs and feature requests and a private swarm of agents working the most popular / high priority.
I suspect it will continue as it has. Where LLM's do use external dependencies, but quite a bit less than people do. Saving it more for the things that are complex and sensitive. They aren't perfect but overall it's better than the non-agent status quo (more dependencies, less re-implementing).
Not sure about this; what would be an incentive for AI vendors to do this ? More libraries used means less tokens spent, so it runs directly against their business model.
LLMs stop me using open source libraries for two reasons
(1) GitHub is now full of slop. The majority of new open source libraries are now generated by AI. Many come from students who are just looking for resume garnish. Others come from devs who are more excited about being an "open source developer" than actually doing open source development
(2) With AI I can probably build something better. And if it's crap? No problem! I can just wait 6 months and rewrite it with gpt_galaxy or Claude Torah or something
At the same time I have no real incentives to contribute back. Anything I create competes immediately with a sea of slop. Any actual work I do gets identified and swallowed up by the plagiarism machine. And to be honest I only occasionally had good experiences with OSS anyway
I think we are looking at the end of open source software. Not the end of free software - in fact that may end up flourishing with LLMs and reverse engineering and the like. But the idea of programmers sharing code, for communal interest and career building, is completely dead.
How about (3): that in-house utility that you want to re-use across projects because it scratches one itch in precisely the way you want it scratched. Its raison d'être does not disappear from the appearance of LLM, it arguably gets stronger. You want the decisions that went into the library to be encoded in immutable form from the perspective of the contexts working on library dependents.
In the old days, both the investment in the library and the additional cost you'd want to sink on it to consider it publishing quality incentivised to keep it truly internal. LLM code however? It might be slop, but it's accurately groomed slop. Prompting the model to roleplay a public library author might even improve the output when it's in fact not published at all. The investment is low, the extra cost of sharing is low, worst that could happen is others actually picking it up and that would not be bad at all.
Is that spamming the public information universe on the faint hope that perhaps future models might come pre-trained with understanding of your library? Sure. But it's a very different thing from the scenarios described in (1).
>Not the end of free software - in fact that may end up flourishing with LLMs and reverse engineering and the like.
LLMs are antithetical to free software, which everyone seems to forget is as much an ethical and social movement as it is a classification of software licensing, and will be until one can examine and audit the training data of a model to exclude the possibility of including any non-free code, and until they include, document and respect the licenses for the sources they do use.
The degree to which they discourage people from actually being able to understand and modify code without an LLM also makes the code they generate implicitly nonfree. All of the freedoms of free software presuppose their utility to a human over their utility to a machine. If people aren't reading, writing and distributing code then it defeats the entire point. That's just proprietary software with extra steps.
I don't know what you call this new paradigm of vibe-coding and just not giving a damn about sources and licenses and copyright, but it isn't free software. And it's probably worth pointing out that said paradigm was created by and is most useful to corporations, not people. Using the plagiarism machine to plagiarize "the man" is still subject to the Rule of Goats[0].
> LLMs are antithetical to free software, which everyone seems to forget is as much an ethical and social movement...
At the core of Free Software there is one simple kernel: the user being in control. Licensing and copyright is used to achieve that, but they're the means, not the goal. The goal is that you, the user, should be in control of your computing, not the developers of the software you are using. Hence software that is free i.e. the software not being bound by the whims of its developers.
From that perspective, IMO, LLMs are far from antithetical to Free Software exactly because they let[0] their users do whatever they want with the software and their computers. That a user can instruct their agent to take the video playback code from MPV and combine it with the UI of XBMC[1] while ensuring it runs on the Raspberry Pi 4 they have connected to their TV (that they "jailbroke" to remove the intrusive ads and other user-hostile software that they replaced with software to their liking) even if they barely have any programming knowledge themselves is a sign of software freedom.
[0] at least the user-aligned ones not bound by restrictions placed on them by their own developers, so perhaps we need a Free Models movement too or something along these lines.
[1] random example, i don't even know of XBMC uses MPV's code already, so assume it doesn't
LLMs don't let their users do whatever they want, users are limited to whatever the model and its training allow, because it makes the decisions based on randomness, not them. Even open weight models have the same limitation. You don't have exact control over what the LLM does, much less (using any of the popular services) its training data therefore you can't even claim that you created, much less own, that code.
And because LLMs can't be audited and can't track or respect the licenses of the software they use as sources, no software generated by an LLM respects the freedoms of Free Software.
>That a user can instruct their agent (...) even if they barely have any programming knowledge themselves is a sign of software freedom.
It isn't, because as I said, freedom presupposes knowledge. If you barely have programming knowledge and you don't understand the code an LLM generates then you can't 1) read it, 2) edit it or 3) redistribute it without an LLM. You are not exercising any freedoms in that case, you're just being a consumer of a service.
>so perhaps we need a Free Models movement too or something along these lines
I said what I think would be the minimum requirements for a model to respect free software - at the very least it should only be trained on free software, it should be provably auditable without hallucination (I don't even think this is technically feasible) and it should respect the licenses of the software it incorporates, including attribution where necessary.
Beyond that there's still the problem that training even open weight models requires expensive infrastructure making it implicitly proprietary. As long as we need data centers at all it's never going to be as free as a text editor.
This does seem to be the case in practice at present, but I wonder if it can be changed. AI models are happy to write you tutorials and explain code to you. And if courts are generally unwilling to enforce copyright on proprietary code that's been laundered through an LLM, so much the better!
IMHO that "career building" part of it was always a bit problematic because of the incentives it could add.
Instead of "here's some small bits of code you might find useful" or "let me share my approach", the personal glory building style of OSS construction can and has tended towards "behold my framework" and a brand, and often even a VC etc funded entity to go with it.
I don't love LLMs just conjuring half-assed things into existence, but I've also been burned too many times by someone else's frameworkization of something that should just be a small modular bit, or part of a standard library. Languages like Rust that have exempted a lot from their standard library that frankly should be there suffer from this the most.
Why create a simple piece of code that does exactly what you want, when an LLM can smash libraries and frameworks together as well as anybody else? I can tell because I'm doing it right now. I'm not happy with the resulting bloat, but there's no denying that you can have the worst of both worlds if you really wanna.
> if we accept the assumption that it is easier to let an LLM generate functionality than to learn the interfaces of existing libraries and frameworks (or let the LLM integrate these libraries or frameworks)
This parenthetical seems like the weakness in the argument to me. Why have an LLM reinvent the wheel rather than let the LLM build on existing libraries?
Carmack once said something along the lines of him not reusing code a lot but rather rewriting it each time.
He is probably not alone, so this "curse" is has never restricted to Lisp completely. There always have been some wood and metal workers besides the many LEGO engineers.
"I also must confess to a strong bias against the fashion for reusable code. To me, “re-editable code” is much, much better than an untouchable black box or toolkit. I could go on and on about this. If you’re totally convinced that reusable code is wonderful, I probably won’t be able to sway you anyway, but you’ll never convince me that reusable code isn’t mostly a menace."
Well, if this means I will get to write code again, even indirectly with LLMs, instead of spending my whole workday merging renovate/dependabot pull requests, then I am all for it.
Application developers don't worry about binaries produced by compilers & in due time the same will be true for code produced by coding agents. I personally do not care what the code looks like as long as it actually has the features that I want & is secure enough for my limited use cases.
It seems like there are advantages as well as problems to this situation. Notably it prevents "monoculture" in software which prevent global worms. If every website implements network protocols from scratch, it would have bugs but possibly not the same bugs and vulnerabilities as it's neighbor. The extreme is every app is its own OS - back to CP/M where running a program involved inserting a disk and turning on the machine, that "interface" had it's charm.
This is rather interesting. But it misses the point somewhat.
The problem is whether or not we want to view code as a medium of communication between other humans vs code as an abstract machine that solves a problem. We know that "programs" in the general sense are fairly objective entities that can be studied objectively, which is where the famous Sussman line about this comes from.
But as a language for expressing execution semantics that a machine can then execute to produce a desirable effect on the real world (a solution to a problem), we would not expect such code to be written with other humans in mind, and as lisp is a strictly more expressive language (both for expressing the meaning of an algorithm that other humans will be able to study and understand and that a machine can execute), and that more people who program are concerned with producing a solution to a problem, the "curse of lisp" is the logical conclusion.
But it's overblown. There exist many excellent and fairly complete libraries for Lisp: Closer MOP, UIOP, ASDF, Bordeaux Threads, etc. And no surprise, these libraries are virtually ubiquitous and considered (at least by me) to be more or less extensions to the language. And these libraries are written in a fairly readable style that anyone can go and try to understand the code.
Secondly, the curse isn't really much of a curse at all. It is more likely to yield more robust programs. The assumption that code reuse through libraries increases software quality doesn't really hold up in practice: libraries that are "complete" tend to add bloat to applications; managing different versions of libraries on the same platform leads to arguments about static vs. dynamic linking, and so on and so forth. Ever had to debug .so versioning bugs on Linux? It's not fun. And yet, that is par for the course if you try to persist in the notion that "libraries and code reuse is always better than reinventing the wheel".
As a reaction to the utter failures that have occurred due to following that ideology, entire programming communities have popped up to fight it, particularly the "Handmade Hero" that became Better Software Conferences as well as "Software Should Work". And while those people aren't a huge fan of Lisp, they really do embody the Lisp mindset of "don't be afraid of building your own system in your own solution's language". And this does usually yield nicer systems to work with because programming is theory building: bringing in different ready-made libraries introduces conflicting theories that requires even more glue code to rectify and the glue code rarely foresees all the discrepancies. And so bugs appear in the integrations between them.
That's not to say that increasing "the curse of lisp" always yields higher quality. The kinds of developers that make great software with that mindset are the ones that are allergic to inconsistency at any level and so squash bugs as quickly as possible and that is what leads their software to be better. But on the practical side of programming work, you feel a lot less inhibited in trying to fix bugs when you're certain you understand the code and how the implementor decided to map the problem domain onto the range of possible solutions. This is where the issue with LLMs comes in: if programming is theory building (which I firmly believe), then what we're really after with using LLMs to generate code is a more effective technique of translating high level theory of the problem into the low level code that executes to solve the problem. As such, maybe vibe coding can work if the vibe coder diligently keeps in their mind what they're after. But usually what that means is writing notes about the desirable properties of the resultant application. And then they're just back to theory building... just at a somewhat higher level.
In at least some cases, this is pretty clear cut. Don't roll your own crypto libraries. You may think you know what you're doing. You don't - not enough to produce genuinely secure crypto.
Don't roll your own numeric libraries. There are libraries with decades of improvements to handle corner cases and increase stability. You won't create a better one, not in one lifetime.
So, yes, in some cases code reuse through libraries really does improve software quality.
It fundamentally comes down to this question: Can I, the user of this library, produce a library that is more robust for my purpose than the existing libraries? Those who say "yes" often underestimate the amount of care needed to create code that actually works - that works solidly, that is robust against failures on certain inputs.
But, you say, you don't actually have any of those inputs, so why do you care? You don't - now. But the day may come when your program is going to have to handle those inputs, and when it does, then it's your problem, because you wrote the library.
I broadly agree, but... if I use my hand-rolled HTTP server that only supports a subset of the spec (and use nginx for TLS termination), I would be free and clear of the Brotli attack.
This is fragmentation. The advent of slop has caused many repos to stop accepting contributions, and people maintaining their own forks. I do it too, diligently maintaining repos that the original maintainers were unwilling to, or simply to add my desired features.
Since code is now cheap I think it is not the end of the world, but it does suck a bit for users, who may need to cobble together multiple forks.
I see two main issues with the point op is trying to make.
1. The lisp curse is very lisp-specific because there are so many lisp implementations. But you essentially only have one major rust/ruby/php/java/perl/erlang implementation. Languages like C/C++ only have two major implementations (gcc,llvm/clang).
2. I see so many projects every day where developers pull in gigabytes of software only to use a small number of features. Don’t even get me started on the whole left-pad clown show. Gigabytes of libraries pulled left and right only to use a few megabytes at most.
"I’d rather rely on proven libraries with well-defined abstractions than generate hundreds of lines of code which solve a local problem for me. “Not generated here” is just a different version of “Not invented here”.
I disagree generally, and the problem is two fold..
I've just seen "well-defined" abstractions go wrong too many times. Over- encapsulation, over-abstraction, or just overall over-cleverness in third party packages that has led me to spend large parts of my career just frustrated trying to glue third party dependencies together in a sensible way... and the abstractions only adding complication.
The joy and temptation of having an LLM "just make it" is that this goes away. The functionality fits my existing structure and abstractions and I'm not fighting a war with the architecture that the third party maintainer decided worked best for them. And getting only the parts I need, not your whole giant framework with it. (And yes, I know the LLM could easily hallucinate something far far worse, but let's just... play pretend here for a minute)
The second part is that since the dawn of Maven, NPM, Crates.io generally there has just been this insane explosion in programs of transitive third party dependencies out into the stratosphere, and the cognitive, technical, and legal debt of so many deps generally is not sufficiently recognized by the community. Running "cargo tree" in my project is often frankly depressing. Multiple linked MD5/SHA256 implementations, random number generators, HTTP protocols, people dragging in tokio deps where I don't want them, multiple versions of the same thing compiled and linked... It's exceedingly frustrating, and with the explosion of security concerns we've seen recently, it's also potentially dangerous.
I don't love the idea of having LLMs "just yeet it" esp if people misapply this to things like security/cryptography domains. But I'm also more than skeptical about the actual real world quality of many hallowed third party packages. I've been burned too many times.
Perhaps potentially what replaces "widely used standard implementation of X in a library/npm/crate form" is an "LLM readable recipe for X along with a formal proof / automated test scenario that I can point my clanker at to get a community proven way of doing this" along with a set of prompts, licensing/credit requirements, etc.
Other than that, the Common Lisp ecosystem is doing well thank you very much. Many contributors to SBCL, regular contributions to the important ecosystem libraries, vivid projects (Lem), new editors (mine) and plugins (OLIVE), some companies picking Lisp and somehow supporting the ecosystem (hosting conferences, contracting implementors). https://github.com/CodyReichert/awesome-cl/ · https://github.com/azzamsa/awesome-lisp-companies/ And yes, new vibe-coded projects too. Some are useful (ICL, cl-tuition…), some reinventing the wheel (badly) went nowhere. For some, we'll see.
But then once I read the second paragraph, I agree with you. Lisp is doing fine in terms of important ecosystem libraries. The article sets that as the context when it asserts as fact that "established libraries are incredibly stable, tested by hundreds of thousands of developers, included in thousands of CI pipelines and maintained by experts on the topic."
I see that sbcl itself is alive and doing rather well, but other than that the landscape is fairly grim.
Unironically the most successful modern day lisp is clojure, that mostly piggybacks the jvm and can reuse most java libraries and frameworks.
So you design from the core logic and write shim to external dependencies. Instead of bringing deps from the get go.
I've used more Rust tools, and jobs are outside few niches non-existent.
Today, when mainstream languages have stolen much from Lisp, the benefit of using a language where you can generally just add a gem, egg, or crate is much higher than the benefit of the one thing that is still unique to Lisp: macros. Reflection is inelegant as hell when you compare, but it will get you almost all the way macros do.
At the same time, typed languages have become much more like dynamic languages. The types have disappeared into var and let statements; they are now inferred rather than repeated. Pattern matching has been added, and more—without compromising the benefits for larger teams and larger code bases.
Purists will disagree, but I haven't seen a real world problem solved in Lisp that I couldn't also solve in other languages.
And yes, I am interested in being proven wrong. I want to know what kind of problems still exist that are that hardcore. But I suspect a big part of it is that those who code in this still think the "hello world" for Java looks something like this:
when in reality it now looks like this: and in C# it looks like this: And comes with full first class support for Linux and Mac.Lisp is an intellectual tower and has given an incredible amount to the programming community in terms of ideas and a playground for people to try things. I'm not denying any of that — it should be celebrated. But my thesis is this: most of that has been moved, copied, stolen, and adapted to mainstream programming languages.
Turing completeness makes "I couldn't also solve" an impossibly high bar.
But if what you mean is, you haven't seen macros used in ways that make you wish your day-job programming language had macros, that's fixable.
Why haven't I submitted my patches? Because my agent wrote them and I didn't review them. I would feel bad submitting my patches without looking at them, and I don't have time to look at them.
It's honestly not worth it. I understand both sides here, but people are going to have to come up with new ways of collaborating. Pull requests are likely not it anymore.
KMyMoney stopped importing (some) OFX files for me. I asked an LLM to debug.
It was a wild journey. An obscure bug in some QT library that was triggered because of something completely unrelated that I had installed briefly in 2019 (and uninstalled years ago).
I had it fix the issue by cleaning the leftovers from 2019, but it's still a QT bug. I had it do a writeup, but I'll never submit it. It's fairly sophisticated and I don't know QT code at all.
https://gavinray97.github.io/blog/llm-build-cheaper-than-sea...
As an example, a black hat doesn't get 70% of web sites from one shared bug in PHP. They need to target specific sites, and the exploit isn't likely to work in very many other places.
Which, probably, you will also have to do.
I suspect that a more effective mitigation to having exploitable bugs would be to have an LLM read all your HTTP requests and the actions the web server proposes to take in response and give a Jev-like approve/disapprove decision.
The one situation where they really can't get anywhere profitable is a unikernel webapp. With no source and no shell they find the vuln but the worst they can do with it is possibly poison or exfiltrate data as a one off, or crash the server. Often they can't even figure that out. Can't even leave themselves a reliable way back in.
Of course the traditional problem with unikernels is that you have to port EVERYTHING you need yourself. But the cheapness of this is exactly what we're talking about so.
Looks like security through obscurity's back on the menu.
The advantage there shouldn't be obscurity, so much as very limited attack surface in the whole exposed stack.
The language is restricted to permissions granted on startup and can only access the system via the runtime, which only supports a particular subset of linux syscalls. So, outside of the stdlib, whichever feature you need, you simply use an LLM to port an existing python/rust/JS/TS/Java library to the language and use it.
As the runtime has its own scheduler and GC, I have actually been wondering about the feasibility of skipping the linux kernel and making the runtime itself the "os." Given that so much modern hosting is actually done on virtual hardware, the drivers could be hosted within the runtime as well.
- statically typed from end to end
- closed and open/extensible types
- closed types are immutable, open ones have two variants: immutable and mutable
- arrays have two variants: immutable and mutable
- no OOP/classes. inheritance is nominal and mainly exists to provide access to class membership (is X a Y?)
- functions operating on data
- UFCS exists to provide nice features like "apple".substr(...)
- first class functions and closures
- monomorphized generics
- tailcalls
- operator overloading
- modules and namespaces to control access
- erlang/beam style first class processes. So you `spawn` a function and communicate between processes using send/recv instead of async/await
Processes are completely isolated at the runtime level. Each has its own stack, heap, mailbox and GC.
The bytecode carries all type information. The VM will compile using AOT-on-load.
My own runtime-becomes-a-os idea is still just an idea. I haven't really pursued it with seriousness. Will probably do that once the language and runtime stabilize.
Genetic monocultures like bananas are at risk from a single species of fungus.
One small issue: they didn't exist! It had just hallucinated them!
Which was a real shame, since they were much nicer than the actual stuff I ended up finding.
I just realized, thinking about this now, that these days (and perhaps even back then), I could just say "ok now take this code and build the missing library so it actually works!"
That was, in retrospect, an important bellwether.
What if there was a code sharing site where it was unambiguous what a given snippet of code would be named and where it would be stored, so that a user could just browse it and find a list of implementations for a given problem?
This of course runs up against one what John Ousterhout notes is one of the most difficult problems in computer science --- naming things --- in his book:
https://www.goodreads.com/en/book/show/39996759-a-philosophy...
I've tried to do it on a small-scale with my own work, storing bits of (la).tex code in a single file, but on reviewing it, I'll often find that I have written a given solution a second time and given the macro a slightly different name....
Almost everything else is domain specific where you have to understand the domain first before understanding the code. And modeling a domain is all about tradeoffs between accuracy, performance, specific vs generic,…
Instead of collecting such solutions, a better approach is to master basic data structures, and then improve your modeling capabilities. After that, it’s mostly learning domain as they come (dsp, physic modeling, operating system concepts, relational database, graph theory, 3D geometry, accounting,…)
What's more secure isn't so simple. If you vendor something with a vulnerability then it wont be patched. But vendoring may also mean you took the 2% of the library you need and not the 98% which reduces the surface area for vulnerabilities. And it can cut the other way - you may update a package that was previously good and is now vulnerable (regressions, supply chain attacks).
There is a balance that must be struck, and I think LLM's strike a better balance than people do. Dependencies are a convenience with a cost and LLM's can trivially copy or re-implement things that we cant.
If you aren't using a library function you're not vulnerable to exploits in that code: unused but imported code is inert (possible usable as a gadget if you're vulnerable elsewhere, but at that point you're screwed anyways) and most compilers/runtimes completely eliminate it.
> you may update a package that was previously good and is now vulnerable (regressions, supply chain attacks)
There are better mitigations to this than not using a library.
You still completely ignore the license violations.
The ones that you are imagining to happen? Its not a problem because you could simply observe proper licensing.
>If you aren't using a library function you're not vulnerable to exploits in that code
Not true, just because you arent calling a function doesnt mean it cant run. Read up on Log4Shell.
It's usually more straightforward to just rebuild it from scratch — not to mention infinitely more satisfying — and that was true before AI!
On the other hand, I routinely build Java/JVM projects with dependencies that haven't been updated in a decade that compile without issues.
Somewhere in the middle are most languages, I think.
It depends on the "load bearing" code. Obviously something like crypto is not something we should implement ourselves but core business logic would be "load bearing" in another way and is probably not something you want to farm out to a dependency.
(LLMs accelerate this and "a few years" can easily turn into "a few months".)
I 100% disagree with this. Supply chain attacks are already at an all-time high due to agentic hacking assistance, and that trend shows no sign of slowing. Every dependency you add is another potential attack vector, and just checking the dependency’s code doesn’t solve the issue - your dependencies also have dependencies!
The cost of rolling your own has gotten so low that it’s almost always a better security tradeoff to reimplement the thing yourself. Is that a good thing? I’m not sure.
Also, you clearly don't work with products that have any sort of serious corporate liability. Being able to blame a supplier/library is almost always preferable to assuming the liability yourself
Why stop there? We could eliminate all external dependencies and even build our own compiler and OS, since the cost of rolling own solution has apparently gotten so low, right? Sometimes I feel like I'm living in a completely different dimension from some of the commenters here.
Yes, some dependencies are great. But unfortunately the vast majority are problematic: Not properly supported, badly tested, with unhelpful maintainers, funny licenses, undocumented and, definitely, not being able to cover your requirements 100% so you have to change your requirements to meet the dependencies instead of the other way around.
I asked him with the first disk was, and I remember distinctly he said it was an operating system. I asked him what an operating system was, and he said it was like a program, it was a program technically, that makes it easier to run other programs, just it was a program that didn't do much on its own and you wouldn't use it without another program right afterward. I said it didn't seem easier if you had to load a different disk than the one you wanted to use, and you still had to load the one you wanted afterward. He looked at me disdainfully and patiently explained that all or almost all the software in the future would use operating systems and they whole process would get easier or even invisible but there would always be an operating system in there somewhere at some point in the very near future. I was skeptical, but I nodded along.
I imagined an operating system as a dressed up software library, which was more intuitive to me as a concept.
Looking back, AmigaOS was remarkably complete and modern for 1985: preemptive multitasking microkernel, shared libraries, multimedia support, everything was modular.
I'm not taking sides here, I love Lisp, though I don't use it anymore (except elisp in emacs, but it doesn't count). My point is that debating the name itself distracts from the actual issue.
Any language that's powerful for creating new syntax and doesn't have an equally powerful community repository gets this same effect to some extent.
the lisp dialect growth is also famous, it's less the case these days because people know about it, but it existed
i'd add one thing to your point, is that lispers know how to port concepts from lang a to lang b very easily, support libs, macro layer if need be (see goroutines in clojure) so fragmentation is not as hard it was in mainstream languages (ps: even though today everybody goes into ast / transpiler to experiment)
Coding agents have actually cut both ways for me. I can reinvent the wheel more easily and with higher quality, but I can also integrate an external library much more easily than before.
The other day I needed my system to download SFTP, so I had the agent build an SFTP client. It turned out that I actually needed FTPS, so I had the agent build that too.
Another time I wanted to integrate with Git repos, but instead of trying to reinvent that, I asked the agent to integrate libgit2. Of course, I asked it to add an API that matched the semantics of my system, so from my perspective, the wheel was exactly how I wanted it.
I think what's going to happen is that there will be more vertically integrated platforms that do it all. If you think about it, it's really annoying to cobble together a system from different libraries, each of which might have different semantics.
What we want, as developers, is to have a single, well-designed, integrated platform that exposes all the functionality we need in a consistent way. That's a lot easier now that LLMs can mold code to your exact specifications.
I'm biased, of course, because that's exactly what I'm building with GridWhale.
But even if they were pleasant and did what I needed (this one time I installed a 5000 line lib that was missing 20 lines to do the thing it was actually designed to do.. like, they couldn't have added 20 more lines?!), an issue I have is spoilers.
Like, at university, they constantly said, ok here's a big problem in computer science.. (3 seconds later) .. aaand here's how we solved it! And I'm like agh, no! Spoilers! I want to figure it out, jeez!
That's also how I feel about "computer, just magick my entire program into existence." Spoils the fun of figuring it out!
Of course, there's no tool without flaws. Nevertheless, oh boy, how amazingly practical Lisp-dialects can be in reality (even with all their imperfections), it's just astounding. Sometimes I think maybe it is some kind of ego-defeating stance or whatever, maybe smart programmers just can't believe how the stupidly simple idea of Lisp can ever satisfy their galaxy-sized brains? To me, Lisp is like a boundless sea of incredible idea generation on the spot, turning it into practical reality like some magic. Effortlessly, painlessly, and with joy.
I don't know, maybe I had to go through the pain of learning the intricacies of a dozen different stacks, using them for years, only to discover Lisp later in my life. Perhaps I wouldn't know the difference otherwise?
Just the other day, I was watching a YT video, which I of course fully control directly from Emacs - that is very useful for taking notes. I can mute, pause, speed-up, rewind, toggle subs, etc. All while typing a message, a note, or while looking at a PR diff. I thought, "hmm, floating window would be nice". Without even missing a beat, I opened eca-emacs buffer, I didn't even have to type the prompt - I used speech-recognition. Agent then (running in Elisp REPL), connected to my WMs REPL driven by Fennel, using tools in my Clojure-made harness with its own REPL, tapped into the lua-based mpv, analyzed things on the fly and just built the solution, right there. I didn't even have to compile, restart, reload, refresh anything. Now I can detach a video playing in the browser into mpv window and later put it back. With a press of a key.
It's almost as if Lisp has been quietly waiting for this moment. LLMs and Lisp REPLs make such a great duo - if you're a programmer and haven't seen it in practice yet, you should seriously be experiencing deep FOMO right now.
It seems sort of wasteful of tokens!
REPLs in non-homoiconic languages like Python, and homoiconic Lisp do differ. Every single step in Read-Eval-Print-Loop does. That's why Lispers sometimes refer to Lisp REPL as a "true REPL", because in practical comparison, Python's REPL is more like an "interactive shell". Homoiconic means that code-is-data, and the Read step in REPL doesn't need to do any acrobatics with parsing, building AST, etc, it's just directly reading the code and passing to Eval as is.
Here's a sense of what it looks like to be writing software with Lisp: https://youtu.be/tHgApFhevrQ?t=2272 I purposely moved the marker so you don't have to watch the whole talk. Jeaye here coded his entire presentation from inside a terminal, embedded into a game.
That's how typically Lispers write software - "from inside out". Lisp, uniquely fits for LLMs, because when you give them a Lisp REPL, they stop guessing and start empirically analyzing current state of things and produce working solution faster, costing fewer tokens.
The typical flow with non-homoiconic PLs is is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation technically stateless. It is a whole different story with Lisp.
> It seems sort of wasteful of tokens!
Clojure is proven to be one of the most token efficient languages. https://tokencalculator.ai/most-token-efficient-languages-fo...
> It seems sort of wasteful of tokens!
This was referring to the LLMs' tendency to blat out hundreds of tokens into a single tool call, not to Clojure.
The REPL in the part of the presentation you link to only evaluates two forms, (range 10) and (bounce!), neither of which takes much advantage of homoiconicity.
cchase88 was talking about a degree of homoiconicity in Tcl that has been abandoned by most dialects of Lisp:
(That might look wrong, but it's actually correct. In Tcl the list {name} is exactly the same as the quasiquoted string "name" and the bare word name.) You can't do that in Racket or SBCL, but it does exist in Clojure; it's clojure.repl/source. However, the part of the presentation you link to doesn't use anything similar, and it's not really a mainstream part of Lisp development anymore, the way it was 50 years ago.What I think we'll see happen is that AI vendors will create private ecosystems for sharing code. I.e. you subscribe to Codex and you get access to a suite of libraries that speed up your development process. Behind the scenes there'll be a mirror of the how the current ecosystem works, with agents submitting bugs and feature requests and a private swarm of agents working the most popular / high priority.
(1) GitHub is now full of slop. The majority of new open source libraries are now generated by AI. Many come from students who are just looking for resume garnish. Others come from devs who are more excited about being an "open source developer" than actually doing open source development
(2) With AI I can probably build something better. And if it's crap? No problem! I can just wait 6 months and rewrite it with gpt_galaxy or Claude Torah or something
At the same time I have no real incentives to contribute back. Anything I create competes immediately with a sea of slop. Any actual work I do gets identified and swallowed up by the plagiarism machine. And to be honest I only occasionally had good experiences with OSS anyway
I think we are looking at the end of open source software. Not the end of free software - in fact that may end up flourishing with LLMs and reverse engineering and the like. But the idea of programmers sharing code, for communal interest and career building, is completely dead.
In the old days, both the investment in the library and the additional cost you'd want to sink on it to consider it publishing quality incentivised to keep it truly internal. LLM code however? It might be slop, but it's accurately groomed slop. Prompting the model to roleplay a public library author might even improve the output when it's in fact not published at all. The investment is low, the extra cost of sharing is low, worst that could happen is others actually picking it up and that would not be bad at all.
Is that spamming the public information universe on the faint hope that perhaps future models might come pre-trained with understanding of your library? Sure. But it's a very different thing from the scenarios described in (1).
LLMs are antithetical to free software, which everyone seems to forget is as much an ethical and social movement as it is a classification of software licensing, and will be until one can examine and audit the training data of a model to exclude the possibility of including any non-free code, and until they include, document and respect the licenses for the sources they do use.
The degree to which they discourage people from actually being able to understand and modify code without an LLM also makes the code they generate implicitly nonfree. All of the freedoms of free software presuppose their utility to a human over their utility to a machine. If people aren't reading, writing and distributing code then it defeats the entire point. That's just proprietary software with extra steps.
I don't know what you call this new paradigm of vibe-coding and just not giving a damn about sources and licenses and copyright, but it isn't free software. And it's probably worth pointing out that said paradigm was created by and is most useful to corporations, not people. Using the plagiarism machine to plagiarize "the man" is still subject to the Rule of Goats[0].
[0]https://www.urbandictionary.com/define.php?term=Popehat%27s+...
At the core of Free Software there is one simple kernel: the user being in control. Licensing and copyright is used to achieve that, but they're the means, not the goal. The goal is that you, the user, should be in control of your computing, not the developers of the software you are using. Hence software that is free i.e. the software not being bound by the whims of its developers.
From that perspective, IMO, LLMs are far from antithetical to Free Software exactly because they let[0] their users do whatever they want with the software and their computers. That a user can instruct their agent to take the video playback code from MPV and combine it with the UI of XBMC[1] while ensuring it runs on the Raspberry Pi 4 they have connected to their TV (that they "jailbroke" to remove the intrusive ads and other user-hostile software that they replaced with software to their liking) even if they barely have any programming knowledge themselves is a sign of software freedom.
[0] at least the user-aligned ones not bound by restrictions placed on them by their own developers, so perhaps we need a Free Models movement too or something along these lines.
[1] random example, i don't even know of XBMC uses MPV's code already, so assume it doesn't
And because LLMs can't be audited and can't track or respect the licenses of the software they use as sources, no software generated by an LLM respects the freedoms of Free Software.
>That a user can instruct their agent (...) even if they barely have any programming knowledge themselves is a sign of software freedom.
It isn't, because as I said, freedom presupposes knowledge. If you barely have programming knowledge and you don't understand the code an LLM generates then you can't 1) read it, 2) edit it or 3) redistribute it without an LLM. You are not exercising any freedoms in that case, you're just being a consumer of a service.
>so perhaps we need a Free Models movement too or something along these lines
I said what I think would be the minimum requirements for a model to respect free software - at the very least it should only be trained on free software, it should be provably auditable without hallucination (I don't even think this is technically feasible) and it should respect the licenses of the software it incorporates, including attribution where necessary.
Beyond that there's still the problem that training even open weight models requires expensive infrastructure making it implicitly proprietary. As long as we need data centers at all it's never going to be as free as a text editor.
https://www.threads.com/@kenpopehat
Instead of "here's some small bits of code you might find useful" or "let me share my approach", the personal glory building style of OSS construction can and has tended towards "behold my framework" and a brand, and often even a VC etc funded entity to go with it.
I don't love LLMs just conjuring half-assed things into existence, but I've also been burned too many times by someone else's frameworkization of something that should just be a small modular bit, or part of a standard library. Languages like Rust that have exempted a lot from their standard library that frankly should be there suffer from this the most.
This parenthetical seems like the weakness in the argument to me. Why have an LLM reinvent the wheel rather than let the LLM build on existing libraries?
He is probably not alone, so this "curse" is has never restricted to Lisp completely. There always have been some wood and metal workers besides the many LEGO engineers.
"I also must confess to a strong bias against the fashion for reusable code. To me, “re-editable code” is much, much better than an untouchable black box or toolkit. I could go on and on about this. If you’re totally convinced that reusable code is wonderful, I probably won’t be able to sway you anyway, but you’ll never convince me that reusable code isn’t mostly a menace."
The very nature of training an LLM seems backward-looking.
Or, as a question: has an AI gotten around to generating the Final, One True Programming Language (FOTPL) yet?
lol but still true
The problem is whether or not we want to view code as a medium of communication between other humans vs code as an abstract machine that solves a problem. We know that "programs" in the general sense are fairly objective entities that can be studied objectively, which is where the famous Sussman line about this comes from.
But as a language for expressing execution semantics that a machine can then execute to produce a desirable effect on the real world (a solution to a problem), we would not expect such code to be written with other humans in mind, and as lisp is a strictly more expressive language (both for expressing the meaning of an algorithm that other humans will be able to study and understand and that a machine can execute), and that more people who program are concerned with producing a solution to a problem, the "curse of lisp" is the logical conclusion.
But it's overblown. There exist many excellent and fairly complete libraries for Lisp: Closer MOP, UIOP, ASDF, Bordeaux Threads, etc. And no surprise, these libraries are virtually ubiquitous and considered (at least by me) to be more or less extensions to the language. And these libraries are written in a fairly readable style that anyone can go and try to understand the code.
Secondly, the curse isn't really much of a curse at all. It is more likely to yield more robust programs. The assumption that code reuse through libraries increases software quality doesn't really hold up in practice: libraries that are "complete" tend to add bloat to applications; managing different versions of libraries on the same platform leads to arguments about static vs. dynamic linking, and so on and so forth. Ever had to debug .so versioning bugs on Linux? It's not fun. And yet, that is par for the course if you try to persist in the notion that "libraries and code reuse is always better than reinventing the wheel".
As a reaction to the utter failures that have occurred due to following that ideology, entire programming communities have popped up to fight it, particularly the "Handmade Hero" that became Better Software Conferences as well as "Software Should Work". And while those people aren't a huge fan of Lisp, they really do embody the Lisp mindset of "don't be afraid of building your own system in your own solution's language". And this does usually yield nicer systems to work with because programming is theory building: bringing in different ready-made libraries introduces conflicting theories that requires even more glue code to rectify and the glue code rarely foresees all the discrepancies. And so bugs appear in the integrations between them.
That's not to say that increasing "the curse of lisp" always yields higher quality. The kinds of developers that make great software with that mindset are the ones that are allergic to inconsistency at any level and so squash bugs as quickly as possible and that is what leads their software to be better. But on the practical side of programming work, you feel a lot less inhibited in trying to fix bugs when you're certain you understand the code and how the implementor decided to map the problem domain onto the range of possible solutions. This is where the issue with LLMs comes in: if programming is theory building (which I firmly believe), then what we're really after with using LLMs to generate code is a more effective technique of translating high level theory of the problem into the low level code that executes to solve the problem. As such, maybe vibe coding can work if the vibe coder diligently keeps in their mind what they're after. But usually what that means is writing notes about the desirable properties of the resultant application. And then they're just back to theory building... just at a somewhat higher level.
Don't roll your own numeric libraries. There are libraries with decades of improvements to handle corner cases and increase stability. You won't create a better one, not in one lifetime.
So, yes, in some cases code reuse through libraries really does improve software quality.
It fundamentally comes down to this question: Can I, the user of this library, produce a library that is more robust for my purpose than the existing libraries? Those who say "yes" often underestimate the amount of care needed to create code that actually works - that works solidly, that is robust against failures on certain inputs.
But, you say, you don't actually have any of those inputs, so why do you care? You don't - now. But the day may come when your program is going to have to handle those inputs, and when it does, then it's your problem, because you wrote the library.
Since code is now cheap I think it is not the end of the world, but it does suck a bit for users, who may need to cobble together multiple forks.
1. The lisp curse is very lisp-specific because there are so many lisp implementations. But you essentially only have one major rust/ruby/php/java/perl/erlang implementation. Languages like C/C++ only have two major implementations (gcc,llvm/clang).
2. I see so many projects every day where developers pull in gigabytes of software only to use a small number of features. Don’t even get me started on the whole left-pad clown show. Gigabytes of libraries pulled left and right only to use a few megabytes at most.
I disagree generally, and the problem is two fold..
I've just seen "well-defined" abstractions go wrong too many times. Over- encapsulation, over-abstraction, or just overall over-cleverness in third party packages that has led me to spend large parts of my career just frustrated trying to glue third party dependencies together in a sensible way... and the abstractions only adding complication.
The joy and temptation of having an LLM "just make it" is that this goes away. The functionality fits my existing structure and abstractions and I'm not fighting a war with the architecture that the third party maintainer decided worked best for them. And getting only the parts I need, not your whole giant framework with it. (And yes, I know the LLM could easily hallucinate something far far worse, but let's just... play pretend here for a minute)
The second part is that since the dawn of Maven, NPM, Crates.io generally there has just been this insane explosion in programs of transitive third party dependencies out into the stratosphere, and the cognitive, technical, and legal debt of so many deps generally is not sufficiently recognized by the community. Running "cargo tree" in my project is often frankly depressing. Multiple linked MD5/SHA256 implementations, random number generators, HTTP protocols, people dragging in tokio deps where I don't want them, multiple versions of the same thing compiled and linked... It's exceedingly frustrating, and with the explosion of security concerns we've seen recently, it's also potentially dangerous.
I don't love the idea of having LLMs "just yeet it" esp if people misapply this to things like security/cryptography domains. But I'm also more than skeptical about the actual real world quality of many hallowed third party packages. I've been burned too many times.
Perhaps potentially what replaces "widely used standard implementation of X in a library/npm/crate form" is an "LLM readable recipe for X along with a formal proof / automated test scenario that I can point my clanker at to get a community proven way of doing this" along with a set of prompts, licensing/credit requirements, etc.