musl’s crypt() returns the string * as an error indicator when the key is over 256 bytes long. * is also exactly what the stored field contains for every account with no password, so the “hash” matches and the login succeeds.
Does anyone know why musl does this? Are they the only C library that behaves this way?
For a C library that famously tries to adhere to POSIX as much as possible, this seems like a crazy choice. From POSIX standard:
Upon successful completion, crypt() shall return a pointer to the encoded string. The first two bytes of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
Thanks yeah just woke up, finished making my way through the single-article website and still unclear if other libcs do the same thing (the article says libxcrypt picked it up, but only after musl did it).
Some more clarity ... Multiple implementations have these sentinel values (not OpenBSD). But unfortunately only musl had a sentinel value that was also used in shadow files...
Apparently because some devs didn't check for NULL which resulted in crashes. Weird choice to fix other peoples' bad programming by deviating from the spec.
To be fair (as the sshole site mentions) it is also necessary to fake a password hash check to avoid timing side channels revealing to remote clients whether users exist, so returning a sentinel value is not a horrifically unreasonable solution to make sure even naive code does approximately the right thing.
Though I am surprised that all of the code examples do not use constant-time strcmp(). Actually, I wonder if even constant-time strncmp() would still leak information about whether the account is disabled because in that case both arguments are much shorter? Food for thought.
But yeah, the crypt(3) API is very 80s C, the man page even features a lengthy description of a long-deprecated and utterly broken hashing algorithm using DES that is even implied to be broken in the text itself (and the warning message is so old it refers to "parallel computers" as though they were a new concept). Given that the default mode is to use a broken hashing algorithm, I'm shocked anyone actually uses the libc crypt(3) -- surely it's trivial to just write your own that isn't awful...?
Constant time strcmp doesn't fix it. The problem is that the user doesn't have a password at all (the hash is *). But which hashing algorithm is used is encoded in the password. So they pick a different user's hash to not leak the information that there is no password.
Alternate solution would be to hardcode a specific algorithm, but that could also leak info in many cases, I guess.
The spec only defines a single error state - ENOSYS, that is basically only for systems that don't implement crypt at all.
Error handling is underspecified. The spec doesn't give you enough well-defined behaviour, meaning everyone lists pages of error-handling extensions they have added.
Pedantic, but more "author's understanding of the spec" which isn't always correct.
> assume nonstandard behavior
I go back and forth on this one (spec vs standard behavior). It feels like at some point the most widely used impl -is- the spec, as we've seen in other places (web standards).
> Upon successful completion, crypt() shall return a pointer to the hashed password [...]. Otherwise, it shall return a null pointer and set errno to indicate the error.
IMO this is where languages like Rust shine. The design patterns built into the language for handling errors (Result) - force you to handle errors / function results with sentinels, null pointers, or errnos are so much more error prone than handling a Result<T,E> using a match statement.
crypt() returning `*` is insane especially since these kinds of sentinels are all over /etc/passwd
I guess this is where the culture around rust shines. You can write crypt that returns a string in rust just fine, just like you can write crypt that returns a tagged union in C. Cultural differences and syntactic sugar change which version is idiomatic.
I suppose if people started writing lots of code that crashed on error in Rust some enterprising developer might improve the function to always return a non-error value and set it to ‘*’ in error cases.
The culture is downstream of the design IMO. In general, people write the API that is obvious. In C, returning a string is obvious, while defining a custom OptionalString type is high friction. In Rust, returning Option<String> is as easy and obvious as returning a String.
Sure, you could write deliberately bad code in Rust by unwrapping everything everywhere, but the point is that it is a distinct choice to do so. You can’t forget to handle errors - you have to make the choice to disregard them. If you forget to check for NULL in C it won’t complain and it’s dependent upon you to remember to do so.
In the vast majority of programs, errors and non-results are irrelevant at the call site. You want to program for the happy path and not wrap every single call with error checking code or `try/catch/finally`.
h = file_open(path)
data = file_read(h)
file_close(h)
return data
is the code you want to read. This essentially is:
return file_open(path).success(h -> h, file_read(h)).success(h, data -> file_close(h); data)
A language with thoughtful error handling primitives will support such code.
This is the root of a lot of the evil. Yes, humans want to program for the happy path. It is a lot more fun. However, that bias leads to so much lack of diligence to properly deal with the unhappy path.
Syntax is not going to solve logic errors. No amount of checked exceptions or forced unwrapping of a result will help with TOCTOU or any number of similar vulnerabilities in code.
You ideally want to attack the problem at two ends:
- make the syntax noise-free so you can see what is happening
- build primitives at the syscall and stdlib level for common footguns
You would still need to deal with domain-specific logic errors, but you have at least reduced the surface area exposed to casual attacks.
returning a non-success you either handle it or pass it up the chain, potentially with not a lot of information about what originally caused it. with an exception you can retain the full information about what happened by either rethrowing it, throwing your own exception with it embedded as the cause or declaring that you don't handle it by saying you can throw it.
The question is _where_ the error needs to be handled. It's almost never directly at the call site. And then there's a few issues with checked exceptions and Java language things (RMI, closures, etc) which make checked exceptions cumbersome. An exception (checked or not) might be worse than sum types, but they're still a thousand times better then returning an error in-band.
> The question is _where_ the error needs to be handled. It's almost never directly at the call site.
Which is why exceptions in Java can bubble up the stack? You don't have to catch it in a particular method you can let it go to the caller of the method, and so on.
I agree but I think was also because Java didn't have sum types (or still doesn't? years I haven't read Java) and rethrowing is super easy, while Rust kinda nudges you into the right direction (if you need to rethrow at all, think if you want to rethrow verbatim or wrap with a semantic error type).
The problem with them all always comes when the ever-steady march of abstractions, features, and edge cases results in one routine throwing the same two error classes but in different places, and how you handle them is different despite being the same wrapper. Add a thousand developers and several years of churn, and poof you get extremely complex bugs that result from errors seeming to come from one place, but actually coming from another.
Having been on the receiving end of debugging that a few times, I no longer mind the in-band error handling, although admittedly I'm inexperienced enough to be confident its better. Just for the random code I've had to read where I just need to know exactly how something I didn't write and will never read again works, and whether the error is handled and how, i love reading in-band error handling, even if I too dislike the way an error laden routine must be written. Over time I've come to appreciate elegance "in the large" as opposed to the code I see in any individual function.
I always thought the ideas was great, but type system was holding it back. If you could have expressed that your thrown exceptions are the same as some other method's (e.g. a method of a class given as a parameter), and some other classes, then it could have been a lot less annoying. Powerful, even! But Java did take its sweet time to get even type parameters.
I have written very little Java, though, so this is more of an armchair analyst position.
Amen, people saw the symptoms of "needs more syntactic sugar" and misinterpreted it into "rigorous error handling is bad".
The same way that layered code should react/wrap/translate when it comes to return types, it should do the same for thrown exceptions, and checked-exceptions helps you do that. They're both results of computation.
Not really related to the topic, but AI-popular sentences and language structure stand out to people right away these days right? I noticed some.
"If you cannot update yet, at least disable password logins: set PasswordAuthentication no in your sshd configuration and restart sshd. Either one closes the hole." (only AI would characterize a critical SSH implementation vulnerability as a hole closing game)
An attacker who can reach your SSH port can log in without knowing any password. It takes one connection, works every time, and needs no retries. (It sounds like the writer is bragging about the exploit instead of describing it. AI does this)
The attacker needs the name of an account whose password field is * (an account with no password set, which is normal for key-only accounts), and needs the machine to have at least one other account with a $5$ or $6$ password hash. The attacker then sends a very long password (257 to 1024 bytes). One user with a password is enough: it makes every account without one reachable. (same as above made stronger with the 'statement: statement2' format)
"Users coming from a directory service through musl-nscd (nslcd, winbind, sssd), since they have no shadow entry." (a human would be more clear about what a shadow entry is and give more context, I think)
Hmm the rest of the article (the remaining 70-80%) seems quite human.
Anyway, not sure why I spent 10+ minutes on this. It just jumps at me right away.
An aside but one nice thing I realize about not using LLMs is, I guess, I am not tortured by this kind of thing, because presumably I am not seeing it all the time. I can't even imagine quite being in the mode of reading this kind of text so closely like this, really. Its a security advisory, I don't really have my critical prose-reviewer hat on.
As a grad student I read and graded a lot of student philosophy papers (well before LLMs), and I just remember the exhaustion of it. It made me appreciate how much I could otherwise just read something, get information or enjoyment out of it, and not focus on each sentence construction.
Just to say, I do believe you, I get that if you have to put up with it so much in general, it must prime you to find it everywhere. You have to read so much more now, I'm sure, so you're trained to be in that mode. It makes sense! I am just sorry its like this for you and most everyone else now. It sounds exhausting.
It's nice to see how quickly this is being addressed, and likely monitored for any unintended side effects.
Looking at the unpatched Ubuntu things in my life, and I'm pretty close to asking Claude to update things to see how quickly Alpine can get close, assuming responsiveness can stay this fast and quality.
both alpine and musl are great in my opinion, and as far as I know theres no empirical evidence that it is more or less secure overall than alternatives. would you like to elaborate your opinion?
I have to admit that I am a bit disappointed in this kind of thinking. There's vulnerabilities everywhere as especially the past few weeks have shown.
One occurrence is not a statistic and should not be used as evidence - it's just an example of a vulnerability that due to various bad circumstances lead to a particularity bad outcome. This logic is in my opinion equally flawed as calling alpine safe now because this particular vulnerability has been fixed.
I'm not a big fan of blaming and I think it's just a result of bad circumstances, but it's interesting to see how some seem to blame musl, but one could argue further that musl documented the behavior of their API which was ignored by openssh and so on
i think it's just unbelievable there isn't a bigger push to move away from c to languages that can prevent stuff like this from happening. this and just the pure unsafety. it's 2026. it's like pure negligence at this point, the equivalent of building a bridge with matchsticks because you like using it more.
How would using a language different than C help here? At it's core this bug is a logic bug, where "*" == crypt(x), because crypt in musl started returning "*" in some cases some years ago. This could be Rust. This could be Go.
Another comment said that returning a * was to avoid returning null which would crash users of the library who didn't check for null. That seems like a C smell to me
Not really. This is like if the char* were an Option<String> but you had programs which stupidly didn't check if the Option had something and assumed it always did. As others have said, Rust would not prevent this.
`Option<String>` or `Result<String>` forces the check at the callsite. Unlike C, you cannot forget that the function can error. This absolutely helps prevent issues like this.
Now, the decision to return `"*"` instead of letting poorly written code crash on `NULL` is not good either.
I don't think it's that people forget it while writing the code, it's that people want their program to work with as little effort as possible and decide to ignore it "for now". And that definitely can happen with Option and Result as well. There's no protection for not checking return values, and it's probably not even possible to protect against (in entirety)
Well they would use `unwrap` if they were lazy, but at least that would be quite visible in the code and would not result in undefined behavior due to operating on a null value. And there might be a group of developers who first choose to use .unwrap. I hope it is the minority.
Given how easy it is to just forward that error to the caller, I think most Rust developers would just choose to do that.
While dereferencing a null pointer is technically UB, it practice it leads to a safe trap. So .unwrap is the right comparison. If you always forward the error to the caller, it will also just lead to some random error message at some point, not really different to a crash.
> There's no protection for not checking return values, and it's probably not even possible to protect against (in entirety)
That's exactly what you get with Result and Option, it's impossible to forget.
Compare:
char* hash = crypt(key, salt);
// We crash here on the hash derefence, it's no clear from the code
// that we were supposed to check for NULL or that we forgot.
if (!strcmp(hash, expected)) {
}
To:
let result: Result<Vec<u8>, Error> = crypt(key, salt);
// We MUST write this line, which is a very clear signal
// that we can panic here.
// This can be grepped for or forbidden.
let hash: String = result.unwrap();
// if result == expected { would type error
if hash == expected {
}
Sure, but it's far better than the C code and I don't think in Rust a library would start returning `Ok` with sentinel values to "fix" shit calling code. Maintainers would, rightfully, tell people to fix their code.
Don't get me wrong, I think rust has some great features and does some things extraordinarily good. But at the same time I can see that for 99% of cases it's just as prone to bugs, including security and supply chain attacks, as any other language. I think I could just as much argue that any "->" is clearly de-referencing that needs checking, which is also pretty easy to see.
this is theoretically, empirically, and objectively wrong. akin to saying that you can get into an accident in a truck as well as a motorcycle, so in 99% of cases there is no difference in safety.
different kind of programming languages change the kinds of bugs and their properties, but it does not change the amount of bugs per line of code. in respect to C and rust for example, rust may have less memory bugs - which LLMs seems to be able to spot now - but it did increase supply chain attacks for example.
>because crypt in musl started returning "*" in some cases some years ago
In Rust, the type of the return value would be a `Result<T, E>`, which means the chain of events that led to this would never have started in the first place.
In C the return value would be NULL, except it was changed by musl at some point.
Edit: at the beginning, crypt always returned a value. So hypothetical Rust return type would be a String, and then it would be retained for backward compatibility. Or something else would happen, this is a complex logic bug triggered by a 25 year old chain of events.
That's what led some people to write code where it crashes without checks. Which is still their mistake. But if the type was something that can't be implicitly used like an always-present value, that would have prevented the situation, even with those same people.
I write firmware where the program has to run for years unattended.
You are right, C with manual memory management, no lifetime management, lack of phat pointers and a culture of leet has problems with deranged maddening bugs that other languages don't have.
But this bug first shows how inband signaling is bad news usually. But worse it's a failure to adhere to the spec properly. If you implement a library and you don't adhere to the spec you're going to bone someone.
Perhaps even worse is obviously the spec doesn't come with any standard testing framework. That was also the root of Apples infamous goto bug.
Rust would not save you here as this is a logic bug. Nor would, well, really any other language. I suppose someone might claim that Rust's result type would save you but it really wouldn't: I can easily make the error type a String or pretty much any other type with little consequence.
Rust would absolutely have saved you here. Even with `Result<String, String>`, which is pretty poor design, there's no confusing `Ok(s)` with `Err(s)`. Sure you can go out of your way to write something like
let hash = match result {
OK(s) => s,
Err(s) => s,
};
but at that point I feel like the library is forgiven (except for the poor error design).
@my-huge-pony does make a good point elsewhere in the thread: if the potential error was not foreseen and the code was written originally to return `String` instead of `Result<String, Error>` then even Rust would be in a pickle.
it's unbelievable to me that there isn't a larger movement of developers to move away from c towards better & safer languages. look at how much pushback the rust4linux folks get, primarily from c developers who are simply afraid of change. imagine engineers not adopting safer building practices because "they prefer the old way"....it's disgusting honestly.
what will it take to move away from c? older devs retiring. Planck said that "science advances one funeral at a time" and i fear it will be the same for software.
Moving away from C in any POSIX system is in fact stop being POSIX system as POSIX and linux being unix like system is built on standard that is defined as C library specification.
Rewriting it to Rust is fine attempt, just hard to call it is unix/posix anymore. It may not matter but semantically true.
Cars without seatbelts and airbags chopping off driver's heads in relatively slow collisions? Well guess what, that could happen in a modern car as well if the driver sticks their head out the window while going through a very tight tunnel!
Does anyone know why musl does this? Are they the only C library that behaves this way?
For a C library that famously tries to adhere to POSIX as much as possible, this seems like a crazy choice. From POSIX standard:
Upon successful completion, crypt() shall return a pointer to the encoded string. The first two bytes of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/c...
https://hachyderm.io/@ncopa@fosstodon.org/117564809789053485
Though I am surprised that all of the code examples do not use constant-time strcmp(). Actually, I wonder if even constant-time strncmp() would still leak information about whether the account is disabled because in that case both arguments are much shorter? Food for thought.
But yeah, the crypt(3) API is very 80s C, the man page even features a lengthy description of a long-deprecated and utterly broken hashing algorithm using DES that is even implied to be broken in the text itself (and the warning message is so old it refers to "parallel computers" as though they were a new concept). Given that the default mode is to use a broken hashing algorithm, I'm shocked anyone actually uses the libc crypt(3) -- surely it's trivial to just write your own that isn't awful...?
Error handling is underspecified. The spec doesn't give you enough well-defined behaviour, meaning everyone lists pages of error-handling extensions they have added.
Pedantic, but more "author's understanding of the spec" which isn't always correct.
> assume nonstandard behavior
I go back and forth on this one (spec vs standard behavior). It feels like at some point the most widely used impl -is- the spec, as we've seen in other places (web standards).
> Upon successful completion, crypt() shall return a pointer to the hashed password [...]. Otherwise, it shall return a null pointer and set errno to indicate the error.
> The crypt() function shall fail if:
> [ENOSYS]
> The functionality is not supported on this implementation.
crypt() returning `*` is insane especially since these kinds of sentinels are all over /etc/passwd
I suppose if people started writing lots of code that crashed on error in Rust some enterprising developer might improve the function to always return a non-error value and set it to ‘*’ in error cases.
One is to return NULL for errors.
Another is to make the string result a parameter and use a return code to signal error.
A third, less common but still out there, is an error parameter.
Failure to observe these is on the caller. Forcing the caller to consider the error is probably where rust shines.
But a good C programmer is thinking about these idioms when they encounter a function signature for the first time, or they write one.
This is the root of a lot of the evil. Yes, humans want to program for the happy path. It is a lot more fun. However, that bias leads to so much lack of diligence to properly deal with the unhappy path.
You ideally want to attack the problem at two ends:
- make the syntax noise-free so you can see what is happening
- build primitives at the syscall and stdlib level for common footguns
You would still need to deal with domain-specific logic errors, but you have at least reduced the surface area exposed to casual attacks.
Which is why exceptions in Java can bubble up the stack? You don't have to catch it in a particular method you can let it go to the caller of the method, and so on.
Right idea at the wrong time.
Having been on the receiving end of debugging that a few times, I no longer mind the in-band error handling, although admittedly I'm inexperienced enough to be confident its better. Just for the random code I've had to read where I just need to know exactly how something I didn't write and will never read again works, and whether the error is handled and how, i love reading in-band error handling, even if I too dislike the way an error laden routine must be written. Over time I've come to appreciate elegance "in the large" as opposed to the code I see in any individual function.
I have written very little Java, though, so this is more of an armchair analyst position.
The same way that layered code should react/wrap/translate when it comes to return types, it should do the same for thrown exceptions, and checked-exceptions helps you do that. They're both results of computation.
https://news.ycombinator.com/item?id=50377221
"If you cannot update yet, at least disable password logins: set PasswordAuthentication no in your sshd configuration and restart sshd. Either one closes the hole." (only AI would characterize a critical SSH implementation vulnerability as a hole closing game)
An attacker who can reach your SSH port can log in without knowing any password. It takes one connection, works every time, and needs no retries. (It sounds like the writer is bragging about the exploit instead of describing it. AI does this)
The attacker needs the name of an account whose password field is * (an account with no password set, which is normal for key-only accounts), and needs the machine to have at least one other account with a $5$ or $6$ password hash. The attacker then sends a very long password (257 to 1024 bytes). One user with a password is enough: it makes every account without one reachable. (same as above made stronger with the 'statement: statement2' format)
"Users coming from a directory service through musl-nscd (nslcd, winbind, sssd), since they have no shadow entry." (a human would be more clear about what a shadow entry is and give more context, I think)
Hmm the rest of the article (the remaining 70-80%) seems quite human.
Anyway, not sure why I spent 10+ minutes on this. It just jumps at me right away.
As a grad student I read and graded a lot of student philosophy papers (well before LLMs), and I just remember the exhaustion of it. It made me appreciate how much I could otherwise just read something, get information or enjoyment out of it, and not focus on each sentence construction.
Just to say, I do believe you, I get that if you have to put up with it so much in general, it must prime you to find it everywhere. You have to read so much more now, I'm sure, so you're trained to be in that mode. It makes sense! I am just sorry its like this for you and most everyone else now. It sounds exhausting.
Looking at the unpatched Ubuntu things in my life, and I'm pretty close to asking Claude to update things to see how quickly Alpine can get close, assuming responsiveness can stay this fast and quality.
There's a good chance similar flaws will be discovered elsewhere.
One occurrence is not a statistic and should not be used as evidence - it's just an example of a vulnerability that due to various bad circumstances lead to a particularity bad outcome. This logic is in my opinion equally flawed as calling alpine safe now because this particular vulnerability has been fixed.
I'm not a big fan of blaming and I think it's just a result of bad circumstances, but it's interesting to see how some seem to blame musl, but one could argue further that musl documented the behavior of their API which was ignored by openssh and so on
Not every C vuln is a memory safety problem.
Now, the decision to return `"*"` instead of letting poorly written code crash on `NULL` is not good either.
Given how easy it is to just forward that error to the caller, I think most Rust developers would just choose to do that.
That's exactly what you get with Result and Option, it's impossible to forget.
Compare:
To:In Rust, the type of the return value would be a `Result<T, E>`, which means the chain of events that led to this would never have started in the first place.
Edit: at the beginning, crypt always returned a value. So hypothetical Rust return type would be a String, and then it would be retained for backward compatibility. Or something else would happen, this is a complex logic bug triggered by a 25 year old chain of events.
That's what led some people to write code where it crashes without checks. Which is still their mistake. But if the type was something that can't be implicitly used like an always-present value, that would have prevented the situation, even with those same people.
Easier to formulate invariants and check/prove them. Better language correlates with safer software.
You are right, C with manual memory management, no lifetime management, lack of phat pointers and a culture of leet has problems with deranged maddening bugs that other languages don't have.
But this bug first shows how inband signaling is bad news usually. But worse it's a failure to adhere to the spec properly. If you implement a library and you don't adhere to the spec you're going to bone someone.
Perhaps even worse is obviously the spec doesn't come with any standard testing framework. That was also the root of Apples infamous goto bug.
oh right, because c is a crusty old language with barely any features.
@my-huge-pony does make a good point elsewhere in the thread: if the potential error was not foreseen and the code was written originally to return `String` instead of `Result<String, Error>` then even Rust would be in a pickle.
what will it take to move away from c? older devs retiring. Planck said that "science advances one funeral at a time" and i fear it will be the same for software.
Rewriting it to Rust is fine attempt, just hard to call it is unix/posix anymore. It may not matter but semantically true.
I'll take forking and porting, thank goodness for LLMs.
in theory could someone do it? yeah, just like in theory someone could program a calculator to randomly rm -rf ~ when pressing +.
but in practice would anyone do it? no.