Yud tried to describe a compiler, but ended up with a tulpa. I wonder why that keeps happening~
Yud would be horrified to learn about INTERCAL (WP, Esolangs), which has required syntax for politely asking the compiler to accept input. The compiler is expressly permitted to refuse inputs for being impolite or excessively polite.
I will not blame anybody for giving up on reading this wall of text. I had to try maybe four or five times, fighting the cringe. Most unrealistic part is having the TA know any better than the student. Yud is completely lacking in the light-hearted brevity that makes this sort of Broccoli Man & Panda Woman rant bearable.
I can somewhat sympathize, in the sense that there are currently multiple frameworks where Python code is intermixed with magic comments which are replaced with more code by ChatGPT during a compilation step. However, this is clearly a party trick which lacks the sheer reproducibility and predictability required for programming.
Y’know, I’ll take his implicit wager. I bet that, in 2027, the typical CS student will still be taught with languages whose reference implementations use either:
the classic 1970s-style workflow of parsing, tree transformation, and instruction selection; or
the classic 1980s-style workflow of parsing, bytecode generation, and JIT.
I can somewhat sympathize, in the sense that there are currently multiple frameworks where Python code is intermixed with magic comments which are replaced with more code by ChatGPT during a compilation step. However, this is clearly a party trick which lacks the sheer reproducibility and predictability required for programming.
He probably just saw a github copilot demo on tiktok and took it personally.
@corbin You missed the best bit: one of the current INTERCAL compilers, CLC-INTERCAL (for a superset of the language which adds a bunch more insanity) is implemented IN INTERCAL! It’s self-compiling. Also object-oriented, has quantum-indeterminate operators, and a computed COME FROM statement (also with quantum indeterminacy).
I think we should organize a fundraiser to pay CLC-INTERCAL’s developer @Uilebheist to visit Yud and melt his brain.
@corbin Probably still 5 years too soon but I would hope the 2027 CS student will be taught the usual engineering flow of specification, formal verification and safety analysis, design, some coding and what should be tiny bit of debug during validation at the end.
Reproducability is everything. If your binary isn’t an exact match for the previous tested copy you are doing QA not production.
I am not a rationalist, but I feel like the most rational thing I could possibly do here would be to take your bet.
If, in 2027, CS students are fighting LLM-powered hellspawn compilers that must be tricked, sweet-talked and threatened, I win the bet.
If, in 2027, CS students are working with ordinary compilers with predictable, reproducible results, I win peace of mind.
@datarama@corbin The Go compiler requires reproducible builds based on a small set of well-defined inputs, if the LLM cannot give the same answer for the same question each time it is asked, then it is not compatible with use in the Go compiler. This includes optimizations – the bits should be identical. #golang
Indeed, this is also the case for anything packaged with #Nix; we have over 99% reproducibility and are not planning on giving that up. Also, Nix forbids network access during compilation; there will be no clandestine queries to OpenAI.
Let me know @corbin@defcon.social if you actually get LLMs to produce useful code locally. I’ve done maybe four or five experiments and they’ve all been grand disappointments. This is probably because I’m not asking questions easily answered by Stack Overflow or existing GitHub projects; LLMs can really only model the trite, not the novel.
it very much fucks with me that there’s a nixpkgs working group dedicated to making NixOS an attractive platform for running LLM code. I’m guessing it comes down to two things: nix actually is very good at packaging the hundreds of specifically pinned dependencies your average garbage LLM project needs to run, and operating in the functional programming space makes these asshole grifters feel smarter (see also all the folks who contributed nothing but crypto miners to nixpkgs during that bubble)
@corbin I’m curious how they deal with the Go builder (not compiler specifically) and all its signature verification hoo-hah. There’s ways around that (and those are consistent with “trust nobody”) but it’s not usual case, and not tested nearly as hard as the default path. You can use your own builder, too, that’s also an option (and now I wonder, could we get the Go builder to export a “build plan” for other-tool consumption?)
@Evinceo PCJ? And (lack of) reproducibility really would be a problem for Go, the LLM would need to expose all its random seeds and not have any run-to-run varying algorithms within it. This is not a joke or snark, the bits have to match on recompilation.
I was wasn’t expecting a serious treatment of this very silly idea, my mistake. I submit that it would cause enough difficult to diagnose bugs while just messing with it that you would never get into ‘but are the builds reproducible’ territory.
@Evinceo there’s code generation, and there’s optimization decisions. Optimization problems often have the property that their correctness is easily checked, but choosing the best one is hard. Register allocation is the easy-to-understand example – if modeled as graph coloring, an incorrectly colored graph is trivial to detect.
@corbin it’s a fucking _compiler_. What working or teaching programmer would accept “AI wrangling” in exchange for marginal improvements in the efficiency of the code that’s output? Just chuck some more compute at it…
Heh “2030 : Computer Science departments across the globe are moved from the Sciences to Politics as under-grads no longer program computers they negotiate with them”
He said lifting ideas from a couple of SciFi novels wholesale.
Yud tried to describe a compiler, but ended up with a tulpa. I wonder why that keeps happening~
Yud would be horrified to learn about INTERCAL (WP, Esolangs), which has required syntax for politely asking the compiler to accept input. The compiler is expressly permitted to refuse inputs for being impolite or excessively polite.
I will not blame anybody for giving up on reading this wall of text. I had to try maybe four or five times, fighting the cringe. Most unrealistic part is having the TA know any better than the student. Yud is completely lacking in the light-hearted brevity that makes this sort of Broccoli Man & Panda Woman rant bearable.
I can somewhat sympathize, in the sense that there are currently multiple frameworks where Python code is intermixed with magic comments which are replaced with more code by ChatGPT during a compilation step. However, this is clearly a party trick which lacks the sheer reproducibility and predictability required for programming.
Y’know, I’ll take his implicit wager. I bet that, in 2027, the typical CS student will still be taught with languages whose reference implementations use either:
He probably just saw a github copilot demo on tiktok and took it personally.
deleted by creator
@corbin You missed the best bit: one of the current INTERCAL compilers, CLC-INTERCAL (for a superset of the language which adds a bunch more insanity) is implemented IN INTERCAL! It’s self-compiling. Also object-oriented, has quantum-indeterminate operators, and a computed COME FROM statement (also with quantum indeterminacy).
I think we should organize a fundraiser to pay CLC-INTERCAL’s developer @Uilebheist to visit Yud and melt his brain.
Ow great you mentioning this has already had one sneerclubber have her brain leak out of her ears.
Have you learned nothing? YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT DANGEROUS IDEAS … TREADKILL ;)
@cstross @corbin @Uilebheist
> quantum-indeterminate operators
Wait… What?
@MeiLin @cstross @corbin @Uilebheist What’s so hard to understand? https://esolangs.org/wiki/Quantum_INTERCAL 🤣
@7leaguebootdisk @cstross @corbin @Uilebheist
*has her brain leak out of her ears*
Uhm…
@corbin
Wait till you get to the calculated COME FROM …
@bsdphk @corbin don’t forget the hallowed PLEASE ABSTAIN FROM REINSTATE
@corbin Probably still 5 years too soon but I would hope the 2027 CS student will be taught the usual engineering flow of specification, formal verification and safety analysis, design, some coding and what should be tiny bit of debug during validation at the end.
Reproducability is everything. If your binary isn’t an exact match for the previous tested copy you are doing QA not production.
I am not a rationalist, but I feel like the most rational thing I could possibly do here would be to take your bet.
If, in 2027, CS students are fighting LLM-powered hellspawn compilers that must be tricked, sweet-talked and threatened, I win the bet.
If, in 2027, CS students are working with ordinary compilers with predictable, reproducible results, I win peace of mind.
Either way, I win.
@datarama @corbin The Go compiler requires reproducible builds based on a small set of well-defined inputs, if the LLM cannot give the same answer for the same question each time it is asked, then it is not compatible with use in the Go compiler. This includes optimizations – the bits should be identical. #golang
Indeed, this is also the case for anything packaged with #Nix; we have over 99% reproducibility and are not planning on giving that up. Also, Nix forbids network access during compilation; there will be no clandestine queries to OpenAI.
@corbin I got a 96GB laptop just so I could run (some) LLMs w/o network access, I’m sure that will be standard by 2025.🤪
Let me know @corbin@defcon.social if you actually get LLMs to produce useful code locally. I’ve done maybe four or five experiments and they’ve all been grand disappointments. This is probably because I’m not asking questions easily answered by Stack Overflow or existing GitHub projects; LLMs can really only model the trite, not the novel.
it very much fucks with me that there’s a nixpkgs working group dedicated to making NixOS an attractive platform for running LLM code. I’m guessing it comes down to two things: nix actually is very good at packaging the hundreds of specifically pinned dependencies your average garbage LLM project needs to run, and operating in the functional programming space makes these asshole grifters feel smarter (see also all the folks who contributed nothing but crypto miners to nixpkgs during that bubble)
@corbin I’m curious how they deal with the Go builder (not compiler specifically) and all its signature verification hoo-hah. There’s ways around that (and those are consistent with “trust nobody”) but it’s not usual case, and not tested nearly as hard as the default path. You can use your own builder, too, that’s also an option (and now I wonder, could we get the Go builder to export a “build plan” for other-tool consumption?)
This reads like a PCJ comment, bravo. I’ll do one for rust:
If an LLM cannot insult the user for having the tremerity to try and compile code, it’s not compatible for use with the Rust compiler.
@Evinceo PCJ? And (lack of) reproducibility really would be a problem for Go, the LLM would need to expose all its random seeds and not have any run-to-run varying algorithms within it. This is not a joke or snark, the bits have to match on recompilation.
PCJ -> Programming Circlejerk.
I was wasn’t expecting a serious treatment of this very silly idea, my mistake. I submit that it would cause enough difficult to diagnose bugs while just messing with it that you would never get into ‘but are the builds reproducible’ territory.
@Evinceo there’s code generation, and there’s optimization decisions. Optimization problems often have the property that their correctness is easily checked, but choosing the best one is hard. Register allocation is the easy-to-understand example – if modeled as graph coloring, an incorrectly colored graph is trivial to detect.
So, sadly, not silly enough.
@corbin it’s a fucking _compiler_. What working or teaching programmer would accept “AI wrangling” in exchange for marginal improvements in the efficiency of the code that’s output? Just chuck some more compute at it…
@corbin
Heh “2030 : Computer Science departments across the globe are moved from the Sciences to Politics as under-grads no longer program computers they negotiate with them”
He said lifting ideas from a couple of SciFi novels wholesale.
@corbin anyone looking for a two hour podcast about INTERCAL (and who ISN’T?) can find one here https://futureofcoding.org/episodes/064