shitpost about llms, not how it works but it would be funnytraining an LLM on only zip files and then asking it to just vibe code me a zip file to see what the fuck comes out
shitpost about llms, not how it works but it would be funny@pearl there's no reason this wouldn't mostly work. source : I have a PhD in this awful crap
shitpost about llms, not how it works but it would be funny@[email protected]... so i guess the question is uh. how would i do it? would it be possible to do some bullshit like prompt in english --> zip file? i am legit curious how bad the output would be lmao
shitpost about llms, not how it works but it would be funny@pearl you need to gather all the zip files you can. every zip file on the internet. then you use a GPU cluster to train a giant transformer model. if you're evil you can power it with smog generators. this will likely cost a bajillion dollars. in the end you will have a giant pile of matrices that would be able to output new zip files which are based on zip files in the training set, and they will mostly not be corrupted and unreadable
shitpost about llms, not how it works but it would be funny@pearl the way that LLMs handle prompts is not actually how it seems. It's just text that essentially says "User: (what you typed) Assistant:" Which it then completes in a loop until the loop sees the model output User: again
shitpost about llms, not how it works but it would be funny@gildilinie that part i knew, but wondering how a lot of those text-to-image models work and if a similar theory could be applied to making text-to-zip
@pearl@gildilinie iirc the gist of diffusion models is (very incorrect but still kinda correct if i remember math4datascience classes): you have like a set size canvas of noise, then you somethings that kinda act like like something that overlays that noise, imagine a prompt "bird with tophat" you'd have a bird filter thing that denoises the image into being bird like, and then a tophat denoise somewhere and probably also a "woodland forest background" denoise thing
