• 2 Posts
  • 29 Comments
Joined 1 year ago
cake
Cake day: June 19th, 2023

help-circle

  • It cannot “analyze” it. It’s fundamentally not how LLM’s work. The LLM has a finite set of “tokens”: words and word-pieces like “dog”, “house”, but also like “berry” and “straw” or “rasp”. When it reads the input it splits the words into the recognized tokens. It’s like a lookup table. The input becomes “token15, token20043, token1923, token984, token1234, …” and so on. The LLM “thinks” of these tokens as coordinates in a very high dimensional space. But it cannot go back and examine the actual contents (letters) in each token. It has to get the information about the number or “r” from somewhere else. So it has likely ingested some texts where the number of "r"s in strawberry is discussed. But it can never actually “test” it.

    A completely new architecture or paradigm is needed to make these LLM’s capable of reading letter by letter and keep some kind of count-memory.


  • I have no idea how to fix the problem, but I’ve read somewhere that burn (a relatively new machine learning framework in Rust) is capable of loading models like stable diffusion. As Burn is built with webGPU and all the shader transpiler-stuff that comes with it doesn’t that mean that it can also run easily on (even older) AMD cards? I think what’s lacking is equal performance as nvidia drivers are heavily optimized already.

    Maybe someone knows more here?








  • Oh, i have a brilliant one:

    A few years ago i spent a lot of time converting .flac-files into .ogg-files in order to put on my oldschool iPod. As I did a lot of repetitive typing - entering $dir / for file in flac ; do convert etc / mkdir -p $somewhere/$artist/$album / mv $somewhere/.ogg->$new_dir/ and so on - I thought: “hm lets just write a loop over loops for all the artists here and then all the albums and at the same time create the nested directories somewhere else… hm actually in the home directory… and later love everything on the iPod at once.”

    so i was in my music folder with the artists-folders i wanted to convert. i did something wrong

    So i did my complicated script directly in the shell. I made something wrong and instead of creating a folder “~/artist/album” I created 3 folders in my current working directory: “~”, “artist” and “album”. hmph dammit gotta try again… but first : i have to clean up these useless folders in the current dir. so i type of course this: "$ rm -r ~ artist album " after about 5 seconds of wondering why it took so long i realized my error. o_O I stopped the running command, but it was (of course) too late and i bricked my current installation. All the half-deleted config files made or impossible to start normally and extremely tedious to repair it by hand, so i reinstalled.



  • I worked as a projectionist in 2009 when the cinema got its first digital projector in order to be able to show Avatar in 3D. At the start of the movie no one actually knew if it would work. Due to the movie being encrypted - with every cinema in Germany waiting eagerly for the password - No cinema was able to play the movie. But everywhere cinemas were packed with people. Because of fuckups somewhere in this incredibly stupid system the movie was delayed about half an hour (IIRC) nationwide. With no-one knowing if it would eventually work - especially nice for the people working at the cinema having to deal with angry audience members.

    At the same time the 2D 35mm film-version we also had started without any problems (it was massive and pretty dicey to carry it around).




    1. I think a more consistent approach would he to not count lines, but filesizes after the code has been minimized with a specific minimizer. I could write everything in one line in many languages, so lines isn’t very clever.

    2. The code has to be compiled and run on a specific architecture and with specific test input (we don’t know if the AoC-example-data is always the same in size or resulting computanional complexity.

    3. The final metric could be: [minified filesize] * [code execution time] * [problem solving time].





  • I’m sorry but, I always find it strange when people talk about nuclear energy as the simplest solution.

    Nuclear energy is extremely expensive compared to wind and solar once you also account for the cost of processing the uranium and then dealing with the radioactive waste afterwards.

    Also take France for example. The EDF has (after being privatized) ran on substance without reinvesting in repairs and renovation so much that last year more than half of its 56(54?) reactors stood still because of problems relevant for their save operation. This was before the last record-breaking summer in 2022 when even more of them didn’t have enough cool water to operate. As a consequence the EDF made mountains of dept because they had to buy so much energy from Germany last summer (from all the solar and wind) that Macron (the famously socialist and anti-market-driven-everything-president of France had to re-nationalize EDF last year. If a neoliberal government like France’s nationalizes the EDF (famous for its highest percentage of nuclear energy in the mix) you can really see how great of a solution it really is.

    Also: where does most of the world’s uranium come from? Russia. So not really much of a difference to the gas. France takes a lot of it from Mali as well (which explains their involvement there. So uranium isn’t that great in this regard as well).

    Also: Nuclear reactors create the most important resource for nuclear weapons automatically.

    In north-east Germany there’s the Wendelstein 7X an experimental stelarator-type fusion generator that since its operation blew all the best estimates for experimentation out of the water. But it can never create more energy than it takes because it’s too small. But it took decades to ensure the funding to even build a small one like this. For a fraction of the subsidies tat nuclear power plants, or gas or coal gets ever year we could’ve build many larger ones that would be much closer to be net positive in power production.

    I’m not against nuclear energy per se. But it’s really annoying to hear all these voices from outside that from thousands of miles away know everything about Germany turning off its power plants.

    The main advantage of nuclear in capitalism is that its central. Everybody having solar power and large fields of wind farms distributed evenly across the country make it less controllable by singular entities.

    I might warm up more to nuclear energy it would be run in a more socialist society where there’s no profit-driven operation that drives companies to skip repairs. The corrosion crisis in France is a direct result of “market forces”.

    If something like Chernobyl happened in France… holy shit. That country has the most tourists in the world and exporting their food into the whole wide world. And -yes - I know that the chernobyl-type reactor (Graphite-mediated and so on) isn’t used in France anymore. As someone who lived half of his life worth in 30km to “Fessenheim” - France’s oldest and now shut down Graphite-Based reactor - I can yell you that you examine the possible impact more closely from time to time and think about it more.

    Solar and Wind are better. But they naturally don’t create market monopolies and dilute power over energy. That’s why they’re not pushed that hard. If a resource is spread out evenly you cannot make money from it. There’s no market. Capitalism doesn’t like this.


  • Yeah it was not specifically only about Deutsche Bahn, but also an observation about one of the multiple problems that drives the enshittification.

    One Point that Deutsche Bahn definitely did was to find out which connections are mostly used by people ( tickets for these connections thereby contribute mostly to DBs revenue) and kind of abandon the less profitable connections. That’s accounting in my book.

    What they did (counting passangers by rail-connections) wasn’t possible before, as DB-tickets were sold not electronically and couldn’t easily (cheaply / with little work-hours) be turned into data sets and analyzed.

    IIRC tickets were priced much differently - they weren’t fixed to specific trains but to connections (no “Zugbindung”). So There wasn’t even (easily available) data to when most travellers were using the trains.

    Today with all the data being generates automatically the accountains know much better what costs and what earns DB money and they prioritize based on that. Once you get into the habit of that even things that are obviously always costs (like fixing rails or bridges) will be outsourced or avoided. (like the supermarket example - it’s obvious that someone has to restuff the shelves, but once you have all the data and see only red numbers you try to separate it away and not do it (so it gets turned into a subcontract with probably unrealistic conditions that some other companies are underbidding each other in order to gain the contract - even if this means that their employees will not earn a living wage from it. It’s a perfect system that also pushes responsibility and blame away from the outsourcing company. they can always blame the sub contracting company for underpaying or not follow safety regulations (even if they can only fulfill the sub contract by operating this way)).