• 0 Posts
Joined 1 year ago
Cake day: August 8th, 2023


  • Machine learning and compression have always been closely tied together. It’s trying to learn the “rules” that describe the data rather than memorizing all the data.

    I remember implementing a paper older than me in our “Information Theory” course at university that treated the creation of a decision tree as compression. Their algorithm considered sending the decisions tree and all the exceptions to the decision tree and the tree itself. If a node in the tree increased the overall message size, it would simply be pruned. This way they ensured that you wouldn’t make conclusions while having very little data and would only add the big patterns in the data.

    Fundamentally it is just compression, it’s just a way better method of compression than all the models that we had before.

    EDIT: The paper I’m talking about is “Inferring decision trees using the minimum description length principle” - L. Ross Quinlan & Ronald L. Rivest

  • I might misunderstand what you mean with “implementing” an LLM, but unless you have a good understanding of deep learning and math I wouldn’t recommend to implement one from scratch. There’s a lot of complex math involved in these kind of topics. If you mean implementing an application around an existing LLM, for example writing a chat website that interfaces with ChatGPT or a local LLM, then it’s doable (depending on you current skills).

  • My first experience with the Sims was jumping behind a random computer at some kind of event that was running the Sims 1. Most of the family had just died because the previous person behind the PC had let the house burn down. Needless to say, I was a bit confused. I’ve played the Sims quite a bit after that, and I honestly like messing around with it.

    I don’t think I’ve ever played a game without cheating a lot of money. I don’t like that the Sims that I made have to go off to work or school, so usually I just build a big fence around the property to keep them all there. From there on it used to devolve into chaos when I was younger. Building huge mazes to access basic necessities, launching fireworks indoors, etc. Nowadays im a bit more behaved though.

    Imo the Sims 4 is the best nowadays. The older ones are showing their age. That being said, the Sims 4 is definitely in need of some competition. It’s inexcusably buggy sometimes, and I personally think there’s a lot more that can be done with a game like this. Hopefully the upcoming competitors can spark some fire into this genre.

  • gerryflap@feddit.nltoMildly Infuriating@lemmy.worldEuro bottles are so much better now
    2 months ago

    Many of the new bottle caps I encounter will actively push back into the closed position, meaning I have to keep them out of the way when pouring if I don’t want to pour over the cap. Since I tend to encounter them on drink cartons rather than bottles, because I don’t drink soda etc, it becomes even more annoying. Bottles you can turn whichever way, but drink cartons need to be kept at a certain angle for optimal pouring. Quite often the cap is in the way and there isn’t really a nice place to put it.

    This is even more frustrating because I never lost these caps anyway, I always threw them away with the packaging. I understand that it probably helps in the bigger picture, but for me personally it solves nothing and is incredibly annoying.

    Edit: two examples

    This one is fine, it snaps into a position that’s handy and out of the way:

    This one is very annoying. It’ll stay in this position and requires constant force to keep out of this position. When opening or closing the packaging the attachment point also rotes, meaning it’s always in the wrong place:

  • I’m not a hundred percent sure, but afaik it has to do with how random the output of the GPT model will be. At 0 it will always pick the most probable next continuation of a piece of text according to its own prediction. The higher the temperature, the more chance there is for less probable outputs to get picked. So it’s most likely to pick 42, but as the temperature increases you see the chance of (according to the model) less likely numbers increase.

    This is how temperature works in the softmax function, which is often used in deep learning.

  • Do people really constantly copy-paste code? If I don’t know something I’ll look it up, but then I’ll read the answer and apply it to the code I’m writing rather than copying it directly. I rarely see a piece of code that I can copy over directly into what I’m doing, and even if I can it’s usually not thr best idea because the naming etc would be inconsistent