you curse the past you for being so selfish

“I have discovered a truly marvelous demonstration of this proposition that this margin is too narrow to contain.” https://www.joh.cam.ac.uk/library/special_collections/early_books/fermat.htm#:~:text=When reviewing his copy of,to fit in the margin

The crux of his argument was that scarcity serves as a catalyst for the perception of value. Constrained access to music prompted people to appreciate and savor it more intensely, leading them to invest greater time in its enjoyment, analysis, and sharing. This phenomenon also resulted in people incorporating their musical preferences into their self-identity, as their selection of albums conveyed significant information about their character.

I don’t think the article argues against physical access though.

Exactly, at the end of the day it’s about using the right tool for the job. Code that’s clear and declarative is easier to maintain, so it makes sense to default to it, but nothing stops you from using low level constructs if you really need to.

I really like this approach for doing non trivial regex https://github.com/VerbalExpressions

const tester = VerEx()
    .startOfLine()
    .then('http')
    .maybe('s')
    .then('://')
    .maybe('www.')
    .anythingBut(' ')
    .endOfLine();

Curious to know what do you think a brain is exactly?

Biological neural networks can do things that artificial ones can’t, and use far less energy.

Indeed, this seems like a big step forward, and here’s a link to the model https://github.com/ridgerchu/matmulfreellm

indeed

Yeah also true, you’re generally not gonna start doing anything if you know you’re getting interrupted anyways.

yup

I’d say it’s not so much that this tech doesn’t have value, but that it gets hyped up and used for things it really shouldn’t be used for. Specifically, the way models work currently, they’re not suitable for any scenario where you need an exact answer. So, it’s great for stuff like generative art or creative writing, but absolutely terrible for solving math problems or driving cars. Understanding the limitations of the tech is key for applying it in a sensible way.

not working due to hallucinations

It’s pretty clear that hallucinations are an issue only for specific use cases. This problem certainly doesn’t make ML useless. For example, I find it’s far faster to use a code oriented model to get an idea of how to solve a problem than going to stack overflow. The output of the model doesn’t need to be perfect, it just needs to get me moving in the right direction.

Furthermore, there is nothing to suggest that the problem of hallucinations is fundamental and can’t be addressed going forward. I’ve linked an example of a research team doing precisely that above.

wasteful in terms of resources

Sure, but so are plenty of other things. And as I’ve illustrated above, there are already drastic improvements happening in this area.

creates problematic behaviors in terms of privacy

Not really a unique problem either.

creates more inequality

Don’t see how that’s the case. In fact, I’d argue the opposite to be true, especially if the technology is open and available to everyone.

and other problems and is thus in most cases (say outside of e.g numerical optimization as already done at e.g DoE, so in the “traditional” sense of AI, not the LLM craze) better be entirely ignored.

There is a lot of hype around this tech, and some of it will die down eventually. However, it would be a mistake to throw the baby out with the bath water.

what I mean is that the argument of inevitability itself is dangerous, often abused.

The argument of inevitability stems from the fact that people have already found many commercial uses for this tech, and there is a ton of money being poured into it. This is unlikely to stop regardless of what your personal opinion on the tech is.

Again, I’m not arguing that open source automatically solves problems, just that since AI is obviously going to continue being developed, it’s better if it’s done in the open.

Open source does actually pave the way towards addressing many of the problems. For example, Petals is a torrent style system for running models which allows regular people to share resources to run models.

Problems like hallucinations and energy consumption aren’t inherent either. These problems are actively being worked on, and people are finding ways to make models more efficient all the time. For example, by using the same techniques Google used to solve Go (MTCS and backprop), Llama8B gets 96.7% on math benchmark GSM8K. That’s better than GPT-4, Claude and Gemini, with 200x fewer parameters. https://arxiv.org/pdf/2406.07394

And here’s an approach being explored for making models more reliable https://www.wired.com/story/game-theory-can-make-ai-more-correct-and-efficient/

The reality is that we can’t put the toothpaste back in the tube now. This tech will be developed one way or the other, and it’s much better if it’s developed in the open.

Couple that with the deteriorating economic situation in US and rampant racism. People are finally starting to see burgerland for the shithole it really is.

mass shootings and police brutality come to mind

If by commit suicide you mean grow faster than any G7 economy making Russia 4th largest economy then sure.

Last I checked, China hasn’t been at war since the 70s, and allocates a tiny fraction of the resources that US allocates to military spending. If you don’t understand why China needs to be able to defend itself against a rabid empire that constantly invades countries, then it’s pretty clear who the actual clown here is. This is what China’s military protects:

Household savings hit major highs across China https://www.chinadailyhk.com/hk/article/315229

90% of families in the country own their home, giving China one of the highest home ownership rates in the world. What’s more is that 80% of these homes are owned outright, without mortgages or any other leans. https://www.forbes.com/sites/wadeshepard/2016/03/30/how-people-in-china-afford-their-outrageously-expensive-homes/

Chinese workers enjoy basic form of the democratic corporate governance system via assemblies of employee representatives. https://www.taylorwessing.com/en/insights-and-events/insights/2024/01/employees-participation-in-corporate-governance-under-the-revised-chinese-company-law

The real (inflation-adjusted) incomes of the poorest half of the Chinese population increased by more than four hundred percent from 1978 to 2015, while real incomes of the poorest half of the US population actually declined during the same time period. https://www.nber.org/system/files/working_papers/w23119/w23119.pdf

From 1978 to 2000, the number of people in China living on under $1/day fell by 300 million, reversing a global trend of rising poverty that had lasted half a century (i.e. if China were excluded, the world’s total poverty population would have risen) https://www.semanticscholar.org/paper/China’s-Economic-Growth-and-Poverty-Reduction-Angang-Linlin/c883fc7496aa1b920b05dc2546b880f54b9c77a4

From 2010 to 2019 (the most recent period for which uninterrupted data is available), the income of the poorest 20% in China increased even as a share of total income. https://data.worldbank.org/indicator/SI.DST.FRST.20?end=2019&locations=CN&start=2008

By the end of 2020, extreme poverty, defined as living on under a threshold of around $2 per day, had been eliminated in China. According to the World Bank, the Chinese government had spent $700 billion on poverty alleviation since 2014. https://www.nytimes.com/2020/12/31/world/asia/china-poverty-xi-jinping.html

Real wage (i.e. the wage adjusted for the prices you pay) has gone up 4x in the past 25 years, more than any other country. https://www.youtube.com/watch?v=Cw8SvK0E5dI