If the issue of resources required nfor training is purely corporate, then why did deepseek use the resources it did for training?

So first of all, there is a difference between using X total resources over three years versus using X total resources over three decades. A more extreme example is the difference between burning a full tank of gas over a few days of driving versus burning all that gas at once by lighting it on fire.

China’s contributions to the AI race are, in my view, tarnished by many of the same problems as Western AI systems, since China is (state) capitalist. (That being said, Chinese companies do usually make their models weight-available, which is commendable, but not enough.)

While I suspect that we have not thrown enough theoretical math at the problem of LLM training (we have thrown lots of empirical math at them; useful for guiding the development of theory, but not theory itself), I also suspect (frankly based on my limited experience training and using these things) that the useful structural properties of LLMs only manifest themselves when you train with large datasets.

So the problem of large datasets is, in fact, an inherent part of the technology. My view is that we have to tackle these things head on.

1. Distributed computing. Internally, LLM companies are computing at such a scale that they already must use distributed computing technologies to spread the load out amongst many machines. My vision would be something like BOINC, where people and collectives who benefit from AI systems can give limited access to the computing power of their systems. Note that AI Horde does something like this for inference. Hence why it is so important to seize the means of production from the capitalist class. Not ask, take! They have the tools and power to do this sustainably, but they chose to do it wrong so they could make a quick bag.

2. Leverage existing resources sustainably. The logic here is that individuals and collectives spreading out the effort won’t necessarily save resources or energy integrated over the whole planet. In fact, as a planet, we might spend more energy on a geographically distributed system if it is done at the scale AI companies are doing it. However, the issue is not just "we’re spending too much energy "; it’s “we’re spending too much energy, too fast, and we are overloading and obliterating local ecosystems, and we are doing it in a way that exacerbates existing climate”. I.e., it’s not just quantity, but also space and computing time whose distribution is mismatched.

Like what the capitalists have pushed on us in basically three years…it really should have taken at least a decade.

So many would rather hope an AI translation app works then to ask a friend to help them translate.

Depends on what I need help translating. Basic Spanish? I’ll ask my mom. Farsi? I don’t know anyone personally who speaks Farsi, to the best of my knowledge. Advanced Spanish? Ain’t no way my mom’s gonna beat a machine, even a pre-2023 AI translator. I could go hire a translator, but I suspect that they have more important things to translate. I.e., translation is usually a routine task, and human translators can show up for the fun stuff (translations that require human supervision).

Or to ask a neighbor to help them with planning a garden.

I wouldn’t ask AI personally. I would probably look on LibGen for botany books, or drop a post on Lemmy since I’m sure there are a few botanists lurking around.

I think using genAI as a convenience limits us from building solidarity with eachother.

It can, but it doesn’t have to. And I think that a lot of those examples are people misusing the tech. In particular, people are using the tech to “solve” problems with simple, well-understood solutions. LLM systems are suited for problems without algorithmic solutions, where either a partial and/or approximate solution is better than none at all.

I do think the current iteration of genAI not being able to credit creators means that information is siloed and harder to extract. Maybe that changes in the future?

I would argue that at least for math, you can actually get really far by telling the AI to cite its sources in the system prompt and giving the harness access to your PDF library (I do this in a virtual machine with a subset of my actual library that I have backed up in case I lose it). (And of course you triple check the citation it gives you.)

Also, the thing about math in particular is that we do not particularly disagree with the capitalists about mathematics. We all agree that 2+2=4 and the indefinite integral of cos(t) is sin(t) + C. And we also widely agree on how to typeset mathematics, namely “rewrite unreadable spaghetti code in LaTeX until the rendered PDF looks gorgeous.” So a LLM with a capitalist bias… isn’t really a problem for math typesetting questions. Conversely, I would never ask a capitalist LLM to, say, summarize the contents of Mutual Aid by Kropotkin, even if I give the system the PDF. (Most capitalist LLMs probably do recall the broad contents of Mutual Aid and the most important texts in anarchist theory, simply because they sucked up all our literature too.) I think there are other walks of life where anarchists and other class-conscious proletarians do not disagree with the capitalists, although personally my disagreements with capitalists are extremely broad, almost universal, so I understand if people have trouble finding common ground with them!

Your understanding of AI systems and their capabilities is probably out of date.

Yeah, I can cop to that. Im not an expert and I don’t claim to be.

That wasn’t just aimed at you … that holds for everyone, myself included. Again, the capitalists are overselling the convergence to AGI specifically…but things are moving incredibly fast. So if you’re not using the systems regularly, it’s honestly going to be very difficult to keep up with the development of the tech. (I’m not saying that you should necessarily start using AI systems regularly just to keep up; it will suffice to just remember that your knowledge might be out of date by the time you finish your argument.)