"Don't You Just Upload It to ChatGPT?"

That's nonsense. There is zero reason to believe that AI (with the current techniques) will ever become reliable enough to let it do its own thing, let alone better than a human. It's been years of development and you still can't trust it to get basic facts correct, not even "well it's better than it used to be". Saying it'll replace humans in 5-10 years is a fantasy (or a prediction that people are stupid enough to fall for hype, I guess).

We would all do well to remember that and remember that each and every advancement and use case regarding AI is the result of choices by people (or the groups of people we call corporations) and are oftentimes motivated by the profit motive, not the best interest of humanity.

We could make different choices up to and including our own Butlerian Jihad where we ban all forms of AI but we could also do everything we can to prevent the worst fallout short of that.

There are only two types of problems in the universe: 1) those posed by the laws of physics 2) those posed by human choices

The problem of AI is one of the latter.

This plague of misanthropic doom is itself pretty depressing. Why do so many people think LLMs are in any way on a path to compete with human brains? Why do you think so little of yourself? The brain is magnificent and complex in ways that we are unable to decipher anytime soon, and it does way more than an LLM. Way, way more.

The sad part is that we haven't figured out how to distribute our resources fairly to these people even thought their services aren't required as often. Instead we just take their wages and give them to the top 0.1%

As one of such people, I think there is a nuance to it. AI is great when you’re translating something to yourself. But when translating things for others, more caution and human judgement is needed. Espesially when translating instruction manuals, where bad wording could cause someone to injure themself.

I can confidently say that LLMs do a better job than the average traditionally published fictions in my country, at least when the original works are in English. Every single time I watch a subbed movie there will be some lines noticeably wrong.

https://www.reddit.com/r/funny/comments/3e786n/chinese_hair_...

On the other hand, a lot of people become extremely put off by the smallest sign of ai slop. And the llms have a tendency to impart their style to any text they touch.

Kinda conceptually similar to how typos and grammatical mistakes aren't a big deal if you're shooting off a quick text or email, but publishing if you've got typos in your advertising copy, in your resume, on your medicine label, etc. it's a real bad look.

There are a number of games I've seen and played on Steam which use MTL to get the game playable in English and universally the translation is absolute shit. Sentences that don't transition into another, wordplay that becomes nonsense and a completely flat affect to everything.

This reminds me of the adage, that ChatGPT is really great at everything except my own work.

Crucially the full translation was part of ChatGPT’s training set. Recall is a pretty solved problem in machine learning.

How well does it translate a French novel published yesterday? Where neither the original novel nor any translations are in the training set yet? Or might not even exist!

I tried asking ChatGPT to translate a letter I wrote in Slovenian this weekend. It got the general gist but missed a lot of the nuance. Completely missed several of the little touches of tone where the right choice of synonym conveys a whole bunch of information.

This isn’t a great test, because Claude almost certainly has multiple translations of The Three Musketeers in its training data.

The number of lies, lies by omission, deceptive distortions, and fallacious argument tactics they generate is absurd, and increasing rapidly. Translation, when done as a service you are paid for, can't be relied on by propaganda bots.

I'm pretty sure the Ellsworth translation is in the corpus. You basically instructed claude to regurgitate it.

The llms all have the more famous books memorized. You can trick them to recite them more or less word for word.

So i guess in the end it just matters how important the work is.

I think you’re missing a big point of translating literary works. A purely “accurate”, phrase-by-phrase translation is often not very good; the actual literary style, the feeling and the allusions and references, often get lost that way. A good translation of literary work requires a lot of deliberate choices by the translator to deviate from literal translations in ways that convey the style of the original, or an extra layer of meaning that would be lost by an “accurate” translation of a phrase. Also, being consistent with these choices matters a lot, which OP claims LLMs are less good at.

Translation is hard. If you're familiar with reading translations from specific languages MTL works have a very specific smell to them, it's a bit hard to describe but it's there. A good translation is miles (kilometers, for those outside of the US) above MTL.

That's not to say that perhaps the latest LLMs will have better translation abilities, but that they are generally crap currently. Maybe they are fine for something very short, but absolutely not for longer content.

Of course as for the poor OP... is this a majority of what working translators are paid to do?

I suspect a lot of translation is just grunt work - technical and business documents. The lack of a cohesive voice with considered style is perhaps not really much of an issue in those. The expectations are just much lower; text that conveys the basic meaning is a much lower bar to clear.

She's probably better than a bot at that stuff, at least for now, but my concern is that it won't be "enough" better for businesses to justify her continued employment. And this is my general feeling about this stuff across society, in basically all domains.

The `cp` program on my computer also has the remarkable ability to produce a faithful translation of The Three Musketeers when provided one as input.

It's functional? I wouldn't say it's poetic, I wouldn't want any AI translator translating art, like say a book or poem, I'd be so uncertain that it would correctly bridge the concepts

A good translator can make stylistic choices that elevate the work and make it fit in their language

(Having read lots of well translated manga and anime, also from what I understand there's a few books I've been told by my bilingual friend's are just chef's kiss quality translations)

Considering translating meaningful art is of some value, on that score I don't think we're there yet

> I do not speak any language outside of English

So you are entirely unqualified to judge this, and you acknowledge yourself that your test is flawed to the point of being completely useless. Yet you make grandiose statements about the quality.

I wrote up my experiences of translating Lorca and Cavafy poems here[1]. tl;dr: I have developed a massive new respect for translators; however much they're being paid, they probably need to be paid more!

[1] - https://rikverse2020.rikweb.org.uk/blog/adventures-in-poetry...

Update: in case it’s not obvious, I am sorry. I could not help it.

I have also taken to being sloppier in my prose, as I’ve had stories rejected for being “written by AI” - when they’re shorts I wrote more than a decade ago. Reworked them to sound like a moron, accepted. Sigh.

> In my Ottawa life, every Tuesday evening, I take two gym classes back to back—boxing and the pompously named “body sculpt,” which makes me discover muscles I didn’t know I had.

The em-dash matches how you'd speak out loud.

You'd say "I take two classes every Tuesday back to back, boxing and 'body sculpt'. Weird name." (Parts of that sentence did flow oddly, but not because of the em-dash).

Grammarians say you can't make those separate sentences without adding some extra words, and because of blah-de-blah-blah-blah, someone might say you can't join them with a comma. So we have an em-dash.

Rewriting the sentence would make it flow less naturally, not more.

Either way, I'm not reading it, it's a clanker or a clanker collaborationist.

I mean, how would you even write an em dash? There's no button in the keyboard for em dashes, it's not in ascii, it's just not something we write in internet text with, it's a safety watermark put into LLMs by OpenAI to help making LLM generated content identifiable as such.

If for some reason you are an em dash lover that was hurt by the LLM debacle, I'm so sorry for your loss, but look who's on your side, give the em dash a funeral and let it go.

Also, I like em dashes.

And if this is my worst sin, so be it.

I still love the tool, but remain as convinced as ever that AGI does not lie at the end of this particular path.

In my experience this is a real problem. Just yesterday I asked my LLM to create a piece of software that could help me build an 'ambilight-like experience' through my home assistant. It did something that seems to work as I expected, but there is a lot of theory that I just brushed past. It would be pretty easy for me to assume that I would be able to replicate this feature from scratch 'now that I understand the problem'.

The other person in the gym was right, did you you just dump it in the latest model?

Maybe AGI is possible and we'll have software defined human intelligence that's completely autonomous but that's not coming in the next slightly better RL trained LLM and if existed likely wouldn't be under our control anyway

Every month a new guy discovers LLMs; discovers a skill the current LLMs require to get good results; and writes about the future jobs that will always be available for smart people like HIM, that are SKILLED in using LLMs.

The next generation of AIs doesn't need his fancy prompt. The image model goes from needing to type in just the right set of weird words and cryptic sorcerous invocations, to most people being able to type in English what they want and get a pretty good result.

There are still tasks that require careful invocation. But they are a much smaller fraction of all the tasks people are trying to do, or you can get a bleh result without the elaborate invocation to get it really good. And to improve on the bleh result you need to be substantially more of an expert than back when the Guy was memorizing a rule about adding "trending on Artstation" to the image prompts, as would always require a human paid to do that.

Another generation of AIs comes out. The next generation of Clever Skills is obsolete. Image models just obey the instructions for compositing panels without mixing them up, and you don't need to be an expert to get them to do it right. Another human value-add is gone. A wider set of tasks require no human expert.

Now a new Guy notices LLMs have become useful in his field for the first time. He discovers they require SKILL to use CORRECTLY. He posts about how there will always be jobs for humans who are SKILLED in using LLMs like HIM.

But it is not an infinite cycle. It is not the same each time it repeats. Now the Guy is a highly paid programmer or a career mathematician in 2026, instead of a graphic artist in 2023.

In six months the models will no longer require his vaunted Skills.

And by then there will be another Guy.

But the process doesn't continue forever. The Guys are coming from fields that were harder and harder for AIs. The brief centaur eras are shorter and shorter.

Today it is writers who are laughing at how bad the LLMs are at their job, and who will perhaps soon be posting about how it takes Skill to get an LLM to do their job Correctly. But the models are coming faster, and the eras of kinds of human value-add in each field are shortening.

There is a point when you run out of Guys, either because the centaur eras are too short for people to develop SKILLs and post to Twitter about them; or because there are not lands left for AIs to conquer; or because ordinary people are not reassured by some Nobel laureate proclaiming there will always be jobs for Nobel laureates with the SKILLS to prompt robotized biology labs Correctly.

But we'll never run out of amateur economists who assert entirely without a brief contemporary example that there will always be jobs for humans skilled at operating AIs!

We'll run out of professional economists saying it when nobody is paid for that work anymore.

I guess we'll also run out of amateur economists when they're dead.

Source: https://x.com/allTheYud/status/2057136382817231151

I'm not sure how to formulate it yet but it seems there is some Peter Principle/Gell-Mann Effect corollary that is AI-related we can say here.

Perhaps: "AI rises to the level of its users' incompetence."

Or: "Confidence in AI output is inversely proportional to one's ability to verify it"

AI produces output that is very convincing to a non-expert, and (dangerously), it's so good at looking like an expert, they might believe that it is an expert. But the moment you ask someone to use it for something they're an expert in themselves, the holes appear wide, consistent & obvious.

My favourite moment of seeing this in action was watching AI-worrier TV host/comedian Bill Maher. He has spent years talking about the dangers of AI taking everyone's jobs, destroying civilisation, ruining the economy, starting wars, "it's just getting better and better all the time", and so on. But one night he let slip a tell. "It's no good at writing jokes. Not yet, anyway". There you go, Bill... connect those dots...

There is real utility in it being a tool to help experts apply their expertise, as in this story where it speeds up some tasks to help the translator do part of the work, enhance their expertise, allow them to be more productive.

It's a better screwdriver, a better hammer, in the hands of somebody who knows what needs a screwdriver or a hammer. It doesn't replace them. It can't replace them. It's a tool that enhances the human, not an alternative.

I don't understand why this is not widely understood yet, but I'm sure it will in due course.

And I don't expect this to change. Even if the latest model scores 100% on every benchmark, all that really tells us is that it's now more productive/efficient than it was before at helping experts do that work, not that it can replace everyone in that category of work.

Each time the frontier models get better, I see another wave of AI doubters suddenly become believers. People say things like, "AI couldn't code last year, but now I use it for everything!" Interesting. Now we know how that the person who said this has the coding skills of a Claude Opus 4.5 or whenever the frontier was when they flipped.

Meanwhile, the rest of us keep using AI as simple tools, like the person in the article. I wonder how long it will take before computers can program better than me, and I flip too.

There is an interesting third group emerging: People who acknowledge the quality problem, but think they can deal with it by applying more AI to the output.

This takes the form of people who spin up a lot of "agents" and give them personalities like security director or quality director (which are unnecessarily complex and maddeningly unpredictable ways to trigger an LLM session for doing a security review or a quality check pass).

It also includes the person who knows that their app is full of bugs, but thinks it's not a problem because they can have the AI fix the bugs as they show up. People in this class haven't encountered security breaches or data loss bugs yet. They think it's all about having Claude fix that div that isn't centered or handle that error code that shows up some times.

Most? Perhaps it's depression, but I look back at my career and wonder if any code I've ever been paid to write is beyond what current AI can do.

Sure, this leaves me with the non-coding tasks of UX taste, and code review + a few other forms of QA (and, when self-employed, project management, game design, etc.), but man, I'm someone who actually learned to read in part on the Commodore 64 user manual (as in, trying to understand what PEAK and POKE meant concurrent with having "Jack and Jill go up the hill" picture books).

(And no, I'm not claiming LLMs make bug-free code, I see the bugs LLMs make during my code review of their output and some of them are awful, hence "this leaves me with …").

I've heard numerous cases where AI solved medical issues that doctor couldn't.

It seems inevitable that we ask for more AI assistance on topics we don't understand. And therefore have the least context to correct. Result: a flood of poor quality information.

In areas we DO understand, we'll either not ask AI at all, or treat its results with a higher degree of skepticism. Result: a lack of high quality information.

Inevitably this means a higher volume of non-expert prompts gets translated into the next generation of internet content. AIs are pumping out more novice-level text and less expert guidance.

The result will be an internet full content written from the perspective of an ignoramus; not addressing any complex issues, staying surface level on every topic. Which will cascade into future models, etc.

Likewise, AI is oblivious to it's own mistakes, much like said professionals can be at times.

Not that AI is actually thinking, but rather the collective corpus of text yields greater insights (knowledge of the crowd, not wisdom of the crowd) than a lower-average person in that same industry.

I swear that the intensity and time I've had to take with code reviews has gone up because LLMs are so good at making flawed code look good. I assume the same goes for everything else we use LLMs for.

Is it really true for most people that they are using their core advanced skills 90% of the time? I'm curious about how people feel about this.

I'm a professor, which is supposed to be an intellectually demanding job. I do research in NLP/AI, and I don't think AI will replace my core intellectual tasks in the near future, but I don't think my core intellectual tasks represent even 10% of my time. Most of the time is taken by various things like writing bureaucratic reports, writing and polishing grant applications, grading exams and exercises, designing a poster, planning a course's calendar for a given year, creating a figure for slides, writing assignments and exams, attending teaching coordination meetings... which definitely are or should be automatable. Probably even teaching the same lesson for the umpteenth time also is from an objective point of view, we'll probably be kept doing it due to the human factors driving motivation but not because a lecture given by a human is intellectually superior.

This is more "humans are special" hubris imo. Not saying it's gonna happen tomorrow but look at the advancements from just 2019 to now.

It's unwise to say it'll never happen.

Yourself included??

Re. not training on obscure languages, the current thing seems to be to chuck all digital information available into the training so although they probably don't hire a human, the specialist dictionaries are probably in there.

Reminds me of the first time I saw a coding agent stumble through an issue in 2023 maybe? and went "this is a big deal", similarly when OG gpt started making jokes that actually kinda worked.

Updated modern version of the classic "make me a greentext", apologies for slop-posting, but it seems relevant:

    > be me
    > senior software engineer
    > in charge of making sure the tickets get, in fact, implemented
    > occasionally have to open the IDE and write some code myself
    > one day i open the IDE and the ticket is already closed
    > the agent did it overnight
    > no steering, no review notes, nothing left for me to do
    > distress.jpg
    > ask my manager what to do
    > he says "just focus on the high-level architecture stuff"
    > i say "what high-level architecture stuff"
    > he says "i don't know, you're the senior engineer"
    > rage.jpg
    > quit my job
    > become a prompt engineer, nice and simple, just tell it what to build
    > first day on the job, sit down to write the prompt
    > AI already wrote it

It's worth noting that you can substitute "dollars" for "context" in that sentence, which seems to be where many of these impressive achievements are coming from. As ever, it's unclear whether these models will get cheaper while remaining better, since all of the recent breakthroughs appear to be of the "think more" kind. For translation specifically, I'd be very surprised if the "think more" LLMs would help given the per-unit cost expected of the output.

Mathematics is famously rigorously defined, it's roughly analog to AI beating humans at chess. Sure it's impressive, but it's also something you'd expect machines to be good at.

I'm curious, do you have a graduate degree in mathematics?

I agree but it's useful to remember that 1. brains and especially the human brain are enormous and 2. individual tokens carry significantly more meaning than individual tiny muscle twitches so even extremely primitive "cognition" can look like it's doing more work than it actually is.

Even small, dumb, local models are excellent at translation already. Frontier models are on par or better than the human translations we've tested them against at work.

I'm a real person.

And I'm shit at recognizing people I don't interact with. PIcture 50 of us in black legging in front of a mirror...!

The unreliability is something that seems like it might be a temporary early stage.

Also, it is positively charming that you think most of us took the time to properly read something before coming here to espouse an opinion. :-)

I do admit testing AI. Hell, most of the time, I don't have the choice anymore—I don't use it but several of my clients send AI-translated documents. Do I just send back a CHatGPT version? Hell no. This is why and how I know it's not reliable or good.

It's not exactly taboo to use AI, is it? IT doesn't have to be all or nothing. AI is great for my glossaries. AI is shit to translate.