Recent comments in /f/MachineLearning

friuns t1_jdz8ef6 wrote

Reply to [D] FOMO on the rapid pace of LLMs by 00001746

I feel you on the FOMO with LLMs. It's like we're all aboard a speeding train, right? Don't stress too much, though! Remember, innovation is a collective journey, and there's always room for exploration, even with limited resources. Keep an eye on new techniques, distillation, and distributed compute - the ML world is full of opportunities to hop in and make a difference! Let's embrace the excitement and keep learning together!

4

Cherubin0 t1_jdz7s1i wrote

Reply to [D] FOMO on the rapid pace of LLMs by 00001746

The do called botter lesson just shows that the research is still at the start. Of course line fitting gets better the more data points you have. We are still just line fitting.

2

spiritus_dei t1_jdz7rmz wrote

Reply to comment by MootVerick in [D] FOMO on the rapid pace of LLMs by 00001746

I think this is the best formulation of the question I've seen, "Can you imagine any job that a really bright human could do that a superintelligent synthetic AI couldn't do better?"

Everyone loves to default to the horse and buggy example and they always ignore the horse. Are programmers and researchers the blacksmiths or are they the horses?

It's at least 50/50 that we're all the horses. That doesn't mean that horses have no value, but we don't see horses doing the work they once did in every major city prior to their displacement by automobiles.

We also hear this familiar tome, "AI will create all of these news jobs that none of us can imagine." Really? That superintelligent AIs won't be able to do? It reminds me of a mixed metaphor. These two ideas are just not compatible.

Either they hit a brick wall with scaling or we all will be dealing with a new paradigm where we remain humans (horses) or accept the reality that to participate in the new world you become a cyborg. I don't know if it's possible, but may be the only path to "keep up" and it's not a guarantee since we'd have to convert biological matter to silicon.

And who wants to give up their humanity to basically become an AI? My guess is the number of people will shock me if that ever becomes a possibility.

I'm fine with retirement and remaining an obsolete human doing work that isn't required for the fun of it. I don't play tennis because I am going to play at Wimbledon or even beat anyone good - I play it because I enjoy it. I think that will be the barometer if there isn't a hard limit on scaling.

This has been foretold decades ago by Hans Moravec and others. I didn't think it was possible in my lifetime until ChatGPT. I'm still processing it.

13

ghostfaceschiller t1_jdz6vzn wrote

Reply to [N] OpenAI may have benchmarked GPT-4’s coding ability on it’s own training data by Balance-

I think this was shown awhile ago (like a week ago, which just feels like ten years)

While I do think this is important for several reasons, personally I don't see it as all that impactful for what I consider AI capable of going forward.

That's bc pretty much all my assumptions for the next couple years are based on the idea of systems that can loop and reflect on their own actions, re-edit code based on error messages, etc. Which they are very good at

76

fiftyfourseventeen t1_jdz6eu7 wrote

Reply to comment by ---AI--- in [D] FOMO on the rapid pace of LLMs by 00001746

The only way you are training your own GPT 3 level model for 600 is by spending 300 bucks on a gun, 300 bucks renting a u haul and heisting a datacenter

Edit: maybe cheap out on the gun and truck, can't forget about electricity costs of your newly acquired H100s

6