It’s just Alpaca with a different base model. Databricks boasted too much.

Ah. I’m trying to train the Dolly model created developed databricks.

You can just follow Stanford Alpaca’s github instructions, as long as you have LLaMA weights. It’s straightforward.

But which cloud service did you use to train them?

I tried using databricks to train a model but the setup was too complicated.

I’m wondering is there a more straightforward platform to train on?

I think it was aleady well known that it would fix its own errors when provided the error message, this is not a breakthrough

No I don’t use databricks. I only tried LLaMA and Alpaca.

With databricks?

I trained the model using cloud

My bad. Did you train the model locally on your PC or using cloud?

From my understanding its limited to no commercial use, so you can use it for what you need, but not commercially.

But what if you're training a model for a narrow use-case and don't intend for anyone to use it except for a niche set of users? Is that enough to be in the clear? Or is any use of OpenAI's model output to train a model for any purpose a no-no?

I write crimson notes on my cave wall.

My annotations make too little sense for anything but me to possibly understand them :)

I save them as pdfs and write notes in the margins.

Hey, I want to fine tune a model to solve a pharma related problem. I want to know if I can fine tune my model with this.. Could you please share your contact details so that I can learn about this more?

Even worse, gpt-4 doesn't know it is gpt-4.

I have chatgpt-plus and the above answer is generated using the gpt-4 model.

wasn't he rather comparing the parameters to the volume of the first computer and not their transistor count?

I’m unable to connect to hamata.so. Can you tell me what kind of analysis you want to do?

for completeness, you should also add all those proprietary models: Megatron-Turing (530B, NVIDIA), Gopher (280B, Google), Chinchilla (70B, DeepMind) and Chatgenie (WriteCream)

Bigger models are more sample efficient for a given amount of data.

Scale is a triangle of three factors; model size, data size, and compute size. If you want to make more efficient use of data, you need to increase the other two.

In practice LLMs are not data limited right now, they're limited by compute and model size. Which is why you see models like LLaMa that throw huge amounts of data at smaller models.

ChatGPT doesn’t know that GPT-4 has multimodal input though, right? I assume based on “not [designed] to analyze images or visual data” this is the case.

I see a future where LLMs or llamas that are multimodels or any other new kind artificial intelligence run on esp32 level of hardware i don't know how that will work but I'm pretty sure we are heading there

Not going to deny that GPT-4 looks impressive, but, they could set up 10 bajillion-quadrillion parameters, question is, do they have the data to effectively utilize all of these? Maybe its time to start looking into decreasing number of parameters, and making more efficient use of the data.

Lets integrate winSpy with it

What is the minimum hardware requirement to fine tune like Stanford Alpaca? I am thinking to build a workstation to do some DL exploration and fine-tuning work. For fine-tuning, I have around 10k samples.

Where does Google’s LAMDA fit on this chart?