so after the "ollama create ..." command i ended up with the error "Error: json: cannot unmarshal array into Go value of type string" I updated ollama (it was as per this date some 4 weeks old) and after the update i did have error "Error: vocabulary is larger than expected '128257' instead of '128256'" Then in the config.json i changed the value -> "vocab_size": 128256 to 128257 and ran again, which then resulted in success. I cant piece this together, but lets test the results, which are not succesvol. The model errors out sadly. But, doing same excise with a different NousResearch model, it all worked same as in video demonstrated. Tnx Mark.
Oh that's weird, I wonder what happened since I made it. I know this technique only works with a certain set of model architectures, but I thought it'd 'just work' with any Llama fine tunes. Could you paste the link to the NousResearch model that you got working?
@@learndatawithmark Actually, I did not solved it properly, The transformation of the model was okay my editing the configuration file but when I went for running the model it gives the error I posted above.
12 дней назад
Is Modelfile should be the same for all models in huggingface? Some of them doesn't show template or prompt format. How we can know what to write in Modelfile?
Tried multiple times to convert the model to GGUF for llama.cpp, but the provided python code kept erroring. I will give this a try although llama.cpp is more efficient. Thanks!
I can't get find help about 'Error: llama runner process has terminated: error loading model: check tensor dims: tensor 'token_`embd.weight' has wrong shape; expected 4090, 128257, got 4096, 128256, 1,' can you assist me?
Hello, Thanks for the great videos. It's been about several ours I am browsing in your channel. Just a question is it possible to use Ollama and doing fine-tuning with that?
how to tinstall modeldownloader? i try git clone, and then try hfdownloader in cmd, its still error its not recognized as an internal or external command. thx
I used the instructions from the Quick Download section - github.com/bodaay/HuggingFaceModelDownloader?tab=readme-ov-file#quick-download-and-run-examples-linuxmacwindows-wsl2
Hi Mark, thank you for your great video! I tried it to download black-forest-labs/FLUX.1-dev model and I succeeded using Hugging Face CLI download. But I never manage it to convert the model to Ollama with a Modelfile file created as you explained and pointing to the directory where the model has been downloaded and stored locally on my Mac, the Ollama create command always says "Error: no safetensors or torch files found" ??? Any idea would be welcome, thanks!
I think the Hugging Face quantizer only works for a subset of models listed on this page - github.com/ollama/ollama/blob/main/docs/import.md But if you want to run Flux locally, perhaps you could have a look at this GGUF version - huggingface.co/city96/FLUX.1-dev-gguf Or if Ollama doesn't work, llama.cpp might be able to run it.
hello, im stuck at the quantize part can you help? I'm using terminal on macos with ollama. pls send me the terminal commands to quantize safetensors llm with the create -q command on ollama(Q5_K_M). thank you
All the code that I used is here - github.com/mneedham/LearnDataWithMark/tree/main/ollama-own-model Let me know which bit didn't work and what error you get?
Thank you so much. I am having problem running models downloaded from hugging face having safetensor file. I have these files in oobabooga/text-generation-webui. I have to use this for ollama. I followed everything, even created modelfile with path to safetensor directory, but it is not running >> ollama create model_name -f modelfile. Please help me.
@@learndatawithmark Thank you so much for quick response. Your videos have helped me a lot. I am running this on windows 11. I did follow steps: 1) created Modelfile with this script: "FROM C:\Users\PARTH\Downloads\text-generation-webui-main \text-generation-webui-main\models\TheBloke_dolphin-2.7- mixtral-8x7b-AWQQ TEMPLATE = """{{ if .System }}system {{ .System }}{{ end }} user {{ .Prompt }} assistant """ PARAMETER stop PARAMETER stop PARAMETER stop " 2) I ran following command on terminal opend from where this modelfile is stored. "ollama create mixtral:dolphine -f .\Modelfile" 3)It showed me this error: "Error: command must be one of "from", "license", "template", "system", "adapter", "parameter", or "message"" 4) I only made file with FROM statement without parameter and template, It ran, but gave this error: "C:\Users\PARTH\.ollama>ollama create mixtral:dolphine -f .\Modelfile transferring model data unpacking model metadata processing tensors Error: couldn't find a layer name for 'model.layers.0.block_sparse_moe.experts.0.w1.qweight'" 5) I ran again with another models, but gave same error: "C:\Users\PARTH\.ollama>ollama create slimorca:13b -f .\Modelfile transferring model data unpacking model metadata processing tensors Error: couldn't find a layer name for 'model.layers.0.mlp.down_proj.qweight'"
Thanks for the video, I wanted to do this since long time ago. just need a bit of help with this error: Error: json: cannot unmarshal array into Go value of type string (see details below) $ time ollama create \ > -f Modelfile NousResearch_Hermes-3-Llama-3.1-8B:q4_0 \ > --quantize q4_0 transferring model data converting model Error: json: cannot unmarshal array into Go value of type string real 34m12.658s user 1m52.414s sys 2m1.768s Modelfile : FROM ./downloads/NousResearch_Hermes-3-Llama-3.1-8B TEMPLATE """{{ if .System }}system {{ .System }}{{ end }} user {{ .Prompt }} assistant """ PARAMETER stop PARAMETER stop PARAMETER stop
so after the "ollama create ..." command i ended up with the error "Error: json: cannot unmarshal array into Go value of type string" I updated ollama (it was as per this date some 4 weeks old) and after the update i did have error "Error: vocabulary is larger than expected '128257' instead of '128256'"
Then in the config.json i changed the value -> "vocab_size": 128256 to 128257 and ran again, which then resulted in success. I cant piece this together, but lets test the results, which are not succesvol. The model errors out sadly. But, doing same excise with a different NousResearch model, it all worked same as in video demonstrated. Tnx Mark.
Oh that's weird, I wonder what happened since I made it. I know this technique only works with a certain set of model architectures, but I thought it'd 'just work' with any Llama fine tunes. Could you paste the link to the NousResearch model that you got working?
I faced the same problem and now it is solved. Thanks
ollama run my_test_model
Error: llama runner process has terminated: error:check_tensor_dims: tensor 'token_embd.weight' has wrong shape; expected 4096, 128257, got 4096, 128256, 1, 1
@@md.mahadehasan737 How did you solve it?
@@learndatawithmark Actually, I did not solved it properly, The transformation of the model was okay my editing the configuration file but when I went for running the model it gives the error I posted above.
Is Modelfile should be the same for all models in huggingface? Some of them doesn't show template or prompt format. How we can know what to write in Modelfile?
Tried multiple times to convert the model to GGUF for llama.cpp, but the provided python code kept erroring. I will give this a try although llama.cpp is more efficient. Thanks!
I can't get find help about 'Error: llama runner process has terminated: error loading model: check tensor dims: tensor 'token_`embd.weight' has wrong shape; expected 4090, 128257, got 4096, 128256, 1,' can you assist me?
Stuck in the same problem.
00:06 its now 1 million from 600,000 in 4 month a go.
Almost doubling every 6 month
What is the command to enter seftonsur in olama in windows
Hello, Thanks for the great videos. It's been about several ours I am browsing in your channel. Just a question is it possible to use Ollama and doing fine-tuning with that?
how to tinstall modeldownloader? i try git clone, and then try hfdownloader in cmd, its still error its not recognized as an internal or external command. thx
I used the instructions from the Quick Download section - github.com/bodaay/HuggingFaceModelDownloader?tab=readme-ov-file#quick-download-and-run-examples-linuxmacwindows-wsl2
Hi Mark, thank you for your great video! I tried it to download black-forest-labs/FLUX.1-dev model and I succeeded using Hugging Face CLI download. But I never manage it to convert the model to Ollama with a Modelfile file created as you explained and pointing to the directory where the model has been downloaded and stored locally on my Mac, the Ollama create command always says "Error: no safetensors or torch files found" ??? Any idea would be welcome, thanks!
its not an llm :), its a diffusion model
I think the Hugging Face quantizer only works for a subset of models listed on this page - github.com/ollama/ollama/blob/main/docs/import.md
But if you want to run Flux locally, perhaps you could have a look at this GGUF version - huggingface.co/city96/FLUX.1-dev-gguf
Or if Ollama doesn't work, llama.cpp might be able to run it.
Or maybe take a look at this actually - pypi.org/project/stable-diffusion-cpp-python/
hello, im stuck at the quantize part can you help? I'm using terminal on macos with ollama. pls send me the terminal commands to quantize safetensors llm with the create -q command on ollama(Q5_K_M). thank you
All the code that I used is here - github.com/mneedham/LearnDataWithMark/tree/main/ollama-own-model
Let me know which bit didn't work and what error you get?
I can't get find help about 'Error: llama runner process has terminated: signal: aborted' can you assist me?
I get this: Error: llama runner process has terminated: exit status 0xc0000409.. Could be nice if someone figured out whats wrong here :D
Have you tried posting it on the Ollama GitHub repository? I haven't come across that error yet!
Hi, I get error "Error: unknown data type: U8", has anyone solved similar problems?
When do you get that error"
@@learndatawithmark on converting model stage, after run "ollama create ..."
Figured out the problem. The problem was that I made fine-tuning on quantized model, and it is not good)
Nice
i keep getting incorrect function, any advice?
Incorrect function? Can you explain a bit more
I'm taking instruction on how to run an AI model from a Buddhist monk in perfect English. The future is here.
Maybe I need to rename as MonkAI
Thank you so much. I am having problem running models downloaded from hugging face having safetensor file. I have these files in oobabooga/text-generation-webui. I have to use this for ollama. I followed everything, even created modelfile with path to safetensor directory, but it is not running >> ollama create model_name -f modelfile. Please help me.
What happens when you run the command?
@@learndatawithmark
Thank you so much for quick response. Your videos have helped me a lot.
I am running this on windows 11. I did follow steps:
1) created Modelfile with this script:
"FROM C:\Users\PARTH\Downloads\text-generation-webui-main \text-generation-webui-main\models\TheBloke_dolphin-2.7- mixtral-8x7b-AWQQ
TEMPLATE = """{{ if .System }}system
{{ .System }}{{ end }}
user
{{ .Prompt }}
assistant
"""
PARAMETER stop
PARAMETER stop
PARAMETER stop "
2) I ran following command on terminal opend from where this modelfile is stored.
"ollama create mixtral:dolphine -f .\Modelfile"
3)It showed me this error:
"Error: command must be one of "from", "license", "template", "system", "adapter", "parameter", or "message""
4) I only made file with FROM statement without parameter and template, It ran, but gave this error:
"C:\Users\PARTH\.ollama>ollama create mixtral:dolphine -f .\Modelfile
transferring model data
unpacking model metadata
processing tensors
Error: couldn't find a layer name for 'model.layers.0.block_sparse_moe.experts.0.w1.qweight'"
5) I ran again with another models, but gave same error:
"C:\Users\PARTH\.ollama>ollama create slimorca:13b -f .\Modelfile
transferring model data
unpacking model metadata
processing tensors
Error: couldn't find a layer name for 'model.layers.0.mlp.down_proj.qweight'"
Thanks for the video,
I wanted to do this since long time ago.
just need a bit of help with this error: Error: json: cannot unmarshal array into Go value of type string (see details below)
$ time ollama create \
> -f Modelfile NousResearch_Hermes-3-Llama-3.1-8B:q4_0 \
> --quantize q4_0
transferring model data
converting model
Error: json: cannot unmarshal array into Go value of type string
real 34m12.658s
user 1m52.414s
sys 2m1.768s
Modelfile :
FROM ./downloads/NousResearch_Hermes-3-Llama-3.1-8B
TEMPLATE """{{ if .System }}system
{{ .System }}{{ end }}
user
{{ .Prompt }}
assistant
"""
PARAMETER stop
PARAMETER stop
PARAMETER stop