4:14 where do you personally find examples like this to train your models or do you come up with them by yourself? and what is a .jsonl file? (compared to .json)
A jsonl file is very similar to a json file. It’s got a new json object on each line of the file. It’s almost like an array of json objects. For examples, it depends on the problem. This video had 10 examples. The first 3 I copied from the openai docs. The last 7 were generated by gpt4 when I asked it to create more examples like the first 3. You could manually create the examples. You could scrape data to create it too. Eg if you want to create a bot that tweets like Elon musk you could use the Twitter api to get his tweets and either manually write prompts for that or come up with another method like gpt4 generating the prompts.
Is there any tool which can convert json (it can have complex nested data) to jsonl for make it ready for fine-tune e.g "appetizers": [ { "id": 1, "name": "Beef Dumplings", "price": 10.45, "description": "6 PCS", "options": { "Dumplings": [ { "id": 1, "name": "Steamed", "price": 0 }, { "id": 2, "name": "Fried", "price": 0 } ]
4:14 where do you personally find examples like this to train your models or do you come up with them by yourself? and what is a .jsonl file? (compared to .json)
A jsonl file is very similar to a json file. It’s got a new json object on each line of the file.
It’s almost like an array of json objects.
For examples, it depends on the problem. This video had 10 examples. The first 3 I copied from the openai docs. The last 7 were generated by gpt4 when I asked it to create more examples like the first 3.
You could manually create the examples. You could scrape data to create it too. Eg if you want to create a bot that tweets like Elon musk you could use the Twitter api to get his tweets and either manually write prompts for that or come up with another method like gpt4 generating the prompts.
Is there any tool which can convert json (it can have complex nested data) to jsonl for make it ready for fine-tune e.g "appetizers": [
{
"id": 1,
"name": "Beef Dumplings",
"price": 10.45,
"description": "6 PCS",
"options": {
"Dumplings": [
{ "id": 1, "name": "Steamed", "price": 0 },
{ "id": 2, "name": "Fried", "price": 0 }
]
This seems to do it from a quick search:
codebeautify.org/json-to-jsonl-converter
Any LLM will also do it well.
nice video mate!
Thank you. Glad you enjoyed it :)