Fine-Tuning with ReFT: Create an Emoji LLM for Medical Diagnosis

[@thisurawz]

Is ReFT performing better than LoRA? i mean the Accuracy mainly. Moreover, which is the best performing one among LoRA, DoRA and ReFT when we compare the Accuracy

[@charitasri8162]

similar to this..can i take the dataset as text and label..and in text i will writing patient symptoms,medical history, medical report results and in label column it has the disease..and when we take the user input symptoms,medical history,medical reports and output should be the medical diagnosis and treatment....? can you please help me

[@muhammedajmalg6426]

thanks for sharing

[@jaganbaburajamanickam7502]

In your code "reft_config = pyreft.ReftConfig(representations={
"layer": 8, "component": "block_output",
"low_rank_dimension": 4,
"intervention": pyreft.LoreftIntervention(embed_dim=model.config.hidden_size,
low_rank_dimension=4)})"
How do we choose the layer and low_rank_dimension value ?
It would be nice if you could describe more about the arguments and give some suggestions/examples how and what to adjust when the output is not exepcted

[@AIAnytime]

Thanks for the tip, can you watch this video of mine for more understanding on Hyper parameters: ruclips.net/video/y-9G41zELIY/видео.html