Fantastic presentation, Max and Rowan! The depth of your analysis and the clarity with which you presented the complexities of evaluating LLMs is truly commendable. It's evident that a lot of thought and effort went into this research. I'm particularly intrigued by your approach to using LLMs as evaluators. It opens up a plethora of possibilities but also brings forth some ethical considerations. How do you account for systemic biases in evaluation metrics when using LLMs as evaluators? Given that traditional metrics might not capture the fairness aspect adequately, have you considered incorporating fairness metrics or mitigation methods in your evaluation process?
![[Webinar] Navigating the LLM Risk Landscape in Financial AI](http://i.ytimg.com/vi/bXWBNlgpDGE/mqdefault.jpg)
![[Webinar] Navigating the LLM Risk Landscape in Financial AI](/img/tr.png)
Fantastic presentation, Max and Rowan! The depth of your analysis and the clarity with which you presented the complexities of evaluating LLMs is truly commendable. It's evident that a lot of thought and effort went into this research.
I'm particularly intrigued by your approach to using LLMs as evaluators. It opens up a plethora of possibilities but also brings forth some ethical considerations. How do you account for systemic biases in evaluation metrics when using LLMs as evaluators? Given that traditional metrics might not capture the fairness aspect adequately, have you considered incorporating fairness metrics or mitigation methods in your evaluation process?
I was looking for a good summary around LLM evaluation metrics.. I see a lot of them captured here well