Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees (TMLR)

In our recent TMLR paper, we develop a new method for concept based eXplainable AI (XAI). The completeness axiom renders the explanation of a post-hoc eXplainable AI method only locally faithful to the model, i.e. for a single decision. For the trustworthy application of XAI, in particular for high-stake decisions, a more global model understanding is required. To this end, concept-based methods have been proposed, which are however not guaranteed to be bound to the actual model reasoning. To circumvent this problem, we propose Multi-dimensional Concept Discovery (MCD) as an extension of previous approaches that fulfills a completeness relation on the level of concepts. Our method starts from general linear subspaces as concepts and does neither require reinforcing concept interpretability nor re-training of model parts. We propose sparse subspace clustering to discover improved concepts and fully leverage the potential of multi-dimensional subspaces. MCD offers two complementary analysis tools for concepts in input space: (1) concept activation maps that show where a concept is expressed within a sample, allowing for concept characterization through prototypical samples, and (2) concept relevance heatmaps, that decompose the model decision into concept contributions. Both tools together enable a detailed global understanding of the model reasoning, which is guaranteed to relate to the model via a completeness relation. Thus, MCD paves the way towards more trustworthy concept-based XAI. We empirically demonstrate the superiority of MCD against more constrained concept definitions.
Find more information here:
Paper: openreview.net/forum?id=KxBQP...
Code: github.com/jvielhaben/MCD-XAI
Subscribe for more content like this!
Follow us on social media:
LinkedIn: / fraunhofer-hhi
Twitter: / fraunhoferhhi
Instagram: / fraunhofer_hhi