The Quest for Reliable Metrics of Responsible AI

TVR, Maria Maistro, Tuukka Ruotsalo, Christina Lioma

The Quest for Reliable Metrics of Responsible AI: 4 upvotes on Hugging Face Daily Papers, #24 of 30 papers on 2025-10-30. Day-by-day upvote history.

The development of Artificial Intelligence (AI), including AI in Science (AIS), should be done following the principles of responsible AI. Progress in responsible AI is often quantified through evaluation metrics, yet there has been less work on assessing the robustness and reliability of the metrics themselves. We reflect on prior work that examines the robustness of fairness metrics for recommender systems as a type of AI application and summarise their key takeaways into a set of non-exhaustive guidelines for developing reliable metrics of responsible AI. Our guidelines apply to a broad spectrum of AI applications, including AIS.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.