A Guide to Challenges and Recommendations for the Implementation of a Fairer AI
Fundación Vía Libre (2023), 19 pp.
CC BY-SA
"The main recommendations we propose are:
1. Before proceeding with any development, analyze the need and social value of it. Consider the potential benefits and drawbacks of implementing the project.
2. Define the problem to be solved in non-technical terms, based on the joint reflection of the parties interested in developing an AI application. Consider the possible secondary uses of the application of the project and its possible harmful effects.
3. Analyze the data sources with which the model may be trained, taking into consideration the protection of personal data and intellectual property. Pay close attention to potential biases to ensure that all groups are adequately and non-discriminatorily represented, with a special focus on minorities.
4. When training and evaluating predictive models, incorporate a methodology for checking well-known limitations of models that lead to pernicious results: underrepresented classes, majority classes, a tendency to overfit, etc. Incorporate also more general error analysis to detect previously unknown limitations.
5. Carry out deployment in phases, accompanied by the corresponding monitoring mechanisms for early detection of pernicious effects of the implementation of the system in actual contexts, incorporating different perspectives into the analysis of the behavior of the system.
Equity analysis must be transversal, i.e., a reflective resource present throughout the entire system development and construction process, rather than being applied only when evaluating the discriminatory impacts of a system after it has been deployed." (Executive Summary)
1. Before proceeding with any development, analyze the need and social value of it. Consider the potential benefits and drawbacks of implementing the project.
2. Define the problem to be solved in non-technical terms, based on the joint reflection of the parties interested in developing an AI application. Consider the possible secondary uses of the application of the project and its possible harmful effects.
3. Analyze the data sources with which the model may be trained, taking into consideration the protection of personal data and intellectual property. Pay close attention to potential biases to ensure that all groups are adequately and non-discriminatorily represented, with a special focus on minorities.
4. When training and evaluating predictive models, incorporate a methodology for checking well-known limitations of models that lead to pernicious results: underrepresented classes, majority classes, a tendency to overfit, etc. Incorporate also more general error analysis to detect previously unknown limitations.
5. Carry out deployment in phases, accompanied by the corresponding monitoring mechanisms for early detection of pernicious effects of the implementation of the system in actual contexts, incorporating different perspectives into the analysis of the behavior of the system.
Equity analysis must be transversal, i.e., a reflective resource present throughout the entire system development and construction process, rather than being applied only when evaluating the discriminatory impacts of a system after it has been deployed." (Executive Summary)
