This is an extension of our AdaFair algorithm (CIKM 2019 LINK:https://aiml-research.github.io/files/19.CIKM.pdf) to other parity-based fairness notions. We propose an ensemble approach to fairness that alters the data distribution over the boosting rounds “forcing” the model to pay more attention to misclassified instances of the minority. This is done using the so-called fairness cost which assesses performance differences between the protected and non-protected groups. The performance is evaluated based on the partial ensemble rather than on the weak model of each boosting round. We show that this cumulative notion of fairness is beneficiary for different parity-based notions of fairness. Interestingly, the fairness costs also help with the performance on the minority class (if there is imbalance). Imbalance is also directly tackled at post-processing by selecting the partial ensemble with the lowest balanced error.