(Summarizing the discussion from #455)

From my understanding, the matched-pairs rank biserial correlation according to Kerby 2014 is calculated as the rank sum difference between favorable and unfavorable evidence for the Wilcoxon signed-rank test (see for example Table 2 on p. 6). This means wether or not a difference in value is considered favorable or unfavorable should depend on the hypothesis.

In Pingouin, differences larger than zero are always considered favorable evidence.

Kerby (2014) states that:

In words, the formula states that the nonparametric correlation equals the simple difference between the proportion of favorable and unfavorable evidence; in the case of the Mann-Whitney U test, the evidence consists of pairs.
An advantage of the simple difference formula is that it expresses one effect size (the rank-biserial correlation) in terms of another easily understood measure of effect size (the common language effect size). The other formulas for the rank-biserial express it in terms that are less easily interpreted. A second advantage of the simple difference formula is that it gives meaning to the direction of the sign. A positive correlation means that the data were in the predicted direction; a negative correlation, that they were against the predicted direction.