Hi Alexei. I haven't read Dalio's Principles yet - it's on my list. In general it's hard to argue with a process where you get a bunch of people to debate something a point and arrive a consensus. In a perfect world, the "believability weights" should be based on the facts marshaled in the argument rather than the person doing the marshalling (ie don't shoot the messenger) and the way one updates their "believability" weights might be modeled in a Bayesian framework (the more data, the less flexiblity to update).

As for Brier Scores, I find them helpful in assessing probabilistic predctions for binary outcomes, but outcomes in capital markets tend to be continuous, ie. alpha over a certain period. Maybe there's a Brier extension for continuous outcomes?