16 February 2019

A Stockfish Experiment

Last year, in the aftermath of the Carlsen - Nakamura chess960 match, Chess.com published an article, What's The Most Unbalanced Chess960 Position?. The essence of the piece was:-
Recently, the Norwegian "supercomputer" Sesse analyzed all 960 variations using Stockfish 9. At a depth of 39-40 ply for each position, which took about two to three hours each, its findings were published.

I downloaded the referenced spreadsheet and compared it with the data currently displayed on the CCRL (see link in the right sidebar). The results are shown below.

The first column shows the start position, followed by the Sesse evaluation in centipawns, followed by the CCRL overall percentage score for White and the precentage of draws. For example, the top row says that BBNNRKRQ was evaluated by Sesse at 0.57 (a little more than a half-Pawn), with a CCRL result of 51.0% in White's favor and 15.3% of games ending in a draw.

The chart gives the top-25 positions flagged by Sesse, down to a cutoff of 0.40. I could have added the numeric ID for each start position and will do so if I ever come back to the data.

At first glance, there doesn't seem to be a high correlation between the Sesse results and the CCRL results. The first position where there is agreement between the two sources is QBBRKRNN. I once discussed this position in A Difficult SP for Black (April 2013). By coincidence, I'm currently playing the position in a pair of correspondence games and hope to have more to say about it in the future.

As for the Sesse results, they confirm that engines just don't evaluate chess960 start positions particularly well. Long term positional considerations are not the engines' strong point.

No comments: