Evolving Evaluations

The previous post Myth No.6 - 'Forced Wins for White' (March 2023) introduced 'the Molas study', a data scientist's effort 'to find if there’s a [chess960] *start position* that's better than the others'. One of the datasets used in the study was:-

Stockfish evaluation at depth ~40 for all the starting positions

This is also known as the 'Sesse' resource and I gave its URL in the post. The Molas study concluded,

Stockfish evaluations don’t predict actual winning rates for each variation

This didn't surprise me. If you consider that each start position (SP) leads to a mega-zillion possible games and that Sesse reduces each SP to a single two-digit number, much more surprising would be to find a meaningful correlation between an SP's W-L-D percentages and its Sesse value.

I discussed the Sesse numbers once before in A Stockfish Experiment (February 2019). That post mentioned another discussion, What's the Most Unbalanced Chess960 Position? (chess.com; Mike Klein; March 2018 / February 2020). Fun Master (FM) Mike observed,

Let's now take the most extreme case the other way -- the position where Sesse claims White enjoys the most sizable advantage. The lineup BBNNRKRQ delivers a whopping +0.57 plus for the first move. The advantaged is so marked that some chess960 events may even jettison this arrangement as a possible option (a total of four positions are +0.50 or better for White, but none are as lopsided as this one).

That position, also known as 'SP080 BBNNRKRQ', has received some notoriety thanks to Sesse, so I decided to investigate further. I downloaded the SP080 file from the CCRL (see link in the right sidebar), loaded it into SCID, and discovered that it contained 554 games. SCID gave me percentages for White's first moves, which I copied into the following chart.

There are 11 first moves for White listed in the top block of the chart. I then expanded the first two of those moves -- 1.g3 (65.7% overall score for White) and 1.Nd3 (59.7%) -- into the second and third blocks of the chart to see how Black has responded to those moves.

You might be wondering why I said there were 554 games in the file, but the SCID extract counts only 519 games. SCID was designed to handle the traditional start position (SP518 RNBQKBNR) and knows nothing about chess960 castling rules. SP080 allows 1.O-O on the first move, which SCID rejects. The 35 missing games (554 minus 519) are games that started 1.O-O. When I'm using SCID for a chess960 correspondence game, I have a technique to account for this anomaly, but I won't go into details here.

Similarly, the charts for 1.g3 and 1.Nd3 show '[end]' as one of the first moves for Black. These are games where Black played 1...O-O on the first move. The corresponding percentage scores are among the worst for Black, showing once again that early castling is a risky strategy.

If I were playing SP080 in a correspondence game, I would analyze both 1.g3 and 1.Nd3. A promising continuation after 1.g3 is 1...c5, which the score '43.9%' says, 'Favors Black'. Of course, I would have to look at White's second moves in this variation, where one move will appear to be superior to the others. And so on and so on.

To be useful, the SCID tool needs to be handled intelligently. I recently blundered into a wrong evaluation that I doumented in The CCRL Is Unreliable (Not!) (December 2021). I'm hopeful that some day a tool will appear that rivals SCID functionality *and* that understands chess960 castling. For now, I make do with the software I have.

For a look at two more SPs where evaluations have shifted with experience, see SP864 - BBQRKRNN and SP868 - QBBRKRNN, which are both attachments to this blog. One lesson I've learned from playing chess960 for almost 15 years : nothing is fixed in stone.

Myth No.6 - 'Forced Wins for White'

Upon encountering chess960 for the first time, one of the first questions a new player asks is 'Are all 960 positions fair?'. I included a statement of this concern in Top 10 Myths About Chess960 (May 2012), where one bullet said,

Some start positions are forced wins for White

Remembering that I wrote this more than 10 years ago, at a time when I wasn't absolutely 100% sure that such unfair positions didn't exist, my standard response to the statement was, 'Which positions are forced wins? Please provide a specific example'. I never received a single example. Ten years later I can say with more confidence -- although still not 'absolutely 100% sure' -- that while some positions are difficult for Black to play, none of the 960 positions is lost before a single move is made.

In January a new study titled, Analyzing Chess960 Data | Alex Molas | Towards Data Science (towardsdatascience.com), appeared. Its subtitle announced,

Using more than 14 million chess960 games to find if there’s a variation that's better than the others.

There is considerable knowledge presented in the study and I don't pretend to understand all of it. I might well need several posts to unravel its subtleties, so I'll start by summarizing its references; in the following discussion, '>>>' means a direct quote from 'Analyzing Chess960 Data'.

>>> 'The original post was published here...'

• Discovering the best Chess960 variation (amolas.dev; Alex Molas; 'Posted Jan 11, 2023; Updated Jan 17, 2023')

[NB: I'll come back to this reference later; see '(A)' below. First I need to point out that there's an important issue with terminology. When chess players use the term 'variation', they mean a sequence of play arising from a specific position; e.g. 'In this position I had two variations and I had to work out which variation was better for me.' • In the Molas study, I'm convinced that the word 'variation' refers to one of the well-defined 960 start positions that are legal for chess960. I read the subtitle of the towardsdatascience.com article as 'to find if there’s a *start position* that’s better than the others' and the title of the amolas.dev post as saying 'Discovering the best chess960 *start position*'. I won't repeat this caveat each time, but it's important and helps to understand the discussion.]

>>> 'Ryan Wiley wrote this blog post where he analyzes some data from lichess..'

• Using Lichess's Public Data To Find The Best Chess 960 Position (lichess.org/rdubwiley; August 2022)

>>> 'There’s also this repo with the statistics for 4.5 millions games (~4500 games per variation)...'

• GitHub - welyab/chess960-win-by-position-setup: Analyzes the results of more than 4.57 million chess games in the chess960 variant (github.com/welyab)

[NB: There's an issue with the word 'variant' here, but it's not as important as the previous 'NB'. Chess960 purists will know what I'm talking about.]

>>> 'In this spreadsheet there’s the Stockfish evaluation at depth ~40 for all the starting positions...'

• Fischer Random - 960 startposisjoner - Google Sheets (docs.google.com; Sesse)

>>> 'There’s also this database with Chess960 games between different computer engines. However, I’m currently only interested in analyzing human games, so I’ll not put a lot of attention to this type of games...'

• CCRL 40/2 FRC (computerchess.org.uk/ccrl)

>>> 'Lichess -- the greatest chess platform out -- maintains a database with all the games that have been played in their platform...'

• lichess.org open database (database.lichess.org)

>>> 'To do the analysis, I downloaded ALL the available Chess960 data (up until 31–12–2022). For all the games played I extracted the variation, the players Elo and the final result...'

• Chess-960-lichess | Kaggle (kaggle.com; 'Dataset with the results of more than 14M games of Chess960. From lichess.')

>>> 'The scripts and notebooks to donwload [sic] and process the data are available on this repo...'

• GitHub - alexmolas/chess-960 (github.com/AlexMolas)

At this point the article launches into 'Mathematical framework; 'Bayesian A/B testing; [...]'. This, of course, is the essence of the study and I won't go any further in this current post. Let's get back to '(A)', where there's another key reference.

>>> 'This post got some attention in Reddit...'

• There are no better initial positions in Chess 960 (reddit.com/r/chess; 11 January 2023)

I could end the post here, but I need to make an admittedly subjective observation. There are two example of bias in the above references.

The first bias is 'I’m currently only interested in analyzing human games'. Huge caveat here. In my not-so-humble opinion, the CCRL is the best source of chess960 opening theory. Period. Full stop. The CCRL engines are rated at least 1000 points higher than most human players on Lichess. The engines don't make simple tactical errors and they calculate deeper into every position than any human can. If there is an unfair chess960 start position, the engines will find it, just like they find errors in most games played between humans.

I can understand ignoring the engines because humans grapple with different challenges in chess960 openings, but the purpose of the study was 'to find if there’s a *start position* that’s better than the others'. Ignoring the experience of the best players on the planet is severely limiting.

The second bias is 'Lichess -- the greatest chess platform out'. The main alternative here is Chess.com. Why ignore games played on the world's largest chess platform? Maybe there's a good reason, but I can't think of one. On a personal note, last year I investigated which of the two sites would be better to continue my own chess960 correspondence play. I determined that Chess.com was more serious about eliminating human players who cheat by using engines in games with other humans. Since my goal was playing no-engine games, I went with Chess.com. How much of the Lichess data involves concealed engine use?

Biases notwithstanding, the Molas study is an important step in evaluating the fairness of all 960 positions in chess960/FRC. I'm looking forward to understanding it in more depth.

Chess.com Reviews a Chess960 Opening

In last week's post, Chess.com Pinpoints a Tactical Error (February 2023; see the post for a copy of the game's PGN), I used 'Chess.com's Game Review Tools' to find out where I had made the first mistake in losing a chess960 game. This week I'll use the same tools to extract comments on the opening moves.

The following diagram shows the start of the same game featured in the 'Tactical Error' post. There's more I can say about the look and feel of the 'Analysis' tool itself, but I'll save that for a series I'm doing on my main blog. The most recent post in that series was Chess.com's Game Review Tools PGN (February 2023).

Shown on the left is the start position for the game, 'SP350 NRKQRBBN'. On the right is a summary of the overall quality of the players' moves ('Brilliant', 'Great Move', ..., 'Blunder'). For example, the tool considers that both players made one 'Great Move'.

AV vs. bemweeks | Analysis (chess.com)

The following table shows the tool's comments on the first 12 moves of the game (24 ply deep). I stopped the analysis when I reached the move where I committed the 'Tactical Error'.

Move	Short Comment	Long Comment	Eval
1.e4	is excellent	This prepares the bishop for development. This threatens to reveal an attack on a pawn.	+0.13
1...e5	is good	This prepares your bishop for development.	+0.30
2.f4	is excellent	This exposes an attack, threatening a pawn.	+0.27
2...Nb6	is excellent	Your piece jumps in to protect a pawn!	+0.41
3.fxe5	is best	Right on target.	+0.41
3...f6	is an inaccuracy	You are threatening to attack a trapped rook.	+1.09
4.Nb3	is a mistake	This loses a pawn.	+0.05
4...fxe5	is best	That wins a free pawn!	+0.05
5.Ng3	is excellent	One of the best moves.	-0.02
5...Qf6	is best	You activate your queen by moving it off of its starting square.	-0.02
6.Be3	is good	This moves the bishop to a better location, allowing it to control more squares.	-0.30
6...O-O-O	is good	Your rooks can see each other now, allowing them to provide mutual defense.	+0.02
7.d3	is best	That's what I would have recommended.	+0.02
7...d5	is best	You are threatening to kick a bishop.	+0.02
8.Qd2	is an inaccuracy	This ignores a better way to develop a queen off its starting square.	-0.47
8...Qc6	is good	You are threatening to kick a bishop.	-0.10
9.Bg5	is good	This wins a tempo by threatening a rook and forcing it to move away.	-0.38
9...Be7	is a mistake	You are threatening to win material.	+0.56
10.Bxe7	is best	After all captures, this is an equal trade.	+0.56
10...Rxe7	is best	You trade off equal material.	+0.56
11.Nf5	is good	This attacks a rook, winning a tempo when it moves away. This threatens to fork pieces.	+0.27
11...Red7	is good	You have now doubled your rooks, allowing them to team up to create threats.	+0.62
12.exd5	is best	This exposes an attack, threatening a pawn.	+0.62
12...Qxd5	is a mistake	You overlooked a better way to recapture a piece.	+1.63

There's much more I could say about the comments, but it would not be useful at this point. Here are a few comments that jumped off the screen at me.

• 8...Qc6; 'You are threatening to kick a Bishop.' • The move defends against a nasty x-ray threat.
  
  
• 9.Bg5; 'This wins a tempo by threatening a Rook and forcing it to move away.' • The move doesn't win a tempo, but it might lose a tempo by forcing the Rook to a better square.
  
  
• 9...Be7; 'Is a mistake. You are threatening to win material.' • It's a mistake to win material? Something does not compute here.

And so on. The long comments are generally lame and show little understanding of chess960 opening objectives. What happened in the fight for the center?

The most valuable part of the exercise is to see the change in evaluation from one move to the next. It reaffirms the severity of my mistake on the 12th move.

Chess.com Pinpoints a Tactical Error

In last month's post, The Fascinating World of Chess960 (January 2023), I discussed a shift in focus for my own chess960 games. In a nutshell, to play chess960 I switched from a site allowing engines to a site forbidding them. I finished the post saying,

That's the background for a series of posts that I plan to write for my games on Chess.com. There are several aspects to be covered, e.g. Game review tools

I introduced those tools on my main blog in Chess.com's Game Review Tools (February 2023), using an example chess960 game. I'll use the same game in this current post. After four wins, it was the first game I lost on Chess.com starting June 2022, when I adopted the no-engine approach.

The 'Game Review Tools' post used numbers to identify the different screen shots and I'll follow the same numbering scheme in this current post. There are three different review tools that I numbered '02', '03a', and '05a'. The '02' tool shows the moves of the game and the times used for each move. It also provides (1) an entry into the '03a' and '03b' tools, and (2) a PGN download of the moves of the game, without variations or comments.

At this point, I don't see much difference between the '03a' and '05a' tools. The differences seem mainly cosmetic, so I'll continue with the '05a' tool. It's accessed via a feature called 'Saved Analysis'. In the game I was outplayed tactically and didn't know where I had gone wrong.

I had Black in a game starting 'SP350 NRKQRBBN'. I was pleased with my position after the first few moves and thought that I had equalized, maybe even gained a slight advantage. Then suddenly I had an inferior game. Why?

The '03a' and '05a' tools offer commentary on each move played in the game. I won't discuss the early comments in this post, because I'm not yet convinced they are helpful. The critical position is shown in the following screenshot, Black to move.

AV vs. bemweeks | Analysis - Chess.com • After 12.e4-d5(xP)

White has just captured a Pawn on d5 with 12.exd5, and Black has four possible recaptures. The move 12...Rxd5 is a '??' blunder, allowing the family fork 13.Ne7+. That leaves three other moves. Since White's last move also discovered an attack on the e5-Pawn, I played 12...Qxd5, protecting that Pawn. That was a mistake, which the tool flags with the remark '(?) Qxd5 is a mistake'.

What's better? The tool suggests 12...Nxd5. Now if 13.Rxe5, Black has 13...Qf6, when White is in trouble. I didn't see that possibility during the game and never recovered. Following is the PGN as provided by the '02' tool.

[Event "Let\\'s Play! - Chess960"]
[Site "Chess.com"]
[Date "2022.07.05"]
[Round "?"]
[White "Andreasvinckier"]
[Black "bemweeks"]
[Result "1-0"]
[Variant "Chess960"]
[SetUp "1"]
[FEN "nrkqrbbn/pppppppp/8/8/8/8/PPPPPPPP/NRKQRBBN w EBeb - 0 1"]
[WhiteElo "1977"]
[BlackElo "1907"]
[TimeControl "1/86400"]
[EndDate "2022.07.23"]
[Termination "Andreasvinckier won by resignation"]
[initialSetup "nrkqrbbn/pppppppp/8/8/8/8/PPPPPPPP/NRKQRBBN w EBeb - 0 1"]

1. e4 e5 2. f4 Nb6 3. fxe5 f6 4. Nb3 fxe5 5. Ng3 Qf6 6. Be3 O-O-O 7. d3 d5 8. Qd2 Qc6 9. Bg5 Be7 10. Bxe7 Rxe7 11. Nf5 Red7 12. exd5 Qxd5 13. Ne3 Qc6 14. g3 Be6 15. Qa5 Bxb3 16. axb3 e4 17. Bh3 exd3 18. O-O-O dxc2 19. Bxd7+ Rxd7 20. Rxd7 Qxd7 21. Qxa7 Nf7 22. Qa5 Nd6 23. Nd5 1-0

The Chess.com tools offer several different PGN downloads. I'll discuss those in another post on my main blog.

The Unusual World of Chess960

If the previous post was about The Fascinating World of Chess960 (January 2023), then this post is about the other side of the coin. Here are a few topics I've noted in the recent past that don't fit in with any other discussions.

FRC Human Openings

While the most recent TCEC Superfinal ('S23 Sufi') was underway, the TCEC announced upcoming events as:-

!next • Now !Sufi * FRC human openings bonus?, Subfi, ...

I reported the Sufi results in TCEC S23, CCC19 Rapid : Stockfish Wins Both (November 2022), but made no mention of the subsequent FRC bonus. Curious what the title 'human openings' meant and finding no explanation in the usual places, I followed my standard procedure on the event's PGN: I downloaded it to my PC, loaded the headers into a database, and ran some simple queries.

I discovered that 48 games were played and that they included only the following six start positions (SPs). The numbers in parentheses show how many times each SP was played.:-

• BRKRQNNB (x2)
• NQRBKRBN (x6)
• NRKQBRNB (x12)
• NRNKBRQB (x12)
• QRBNKBRN (x10)
• RNBBKQRN (x6)

A year ago I ran a similar analysis and recorded my findings in TCEC FRC4 Unbalanced Books (January 2022; 'Is there a correlation between 'unbalanced books' and the degree of scrambling?'). The same question could be asked again, but first I checked whether there was any overlap between the 'unbalanced books' SPs and the 'human openings' SPs.

I found one position on both lists: SP317 NQRBKRBN. Four games from a year ago plus six games from the current exercise makes enough games for further exploration.

'Nyuk, Nyuk' from NIC

Seen on the cover of New in Chess 2022/8 (newinchess.com; New in Chess): The woman on the left, wearing a 'Random Fandom' t-shirt (nice; I love it!), says, 'They call it chess960'. The man on the right, with chess pieces tatooed on his arm, says, 'Oh wow, that's a lot!' Wait until he discovers chess960-squared, aka DFRC.

Google returns three times more references to 'Fischer Random' in the magazine than it does to 'chess960'. I suppose NIC respects the preference of its contributors. Follow the money?

Google Gets Its Wires Crossed

I recently noticed a real mystery in Google search. The home page for this C960/FRC blog showed up in the results for a search on 'chess960' with the following description:-

Chess960 (FRC) • 'It is played in the Chess 960 or Fischer Random format. Created by Bobby Fischer in the late 1990s, Chess 960 shuffles the pieces on the home rank, with 960 ...'

That's all very nice, except I didn't write it. I never split the word 'chess960' into two parts and I try not to capitalize it. Why not? Because the word 'chess' hasn't been capitalized in a long, long time. As for 'created in the late 1990s', everyone knows that it was created by Fischer in the early to mid-1990s and announced in 1996.

So where does the description come from? Apparently from 2022 Champions Showdown - Chess 9LX (uschesschamps.com):-

What is Chess 9LX? • Chess 9LX is a tournament hosted by the Saint Louis Chess Club. It is played in the Chess 960 or Fischer Random format. Created by Bobby Fischer in the late 1990s, Chess 960 shuffles the pieces on the home rank, with 960 representing the number of possible starting positions. Players will not know the order of the home rank until 15 minutes before each round.

Thanks, Google, but please note that I have nothing to do with writing copy for the Saint Louis Chess Club. I could also mention that the moniker '9LX' is an aberration, but I won't. To the Saint Louis Chess Club: Please note that I haven't plagiarized your text.

More?

There are a few more remarks I could make on minor points, but that's enough for this post. The first rule of chess960 is the same as the first rule of chess: 'Just have fun!'

The Fascinating World of Chess960

Last month's post, Christmas Eve (December 2022), wasn't just about creating a new category for Posts with label MW's games (that's me). It was also about switching to a different online service for playing chess960.

I recorded my first game of chess960 in a post on my main blog, titled Chess960? I'm Hooked! (September 2008). The PGN embedded there says the game was played on SchemingMind.com (SM), a site for correspondence chess that is particularly strong in its support of chess variants. I continued playing there until 2016.

A few years after getting hooked, I started playing on another correspondence site, The Lechenicher SchachServer (December 2012; LSS), for reasons explained in that post. I continued playing on both SM and LSS until 2016, when I ran into a problem on SM and decided to leave. To make a long story short, SM has a no-engine policy, but makes little effort to enforce it. LSS allows engines for most of its events and I preferred the clarity of LSS.

I continued playing on LSS until last year. I was playing chess960 in a couple of multi-stage events, where success in one stage promotes a player to the next stage. As the two events were winding down, the site announced the next stages. Unfortunately for me, the start of both events coincided with a pair of two-week vacations that I had been planning for some time. Since LSS events allow only two weeks of vacation on a fixed number of days for a game (no increments), I was faced with an immediate time deficit in all new games. I decided to skip the next stages, essentially taking a year off from serious play.

As my active games gradually came to a conclusion, in my free time I started using an engine to analyze my old correspondence games from the pre-engine, pre-chess960 era. I was amazed that my moves were generally approved by modern engines. I could often recollect the reasoning and emotions behind my moves and realized that using an engine had turned me from a chess player into an engine operator. After 14 years of playing chess960, I hadn't gained much insight into its subtleties, because I was essentially playing what the engine instructed me to play, often without understanding why.

I decided to switch to a site that didn't allow engine use. SM was out because it doesn't enforce its policy. Then I remembered Chess.com, which has a good reputation for vigorously enforcing its no-engine policy, even if it leads to controversial decisions. I had played a few games of chess960 there in 2009-2010 and more recently in 2019, an experience documented in Playing the FWFRCC (June 2019).

I switched to Chess.com in May 2022, playing one or (maximum) two games of correspondence chess at a time. What a difference! Where my last years with LSS involved struggling against players with far more powerful engines than I was using, at Chess.com I was using my own head to play real chess against other players doing the same. After all, that's what had attracted me when I first started playing chess so many years ago.

So far I've played about a dozen games on Chess.com, never once tempted to use an engine. I also know full well that if I do use an engine and am caught, I will lose the premium membership that CEO Erik gave me when I was writing a review of the site for About.com in mid-2008.

That's the background for a series of posts that I plan to write for my games on Chess.com. There are several aspects to be covered:-

• Insights from my games
• The correspondence play interface
• Game review tools
• The site's custom anti-cheating measures
• And more...?

I'll wander through these topics in future posts, some of them on my main blog. Thanks to both SchemingMind and LSS for the terrific support of chess960 throughout the years. May they continue to introduce keen chess players to the fascinating world of chess960.

Christmas Eve

Out with the old!

• Showing posts with label LSS.
• Showing posts with label Schemingmind.

In with the new!

• Showing posts with label MW's games.

That would be more appropriate for next week's New Year's Eve post, but so be it. I needed to make the change now.