I'm using the HEAD from 2019-09-23 (https://github.com/SimpleMachines/SMF2.1/commit/8079796516a5f2944fea476a1fd04a0bda6d1fb7).
When I aptitude install enchant php5-enchant (Debian 8), restart my webserver, enable the then-newly-available option "Forum → Posts and Topics → Post Settings → Enable spell checking", begin a new post and press "Spell check", a blank window opens and my post is submitted as if I had pressed "Post". Nothing in the Error log, lighttpd access/error logs or postgres logs.
Did you install a dictionary? More specifically the dictionary that would match the language pack you have installed?
Good catch, but that doesn't seem to be the whole problem, the behavior is still very interesting.
My forum is english/german. An english dictionary ("hunspell-en-us") was installed by default. I now manually installed "hunspell-de-de" too.
Before, with only the english dictionary, the popup window really was just an empty white window. Now, the popup window does have contents, but spell checking only seems to work sometimes: First, as you kinda implied, spell checking only seems to work for the language that is selected in the UI at that moment. But that's not all.
When the first word is spelled correctly, the text will just be posted (and the quick reply field emptied as before (but I forgot to mention it before ;))), and the popup is still kinda empty (three empty text fields).
When the first word is spelled incorrectly and (<assumption0>) the spellchecker has a suggestion (</assumption>), the post is submitted in the background, but I can actually begin spellchecking my text. When I finish spellchecking the complete text, my corrected text is re-inserted into the quick reply field again (during spellchecking, it's empty).
: I assume this because f.e. it doesn't work (only displays the three empty text fields) in cross-language situations (UI set to english, text in german, or vice versa).
Errors in the console? In 2.0 (and if memory serves it's not wildly different in 2.1), you get the popup and if there are no spelling mistakes, you get a JS alert saying so, and when you acknowledge it, it closes both the alert and the popup.
No. Still like I posted above: nothing at all, not in the SMF Admin Error log, not in the lighttpd logs, nor the postgres logs.
I got no such JS popup in the problematic case (only after successfully finishing the spellcheck I got one that worked like you describe; don't remember what it said, though).
Ah, I misread... late night brain... There are no errors AFAICT, see attachment.
Did you mean https://github.com/SimpleMachines/SMF2.1/issues/5847 ? ;)