So you're not actually typing in the name with the funny symbols, but replying to a PM or otherwise letting the system fill in the name? It also happens when you try to type in the name, with a character such as a (tm) cut and pasted from your own forum?
I seem to recall seeing complaints long ago about PM having trouble with user names containing odd characters. Have you done a thorough search of this forum? I don't remember if it was happening in your circumstances, or only when trying to type in a name (or use the auto-complete). Maybe if you find those discussions there will be some information there that can point us in the right direction.
By the way, was your forum originally in another encoding (Latin-x) and you converted it over to UTF-8? Are any of these offending names dating from pre-UTF-8 times? If so, they probably have the wrong byte(s) for the special (non-ASCII) characters. I don't know if the conversion to UTF-8 takes care of them. Can you tell if the characters in question are a single byte or multiple bytes? You might have to cut and paste them into a file and examine it under a hex editor, or an editor that can switch between UTF-8 and Latin-1. You might even be able just to display their name in your forum and see what switching your browser to Latin-1 does (View > Character Encoding)... do the special characters change to a stream of odd accented characters? If so, they are in UTF-8. If not, they never got converted to UTF-8, and may still be Latin-1 or even CP-1252 (originally cut and pasted from Word).
In your original post, you showed some text that was a string of ? and invalid character markers. It is not in UTF-8 and you are trying to display it in UTF-8. Is this text Arabic? Where did it come from -- language support file or a post?