Search issue - doesn't search my name!

[Adish - (F.L.A.M.E.R)]

If you search my name via the search function:
F.L.A.M.E.R
"F.L.A.M.E.R"
(F.L.A.M.E.R)
"(F.L.A.M.E.R)"

no results are given out. I havent got a chance to test it out on a virgin 2.x or even 1.x but this forum (sm.org) having my name in many boards does not return with any results.

I'll have a test if I can get hold of a computer anytime soon and post up results if anyone else doesnt test it out before me.

[Illori]

seems the search function also produces no results when searching for something like i. which could be the issue (F.L.A.M.E.R) is seeing.

[Adish - (F.L.A.M.E.R)]

I scared the search engine... Boo! Muahaha

[Arantor]

Before we can begin to debug this, let's take stock of a few things.

Firstly, I can guarantee your local install will perform differently to here - this server uses Sphinx. Sphinx tokenises words (building hashes out of word units) and IIRC (though SleePy can confirm), the . character is not considered a word character, so it's only interested in the individual letters. And, again IIRC, the minimum word length is 3 (certainly that's the Sphinx default), which means it won't even have indexed your name.

Same deal for & *here*. Your mileage will vary in other systems seeing how there are 3 different ways that SMF will perform a search depending on backend, and what characters are being stripped first.

[Aleksi "Lex" Kilpinen]

The search on sm.org has never been any good with any special chars. Stick to basic alphabets and your golden though

[Arantor]

It's configurable though, as part of the Sphinx configuration, you give it a list of characters it should treat as parts of words. Trouble is, if you add something like ; to the list, it becomes part of a word, so $variable = $othervariable; - the ; is part of the 'word' othervariable.

[Joshua Dickerson]

Common English punctuation shouldn't be part of the word.

Related to: http://www.simplemachines.org/community/index.php?topic=409650

[Arantor]

Common English punctuation shouldn't be part of the word.

Correct, but that's why it ignores the . in F.L.A.M.E.R. but on a technical forum wouldn't you want to consider punctuation as well, possibly?

In any case you can't tell Sphinx to differentiate between when it should, and when it shouldn't consider characters as punctuation, they're either parts of what Sphinx considers a wordunit or they're not, simple as that.

[SlammedDime]

The Sphinx Search API we use here strips all non-alphanumeric characters from the search string and replaces them with empty spaces and then searches on that.  In the case of your name, "(F.L.A.M.E.R)" becomes "F L A M E R", and since our minimum word length is set to '3' in the sphinx config file, your name will never show up when searched for...

Change your name, it's better that way for everyone...

[Arantor]

...I thought I already explained this a few posts up? (though I won't deny you explained it a little more simply than I did... and I think you'll find certain characters are still indexed, that it's not just strictly alphanumeric)

[SlammedDime]

...I thought I already explained this a few posts up? (though I won't deny you explained it a little more simply than I did... and I think you'll find certain characters are still indexed, that it's not just strictly alphanumeric)

Your post was relating to how Sphinx indexed the words, I was pointing to how the search string is 'reformatted' before being sent to Sphinx.  Here on SM.org, it doesn't matter if you try searching for "!#)*#(#HI!@#(*(*#$", or "...HI...", Sphinx will be sent "HI" either way.

[Arantor]

Funny, it certainly used to bother it which is why there were all the 'unable to access the search daemon' issues which were Sphinx-side, not SMF side...

[SlammedDime]

When I rewrote the API recently to use SphinxQL, I modified how search strings were formed.  Instead of relying on what SMF was passing in to the API, I took the raw search string and cut out all the non-alphanumerics and used that instead (you have access to big boards, take a gander at the file).

[Arantor]

Oh, so it's using the new API here? Interesting.

What's more interesting is that your API is nice enough to pass true alphanumeric characters to Sphinx, looking at the regexp you put in, but that the default configuration file that's built just discards anything that isn't 0..9, A..Z, a..z or _. (I know the configuration here has a much more thorough charset_table though)

[SlammedDime]

Yea, our charset_table is very verbose here, meant to encompass as many utf8 characters as possible (which I had to take into account when doing the regex).  The default config file though shouldn't be construed to be an 'end all' solution though, it forms the basis to get sphinx up and running and operating with SMF... as with anything that can serve a wide range of audiences, some fine tuning is necessary (although I'm sure SM.org could share it's finer details of sphinx.conf and maybe we could create a more robust solution).

[Arantor]

There's not really a lot of need to share, it's basically just a dump of all the characters listed on the Sphinx wiki for all the different regions.

[vbgamer45]

Not sure how big of issue this is. Seems like all major search engines have the same issue. Tried searching his name or any name with different  punctuation  on google and it fails as well.

[Adish - (F.L.A.M.E.R)]

When we use "F.L.A.M.E.R" or "(F.L.A.M.E.R)", shouldn't it mean that you need the the word being searched to be exactly the same? Whereas in F.L.A.M.E.R or (F.L.A.M.E.R), they are considered to be as full stops?

Also, I think it is a better idea to treat F.L.A.M.E.R as FLAMER instead of F L A M E R, that is what all the search engines do.

As a note: We can search (F.L.A.M.E.R) or F.L.A.M.E.R without a warning saying you need at least 3 characters for the search but gives no results. So it does consider it in some other way..

The funny thing: Members search gives my name as the result for both (F.L.A.M.E.R) and F.L.A.M.E.R

[coloradorockies]

Your name so popular (About 4,530,000 results ) next time try more unique and you will success with search result. 
service 

[SlammedDime]

LOL... that just screams "I'm a bot, ban me now!!!!!"