Search function not working properly

[Ihmeliö]

The search function in our forum does not seem to function correctly. From what I gather, it doesn't search the topic subjects at all (not even with the "Search in topic subjects only" field ticked), so it's only possible to search from the topic contents.

Our forum is located at https://www.tietoiseen.fi/foorumi/
You have to accept the security sertificate
I created an account for you to be able to try out the search;
username: testuser
password: password

[Chas Large]

In Admin > Forum > Search-method > Settings, Click the Create Fulltext Search link, then change the Search Index to Full Index, finally click the Save button.

This will search both the Title and the Body of posts.

You can also change the weighting of the search method to make the search daemon look more towards the title then the body or vice versa.

[Ihmeliö]

There already was a fulltext index; I tried to enable it but the search still doesn't work; I removed it and created a new one, still no change.

I also tried to enable force, to no avail.

For example, I'm trying to search with the word "musiikkikappaleita", which is the title of a topic in the "Taide ja kulttuuri » Musiikki" subforum. "No results found"

[Chas Large]

Well I would have to say your search facility IS working. I just logged in with the credentials you gave earlier and did a search for the title of the 5th topic "Tiede". Attached are the results I obtained.

Is this not the problem you state in your original post?

Have you cleared your browser cache? sometimes that can help.

[Ihmeliö]

So apparently, the search does not search all the topic subjects - did you try searching for the word I referred to? It's not the only one that ends up with "no search results"...

[Chas Large]

I just did a search for "musiikkikappaleita" and the result was empty. Then I looked manually at all the topics on the 3 pages of

Tie² » Keskustelu » Taide ja kulttuuri » Musiikki

and could not find it.

Shortening the search to musiikk I found the attached results.

It would seem the Topic with the title containing "musiikkikappaleita" has been deleted or renamed, unless my eyes are deceiving me.

[Ihmeliö]

No, the topic hasn't been renamed, and for me shows correctly under the topic listing. It's the 13th topic at the time of writing:

edit: I also tried logging in with the testuser account, and also found the topic in the correct place, under Taide ja kulttuuri >> Musiikki.

[Chas Large]

Ah yes, so it is. As we say in the UK, "Should have gone to specsavers"

This may be an odd question, but have you tested this in the SMF default theme?

[Ihmeliö]

I tried the default theme. Doesn't work there, either.

[Chas Large]

I've done a little more careful searching and it is finding the topics before and after the "Yksittäisiä musiikkikappaleita" post in Musiikki, so it seems to be working.

Can you tell me what Weighting factors you have set. Admin > Forum > Search > Weights.

The Search daemon has some quirks but I've never seen it not find a topic amongst others that it can find. The only odd parameter about that topic is there are 217 replies whereas the other topics have 5 or less. The weighting factors could be causing this to be ignored. Try increasing the value of Relative search weight for topic length: or Relative search weight for age of last matching message:

[Ihmeliö]

I tried searching for other, longer topics, which the search for some reason did find.

The current values are:
Relative search weight for number of matching messages within a topic:  30%
Relative search weight for age of last matching message:  20%
Relative search weight for topic length:  15%
Relative search weight for a matching subject:  25%
Relative search weight for a first message match:  10%
Relative search weight for a sticky topic:  0%
(I didn't try changing them yet)

[Chas Large]

On the two main forums that I administer the weights are set to:

Relative search weight for number of matching messages within a topic:    30%
Relative search weight for age of last matching message:    25%
Relative search weight for topic length:    20%
Relative search weight for a matching subject:    15%
Relative search weight for a first message match:    10%
Relative search weight for a sticky topic:    0%

The search seems to work fine on both of them.

[Ihmeliö]

Tried to change that - still doesn't work...

[Ihmeliö]

Ah - the forum was converted from PHPBB - maybe this is important?

[Chas Large]

I suppose it's possible but I don't know what's involved in the conversion. If the topic that can't be found was originally on phpBB and the ones that can be found are not, it looks likely. You could try splitting it in half and seeing the newly created second half can be found. You can always merge them back together if not. Might be worth putting the forum in Maintenance Mode whilst you try that to prevent new posts being added.

[Ihmeliö]

I split the last post in the topic. Afterwards, the search did find the individual post I had split. After merging, the search found (only) the first post in the topic. So now the search "musiikkikappaleita" shows up the correct thread as a result.

So maybe it's really got something to do with the conversion. Could it be possible that the older posts are with a different character encoding, or smth like that?

[Chas Large]

It's possible but I don't know enough about the differences in character encoding to begin to tell you how to check it. What you could do is start a new topic with the same title and copying the contents of the first post in the old topic. Then split off all the remaining posts from the original and merge them with the new topic, finally deleting the original topic. This should then allow the search to find all the posts in the topic if required.

[Ihmeliö]

It's possible but I don't know enough about the differences in character encoding to begin to tell you how to check it. What you could do is start a new topic with the same title and copying the contents of the first post in the old topic. Then split off all the remaining posts from the original and merge them with the new topic, finally deleting the original topic. This should then allow the search to find all the posts in the topic if required.

With over a thousands topics, that's quite a lot of work to do by hand. Too much.

[Allusion]

What are the PHP and MySQL versions?

[Chas Large]

It's possible but I don't know enough about the differences in character encoding to begin to tell you how to check it. What you could do is start a new topic with the same title and copying the contents of the first post in the old topic. Then split off all the remaining posts from the original and merge them with the new topic, finally deleting the original topic. This should then allow the search to find all the posts in the topic if required.

With over a thousands topics, that's quite a lot of work to do by hand. Too much.

Sorry didn't realise the problem was with multiple topics. I suppose you could try changing the character set to a different format:
Admin > Maintenance > Database >  Convert to UTF-8
This may recode all the topics and enable them to be found in search. Do a full backup of the dB and Forum files before doing this so that you can restore it if it causes problems. If possible, you could set up a test version of the forum using a backup of the main forum dB and converting that to see if it works.

[Allusion]

Admin > Maintenance > Database >  Convert to UTF-8

The forum seems to be in UTF-8 already.

On the other hand, running the "Convert HTML-entities to UTF-8 characters" task in Admin > Maintenance > Database might help.

[Ihmeliö]

I believe I've found figured out the problem, explaining the seemingly random search inefficiencies: When using the search function from a certain area, it only searches from within that area - when searching from the front page, it searches everywhere.

Maybe this should be emphasized more, somehow somewhere?

[Deaks]

moikka Ihmeliö, do you still need help with this?