Search results limited to one page on 2.0.3

[Tomcraft]

Hi there,

the pagination on search page in my forum stopped working. It worked before without having changed anything except the latest update from 2.0.2 to 2.0.3 but I'm not sure when exactly it stopped working.

I found these two topics:

• Search results limited to one page on 2.0b3
• [4275]Search not showing up 2nd page

I haven't activated a search-index yet. Maybe there are to many topics on my forum for the search-pagination to work properly?

Installing "SMF Search Enhancement Mod" fixes the search-pagination without the need to activate the search-index, but I'm not sure if I'm willing to have this mod installed. I guess the search-pagination should work without the need of any external mods.

I started playing around with the search-index in a testing environment, but since I have loads of topics and posts on my forums, the indexing lasts hours...

Any help is appreciated

Regards, Torsten

[Arantor]

Define 'stopped working'. Do you not get pagination? Do you get multiple pages that don't work? Do you get a hard limit on the number of results back?

[Tomcraft]

Ah sorry. The pagination in other forum sections works fine! Only the pagination using the search doesn't work like it should.

I have set the limit for the search results to 1200 and 30 results per page.

Regards, Torsten

[Arantor]

That wasn't what I asked. You've said twice that it 'doesn't work like it should' and yet you've not explained what it isn't doing that it should be.

[Tomcraft]

Haven't I?

When using the search for example to look up the word "test" I get 30 results and only 1 page in the pagination (see screenshot). There should be about 40 pages using the search term "test" on my forums.

Funny thing is that when changig the last parameter (;start=40) in the URL I get different search-results.

To test under controlled conditions I always use the same search term "test".

I wonder why the is only one page n the pagination?

Regards, Torsten

[Arantor]

Haven't I?

Nope, you've just said 'it doesn't work' and 'it doesn't work like it should'. That's the equivalent of going to the doctor and saying 'I'm ill', doesn't provide any symptoms or anything.

When using the search for example to look up the word "test" I get 30 results and only 1 page in the pagination (see screenshot). There should be about 40 pages using the search term "test" on my forums.

Ah, symptoms and details!

;start=40... for 30 per page, that's a little unusual. What happens if you use start=30 (which should be page 2)

It sort of sounds like it's using an incomplete index. Did you let it finish building the index, and then tell it to use it? Or have you since told it to use a different index?

[Tomcraft]

[...]

Haven't I?

Nope, you've just said 'it doesn't work' and 'it doesn't work like it should'. That's the equivalent of going to the doctor and saying 'I'm ill', doesn't provide any symptoms or anything.

When using the search for example to look up the word "test" I get 30 results and only 1 page in the pagination (see screenshot). There should be about 40 pages using the search term "test" on my forums.

Ah, symptoms and details!
[...]

Sorry... I thought i would be specific enough to say search pagination doesn't work as "pagination" is self explaining. :-)

[...]
;start=40... for 30 per page, that's a little unusual. What happens if you use start=30 (which should be page 2)
[...]

This was just a test... I also could have added start=188 to see if I get more and other search results as seen on page 1.

[...]
It sort of sounds like it's using an incomplete index. Did you let it finish building the index, and then tell it to use it? Or have you since told it to use a different index?

Shouldn't it also work without having the search index function enabled?
I used the two different index functions and it work some time sporadic but I couldn't find what was the critical switch I enabled for the search pagination to work again. So I recover my backup in the testing environment and start again.

Regards, Torsten

[Arantor]

I thought i would be specific enough to say search pagination doesn't work

Which is why I asked my first questions. There are various ways in which it 'may not work' and I wanted to know what.

This was just a test... I also could have added start=188 to see if I get more and other search results as seen on page 1.

Yes, I realise that. I was trying to help narrow down something important, but hey, it won't make any difference if it's just a test, right? There is a reason I asked specifically for that. The code reacts differently depending on whether you're on a page boundary vs not being on a page boundary (i.e. 30 for 30 per page)

Shouldn't it also work without having the search index function enabled?

-sigh- You came here asking for information. I'm trying to find out exactly what's going on using my knowledge of how SMF works. Questioning my logic isn't going to get you a solution any faster, but since I apparently need to explain my diagnosis logic to you as well...

You're seeing 30 results when you should be seeing hundreds more. This suggests to me that it's not going through the entire set of posts for some reason. Thus, the theory that you've said you started to build a custom index but given how long it took, it may not be a complete index and thus you might not have all the posts being found.

Do you have any boards set to ignore?

[Tomcraft]

[...] The code reacts differently depending on whether you're on a page boundary vs not being on a page boundary (i.e. 30 for 30 per page)
[...]

Dunno exactly what you mean. As I said the should be about 40 pages haven 30 results each as I remember when the search worked fine.

[...]

Shouldn't it also work without having the search index function enabled?

-sigh- You came here asking for information. I'm trying to find out exactly what's going on using my knowledge of how SMF works. Questioning my logic isn't going to get you a solution any faster, but since I apparently need to explain my diagnosis logic to you as well...
[...]

Sorry... I answered your questions later after I edited my last post. I was very surpised how fast you answer.

[...]
You're seeing 30 results when you should be seeing hundreds more. This suggests to me that it's not going through the entire set of posts for some reason. Thus, the theory that you've said you started to build a custom index but given how long it took, it may not be a complete index and thus you might not have all the posts being found.
[...]

Well... I can test that by counting the entries containing the search term "test" in the database table for the custom index.

[...]
Do you have any boards set to ignore?

No, I didn't even know I can ignore some boards from the search.

As far as I remember I got the search pagination only to work in my testing environment with the custom index which is not recommended as far as I can interpret the postings in the SMF forum.

I currently create a new custom large index and see if it works again.

Regards, Torsten

[Arantor]

No, I didn't even know I can ignore some boards from the search.

Actually, there are two different ways. You can do so by default by selecting boards in your profile to be ignored. For example here, I have all the non English boards set to ignore, and they're all excluded from search by default because of that. Then there's the ability to set which boards you're actually searching from in the advanced search itself.

As far as I remember I got the search pagination only to work in my testing environment with the custom index which is not recommended as far as I can interpret the postings in the SMF forum

On the contrary, it is recommended for most forums from a performance point of view, and only recommended not to use it if you have Sphinx available. It outperforms MySQL's fulltext index quite a lot, actually.

[Tomcraft]

No, I didn't even know I can ignore some boards from the search.

Actually, there are two different ways. You can do so by default by selecting boards in your profile to be ignored. For example here, I have all the non English boards set to ignore, and they're all excluded from search by default because of that. Then there's the ability to set which boards you're actually searching from in the advanced search itself.
[...]

Ah okay... I have not ignored boards in my profile and I always have all boards marked when searching them.

[...]

As far as I remember I got the search pagination only to work in my testing environment with the custom index which is not recommended as far as I can interpret the postings in the SMF forum

On the contrary, it is recommended for most forums from a performance point of view, and only recommended not to use it if you have Sphinx available. It outperforms MySQL's fulltext index quite a lot, actually.

There was a hint saying that for example using the search term "test" would not show results containing "tests" when using a custom index.

The main question for me remains: Why does the search pagination suddenly stop working? Is there a hard limit when the database reaches a certain size?

Why does the fulltext index not work for me? I only got the pagination back twice using a large custom index or without any index when installing the mod "SMF Search Enhancement Mod". But I don't want to use this mod to have a basic function working again since I believe this has to work without this additional mod.

Regards, Torsten

[Arantor]

Why does the search pagination suddenly stop working? Is there a hard limit when the database reaches a certain size?

There isn't, to my knowledge, but you've not given me a lot to work on.

Why does the fulltext index not work for me?

Fulltext is actually kind of buggy but it's a MySQL construction, not an SMF one; the query is handed over entirely to MySQL to deal with.

It's kind of hard to know what to say because it's not clear what it might be causing, but I'm not entirely what questions to ask next. See how the custom index works for you.

[Tomcraft]

Okay... seems like I had a curious caching problem. After activating the fulltext index I needed to use a different search term than "test". After using a different search term like "help", the pagination starts working fine with the search term "test".

Regards, Torsten

[Storebror]

Glad that you got that issue sorted Tomcraft, but unfortunately I have to second that and to me the issue persists no matter what I do.
I've tried search function with and without index, i.e. Search Method "No Index" and Search Method "Custom Index" with a fully built large custom index.
I've tried with enabled and disabled caching.
I've tried a whole lot of different search terms (test, help, hello, admin etc.).
Our board contains nearly 200.000 messages.
Whatever I do, I always get back the number of results configured in "Number of search results per page", but the resulting page shows just one page in the pagination list, further result pages are missing.
E.g. I configure "Number of search results per page = 20" and "Maximum results to show = 500" I'll get back a single page with 20 results, no further page links.
Same thing if I configure "Number of search results per page = 100" and "Maximum results to show = 500" I'll get back a single page with 100 results, no further page links.

I tried both the default "Curve" theme and the "Core" theme, both to no avail.

As soon as I tick "Search in topic subjects only" everything is back normal and I get multiple result pages, but if I leave that field unticked, only one page of results is returned.

Any help would be much appreciated!

Best regards - Mike

[Tomcraft]

Did you try this?

[...] After activating the fulltext index I needed to use a different search term than "test". After using a different search term like "help", the pagination starts working fine with the search term "test".
[...]

Regards, Torsten

[Storebror]

Hi Torsten,

Thank you very much for your reply.
In fact I haven't tried the fulltext index yet since as much as I got from SMF expert's explanations the fulltext index relies on MySQL indexing and is supposed to be "buggy" (for whatever that means).
I tried "No index" and "Custom Index" (large type for the latter).
For these two I tried different search terms many many times but each to no avail.

I've seen this being filed as a bug report for SMF 2.1 project by developer "Spuds" here:
http://dev.simplemachines.org/mantis/view.php?id=4945

According to that bug track it should have been resolved, but I get that it has been resolved for 2.1 only.
However since almost a year passed since this bug has been resolved for 2.1, I wonder whether there might be an official fix available for the current release 2.0.3 as well, particularly since the search function is really something essential for a well populated board.

Best regards - Mike

[cpnsaiful]

It is right Ur self way.

[Storebror]

It is right Ur self way.

Sorry but I didn't get what you wanna tell me/us with that post.

Best regards - Mike

[Storebror]

*bump*
Just to let anyone know, I think I managed to fix this by myself by taking the details listed in the bug report at
http://dev.simplemachines.org/mantis/view.php?id=4945
together with latest 2.1 release sources from GitHub and comparing these to the 2.0.3 sources.
I eventually changes line 1670 in Sources/Search.php from

Code Select Expand

else
to

Code Select Expand

elseif ($_SESSION['search_cache']['num_results'] == -1)

The only annoyance remaining is that anyone who used the search function before applying that fix will have to log out and re-login in order to get the session's search cache cleared, otherwise when searching for a term which has been in the cache before and returned the single page issue, this issue would persist for that term.

Best regards - Mike

[Tomcraft]

Will this be fixed in 2.0.4?

Regards, Torsten

[Storebror]

From the bug report I get that it will be fixed in 2.1.

Best regards - Mike

[Arantor]

That would be an interesting conclusion, since bug 4275 was marked as solved nearly 3 years ago...