Major bug in SMF 1.1.3 Search function!

[ioszilla]

The problem was mentioned in this thread: Search only works after "Clear Logs"

And Eleglin's forum: http://forums.archivesdegondor.net/index.php, on which I've been working to find a fix, was also deeply affected by this bug.

This bug exists (at least) in the *official* SMF 1.1.3 search function, and will affect all SMF 1.1.3 forum owners without MySQL privilege to CREATE TEMPORARY TABLES.

This bug is most likely to immensely disrupt the search function on affected forums - by injecting totally unrelated matches (up to 254 of them) into search results -- regardless of search configurations in Admin panel.



How to fix

Edit Sources/Search.php

Find (line #1351 - 1368):

            db_query("
               INSERT IGNORE INTO {$db_prefix}log_search_results
                  (ID_SEARCH, ID_TOPIC, relevance, ID_MSG, num_matches)
               SELECT
                  " . $_SESSION['search_cache']['ID_SEARCH'] . ",
                  t.ID_TOPIC,
                  1000 * (
                     $weight[frequency] / (t.numReplies + 1) +
                     $weight[age] * IF(t.ID_FIRST_MSG < $minMsg, 0, (t.ID_FIRST_MSG - $minMsg) / $recentMsg) +
                     $weight[length] * IF(t.numReplies < $humungousTopicPosts, t.numReplies / $humungousTopicPosts, 1) +
                     $weight[subject] +
                     $weight[sticky] * t.isSticky
                  ) / $weight_total AS relevance,
                  t.ID_FIRST_MSG,
                  1
               FROM ({$db_prefix}topics AS t, {$db_prefix}" . ($createTemporary ? 'tmp_' : '') . "log_search_topics AS lst)
               WHERE lst.ID_TOPIC = t.ID_TOPIC" . (empty($modSettings['search_max_results']) ? '' : "
               LIMIT " . ($modSettings['search_max_results'] - $_SESSION['search_cache']['num_results'])), __FILE__, __LINE__);

Replace with:

            db_query("
               INSERT IGNORE INTO {$db_prefix}log_search_results
                  (ID_SEARCH, ID_TOPIC, relevance, ID_MSG, num_matches)
               SELECT
                  " . $_SESSION['search_cache']['ID_SEARCH'] . ",
                  t.ID_TOPIC,
                  1000 * (
                     $weight[frequency] / (t.numReplies + 1) +
                     $weight[age] * IF(t.ID_FIRST_MSG < $minMsg, 0, (t.ID_FIRST_MSG - $minMsg) / $recentMsg) +
                     $weight[length] * IF(t.numReplies < $humungousTopicPosts, t.numReplies / $humungousTopicPosts, 1) +
                     $weight[subject] +
                     $weight[sticky] * t.isSticky
                  ) / $weight_total AS relevance,
                  t.ID_FIRST_MSG,
                  1
               FROM ({$db_prefix}topics AS t, {$db_prefix}" . ($createTemporary ? 'tmp_' : '') . "log_search_topics AS lst)
               WHERE " . (!$createTemporary ? 'lst.ID_SEARCH = ' . $_SESSION['search_cache']['ID_SEARCH'] . ' AND ': '') . 'lst.ID_TOPIC = t.ID_TOPIC' . (empty($modSettings['search_max_results']) ? '' : "
               LIMIT " . ($modSettings['search_max_results'] - $_SESSION['search_cache']['num_results'])), __FILE__, __LINE__);

Explanation

WHERE lst.ID_TOPIC = t.ID_TOPIC" . (empty($modSettings['search_max_results']) ? '' : "

really should have been

WHERE " . (!$createTemporary ? 'lst.ID_SEARCH = ' . $_SESSION['search_cache']['ID_SEARCH'] . ' AND ': '') . 'lst.ID_TOPIC = t.ID_TOPIC' . (empty($modSettings['search_max_results']) ? '' : "

If smf search is not able to create any temporary mysql table on the server, it will have to insert temporary (subject matches) results into and read from {$db_prefix}log_search_topics table (instead of {$db_prefix}_tmp_log_search_topics).

log_search_topics and tmp_log_search_topics table have different structures. tmp_log_search_topics is a memory heap table containing only ID_TOPIC column and is created/destroyed just for the current search - the table is temporary and does not keep results from previous searches.

However log_search_topics table *does* keep results from previous searches, there are two columns: ID_SEARCH, ID_TOPIC. Every time there's a new search operation, a new ID_SEARCH ($_SESSION['search_cache']['ID_SEARCH']) is assigned to uniquely identify temporary results generated for the current search.


As log_search_topics table (which keeps previous search results) is used, the line below from official SMF 1.1.3 Search.php would fail to restrict results to current search session:

WHERE lst.ID_TOPIC = t.ID_TOPIC" . (empty($modSettings['search_max_results']) ? '' : "

Now without the lst.ID_TOPIC = t.ID_TOPIC condition, the search will indiscriminately fetch all previous (unrelated) subject matches and insert them to current search results -- as long as there's one subject match for your search ($numSubjectResults !== 0) !


If your forum is affected by this bug, follow my "How to fix" instructions above or wait for an SMF official patch. 


p.s. I have also updated my smf search enhancement mod: http://custom.simplemachines.org/mods/index.php?mod=887 to patch this.

[nitins60]

Thanks for Bringing it up

[mkress]

Edit Sources/Search.php

Find (line #1896 - 1913):

i think that are the wrong lines - because standard soruces/search.php has only 1769 lines :-)

should be around line 1347 and following...


thank you for that fix!!!!

mfg
Markus

[ioszilla]

i think that are the wrong lines - because standard soruces/search.php has only 1769 lines :-)

should be around line 1347 and following...

thank you for that fix!!!!

mfg
Markus

Fixed. Thanks!! I'm glad it's working for you.

[Eleglin]

I confirm this bug was crippling the function search on my forum. Before the modification of wordzilla, there were many irrelevant results. Since i have installed this modification, there are no more errors.

This bug doesn't appear when I was on SMF 1.0.x and there on SMF 1.1 RC3. Many tests suggest that it appears since the function for results with subject-only matches has been "partially" repaired on SMF 1.1. final

So I thank Wordzilla for his help ! 

[ioszilla]

This is how SMF 1.1 RC3 works:

IF (found matches in message body)
         Insert body matches into temporary results table
ELSEIF (found matches in message subject)
         Insert subject-only matches into temporary results table
ELSE
         Report no results

Naturally in 1.1 RC3 whenever there are message body matches for your search, all subject-only matches will be ignored.


Luckily SMF 1.1.3 final fixed this control flow joke:

IF (found matches in message body)
         Insert body matches into temporary results table

IF (found matches in message subject)
         Insert subject-only matches into temporary results table
ELSE
         Report no results

And now in 1.1.3 final the problem emerges because in most cases the search function will have to insert subject-only matches into the temporary results table.


SMF 1.1 RC3 only "fixed" the problem because there was another major bug before the above-mentioned bug, and prevented it from raging in its full fury.

[Jorin]

Nice work! Will try this the next days on our forum.

Do you have contact with the team to correct the wrong code/function in the next release of the smf software?

[ioszilla]

Yup, Oldiesmann submitted a bug report on it I think.

[Jorin]

Great, thanks!

[bitmovel]

Looks like this bug is back in SMF 2.0

check my thread

http://www.simplemachines.org/community/index.php?topic=263564.0

they pointed me here, but seems like the same problem

Any update on how to fix this in SMF 2.0 b4?