Faster SMF BBCode Parser

[dougiefresh]

Link to Mod

FASTER SMF PARSER v2.0
By Dougiefresh -> Link to Mod

Introduction
The SMF parser in the SMF 2.0.x line, as well as the SMF 2.1 Beta 1 & 2, have a potentially "fatal flaw", at least where it concerns bbcodes with a large number of parameters.  The original SMF code generates permutations of all possible combinations of the parameters in order to make it easier for the user to use bbcodes, and then checks to see which permutation is the right one.  A bbcode with 14 parameters generates 98,306 permutations, taking little more than half a second on my system PER USE and using 90MB of memory!!  Checking for 15 parameters crashes my forum with an out-of-memory error because it attempts to allocate more than 128MB of memory!

SMF 2.1 Beta 2 Nightly (as of September 4th, 2015) takes a different approach to the "fatal flaw", avoiding the use of a permutation array prior to processing the parameter permutations.  This avoids the possible of a forum crash, plus potentially allows for more parameters to be processed without having to rearrange parameters.

Please note that both this mod and SMF 2.1 Beta 2 nightly leaves the permute function in place, as other mods or future forum versions may need the built-in functionality.

Related Discussion
o REPORT: Potentially "Fatal Flaw" in SMF's bbcode parser

Admin Settings
There are no admin settings to this mod.  To disable, you must uninstall this mod.

Compatibility Notes
This mod was tested on SMF 2.0.10, but should work on SMF 2.0 and up.  SMF 1.x is not and will not be supported.

Changelog
The changelog has been removed and can be seen at XPtsp.com.

License
Copy of SMF 2.1 Beta 2's License as copied from the LICENSE file in that install:

Copyright © 2011 Simple Machines. All rights reserved.

Developed by: Simple Machines Forum Project
Simple Machines
http://www.simplemachines.org

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

* Neither the name of SMF2.1 nor the names of its
  contributors may be used to endorse or promote products derived from
  this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

[dougiefresh]

Uploaded v1.1 - August 19th, 2015
o Modifications made to parse quoted stuff better.

[Biology Forums]

@dougiefresh

Would this also apply to SMF 1.x?

[dougiefresh]

Would this also apply to SMF 1.x?

The portion of the code being replaced in Sources/Subs.php looks the same as v2.0.x.  I would imagine that if you cut and pasted the mod's contents into the SMF 1.1.21 copy of Sources/Subs.php, that it might work....

HOWEVER, I tried to modify the mod to work on SMF 1.1.21, then tried to install it on a clean copy of SMF 1.1.21.  SMF 1.1.21 didn't recognize the mod as a mod package....  So I don't know how to write the package-info.xml for SMF 1.1.21....  I'm not sure that I care to figure it out either.  SMF 1.1.21 is no longer on the SMF main download page, so I'm not sure if that means that it is being discontinued, phased out, or what....

[Biology Forums]

Agreed, I will do it manually.

Thanks for the reply.

[villasg]

Hi @dougiefresh i am again .
I try to install this mod but receive error in second edit [ replace] in Subs.php .
My subs.php are :

Code Select Expand


$preg = array();
foreach ($possible['parameters'] as $p => $info)
$preg[] = '(\s+' . $p . '=' . (empty($info['quoted']) ? '' : '"') . (isset($info['match']) ? $info['match'] : '(.+?)') . (empty($info['quoted']) ? '' : '"') . ')' . (empty($info['optional']) ? '' : '?');

// Okay, this may look ugly and it is, but it's not going to happen much and it is the best way of allowing any order of parameters but still parsing them right.
$match = false;
if (in_array($possible['tag'], array('imgleft', 'imgright')))
$orders = array(BBCode_Float_Params($message, $pos1, $possible['parameters']) ? $preg : array());
else
$orders = permute($preg);
foreach ($orders as $p)
if (preg_match('~^' . implode('', $p) . '\]~i', substr($message, $pos1 - 1), $matches) != 0)
{
$match = true;
break;
}

// Didn't match our parameter list, try the next possible.
if (!$match)
continue;


How can i fix it ?
I think to remove "Image Floating to Left or Right BBCode" mod , install the "Faster SMF BBCode Parser" mod  and after install again the "Image Floating to Left or Right BBCode" mod ? ?
Its ok this solution ?
Thanks

[dougiefresh]

Uploaded v2.0 - September 7th, 2015
o Replaced custom code with SMF 2.1 Beta 2 Nightly's code (Sept 4th, 2015)



Just replace the whole thing with the new code.  The new code in this mod essentially removes the need for the other stuff....  This applies to both v1.x and v2.0 of this mod....

[DSystem]

OLA

Would it be possible to let this mod compatible with "Post and PM Inline Attachments" ?

Updade Subs.php error

Code Select Expand

if (in_array($possible['tag'], ILA_tags()))
$orders = array($preg);
else


[villasg]

I think not seen my last question ..

Hi @dougiefresh i am again .
.....................
Thanks

[dougiefresh]

@DSystem: This mod fixes this issue that caused the need for that particular change in my other mods.  In the latest version, that particular operation is optional and not required if installed AFTER this mod.  If you install this mod after any of my mods having that particular operation, you will have to manually replace the code with the "replace" search spec in this mod before attempting installation.....  I hope this helps you to understand....

Just replace the whole thing with the new code.  The new code in this mod essentially removes the need for the other stuff....  This applies to both v1.x and v2.0 of this mod....

@villasg: This comment was meant to be directed at you, villasg.

[Jade Elizabeth]

Just curious how likely it would be that this would be causing people a lot of issues? On both of my forums BBC is used quite often, and I use it quite....insanely sometimes...so I am curious because I don't think I've encountered a problem before now is all.

Can you post some BBC that's likely to cause the issue (in code tags or through pastebin of course!) or explain it in dummy terms?

[margarett]

@Jade: the problem exists when you create a new BBC which accepts a lot of parameters. SMF has none of those so if you're just using standard BBCs you should be fine

[dougiefresh]

The problem exists in bbcode mods that support a large number of parameters, such as:
> Image Floating to Left or Right BBCode
> Images with Margins and Borders
> Post and PM Inline Attachments
> Visible, Invisible and NoQuote Tags
> Yet Another YouTube BBCode Tag

All of the mentioned mods have been "patched" specifically to avoid the forum crashing permute function that is found in the parser without the SMF 2.1 Beta 2 code (found in v2.0 of this mod, which was taken from the SMF 2.1 Beta 2 copy found on GitHub!)  This "patch" consists of code that reorganizes the parameters to be in the expected order so that generating all permutations of the parameters isn't necessary.

It should be also noted that neither the GitHub version of SMF 2.1 Beta 2 nor v2.0 of this mod will experience this particular issue because of the way that the permute/testing code has been altered.....

Side note: Yes, the listed mods that are affected are mine....  Other mods may be affected, but I'm not gonna look at every mod to determine that.  This topic discusses the issue(s) at length.... and my proposed solutions (which are present in the v1.x of this mod).

[Jade Elizabeth]

@Jade: the problem exists when you create a new BBC which accepts a lot of parameters. SMF has none of those so if you're just using standard BBCs you should be fine

Ohhhh!! Thank you I was so confused. Maybe Dougie you should include this in your description, and put a note that BBC mods should consider using it if they have many parameters .

Thanks both of you for explaining. You've done some really awesome work here Dougie!! Keep up the good stuff!!!

[dougiefresh]

Thanks both of you for explaining. You've done some really awesome work here Dougie!! Keep up the good stuff!!!

I can take full ownership of the version 1.x mods, however, (as stated) v2.x comes from SMF 2.1 Beta 2 from GitHub.  All I did was repackage the relevant changes (which weren't many) and upload it as a mod.  I hope this helps people....

Did I forget to thank the SMF development team?  If so, THANK YOU, SMF DEVELOPMENT TEAM!!!!

[dougiefresh]

Uploaded v3.0 - August 28th, 2017
o Updated mod to use code from SMF 2.1 Beta 3.

[villasg]

After update from v2.0 to 3.0 have problem in my forum .
With the version 2 not have this problem .
Thanks

Look the images
Before


After



Before


After



Error


subs.php
2135 // Okay, this may look ugly and it is, but it's not going to happen much and it is the best way of allowing any order of parameters but still parsing them right.
2136 $match = false;

[dougiefresh]

I'll take a look at beta 3 and see what I missed.  Obviously I missed something....

[dougiefresh]

I've pulled version 3.0.  The changes in 2.1 Beta 3 run much deeper than just a few lines of code and I just don't feel like backporting a whole function.  If you want an even faster parser than what came with the GitHub 2.1 Beta 2, you need to upgrade to 2.1 Beta 3 or better.  Sorry....