Limit The Number of BBC Codes Per Post

[FrizzleFried]

I really need a way to limit the number of BBC codes per post in SMF.   I am running in to WHITE SCREENS OF DEATH on posts because people are using too many BBC codes in a thread.  It appears when I get to about 100 per thread-page,  it white screens the page.   The only way I can get the thread back is to go in to the database and remove a number of BBC codes.

I've asked this a couple years back and there was no "answer"... has anyone come up with a way since then to limit the number of BBC codes per post?

It seems to have gotten worse since my upgrade to 2.0.7.

Thanks!

[Arantor]

It will have gotten worse since 2.0.7.

Predominantly the problem is running out of memory allocation. Though it seems to me that if the parser is leaking memory there are bigger problems at work. I know there is a patch to fix the issues in 2.0.7 currently in testing because against my better judgement, I prepared it.

The real problem is that there are too few people who understand the bbc parser enough to rewrite it without breaking backwards compatibility or introducing new bugs or security issues.

[FrizzleFried]

I'll volunteer to test the fix as well if you're willing...

[Arantor]

I would prefer the team posts they don't have a problem with it before I post it, as provisionally it would become SMF 2.0.8.

[FrizzleFried]

...waiting with bated breath...

...and if you added a way to limit the number of BBC codes per post... I just might send you flowers.

[Arantor]

I see that as, at best, fixing a symptom of a problem, not solving the problem itself.

[Chen Zhen]

FrizzleFried,

  I understand too many anonymous functions being executed in succession within a callback loop is causing the issue. Is this issue occurring within a topic with X number of posts (that contain BBCs and smileys)? If so perhaps lessening the amount of posts shown per topic will ease the existing memory consumption.
( Admin -> Posts and Topics -> Topic Settings -> Number of posts per page in a topic page )

[Arantor]

The problem is that they're not anonymous functions. If they were anonymous functions there wouldn't actually be a problem.

[Chen Zhen]

ref. http://ca3.php.net/create_function

create_function is an anonymous function whereas the newer implementation (PHP 5.3.0+) is called a native anonymous function (a.k.a. closures).

[Arantor]

That's just it, they're not actually anonymous. Each function is a very real function with a name. Functionally, yes, it's an anonymous function but in actuality it's not. This is how come every time you call it, you end up with a new function, rather than re-referencing the existing function (which is what closures do)

Any time you run create_function a new function is created and the string returned is the name of that function. How can something that has a name be anonymous?

For those curious, the return value is a string that begins with the character 0 (not the digit 0 but the byte value 0) followed by 'lambda_' and then a number for uniqueness, and these functions explicitly do not appear in get_defined_functions().

[Chen Zhen]

I was just saying that my previous statement was using the language from the books so to speak. Imo you know what I was referring to as we had a conversation about this issue in another thread several months ago.  Yes I understand your plight of the designation the developers opted for it even though it returns a function name & I suppose its anonymous during its onset is the reasoning behind it. That is what we are given as reference whether one likes it or not.

 

[Dragooon]

I don't think the create_function calls are the problems? The whole thing is in a static array so they are not repeatedly instantiated. The array takes about 134KB of memory on a fresh SMF 2.1 install which remains about constant.

[Chen Zhen]

You want to do a true test?

Set up 2 local SMF installations (installations must use an updated PHP version). Keep one as the default and on the other replace all the create_function callbacks related to smileys and BBCs with closures. Create some posts with 100 BBC's and 100 smileys and run your memory-ticker on those for comparison. 

[Dragooon]

You want to do a true test?

Set up 2 local SMF installations (installations must use an updated PHP version). Keep one as the default and on the other replace all the create_function callbacks related to smileys and BBCs with closures. Create some posts with 100 BBC's and 100 smileys and run your memory-ticker on those for comparison. 

And that didn't do jack . Use this, replace the entire $codes array in Subs.php with:

Code Select Expand

$codes = array(
array(
'tag' => 'url',
'type' => 'unparsed_content',
'content' => '<a href="$1" class="bbc_link" target="_blank">$1</a>',
'validate' => function(&$tag, &$data, $disabled)
{
$data = strtr($data, array('<br>' => ''));
if (strpos($data, 'http://') !== 0 && strpos($data, 'https://') !== 0)
$data = 'http://' . $data;
},
),
array(
'tag' => 'url',
'type' => 'unparsed_equals',
'before' => '<a href="$1" class="bbc_link" target="_blank">',
'after' => '</a>',
'validate' => function(&$tag, &$data, $disabled)
{
if (strpos($data, 'http://') !== 0 && strpos($data, 'https://') !== 0)
$data = 'http://' . $data;
},
'disallow_children' => array('email', 'ftp', 'url', 'iurl'),
'disabled_after' => ' ($1)',
),
);
And create a post with hundred or so URL tags. Doesn't make a difference in memory usage.

[Arantor]

And of course I didn't already do all that when doing the patch I referred to? Give me a little bit of credit.

Also, no, it's not the big array that's the problem. It's other places where create_function gets used in a loop, each iteration declaring a new function.

[Dragooon]

And of course I didn't already do all that when doing the patch I referred to? Give me a little bit of credit.

Also, no, it's not the big array that's the problem. It's other places where create_function gets used in a loop, each iteration declaring a new function.

But wouldn't using closures solve the problem if that was the case?

[Arantor]

If you were putting them in the right place, it might. You're right, that big-ass array is kept static, meaning that new functions aren't declared every iteration.

Unfortunately there are plenty of places create_function is called inside a loop where it will spew new functions every iteration.

[Dragooon]

If you were putting them in the right place, it might. You're right, that big-ass array is kept static, meaning that new functions aren't declared every iteration.

Unfortunately there are plenty of places create_function is called inside a loop where it will spew new functions every iteration.

Sure but that doesn't help with parse_bbc's case. If replacing that big ass array with closures doesn't help, then it's leaking memory somewhere else (since it only returns the parsed body)

[Arantor]

There are other places that leak inside parse_bbc especially given the number of times per page it can be run. For example the long word break code requires such a function.

But yes, the function leaks memory. I patched the worst of it in the prospective 2.0.8 patch but I can't figure out where the worst leaks are coming from yet. Even profiling hasn't really helped.

[Dragooon]

Can...someone help me test this?

Open Subs.php, find the function pregReplacementCurry and replace it with this:

Code Select Expand

function pregReplaceCurry($func, $arity)
{
return function() use ($func, $arity)
{
$args = func_get_args();
if (count($args) >= $arity)
return call_user_func_array($func, $args);
return function() use ($args, $func)
{
$a = func_get_args();
$z = $args;
$a = array_merge($z, $a);
return call_user_func_array($func, $a);
};
};
}
This thing had create_function in create_function in loops with each BBC tag replacement having a possible valid match for smiley. Hence, more BBC tag, more possibility for smileys and this was just a lot of smileys and create_functions...

Look for memory usage and see if it offers any improvement (PHP 5.3+ only)