Advertisement:

convert.php memory-aware

Aloittaja emanuele, lokakuu 06, 2013, 06:24:00 IP

« edellinen - seuraava »

emanuele

While working on a conversion I hit several times the memory limit (I was using either html_to_bbc and preparsecode in order to properly fix all the tags stored in html O:-)) and had to lower the limits of the query to an "insulting" level (5 messages at a time) increasing terribly the time necessary to complete the conversion.

So, I had a quick look at the code and decided to try to add a memory detection and try to avoid using too much of it, while still process the maximum amount of messages during the same round.

The result is the attached convert.php, with the normal convert it took me about 6/8 hours to finish the conversion, with the one I think a couple of hours.
Probably the reason is that most of the messages do not require a very high amount of memory to be processed, but few may require a very high amount and if those are (by chance) all together the script goes out of memory.

This script checks the amount of memory available after each "item" processed during the conversion, if the amount of free memory is more than 30% of the total (30% should be safe in most of the cases and if it isn't... well, something is wrong anyway :P) it goes on processing another "item", otherwise it stops and takes a break.

Here for testing/comments/etc.

In order to take maximum advantage from this, you can remove the numbers after the name of the table that are normally used to limit the number of "items" processed at a time.
If you don't know what I mean, probably is better if you don't use this script, so I'm going into details. :P


Take a peek at what I'm doing! ;D




Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Advertisement: