Fatal Error: Call to undefined function idna_maps_not_std3

[WIMorrison]

Is there a permanent fix to this being made available? It happened a couple of day after upgrading to 2.1.6.

I have commented out the function as recommended above (Thanks for this!!) but I don't know what else might be broken by getting rid of this function

[Aleksi "Lex" Kilpinen]

It is a permanent solution until an official patch is released, nothing should break.

[alastairnyght]

My forum also went down because of this. Worked alright for a while but then this hit us too.

[Rheingauner]

I've had to comment out the call to the function in Class-Punycode.php, Lines 536/537.

SMF 2.1.6.

This is a quick and dirty solution  , our Forum is running again without a lot of errors.

Thx

[zappaDPJ]

We also encountered this error which occurred a day or two after upgrading from 2.1.4 though 2.1.5 to 2.1.6.

[@rjen]

If you read this topic at least 10 different fora...

[dsanchez]

Same issue here after upgrade to to 2.1.6. (error did not show inmediately)

Applying the patch of post # 1 now...

[MegaBrutal]

Happened to me too, but only today, after several days of the upgrade, so I'm not sure if 2.1.6 has anything to do with it. According to error log, it started just around the time I went to sleep. I noticed some files have changed in the SMF installation directory!

Code Select Expand

# ls -lt Sources/Unicode
total 1132
-rw-r--r-- 1 www-data www-data 250748 Jul 13 01:16 Idna.php
-rw-r--r-- 1 www-data www-data  39292 Jul 13 01:16 QuickCheck.php
-rw-r--r-- 1 www-data www-data  95317 Jul 13 01:16 RegularExpressions.php
-rw-r--r-- 1 www-data www-data 107149 Jul 13 01:16 CaseFold.php
-rw-r--r-- 1 www-data www-data 106819 Jul 13 01:16 CaseUpper.php
-rw-r--r-- 1 www-data www-data 105111 Jul 13 01:16 CaseLower.php
-rw-r--r-- 1 www-data www-data  23574 Jul 13 01:16 CombiningClasses.php
-rw-r--r-- 1 www-data www-data  38384 Jul 13 01:16 Composition.php
-rw-r--r-- 1 www-data www-data 140938 Jul 13 01:16 DecompositionCompatibility.php
-rw-r--r-- 1 www-data www-data  83748 Jul 13 01:16 DecompositionCanonical.php
-rw-r--r-- 1 www-data www-data    412 Jul 13 01:16 Metadata.php
-rw-r--r-- 1 www-data www-data   7741 Apr 27  2023 CaseTitle.php
-rw-r--r-- 1 www-data www-data  92225 Apr 27  2023 DefaultIgnorables.php
-rw-r--r-- 1 www-data www-data    217 Apr 27  2023 index.php

Is it a ******ing cyber attack?

Interesting errors right at the time when the files changed:

Code Select Expand


Hiba
A hiba típusa
Cron
Hibaüzenet
2: unlink(/tmp/Metadata.R04GQs): No such file or directory
Fájl
/var/www/asperger/Sources/Subs-Admin.php
Sor
1928
A hibát okozó oldal címe
https://asperger.hu/index.phphttps://asperger.hu/cron.php
Backtrace információ

    #0: smf_error_handler_cron()
    Híva innen: ismeretlen, -1. sor
    #1: unlink()
    Híva innen: /var/www/asperger/Sources/Subs-Admin.php, 1928. sor
    #2: safe_file_write()
    Híva innen: /var/www/asperger/Sources/tasks/UpdateUnicode.php, 583. sor
    #3: execute()
    Híva innen: /var/www/asperger/cron.php, 250. sor
    #4: perform_task()
    Híva innen: /var/www/asperger/cron.php, 132. sor

Similar unlink() errors follow for different files:

Code Select Expand

2: unlink(/tmp/DecompositionCanonical.UnB9S2): No such file or directory
2: unlink(/tmp/DecompositionCompatibility.fxUqfu): No such file or directory
2: unlink(/tmp/Composition.D2HWqG): No such file or directory
2: unlink(/tmp/CombiningClasses.jljLU2): No such file or directory
2: unlink(/tmp/CaseUpper.eoGGUZ): No such file or directory
2: unlink(/tmp/CaseFold.HWV3s1): No such file or directory
2: unlink(/tmp/RegularExpressions.UlkIRX): No such file or directory
2: unlink(/tmp/QuickCheck.bkevcW): No such file or directory
2: unlink(/tmp/Idna.QHUy9K): No such file or directory

Note the same filenames those are changed at 1:16, just with random temporary extension (Sources/Unicode/Idna.php <-> /tmp/Idna.QHUy9K).

My theories:

• (Pessimistic:) This is a ******ing vulnerability that has been exploited and we have no idea of its scope, e.g. what did it do beyond corrupting files and whether data has been stolen.
• (Optimistic:) Cron tried to upgrade the Unicode library and it has failed for some reason.

[shawnb61]

It's #2.

The unicode tables are now updated straight from the source.  Part of the weekly maintenance task.

But the unicode folks made a huge change in direction in a recent revision (#33) of unicode 6 - while the smf code is operating under the prior rules.   

So...  A fix is coming.

Until then, the temp fix shared above works perfectly.

[MobileCS]

Are there any known issues with manually trying to run the Weekly Maintenance in scheduled tasks? It does nothing here.

My web files are intentionally not writable by www unless I'm doing an update, so I've now seen this twice in the error log:

Code Select Expand

A new version of Unicode is available, but SMF could not update to it. Please make sure /example.com/httpdocs/forum/Sources/Unicode and all the files in it are writable. SMF will try to update its Unicode data files again automatically.
After each time I've seen this, I've made my forum files owned by www and then manually ran the weekly maintenance. I've waited 10 minutes and nothing happens. No errors reported.

How I know it's not working here, as I have not patched my forum for the idna_maps_not_std3() bug and the files in the Unicode folder still have the same last modification date.

[Sesquipedalian]

Are there any known issues with manually trying to run the Weekly Maintenance in scheduled tasks?

Nope.

For your case, it might help to know two things:

• The task to update the Unicode files happens in the background after the Weekly Maintenance task runs, not during the Weekly Maintenance task. Weekly Maintenance itself just adds the Unicode update task to the task queue so that a subsequent call to cron.php will run it. This is because the Unicode update process can take an unpredictable amount of time to complete, since it requires fetching a bunch of data files from an external source and is therefore subject to all manner of possible connectivity issues.
• The Unicode update task checks whether the necessary files are writable at the start of the process and reports the error you quoted if the files are not writable. At the end of the process, once the new versions of all the data files have been built, they are moved into place. If for any reason the files cannot be moved into place, the operation is aborted silently.

In light of these two facts, it is entirely possible that if you changed the file permissions, manually ran Weekly Maintenance, and then changed the file permissions soon afterward, the Unicode update might end up failing silently. This would be because the Unicode update task started before you reverted the file permissions, but it didn't finish until after you reverted the file permissions. That specific sequence of events would cause the update to fail silently.

[zappaDPJ]

The unicode tables are now updated straight from the source.  Part of the weekly maintenance task.

But the unicode folks made a huge change in direction in a recent revision (#33) of unicode 6 - while the smf code is operating under the prior rules.   

I'm sure your knowledge in this area goes way beyond mine but I can't wrap my head around why the issue as described didn't take out every SMF forum on the planet.

[shawnb61]

I'm sure your knowledge in this area goes way beyond mine but I can't wrap my head around why the issue as described didn't take out every SMF forum on the planet.

I'm actually at the other end of that spectrum...  I'm unclear how it's taking out sites at all.  The issue is specific to Punycode usage, and even then to a specific subset of characters.  I could see that possibly affecting some posts with Punycode links, etc. 

But take down a site?  Maybe if folks are using Punycode in their boardurls?  I haven't applied the fix to any of my test environments and they're all fine.

[MobileCS]

Are there any known issues with manually trying to run the Weekly Maintenance in scheduled tasks?

Nope.

For your case, it might help to know two things:

• The task to update the Unicode files happens in the background after the Weekly Maintenance task runs, not during the Weekly Maintenance task. Weekly Maintenance itself just adds the Unicode update task to the task queue so that a subsequent call to cron.php will run it. This is because the Unicode update process can take an unpredictable amount of time to complete, since it requires fetching a bunch of data files from an external source and is therefore subject to all manner of possible connectivity issues.
• The Unicode update task checks whether the necessary files are writable at the start of the process and reports the error you quoted if the files are not writable. At the end of the process, once the new versions of all the data files have been built, they are moved into place. If for any reason the files cannot be moved into place, the operation is aborted silently.

In light of these two facts, it is entirely possible that if you changed the file permissions, manually ran Weekly Maintenance, and then changed the file permissions soon afterward, the Unicode update might end up failing silently. This would be because the Unicode update task started before you reverted the file permissions, but it didn't finish until after you reverted the file permissions. That specific sequence of events would cause the update to fail silently.

Thank you for explaining how that works. It sounds like the issue might be with connectivity to the external source.

I'd imagine 10 minutes should be more than enough, especially since I'm running cron.php every minute via crontab on my own dedicated server (Xeon CPU, 128 GB RAM, 1 Gbps port).

I'll give it another shot in the morning and report back.

[MobileCS]

It's still not working for me.

I started the weekly maintenance at 11:49AM and it's now 5:58PM and nothing has changed.

[shawnb61]

Maybe nothing changed?  Look at the first couple of files.  If the versions are 2.1.5 or 2.1.6, you're good.  Some of the others may say 2.1.3, but most will be 2.1.5 or 2.1.6.

They typically only have one major release a year, & occasional tweaks.

[MobileCS]

I checked all the files, and almost all of them are 2.1.5 - no 2.1.6 listings.

However, for the last 2 weeks I've been getting this in my error log:

A new version of Unicode is available, but SMF could not update to it. Please make sure /example.com/httpdocs/forum/Sources/Unicode and all the files in it are writable. SMF will try to update its Unicode data files again automatically.

[shawnb61]

You're current.  There were no changes between 2.1.5 & 2.1.6.

I suspect those messages were from prior failed attempts when you had your folders protected.

[MobileCS]

Well that's the thing, the errors have been coming in once per week for the last 2 weeks and my files always had the same last modified time. I'm pretty sure I'll be getting this error again on July 19th unless I leave that folder writable by PHP, which I was trying to avoid by manually running the weekly maintenance now.

[shawnb61]

It needs to go thru the motions & download & inspect the remote files, etc.  So I'd bet that you're going to see that error once a week, whenever the folders are protected.

But it looks like you're current.  You can do a compare to the files in the 2.1.6 upgrade package if you wish to confirm that.