News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

[REQUEST] Forumscraper (json files) import

Started by sgm9, April 05, 2025, 12:59:34 PM

Previous topic - Next topic

sgm9

github TUVIMEN forumscraper
This script, which you run like (sorry can't post external links, not even bogus ones)
python forumscraper https (://) site com/index.php?action=forum
-t 2 --users --reactions --boards --tags --forums -r 10 --retry-wait
10 -m 15 -k -F err.txt -l log.txt --compression none  --cookie SMFC
ookie992=whatever --cookie PHPSESSID=whatever

produces json files with topics (posts) and boards (their topics).
Would be nice to be able to restore from this. I don't have the user info retrieved separately, needed more rights for that, but usernames are found in the posts, which could be imported and then e-mails manually added, password hashes generated, etc. Admin of the forum is MIA for 2 years now and forum host will expire soon, this is why the odd route.

Let me know if anyone is interested in helping, otherwise I'm going to have a spend a day or two and $10 on claude to do it :)
Or if there's a well written more generic importer library, in python, rust or some other easy to read language, so i could just plug the json files as source instead of other forum software db format.

sgm9

Well, I managed to do it with lots of chatgpt, but it's not an universal solution as the scraper still has bugs, so the jsons contained imperfect data.

Kindred

this seems like a way to steal someone else's forum data...   

given that this information is all stored with associated and relational data in the database, I fail to see much use for this...
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Diego Andrés

Indeed the use is likely for forums where you're not an administrator.
There are many use-cases, some as simple as storing a list of topics in a spreadsheet.

SMF Tricks - Free & Premium Responsive Themes for SMF.

sgm9

Can you steal from a dead person with no heirs though ?

Advertisement: