Hi,
I'm trying to make a post to the Forum after logging in.
The topic URL is: http://myforum.com/index.php?topic=5.0
There are already multiple posts in that topic. I want to post a new message. For example, right now there are 7 pages in that topic. So, on the 7th page, I make a new post by clicking reply. I am now trying to automate the same thing using Python script but its not working.
Below is my code:
import requests
cookies = {"PHPSESSID":<id>, "SMFCookie463":<cookie_id>}
url = "http://myforum.com/index.php?action=post;topic=5.90;last_msg=302"
post_data = {"message":"check this out"}
requests.post(url, data=post_data, cookies=cookies)
Do I need to choose a different URL while making the post? If I capture the POST URL from the HTTP POST request when I make a new post, it look like: http://myforum.com/index.php?action=post2;start=90;board=2
Tried with this URL also and it didn't work.
Also, do I need to add some custom HTTP request headers to make it work?
Thanks.
Did you look at the response of the page you got back? The HTML would give you the error...
At a minimum you will also need to pass in a subject because subjects are per-message. I don't *think* it mandates an icon, but if in doubt look at the stuff sent from quick reply as the canonical example of bare minimum.
Quote from: Arantor on October 17, 2020, 08:11:40 AM
Did you look at the response of the page you got back? The HTML would give you the error...
At a minimum you will also need to pass in a subject because subjects are per-message. I don't *think* it mandates an icon, but if in doubt look at the stuff sent from quick reply as the canonical example of bare minimum.
Thanks. So, after I added the subject, I checked the response and what I see is the preview of the message.
So, instead of posting the message directly, it displays the preview as you can see below..
(https://i.imgur.com/wic79FP.png)
Maybe I need to change something in the URL I am sending the HTTP POST request to? SMF Forum does detect the new post.
You need to call post2 instead of post in the action for the url
So... look at what's in the normal post sending to reply to a topic.
When replying to this topic, I see the submit URL is https://www.simplemachines.org/community/index.php?board=8;action=post2 - which *on its own* would be the post to a new topic submit button (note the action=post2 part)
But when I look at all the other things submitted normally from QR:
topic - the topic ID
subject - the subject for the message
icon - an icon, defaults to 'xx'
from_qr - indicates if from quick reply or not, I forget exactly what that does
notify - whether to turn notifications on for the topic
not_approved - relates to the post approval workflow, leave blank if post moderation is not applicable
goback - 0 to redirect to the list of topics, 1 to redirect back to the topic itself after posting
last_msg - the last msg in the topic that you saw (so it knows whether to check the 'something has been posted while you were typing' warning)
seqnum - a value to check if you already submitted or not
plus the session identifier.
PLUS, and this is the thing I think you're missing:
post - with a value, to indicate you pressed the Post button and not preview.
Quote from: Arantor on October 17, 2020, 02:06:38 PM
So... look at what's in the normal post sending to reply to a topic.
Thanks for the details. So I modified the code and used post2 instead of post.
Now, I receive the following error.
Your session timed out while posting. Please try to re-submit your message. (https://i.imgur.com/GTCAAfz.png)
Now, I am sending the following
topic
subject
icon
message
Here is my updated code:
import requests
cookies = {"PHPSESSID":<session_id>,"SMFCookie384":<cookie_value>}
post_data = {"topic":5, "icon":"xx","subject":"Something", "message":"check this out now"}
url = "http://myforum/index.php?action=post2;start=90;board=2"
# I tried both, with and without the referer.
# my_ref = "http://myforum/index.php?action=post;topic=5.90;last_msg=302"
r = requests.post(url, data=post_data, headers={'referer': my_ref}, cookies=cookies)
Quote from: vbgamer45 on October 17, 2020, 02:04:47 PM
You need to call post2 instead of post in the action for the url
Thanks. Yes, I used post2 instead of post now. I receive the error: " Your session timed out while posting. Please try to re-submit your message. "
I checked the PHPSESSID value again and it is correct.
There is also a hidden input field with a session check variable that is uniquely generated. the fieldname and value.
If you have access to the site. I would suggest creating your post script using SSI.php then talk to that script via your Python
Quote from: NeonFlash on October 17, 2020, 02:20:44 PM
Quote from: vbgamer45 on October 17, 2020, 02:04:47 PM
You need to call post2 instead of post in the action for the url
Thanks. Yes, I used post2 instead of post now. I receive the error: " Your session timed out while posting. Please try to re-submit your message. "
I checked the PHPSESSID value again and it is correct.
I didn't say PHPSESSID; I said the session identifier - the one that is shown in the form if you look at, for example, quick reply. That shows you *all* the fields that get sent.
Quote from: vbgamer45 on October 17, 2020, 02:34:06 PM
There is also a hidden input field with a session check variable that is uniquely generated. the fieldname and value.
If you have access to the site. I would suggest creating your post script using SSI.php then talk to that script via your Python
Are you referring to this field as highlighted in image below?
(https://i.imgur.com/UXPke0b.png)
If yes, then I added this also to the POST data, I still receive the session error. Please see below my updated script:
import requests
cookies = {"PHPSESSID":<session_id>,"SMFCookie384":<cookie_value>}
post_data = {"topic":5, "icon":"xx","d6e2b56fb":"268cf9d9b38548f4590d18b8bcbf150c","subject":"Something", "message":"check this out now"}
url = "http://myforum/index.php?action=post2;start=90;board=2"
r = requests.post(url, data=post_data, cookies=cookies)
I still receive the same error about session. Why is that so?
Quote from: Arantor on October 17, 2020, 03:08:45 PM
I didn't say PHPSESSID; I said the session identifier - the one that is shown in the form if you look at, for example, quick reply. That shows you *all* the fields that get sent.
Yes, could you please check my new response? I think I found the hidden session id in HTML form and I used it while sending the HTTP POST request also. But I still receive the session error.
Are these directly from your current session? Because this is a CSRF value based on your current session and will change over time. This sounds to me like you have already outdated values.
Quote from: Arantor on October 17, 2020, 03:12:18 PM
Are these directly from your current session? Because this is a CSRF value based on your current session and will change over time. This sounds to me like you have already outdated values.
Yes, those are from the current session. I just now made a post in the Forum and captured the HTTP POST request headers using the browser's console. I once again verified the cookies (PHPSESSID and SMFCookie384), as well as the CSRF token in the HTML form (generated session id). All are matching. When I make the request inside the browser, it works but from within the Python script.
As you can see below, the hidden session id is the same as the one I specified in the script.
"d6e2b56fb":"268cf9d9b38548f4590d18b8bcbf150c"
http://myforum.com/index.php?action=post2;start=90;board=2
Host: myforum.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:81.0) Gecko/20100101 Firefox/81.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: multipart/form-data; boundary=---------------------------112038716637290886941207617303
Content-Length: 2128
Origin: http://myforum.com
Connection: keep-alive
Referer: http://myforum.com/index.php?action=post;topic=5.90;last_msg=302
Cookie: PHPSESSID=47e516875c49caba4aa35bac00a30cf5; SMFCookie384=a%3A4%3A%7Bi%3A0%3Bs%3A1%3A%221%22%3Bi%3A1%3Bs%3A40%3A%22d266ba3b7037248cd03a307f8e0fa342ce95aa1a%22%3Bi%3A2%3Bi%3A1792174975%3Bi%3A3%3Bi%3A0%3B%7D
Upgrade-Insecure-Requests: 1
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="topic"
5
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="subject"
Re: Something
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="icon"
xx
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="sel_face"
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="sel_size"
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="sel_color"
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="message"
this one please
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="message_mode"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="notify"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="lock"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="sticky"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="move"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="attachment[]"; filename=""
Content-Type: application/octet-stream
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="last_msg"
302
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="additional_options"
0
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="d6e2b56fb"
268cf9d9b38548f4590d18b8bcbf150c
-----------------------------112038716637290886941207617303
Content-Disposition: form-data; name="seqnum"
3005024
-----------------------------112038716637290886941207617303--
I executed my Python script and captured the HTTP POST request headers generated by it. They are as shown below.
POST /index.php?action=post2;start=90;board=2 HTTP/1.1
Host: myforum.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.24.0
Cookie: PHPSESSID=47e516875c49caba4aa35bac00a30cf5; SMFCookie384=a%3A4%3A%7Bi%3A0%3Bs%3A1%3A%221%22%3Bi%3A1%3Bs%3A40%3A%22d266ba3b7037248cd03a307f8e0fa342ce95aa1a%22%3Bi%3A2%3Bi%3A1792174975%3Bi%3A3%3Bi%3A0%3B%7D
Content-Length: 107
Content-Type: application/x-www-form-urlencoded
topic=5&message=check+this+out+now&d6e2b56fb=268cf9d9b38548f4590d18b8bcbf150c&subject=Re%3A+Something&icon=xx
The only differences I see are:
1. Different user-agent
2. No referer in my HTTP request
3. I have not specified the last_msg and seqnum fields.
4. Do I have to adjust the "Content-Type" HTTP request header field? Its not "multipart/form-data" in my case. Maybe the server is not receiving the session id in the correct way and I need to change something in the way I am sending the request.
Could you please check my response above? Why is it not working? Please suggest an alternative. I really need a way to make the posts automatically using a script.
I found this, does it help? https://github.com/Chaturaphut/Python-SMF-Auto-Post/blob/master/smf_post.py
Quote from: live627 on October 18, 2020, 03:18:17 AM
I found this, does it help? https://github.com/Chaturaphut/Python-SMF-Auto-Post/blob/master/smf_post.py
Thanks for sharing but that script does not work. That script is using the same idea as me though.
By the way, I noticed an anomaly. So, I executed my code around 10 times and it worked one time. No other changes were made to the code.
Attempt 1: didn't work (session expired)
Attempt 2: didn't work (session expired)
Attempt 3: worked (post was made)
Why is it that way?
In all the 3 attempts, the exact same cookie and CSRF session id was used. No change was made to the script as well.
Here is the code, can someone verify?
import requests
cookies = {"PHPSESSID":"", "SMFCookie384":""}
post_data = {'topic':'4','subject':'Re: Something','icon':'xx','sel_face':'','sel_size':'','sell_color':'','message':'it works better now','message_mode':'0','notify':'0','lock':'0','sticky':'0','move':'0','e72529b3':'8432df4afac8da681859778fcebd6be1'}
url = "http://myforum.com/index.php?action=post2;start=0;board=2"
r = requests.post(url, data=post_data, cookies=cookies)
I have a similar problem. Did you manage to make some sense of this issue?
Quote from: EthanR on November 03, 2020, 02:47:40 PMI have a similar problem. Did you manage to make some sense of this issue?
https://github.com/oliver-zehentleitner/python-simplemachinesforum
from simplemachinesforum.simplemachinesforum import SimpleMachinesForum
smf_url = "https://www.any-simplemachinesforum.com"
smf_user = "user_name"
smf_pass = "password"
smf = SimpleMachinesForum(smf_url, smf_user, smf_pass)
# 1 = board id:
smf.new_topic(1, "subject", "This is the message to post!")
Please avoid reviving old topics. Topic locked.