News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

Automatic Twitter Embedding - Code provided

Started by dimspace, October 31, 2014, 01:11:55 PM

Previous topic - Next topic

dimspace

Hokay.. Some basic background on this.

We previously used a service from embed tweet to auto embed tweets within our forum but due to various twitter restrictions notably the number of calls this stopped working so we figured out our own solution.

Weve got to the point where we now have working, automated twitter embedding on our site velorooms, but the code is very much a "hack" and requires polishing, and packaging for use by the community.

Firstly, some background on how we did it and why..

QuoteI completed the Twitter embed fix today.

Operating in more or less the same way as the previous implementation, it runs out of two files and a new table in the VR database. The main difference is that the magic happens with VR rather than a third-party service.

Rationale

A quick restatement of why this was necessary: Twitter is happy to supply you the content of a tweet on request but to prevent oversubscription by hacking or lazy site owners, we're limited to 180 requests per fifteen minutes.

On a forum, that limit could be reached pretty quickly with multiple users accessing multiple embedded tweets per page, so we want to cache every request in order not to have to go back to Twitter every single time a tweet is viewed on a page.

JS: http://velorooms.com/vr-js/embed_v2.js

This is the same JS file as for the former EmbedTweet.com solution. I commented out the former code and implemented a bootstrap with jQuery, which is already in general use on the VR website.

The principle for this part is the same as before: scan the current page for tweets in the form of links and grab the tweet ID. For each ID found on the page, we make an AJAX call to a PHP page on the VR server (it would have been good to put all IDs into a single request but Twitter itself won't allow multiple calls, so it wasn't worth it).

PHP / MySQL / JSON tweet loop: http://velorooms.com/tweet-cache.php?id=[Tweet ID]

The PHP page receives the request with the tweet ID attached as a querystring and it checks the new MySQL table tweet_cache for the existence of that tweet ID.

If it isn't already found there, then we know that we have to ask Twitter for the tweet. This comes back as JSON and we store it in the database ready for the next time it's requested. We then pass the result - again as JSON - as a response to the original JS AJAX request and the JS plops it onto the page right where the tweet link was. If the tweet ID was already in the DB, then we pass that back as a reponse instead of bothering Twitter for it

Secondly, the parts involved in making this work.

Part 1 - JS file called Embed_v2 which is in the scripts directory.

$(document).ready(function() {

var oTwitter = $('a[href*="twitter.com"][href*="/status"]');
if (oTwitter.length > 0) {
oTwitter.each(function() {
var oHolder = $(this);
var sStr = $(this).attr('href');
sStr = sStr.replace(/\/+$/, "");
sStr = sStr.substr(sStr.lastIndexOf('/') + 1);
$.getJSON( "/tweet-cache.php?id=" + sStr, function(data) {
oHolder.before(data.html);
oHolder.text('Source: ' + oHolder.text()).wrap('<div class="quoteheader"></div>');
});
});
//$.getScript('https://platform.twitter.com/widgets.js');
}

});


part 2 - tweet-cache.php which is placed in the root directory

<?php

/*

Rough take on Twitter oEmbed for VeloRooms
by L'arri : [email protected]
27 October 2014

*/

// I used SMF items where possible for database interactions and required includes.

define('SMF'1);
ob_start();

foreach (array(
'db_character_set''cachedir') as $variable)
if (isset($GLOBALS[$variable]))
unset($GLOBALS[$variable], $GLOBALS[$variable]);

require_once(
dirname(__FILE__) . '/Settings.php');

// Make absolutely sure the cache directory is defined.
if ((empty($cachedir) || !file_exists($cachedir)) && file_exists($boarddir '/cache'))
$cachedir $boarddir '/cache';

require_once(
$sourcedir '/Load.php');

// Handle SMF maintenance
if (!empty($maintenance) && $maintenance == 2) {
//db_fatal_error();
echo '{"html":"Tweet error (database unavailable)"}';
}

// Create a variable to store some SMF specific functions in.
$smcFunc = array();

// This is where SMF's database connection is initiated...
loadDatabase();

global 
$smcFunc;

function 
check_cache($tweet,$smcFunc){

$request $smcFunc['db_query']('','SELECT html from tweet_cache where tweetid ='.$tweet, array());
if (!$request) {
//db_fatal_error();
echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (database unavailable)</p>"}';
}
if ($smcFunc['db_num_rows']($request) == 0) {
add_cache($tweet,$smcFunc);
}
else {
while ($row $smcFunc['db_fetch_row']($request))
echo '{"html" : "' $row[0] .'"}';
}
$smcFunc['db_free_result']($request);

}

function 
add_cache($tweet,$smcFunc){

$twitterapi_url "https://api.twitter.com/1/statuses/oembed.json?id=";
$twitterapi_url $twitterapi_url $tweet;

$curl curl_init($twitterapi_url);
curl_setopt($curlCURLOPT_RETURNTRANSFER1);
$response curl_exec($curl);
curl_close($curl);

$json_content json_decode($responsetrue);
$json_content preg_replace"/\r|\n/"""$json_content );

$html $json_content['html'];
if(!empty($html)) {
$request $smcFunc['db_insert']('insert',
            
'tweet_cache',
            
array(
                
'tweetid' => 'int''html' => 'text'
            
),
            
array(
                
$tweetaddslashes($html)
           
),
            
array('tweetid','html')
        
);
echo '{"html" : "' addslashes($html) .'"}'
}
else {
echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (does not exist)</p>"}';
}


}

$_GET filter_input_array(INPUT_GETFILTER_SANITIZE_NUMBER_INT);
$qv  $_GET['id'];
if(!empty(
$qv)) {check_cache($qv,$smcFunc);} else {echo '{"html":"<p style=\"color: #666; border: 1px dotted #666; padding: 5px; width: 490px;\">Tweet error (blank ID)</p>"}';}

?>


3 - New Database table to store the data




And finally

4 - The call in display_template.php to load the script (jquery required obviously)

<script type="text/javascript" src="/*scrips directory*/embed_v2.js"></script> '

This then results in automatic tweet embedding with a post.





So obviously this is a start point. The code by Larri's admission is somewhat of a "hack" and can be refined by someone more familiar with SMF, and obviously it would then need to be packaged as a mod. Ideally someone who will be able to handle the upkeep of the modification as twitter adjusts its API etc.

Only requirement is that L'arri remains credited in the code as the original coder and any mod made from this code must be free, no paid mods from our code. 

Like i say, this is just the rough beginnings, but it is all there as working code, just needs refinement, packaging and development.

Rumpa


Cylent1

I would most def be interesting in a mod like this...

dimspace

well the code is there and works, just needs someone to refine it an package it, possibly add an admin panel to enable/disable it.

oh.. i didnt actally say what it does properly..

you post the twitter link, and it automatically embeds the tweet for you

DEK24


vbgamer45

Package/modified and submitted.

Requires cURL extension in php installed. And includes jquery hosted on cloudfare cdn.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Cylent1

Installed no problem!  Where are the options in admin panel if any?

vbgamer45

Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Cylent1


vbgamer45

You just post the twitter tweet in a thread and it auto embeds
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Cylent1

Are you suggesting to use the "embed Tweet" option on the bottom of the tweets?  If so, then it still isn't working for me.

vbgamer45

Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Antes

This mod still under review, you guys can try my mod: http://custom.simplemachines.org/mods/index.php?mod=3837

- It has no external JS library included
- Tweets got directly from Official API
- Support old fashion bbcode as well as auto-embedding.

mark87

Why not just use twitterfeed. com  and skip the coding ? Couple good threads on it and it never gives me any issues ?

Antes

They are different things, that site takes from forum and sends to social media while those two mods embeds the tweets into the forums (on request not automated).

dimspace

Quote from: vbgamer45 on November 29, 2014, 02:06:40 AM
Package/modified and submitted.

Requires cURL extension in php installed. And includes jquery hosted on cloudfare cdn.

thanks for doing this.

as said before, larri and i had done the coding and hard coded but neither of us know how to package smf mods.

dimspace

Quote from: mark87 on January 03, 2015, 10:54:50 AM
Why not just use twitterfeed. com  and skip the coding ? Couple good threads on it and it never gives me any issues ?

the reasons for this code were two fold.

1) makes embedding tweets incredibly simple, you only need to include the tweet link

2) it was built to use the smf database for entries which means on very large forums with a lot of page views you dont get locked out of twitter (basically every time the tweet is viewed its a fresh view and twitter gets very upset if you make multiple views - which is the problem we had), this caches the tweets in the smf database.

dimspace

Quote from: Antes on December 29, 2014, 06:55:48 PM
- Tweets got directly from Official API

This is the problem, for very large forums you end up getting locked out by twitter for making too many calls to the api (we had loads of days where our embedded tweets just vanished when twitter locked us out), which is why an alternate solution was looked for.

Xpresskonami

I install this Auto tweet Embeed. it works fine, but suddenly the mod stop working. please really need solutions to this

vbgamer45

Link to forum with a topic that has a twitter post.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Advertisement: