Advertisement:

Author Topic: [MOD][Pending]Open Graph Embed (Testing wanted)  (Read 8946 times)

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
[MOD][Pending]Open Graph Embed (Testing wanted)
« on: December 04, 2015, 01:05:02 AM »
This is a BBC code that works with the Open Graph standard to embed media and site descriptions into your website.

Instead of installing various mods to embed content from a web site you can embed content with this mod from any site that supports it with one simple BBC tag.

Code: [Select]
[embed]http://url.of.site/you/want/to/embed[/embed]
By placing the URL in between the [embed][/embed] tags, this mod pulls Open Graph information shared freely from the site owner for display on your forum. By doing this your displaying the content the way the site author intended.

This mod caches the data in the database with a short TTL of only 30 days. This mod also uses the cache system with a TTL of 30 minutes.

Popular Supported Sites
- YouTube
- Vimeo

The sites that supply OG information are many, but this is a list of popular sites I have tested so forth.

TODO:
- Add admin option to purge old TTLs or some/all URLs.

Changelog:
0.1
- Initial pre-release
« Last Edit: December 09, 2015, 02:30:03 AM by Colin »

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][WIP]Open Graph Embed (Testing wanted)
« Reply #1 on: December 05, 2015, 11:25:15 PM »
I guess can't edit post anymore.

Here is currently what I have done.

* Moved cache_put for a successful database call to the if statement for max-age to prevent stale data from being cached.
+ Added support for HTML 5 video and audio.
+ Added fallback support for text/html video to iframe.
* Moved $data = array() to else statement. (Introduced this bug myself thinking I made a mistake.  ::) )

TODO
- Add admin option to purge old TTLs or some/all URLs.
- Finish off support for multiple video suggested by OG data.

Currently working on the array for multiple videos suggested in OG. Since I can only pull this information up one by one I have to insert it in a array to sort at a later time when all the details are present.

Code: [Select]
foreach ($xpath->query('//*/meta[starts-with(@property, \'og:\')]') as $meta) {
if(isset($data[$meta->getAttribute('property')])) {
if ($meta->getAttribute('property') == 'og:video:url'|| $meta->getAttribute('property') == 'og:video:type' || $meta->getAttribute('property') == 'og:video:height' || $meta->getAttribute('property') == 'og:video:width') {
if ($meta->getAttribute('property') == 'og:video:url')
$v=$meta->getAttribute('content');
if (isset($v)) {
$data['vids'][$v][$meta->getAttribute('property')] = $meta->getAttribute('content');
}
}
continue;
} else {
$data[$meta->getAttribute('property')] = $meta->getAttribute('content');
}
}

I am saving this information with the og:url property as the unique identifier.

og.php
Components in here will change as I am undecided on how I should handle the OG data for multiple video sources, plus it isn't coded all the way.
Code: [Select]
<?php

if (!defined('SMF')) 
die('Hacking attempt...');

function 
get_og_data($url) {
global $smcFunc;
$maxage 2419200;
$en_url urlencode($url);
if (filter_var($urlFILTER_VALIDATE_URL) === FALSE)
return(array('display'=>$url));
if(($data cache_get_data('og-info'.$en_url1800)) != null) {
return json_decode($datatrue);
} else {
$query $smcFunc['db_query']('''SELECT vars, req_date FROM {db_prefix}og_cache WHERE url = {text:url}', array('url' => $en_url));
if ($smcFunc['db_num_rows']($query) != 0) {
$data $smcFunc['db_fetch_assoc']($query);
$smcFunc['db_free_result']($query);
if ($data['req_date'] > (time()-$maxage)) {
cache_put_data('og-info'.$en_url$data['vars'], 1800);
return json_decode($data['vars'], true);
}
} else {
$data = array();
}
libxml_use_internal_errors(true);
$doc = new DomDocument();
$doc->loadHTML(file_get_contents($urlNULLNULL020000));
$xpath = new DOMXPath($doc);
foreach ($xpath->query('//*/meta[starts-with(@property, \'og:\')]') as $meta) {
if(isset($data[$meta->getAttribute('property')])) {
if ($meta->getAttribute('property') == 'og:video:url'|| $meta->getAttribute('property') == 'og:video:type' || $meta->getAttribute('property') == 'og:video:height' || $meta->getAttribute('property') == 'og:video:width') {
if ($meta->getAttribute('property') == 'og:video:url')
$v=$meta->getAttribute('content');
if (isset($v)) {
$data['vids'][$v][$meta->getAttribute('property')] = $meta->getAttribute('content');
}
}
continue;
} else {
$data[$meta->getAttribute('property')] = $meta->getAttribute('content');
}
}
if (empty($data['og:title']))
$data['og:title'] = $xpath->query('//title')->item(0)->textContent;
if (empty($data['og:description']))
$data['og:description'] = $xpath->query('/html/head/meta[@name="description"]/@content')->item(0)->textContent;
if (empty($data['og:image']))
$data['og:image'] = $xpath->query('/html/head/link[@rel="image_src"]/@href')->item(0)->textContent;
if (empty($data['og:image']))
$data['og:image'] = $xpath->query('/html/head/link[@rel="icon"]/@href')->item(0)->textContent;
if (empty($data['og:url']))
$data['og:url'] = $url;
if (!empty($data['vars']))
unset($data['vars']);
$data['req_date'] = time();
$smcFunc['db_insert']('replace''{db_prefix}og_cache', array('url' => 'text''vars' => 'text''req_date' => 'int'),array($en_urljson_encode($data), $data['req_date']),array('url'));
cache_put_data('og-info'.$en_urljson_encode($data), 1800);
}
if (empty($data['og:title']))
return(array('display'=>$url));
return($data);
}

function 
phase_og_data($data) {
if (!empty($data['display']))
return('<a href="'.$data['display'].'" target="_blank">'.$data['display'].'</a>');

return ('
<div class="og-embed">
<div class="og-embed-title"><a href="'
.$data['og:url'].'" target="_blank">
'
.(empty($data['og:site_name']) ? '' $data['og:site_name'].' - ').(empty($data['og:title']) ? '' $data['og:title']).'
</a></div>'
.(!empty($data['og:video']) || !empty($data['og:video:url']) ? phase_og_video($data) : '')
.(!empty($data['og:audio']) ? phase_og_audio($data) : '')
.'
<div class="og-embed-description">
'
.(empty($data['og:image']) ? '' '<img class="og-embed-image" src="'.$data['og:image'].'" />').(empty($data['og:description']) ? '' $data['og:description']).'<br style="clear:both;" />
</div>
</div>'
);

}
function 
og_bbc(&$bbc) {
$bbc[] = array(
'tag' => 'embed',
'type' => 'unparsed_content',
'content' => '$1',
'validate' => 'og_bbc_validate',
'disabled_content' => '$1',
);
}

function 
og_bbc_validate (&$tag, &$data, &$disabled) {
$data phase_og_data(get_og_data($data));
}
function 
phase_og_video ($data) {
if (!empty($data['og:video:type']) && $data['og:video:type'] == 'text/html') {
return ('
'
.(!empty($data['og:video:url']) ? '<div class="og-embed-media"><center><iframe class="og-embed-video" src="'.$data['og:video:url'].'" frameborder="0" allowfullscreen></iframe></center></div>' '').'
'
.(!empty($data['og:video']) ? '<div class="og-embed-media"><center><iframe class="og-embed-video" src="'.$data['og:video'].'" frameborder="0" allowfullscreen></iframe></center></div>' '').'');
} else {
return ('
<div class="og-embed-media"><center>
<video src="'
.(!empty($data['og:video:url']) ? $data['og:video:url'] : $data['og:video']).'"'
.(!empty($data['og:video:type']) ? ' type="'.$data['og:video:type'].'"' '')
.(!empty($data['og:video:width']) ? ' width="'.$data['og:video:width'].'"' '')
.(!empty($data['og:video:height']) ? ' height="'.$data['og:video:height'].'"' '')
.' class="og-embed-video" controls>
</video>
</center></div>'
);
}
}
function 
phase_og_audio ($data) {
return ('
<div class="og-embed-media"><center>
<audio src="'
.(!empty($data['og:audio']) ? $data['og:audio'] : '').'"'
.(!empty($data['og:audio:type']) ? ' type="'.$data['og:audio:type'].'"' '')
.' class="og-embed-audio" controls>
</audio>
</center></div>'
);
}
?>

Comments, questions?

I am still debating if it should support multiple videos, the first one should always work anyways, just if you have a preference of the format delivered.

What do you think?

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][PENDING]Open Graph Embed
« Reply #2 on: December 08, 2015, 04:47:52 PM »
Version 0.2 release

0.2
! Moved cache_put for a successful database call to the if statement for max-age to prevent stale data from being cached.
+ Added support for HTML 5 video and audio.
+ Added fallback support for text/html video to iframe.
! Moved $data = array() to else statement.
- Removed HTML 5 video support as it isn't fully supported.
+ Added some more meta phasing for og:video:url
+ Added code to disregard text/html video.
+ Added scheduled task to purge old data.
- Remove update support from main function.
! Renamed modification to Open Embed.
! Fixed errors with xpath when property isn't found.


I can't change the title of this post to [PENDING], what do I do start a new post?

Offline Colin

  • Lead Developer
  • SMF Hero
  • *
  • Posts: 7,760
  • Gender: Male
  • SMF Developer
    • colinschoen on GitHub
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #3 on: December 09, 2015, 02:30:22 AM »
I can't change the title of this post to [PENDING], what do I do start a new post?

Edited.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #4 on: December 09, 2015, 09:42:41 AM »
Thanks,

Submitted to mod site - http://custom.simplemachines.org/mods/index.php?mod=4075

Next release 0.3, adding oEmbed standards for more embed support.
http://oembed.com/

This content will be displayed in a iframe due to how the standard works embedding straight into a forum would leave it open to XXS attacks. This problem is not present in the current mod (0.2) because it is using XPath Meta queries. The oEmbed standard uses XML and JSON.

Offline margarett

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 19,761
  • Gender: Male
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #5 on: December 09, 2015, 12:09:08 PM »
Hi nend.

Some considerations just regarding the package (haven't really tested it), which will also become handy in order to get the MOD approved ;)
* Languages: because you already have some file edits, I would suggest you to add your changes to Modifications.english.php instead of loading a new file. As a rule of thumb, if you have no file edits, then it's great if you keep it "edits-free" and use your own file for the language. If you have a LOT of strings, it's always a good idea to have your own language file. But if you have few strings *and* you already have some file edits, why not save a file load and perform an additional edit for the few language strings? ;)

* install.xml, when you add content using "end", you should always make sure you add an extra line break at the end, eg:
Code: [Select]
.og-embed-description {background-color:#eee;padding:2px;}
]]></add>

* package-info.xml, some considerations ;)
- $themedir/languages should be $languagesdir
- at this point I would suggest you don't really need to support "upgrade", as it's a new MOD. If you really want to do it, you need to specify "upgrade from"
- your .php install files need some love ;)
a) The database operation table ($smcFunc['db_create_table']('{db_prefix}og_cache',) should be ran <database> tags, not "code". This way, SMF will log this change and offer the user the possibility to either remove it or not at uninstall (user's choice). The remaining operations in install.php should probably be kept in a dedicated fine which will run in "code" tags ;)
b) After this:
Code: [Select]
$dbresult = $smcFunc['db_query']('', "SELECT COUNT(*) as total FROM {db_prefix}scheduled_tasks WHERE task = 'og_prune'");
$row = $smcFunc['db_fetch_assoc']($dbresult);
You need to run
Code: [Select]
$smcFunc['db_free_result']($dbresult);Or, better yet, don't run this query at all and use:
Code: [Select]
$smcFunc['db_query']('', "INSERT IGNORE INTO {db_prefix}scheduled_tasks(you should use the insert operation of $smcFunc but I wouldn't hold the MOD just because of this :P )
c) in uninstall.php, this
Code: [Select]
$smcFunc['db_drop_table']('{db_prefix}og_cache');Isn't necessary if you create the table under "database" tags ;) SMF will offer the user the possibility to remove the database or not at uninstall
d) in Package Manager you don't need to
Code: [Select]
global $smcFunc;
db_extend('packages');
db_extend('extra');
only to run the .php file directly (via SSI). In this case, though, you might want to add some echo statements (at least the "Done" information)

* og.php looks nice, just noticed a missing tab:
Code: [Select]
if (filter_var($url, FILTER_VALIDATE_URL) === FALSE)
return(array('display'=>$url));
Because you are using DOM stuff, you should note that your MOD requires at least PHP 5 (which should be more than given by now, but you'll never know...)
Se forem conduzir, não bebam. Se forem beber... CHAMEM-ME!!!! :D

Quote
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #6 on: December 10, 2015, 01:12:14 AM »
Thanks Margarett,

Will fix those up tomorrow, I just finished my oEmbed json part of the mod, still need to work on oEmbed XML and photos but isn't vital to mod operation so will release 0.3 without it.

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #7 on: December 10, 2015, 07:26:16 PM »
I am having trouble finding documentation for the <database> tag. Do I put sql in between the tags, what is the syntax?

Thanks.

*never mind, found it

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #8 on: December 10, 2015, 09:18:53 PM »
0.3
+ Added some support to remove autoplay from url string.
+ Added oEmbed JSON support. (Only in a iframe via JavaScript as any other option isn't wise)
! Updated install with Margaretts suggestions.
! Changed video support from iframe to embed
+ Added useragent string. (Some sites are back in the 00s, browser discrimination it's called.)

+ Other stuff I may have forgotten...

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #9 on: December 14, 2015, 07:34:03 AM »
I know everyone is busy being this time of year, but can I get some love on the mod site.

http://custom.simplemachines.org/mods/index.php?mod=4075

Thanks

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #10 on: January 08, 2016, 09:20:03 AM »
Any problems with the mod on the customization site? It has been a month since submission just waiting on approval.

It has been download one time so forth, so I guess someone is testing it.

Offline live627

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 5,265
  • Gender: Male
  • Cat: Destroy!
    • live627 on Facebook
    • live627 on GitHub
    • live627 on LinkedIn
    • @live627 on Twitter
    • livemods
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #11 on: February 27, 2016, 06:02:53 PM »
I know everyone is busy being this time of year, but can I get some love on the mod site.
I'm looking at it now.
Try not to become a man of success, but rather try to become a man of value.
- Albert Einstein

Offline nend

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 1,755
  • 2 deep n2 the code
    • sicommnend on GitHub
    • SIComm.us
Re: [MOD][Pending]Open Graph Embed (Testing wanted)
« Reply #12 on: February 27, 2016, 11:09:47 PM »
0.5 released, it has some unfinished elements in the admin panel. Since I merged the requested changes for approval in my repo it is an early release and I haven't had a chance to complete it. However these unfinished elements are hidden anyways and do not cause any errors or problems.