Advertisement:

Author Topic: Link preview mod?  (Read 2762 times)

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Link preview mod?
« on: October 13, 2017, 05:58:36 PM »
Is there a mod that will create a preview for a link as they are common on social media nowadays, ie include a thumbnail of the linked pages main image, the main headline, a teaser-text, the domain name, etc.
This information can usually be gathered from meta tags of the linked page.

Offline Kindred

  • The Mean One
  • Support Specialist
  • SMF Legend
  • *
  • Posts: 58,431
  • Gender: Male
    • Kindred-999 on GitHub
Re: Link preview mod?
« Reply #1 on: October 13, 2017, 07:17:33 PM »
in short... no
Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #2 on: October 14, 2017, 10:03:17 AM »
Haha, so is anyone interested in getting paid for making one
and open sourcing it?

And on that note, who else would be willing to shell out some money, so we can give the developer a decent payment?

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #3 on: October 15, 2017, 05:26:32 AM »
Version
2.0.14

Description
When writing a link into the post editor an Ajax request is fired and gets the linked page to evaluate its meta-data. Due to cross domain policy this needs to go through a simple PHP-proxy (https://stackoverflow.com/a/2559062/1225787). This proxy needs to be secured at least to only allow requests from localhost. Multiple secure PHP proxy examples for this use case exist and should be used.
The meta-data to be evaluated is semantic markup in JSON-LD format as defined by schema.org. Example data:
Code: [Select]
<script type="application/ld+json">
{
    "@context": "https://schema.org",
    "@type": "Article",
    "publisher": {
        "@type": "Organization",
        "name": "Troy Hunt",
        "logo": {
            "@type": "ImageObject",
            "url": "https://www.troyhunt.com/content/images/2016/04/Troy-Hunt-Profile-Photo.jpg",
            "width": 60,
            "height": 60
        }
    },
    "author": {
        "@type": "Person",
        "name": "Troy Hunt",
        "url": "https://www.troyhunt.com/author/troyhunt/",
        "sameAs": []
    },
    "headline": "What Would It Look Like If We Put Warnings on IoT Devices Like We Do Cigarette Packets?",
    "url": "https://www.troyhunt.com/what-would-it-look-like-if-we-put-warnings-on-iot-devices-like-we-do-cigarette-packets/",
    "datePublished": "2017-10-13T08:16:58.000Z",
    "dateModified": "2017-10-14T09:23:46.000Z",
    "image": {
        "@type": "ImageObject",
        "url": "https://www.troyhunt.com/content/images/2017/10/WeVibe-1.jpg",
        "width": 800,
        "height": 801
    },
    "keywords": "Security, IoT",
    "description": "A couple of years ago, I was heavily involved in analysing and reporting on the massive VTech hack, the one where millions of records were exposed including kids&#x27; names, genders, ages, photos and the relationship to parents&#x27; records which included their home address. Part of this data was collected via",
    "mainEntityOfPage": {
        "@type": "WebPage",
        "@id": "https://www.troyhunt.com/"
    }
}
    </script>
   
If semantic markup does not exist on the page, fall back to meta-tags in Facebook's Open Graph model. Example data:
Code: [Select]
<meta property="og:site_name" content="Troy Hunt" />
<meta property="og:type" content="article" />
<meta property="og:title" content="What Would It Look Like If We Put Warnings on IoT Devices Like We Do Cigarette Packets?" />
<meta property="og:description" content="A couple of years ago, I was heavily involved in analysing and reporting on the massive VTech hack, the one where millions of records were exposed including kids&#x27; names, genders, ages, photos and the relationship to parents&#x27; records which included their home address. Part of this data was collected via" />
<meta property="og:url" content="https://www.troyhunt.com/what-would-it-look-like-if-we-put-warnings-on-iot-devices-like-we-do-cigarette-packets/" />
<meta property="og:image" content="https://www.troyhunt.com/content/images/2017/10/WeVibe-1.jpg" />
The gathered meta-data shall be used to create a "link-preview" in BBCode that shall get inserted into the post-editor after the URL (avoid screwing up fast typers through an Ajax loading blocker). Examples of how this looks on other services are attached as images. However this mod shall not be concerned with styling the link-preview, but merely provide the BBCode to do so. Example of the expected result to get inserted:

Code: [Select]
[quote=Troy Hunt on 13.10.2017, 08:16 UTC (modified 14.10.2017, 09:23 UTC)]
[url=https://www.troyhunt.com/what-would-it-look-like-if-we-put-warnings-on-iot-devices-like-we-do-cigarette-packets/]
https://www.troyhunt.com/what-would-it-look-like-if-we-put-warnings-on-iot-devices-like-we-do-cigarette-packets/

[b]What Would It Look Like If We Put Warnings on IoT Devices Like We Do Cigarette Packets?[/b]

A couple of years ago, I was heavily involved in analysing and reporting on the massive VTech hack, the one where millions of records were exposed including kids&#x27; names, genders, ages, photos and the relationship to parents&#x27; records which included their home address. Part of this data was collected via ...

[img]https://www.troyhunt.com/content/images/2017/10/WeVibe-1.jpg[/img]
[/url][/quote]

Permissions
The mod should have a global setting to enable/disable it. Optional: Integration with the permission system to allow/disallow for user-groups.

Payment
EUR 100, payable through Paypal.
As this will likely not cover development costs, I hope others will hop on board to raise payment for this feature. Come on, you want sweet automatic link previews too, don't you?

License
The complete code for this mod (excluding any third party code like libraries, etc) shall be released under an open source license that allows anyone to use/copy/modify (with attribution)/distribute this mod. As for the particular license model I don't really care as long as these conditions are met.

Offline digger

  • Sr. Member
  • ****
  • Posts: 761
  • Gender: Male
    • realdigger on GitHub
    • SMF Russian Community
Re: Link preview mod?
« Reply #4 on: October 15, 2017, 08:39:21 AM »
I'm working on a similar mod for autoembed opengraph data but it's not finished yet.

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #5 on: October 15, 2017, 10:42:45 AM »

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #6 on: August 13, 2019, 08:59:39 AM »
Since no one came up with a solution so far, I wrote it myself. I'm releasing it here as a free recipe. I wrote this for my own website and have no interest in creating and supporting a mod. If anyone else wants to, be my guest.

linkpreview.php - This is a slightly modified version of this one by abazad: https://github.com/abazad/LinkPreview/blob/master/LinkPreview.php
I fixed some encoding issues and added "security" in so far that only logged in users can retrieve preview data.
It retrieves the Title, Description and Main Image from a URL passed to it, providing all necessary data in JSON format to create link previews for most pages.
Code: [Select]
<?php
require_once($_SERVER['DOCUMENT_ROOT'].'/SSI.php');

if ($context['user']['is_logged']) {
function getTitle($linkSource) {
// The page should only have one title, so lets go ahead and get it!
return addslashes($linkSource->getElementsByTagname('title')->item(0)->nodeValue);
}

function getDescription($linkSource) {
// Description is in the meta tags, so lets iterate through them, until we find it
$metas $linkSource->getElementsByTagname('meta'); 
foreach ($metas as $meta) {
if ($meta->getAttribute("name") == "description") {
return addslashes($meta->getAttribute("content"));

}
}

function getImage($linkSource$linkurl) {
// Some sites hard code image urls for Twitter and Facebook, lets just use those.
$metas $linkSource->getElementsByTagname('meta'); 
foreach ($metas as $meta) {
if($meta->getAttribute("name") == "twitter:image") {
return addslashes($meta->getAttribute("content"));
} else if($meta->getAttribute("property") == "og:image") {
return addslashes($meta->getAttribute("content"));
} else {
// Do Nothing
}
}
// If this site doesn't do that, lets just grab the first image, because I don't know what else to do
$firstImage "";
$imgs       $linkSource->getElementsByTagname('img');
foreach ($imgs as $img) {
$firstImage $img->getAttribute("src");
if (strpos("http://"$firstImage) === false) {
$firstImage $linkurl $firstImage
}
break;
}
return addslashes($firstImage);
}

function loadSource() {
$html filter_var($_GET['link'], FILTER_VALIDATE_URL);
$html file_get_contents($html);
$html mb_convert_encoding($html'HTML-ENTITIES'"UTF-8");
$dom = new DOMDocument();
$dom->loadHTML($html);
$dom->preserveWhiteSpace false
return $dom;
}

error_reporting(0);

$urlSource loadSource();

$arr = array('title' => getTitle($urlSource), 'description' => getDescription($urlSource), 'image' => getImage($urlSource$_GET['link']));
echo stripslashes(json_encode($arrJSON_UNESCAPED_UNICODE JSON_UNESCAPED_SLASHES JSON_HEX_QUOT JSON_HEX_APOS));
}
?>

linkpreview.js - this is the main work I did. It adds a button that will fire the linkpreview function. The function filters URLs from the editor text, sends them to linkpreview.php, retrieves JSON, and finally replaces every URL in the editor with one that is enriched with preview data from the JSON.
Code: [Select]
var linkPreviewButton = jQuery('<img id="BBCBox_message_button_1_0" src="/Themes/default/images/bbc/linkpreview.gif" alt="Insert Link Previews" title="Insert Link Previews" style="cursor: pointer;background-image: url(&quot;/Themes/default/images/bbc/bbc_bg.gif&quot;);" width="23" height="22" align="bottom">')
.click(linkpreview)
.insertAfter('[id*="BBCBox_message_button"][src*="url"]');

function linkpreview() {
var regex = /((http(s)?:\/\/.)?(www\.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}\b([-a-zA-Z0-9@:%_\+.~#?&//=]*))/gim;
var message = jQuery.trim($("#message").val());
var richLink;
var matchedUrls = [];
match = regex.exec(message);
while (match != null) {
matchedUrls.push(match[0]);
match = regex.exec(message);
}
matchedUrls.forEach(function(url) {
jQuery.ajax({
url: '/Themes/default/linkpreview/linkpreview.php?link='+url,
dataType: "json",
success: function(data){
console.log(data)
if (data.title || data.description || data.image) {
richLink = url;
if (data.title != '') richLink = richLink + '\n[b]'+ data.title + '[/b]';
if (data.description != '') richLink = richLink + ' - [size=1]' + data.description + '[/size]\n';
else richLink = richLink + '[/size]\n';
if (data.image != '') richLink = richLink + '[img width=600]' + data.image + '[/img]';
richLink = richLink + '\n';
message = message.replace(url, richLink)
jQuery('#message').val(message);
}
}
})

})
}

As you can see it requires jQuery. It could be rewritten to pure JS easily, but I have jQuery loaded anyway, so meh.
I recommend loading linkpreview.js before the closing BODY element. You may have to change some file paths, depending on where you store these files.

You may also note that the preview function does not fire automatically when sending the message as is often the case in other social media. I don't think it's desirable to automatically screw around in forum posts. So it's a button instead, retaining full user control.
« Last Edit: August 13, 2019, 09:24:19 AM by Kolya »

Offline Antechinus

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 24,471
  • Master of BBC Abuse
Re: Link preview mod?
« Reply #7 on: August 13, 2019, 04:42:22 PM »
Nice work. I like the idea of having it controlled by a button. I tend to loathe automatic pop-ups regardless of what they do.

Online njtweb

  • Sr. Member
  • ****
  • Posts: 917
Re: Link preview mod?
« Reply #8 on: August 14, 2019, 07:31:09 AM »
Since no one came up with a solution so far, I wrote it myself. I'm releasing it here as a free recipe. I wrote this for my own website and have no interest in creating and supporting a mod. If anyone else wants to, be my guest.

linkpreview.php - This is a slightly modified version of this one by abazad: https://github.com/abazad/LinkPreview/blob/master/LinkPreview.php
I fixed some encoding issues and added "security" in so far that only logged in users can retrieve preview data.
It retrieves the Title, Description and Main Image from a URL passed to it, providing all necessary data in JSON format to create link previews for most pages.


As you can see it requires jQuery. It could be rewritten to pure JS easily, but I have jQuery loaded anyway, so meh.
I recommend loading linkpreview.js before the closing BODY element. You may have to change some file paths, depending on where you store these files.

You may also note that the preview function does not fire automatically when sending the message as is often the case in other social media. I don't think it's desirable to automatically screw around in forum posts. So it's a button instead, retaining full user control.

Do you have a working SMF site where we can see this in action? Personally I think I have an idea what you're talking about but want to see it working for clarity.

Thank you

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #9 on: August 14, 2019, 07:42:26 AM »
If you want to see the result, it looks like this. You enter the URL (in this case a Kotaku article), click the button and the JS function adds the title, description and image directly in the editor. So you can still edit this data. Works for multiple URLs (unlike all other social media I know). Of course you can change formatting relatively easily. The BBC tags and line breaks are in preview.js


https://kotaku.com/hundreds-of-players-trying-to-cheat-at-apex-legends-and-1837206180
Hundreds Of Players Trying To Cheat At Apex Legends And Counter-Strike Get Their Private Data Stolen - Since February, a number of ne’er-do-well Apex Legends and Counter-Strike: Global Offensive players hoping to download cheats have in fact been infecting their computers with credential-stealing malware, security firm Sophos discovered.n


Online njtweb

  • Sr. Member
  • ****
  • Posts: 917
Re: Link preview mod?
« Reply #10 on: August 14, 2019, 08:29:30 AM »
Thank you, I use descriptive links mod right now which when I paste a link to an external article, after I submit it changes the URL into the name of the article. I would think this code modification would probably conflict with the descriptive links mod.

Offline Kolya

  • Full Member
  • ***
  • Posts: 433
  • systemshock.org
Re: Link preview mod?
« Reply #11 on: August 14, 2019, 09:31:08 AM »
Actually they should be compatible, because this linkpreview happens before submitting and leaves the URL untouched.

However it would be trivial to create a linked title with this linkpreview. You just need to build the richLink in linkpreview.js a bit differently. All the ingredients are already there: the url and data.title
So you just need to create a BBC Link of these:
Code: [Select]
richLink= '[url=' + url +  ']' + data.title + '[/url]'; Something like that.