SMF 2 upload large video files and play them in post

[Arantor]

Oh, I saw that. Except you appear to have missed the part where he's still talking about protecting the directory or the part where he talked about it not being visible in the DOM document. If they're level 0 geeks, protecting the directory is unnecessary because they wouldn't even figure that much out. Maybe they're not as clueless as he suggests?

It really doesn't matter what you do with the client side, no matter how convoluted you make it, there's a woefully trivial way around it (Chrome: press F12, press Network, watch all the URLs requested stack up including video URLs). So it's pretty much impossible without building a lot more stuff that can't be done on a typical shared hosting, at least not giving the user what he asks for - only crude, convoluted and still-not-doing-what-he-asked alternatives.

[playjam]

I especially registered an account for this post, because I don't like it when people say "PHP can't do this". Even on a shared host it is possible to implement video upload and a streaming protocol with sufficient security in php, which gets around the typical file size and max run time limitations.  However on a typical shared host you will also run into other limitations (e.g. max number of database connections, bandwidth allotted to your account, etc.) which will just be a hassle to work around and in the end it would be faster and cheaper spending more money on your own virtual server (from 10$ per month) . If you're running your own virtual server anyway it would make more sense making use of existing streaming servers which only use php for the front end. So possible in PHP, yes, but not the easiest or cheapest solution.

@Arantor: "sufficient" security meaning preventing  90% of the users just downloading the video file. There will always be the 10% who will get around any restrictions, e.g. by "simply" writing their own custom player and accessing the decoded video samples before outputting to the renderer (on Windows you'd need to have a good understanding of DirectShow for this).

[Arantor]

Um, I've done some crazy stuff in PHP in the past, including having written my own webserver in PHP. I'm only too aware what PHP can do, and I guess you didn't read the in-depth part of what I was saying.

The problem is PHP is just not designed to be used for that sort of thing, it is designed for loading, serving a page and ending. Yes, you can do other things with it but it's not really designed for streaming because of the way it ties up connections on Apache. It's a bit different on nginx with PHP-FPM but still you run into the same general problem that you're tying up a connection with an inordinate amount of RAM used to do it.

I was never disputing that it *couldn't* be done in PHP, because it could. But not in the confines of the setup under consideration and not without significant problems along the way.

[The Wizard]

Hello:

As Arantor has pointed out what dadaas wants to do in php is imposable or at the very least very, very crazy.

That being said here is a solution -

1. Set up a private board on your SMF. Allow only accesses to people you trust - In other words you have to approve them yourself by hand before they can gain accesses to the board.

2. Create a master file outside of SMF. Name the file something no one would bother looking at. In other words don't name it "Master Video File" Unless you into flashing neon signs saying video piratcy here!

3. create a  ".htaccess" username/password file in your master file - if you don't know how to do this do some research on how to it's simple. (you could even add a track logins deal as well)

4. Convert your video to a flash file. - Why? To make your video file smaller and you still have nice quality. (there are programs out there that do this)

5. Come up with a code to name your videos - In other words don't name them "Iron Man 3"

6. Upload your flash file and include a flash player. When you convert your video you can choose to have a player created along with it.

7. post the links to each video in the SMF protected zone.

Legal Notice: I am not suggesting you use this method for any kind of illegal activity. This is just a theoretical idea.

Wiz

[playjam]

Um, I've done some crazy stuff in PHP in the past, including having written my own webserver in PHP. I'm only too aware what PHP can do, and I guess you didn't read the in-depth part of what I was saying.

I respect what you have done in the past. However I would still like to point out, that there is no need to keep the connection open for video streaming if the video client requests the video in chunks and implementing a check if the user has access rights with every request is trivial. You do not need to have any long running process which shared hosting environments usually do not support. Having said it is possible in PHP (which is a wonderful language since 5.2.3) I totally agree, that it is not a good idea to run a video website on a shared hosting account.

[playjam]

[...] Name the file something no one would bother looking at. [...]

Security through obfuscation is not a recommended approach.

[playjam]

Yea i know smf is not designed for this. But i think i can upload videos for example in folder videos using ftp.
Then please if you or anyone know how to protect this folder from being viewed by direct access.

And then i just insert flash inside post, using bbc code, i think i have seen this before in smf.

Any ideas if this will work?

Lookup "pseudo streaming flv php" on stackoverflow, there are a few examples there which will help you start out in the right direction.

[Arantor]

I assume you're referring to http://stackoverflow.com/questions/13246280/pseudo-streaming ?

That isn't streaming. That just opens a conventional connection and dumps the entire file in 16KB blocks at Apache. Which means the connection is still open for the time the user has to download the file while PHP serves the file, or it's held by Apache, configuration depending. Either way you're tying the connection up for the length of the download, and that means a flat minimum of an 8MB PHP instance and one of Apache's child connections, which means if a lot of users are doing this, it's simply down to which users hit MaxChilds or oomkiller first.

The other method that you seem to be suggesting is to have the server pull the file in chunks with repeated polls. Given the code required by SMF in a bootstrap followed by the flat minimum of 2 queries to do minimum possible authentication (and, typically, it will be nearer to 4 not 2), every chunk load, that's a non trivial load being attached.

[playjam]

The second  method is what I am suggesting: Have the client request chunks, do not keep the connection open. We demonstrated this method (however written in C/C++ and Java) streaming live TV with a HTTP 1.0 server on the CeBIT 1999 and with seeking capability on the CeBIT 2000. Obviously you would need to do some clever thinking how to authenticate without hitting the database every single time.

[Arantor]

Well, not being funny but you're so smart, how would you do that?

[playjam]

even god calls me god

E.g. cache the db request. Anything is faster than a database query on a (typical) shared host.

[Arantor]

What exactly do you think you can cache? The whole point is that you supply a token of some kind to the site that evidences that you are logged in. That token has to be validated somehow. Additionally, you need to update the session details because it all revolves around actually having that session... and SMF puts sessions into the database for security reasons (and has done so for the last decade)

[playjam]

The video streaming script only needs to adhere to SMFs method of authentication for the initial request. After that you roll your own.

[Arantor]

So it still has to issue a token of some kind and still has to authenticate that token... which still has to be in the database because nowhere else can be considered either reliable or safe.

But that still brings a new database query to check token against video. Better than the 4 that SMF could make. But I'm still not going out of my way to implement something that's going to cause plenty of issues on shared hosting for a handful of users, despite your apparent insistence that I should (the whole fact of signing up just to have this argument is... curious)... honestly I have more important things to attend to for SMF as a whole than this right now.

[playjam]

You must be joking, I would not dream of telling you what to do in your spare time. I simply strongly object to putting the blame on PHP.

[Arantor]

Where am I joking? I just don't get why you signed up purely to have this debate.

And as for using PHP, considering that I have been using PHP for 10 years (including strangeness like PHP-GTK as well as writing daemons, and I hold a current ZCE certificate) I'm only too aware of its limitations, especially given the overall context in which it is being used. If it were purely PHP on a shared host it would be problematic in itself because of the way most shared hosts measure CPU and I/O use - but that's not the real problem here.

[playjam]

I signed up because you were giving the right answer for the wrong reasons. As I mentioned earlier it is possible to achieve what dadaas wants to do purely in PHP with an appropriate client, even on a limited shared host. If he wants to do this, he can go ahead. I think SoLoGHoST and I have given enough hints how to approach the matter short of actually writing down the code. However due to the amount of time and expertise required, I estimate it would be less costly to use non-PHP existing solutions especially since he probably can install the stuff on his VPS.

[青山 素子]

Time for me ot chime in. Arantor is correct. Trying to build a streaming solution around PHP is not a very good idea. It just wasn't designed to do this kind of stuff. Anyway, here's the options you have for streaming videos:

1. RTSP, RTP, or some similar streaming protocol: This was designed to stream media files. It has all the awesome stuff you need to do things like make downloading the source difficult. The problems are that it requires special server software and some firewalls block it.

2. Pseudo-streaming: This is a great low-cost solution and is basically an implementation of progressive download. Format your video for quick loading (there are tools for this) and both Flash and HTML5 video support it. You run it over HTTP, so it's easy to set up. Downside is that the URL to the media is easy to find. You could use PHP to authenticate the session, but the client can just save the video source as it's being sent. You'd have to allow direct access to the file from outside SMF's system as SMF doesn't support things like partial downloads on attachments or anything going through that mechanism.

3. HLS (HTTP Live Streaming): This seems to be what playjam was talking about for "breaking it into chunks, as that's the way it works. This could possibly be implemented in PHP, but it certainly couldn't be made inside SMF. It seems most existing software for HLS server functions are actual servers, not some PHP script. This is likely for a very good reason - PHP isn't suited to this use.

4. Some custom tool that uses a proprietary solution involving authentication, etc. This probably wouldn't work well in PHP and you'd be tied to either a special browser plugin or some client-side software like Flash or Java applet. Of course, that rules out mobile use without developing a special native mobile application just for your stuff. Too much work unless you're very rich or have very special needs (Netflix and Hulu are examples).

So, yeah, all four solutions I can think of are basically bypassing PHP and using either the underlying http server (pseudostreaming) or are using a special server software to handle the protocol (RTSP, HLS).

Oh, yes, I can even back up my words with qualifications. One of our clients at work had us design a user-generated-content video site catering to 3D videos. We went through all the different possibilities of streaming videos, including working with providers like Akamai, EdgeCast, etc. As a result, I know way too much about video streaming technologies, including some interesting issues with Akamai's RTSP and pseudostreaming servers. (I also know way too much about post-processing 2D video to "convert" it to 3D stereoscopic and all the fun stuff involved in encoding that for proper streaming situations.)

[SoLoGHoST]

For streaming video... youtube, vimeo, or jwplayer.  All of these support most, if not all, platforms, including mobile devices.  Like I said, bitsontherun.com uses jwplayer (latest version) and offers a way to upload videos using the API, with a php form that you can create and use to allow video uploads.

There is a free version of the jwplayer that you can use where they put their logo in it.  It is worth it though.

[Arantor]

And the client side was not, and is not, the problem. There are a bunch of clients... but the problem is ALWAYS the server component serving them.