Home Streaming Media Server

[BrianG]

I want to build a home media streaming server, but can't seem to find a free solution that'll give me what I'm looking for:

- Runs on Linux (preferably Ubuntu)

- Able to stream video and audio files.

- Accessible via computer/laptop (Win and Mac), smartphone, Wii, Xbox, etc. The Wii and Xbox is not required, but would be a plus.

- Able to transcode so the media can be played on any of the devices above.

- Able to mount and play DVD files in ISO format automatically.

- Able to automatically create/update media list as files are added to the library.

- Web browser interface to browse and play the media from the server to avoid having to install specific software on the devices.

Basically, I'm looking for an interface something like Netflix where you can browse for movies and play them directly in the web browser. This won't be available to anywhere outside my local LAN, so I don't have to worry about bandwidth issues. I just want to be able to have my media available anywhere in the house on any device.

VLC seems like the closest thing so far. But, I don't think it mounts DVD ISO files, and everytime I add media, I'd have to update the "playlist" on the server. It's browser interface doesn't look like it will do what I want either. I may have to play with this to be sure.

Opera Units also looks like a decent contender, but seems to require the Opera browser. I don't want to be tied to any specific software on the client - the server should be doing all the work.

[brainanator]

Wow, you really are asking for a lot with this one. Some day I want to set up a bad-ass system like what you're talking about so I'll be watching this thread. Sadly I don't have much to contribute...

[bdebde]

Why must it run on linux? The free part?

I am in process of trying to do the same, with my windows home server... starting to look as though one program won't do it all, but I can run multiple media servers, just need to find the right ones.

[BrianG]

Yeah, Linux because it's free. Otherwise I'd have to shell out another $140 for a "system builders" license of Win7 Pro.

I've done more research on this and just can't seem to find what I'm looking for. I can't be the only one that wants something like this?

Since I want the client devices to use a web interface via browser (which wouldn't require special client software), I suppose I could bite the bullet and get a Windows license. Then, I can install IIS, develop an ASP-driven site using the filesystem object to dynamically create the file listings. I could use Linux, Apache, and PHP to do the same, but I'm not nearly as proficient in PHP as I am in ASP.

Anyway, streaming MP3's should then be fairly simple, but the video streaming is still a challenge. Since all my videos are in ISO disc-image format, I'd have to find a way to automate the mounting of the image and stream it from the virtual disc. And then, two people want to stream videos at the same time, the system would have to be smart enough to "know" that one virtual drive is already mounted/in use, and use the next available virtual drive.

And I still don't know how to stream that video over HTTP protocol. I suppose I could stream the content using Flash (like YouTube), or Silverlight (like Netflix), but then I have to find something (preferably free or cheap) that does the transcoding.

[skellyo]

Not sure if it'll do anything with an ISO image, but you should check into FFMPEG for transcoding.

[Thirdgen89GTA]

I can help with this!

• Runs on Linux (preferably Ubuntu)
• Able to stream video and audio files.
• Able to transcode so the media can be played on any of the devices above.
• Able to mount and play DVD files in ISO format automatically.

There are several options available to you. However, there are some things that either aren't implimented yet, or aren't implimented well. This means you will have to run multiple background help apps or services to meet all yoru needs.

First off, we should start with a Hometheater PC, or mini PC.

Your best bet is to buy a Home Theater PC case that includes an IR Receiver. You'll also need a remote thats compatible with it. If I were you I'd look towards cases that are compatible with the Logitech Harmony line of remotes.

Next you'll need your OS. Since you want to run linux cause its free that puts some forks in the road.

You can use the maintained "Mythbuntu" system which is really just a version of Ubuntu that boots directly to a HTPC type interface. Mythbuntu has some codec incompatibilities due to licensing issues. But strangely Boxee and XBMC get around this despite both having their roots in Linux OSs.

The other option is to run a basic linux system and run a HTPC program like Boxee, XBMC, or another application like that on top of it.

Mythbuntu features/options/drawbacks
Good support for multiple Server/Client installs. It supports a backend server that handles all transcoding and file server duties. It can also play DVD.iso's stored on the file server. Because it supports a back-end and front-end setup you can use a powerful system to do all of the transcoding/serving and hide it in a closet if its noisy. You can then have a relatively weak front-end PC thats hooked up to your home theater. Since the actual transcoding (the most CPU intensive HTPC task) is handled onthe back-end server its just streamed to the client HTPC. I recommend a wired gigabit network as a standard! To transode HD video you will need some serious back-end hardware if you want to maintain good framerates and avoid audio/video sync issues. This also supports "apps" like Pandora/Last.FM/etc....

Mythbuntu is not as easy to setup as Boxee or XMBC though.

XBMC and Boxee type systems.
These are much simpler to setup. But they aren't nearly as configurable and they don't support back-end decoding. So the client PC has to be much more powerful. Out of the box though they will play nearly every format out there, and both include tons of applications. XBMC and Boxee will run on Windows/Ubuntu/Mac installs and can reference file servers via SMB/AFP/NFS protocols, or local content. It "installs" just like a normal application. They also have android and iPhone based apps that can act as remote controls over a WiFi connection.

That covers the basic HTPC type setup. Now onto your other requirements.

• Able to automatically create/update media list as files are added to the library.
• Web browser interface to browse and play the media from the server to avoid having to install specific software on the devices.

This is where it gets tricky because there is no "all-in-one" solution.

What you will need to do is install multiple "services" that run in the background to handle these types of tasks.

What you are looking for now is a webserver that can server up media content. You can look into the ORB software or Twonkey Media Server. The problem with both of these service apps is that they don't happily mesh with ANY existing media programs so their "ratings" and "smart playlists" don't transfer over. Aka if you rate a song, you will have to separately rate and create a similar playlist on another machine. This seems simple, until you get to massive libraries. Then it becomes a nightmare to keep ratings and playlists synchronised. They are also not free!

This is my setup at home, and it works pretty well and is easy to keep sync'd up.

• 1 Linux PC running iTunes via WINE.
• 1 Macbook Pro (considered the "Master" library for iTunes)
• 1 MacMini hooked up to Home Theater via a DVI->HDMI converter and a SPDIF TOSlink cable. Video is passed through the DVI interface which is pin compatible to HDMI. But no audio passes through. For that i use the SPDIF link to transmit DD5.1 and PCM audio to the receiver.
• External 2TB USB drive hooked up to MacMini.
• The MacMini runs a paid app called "Connect360" which shares iTunes library content as well as smart playlists. The smartplaylists were the "killer feature" I wanted.

The Mac Mini hosts all Video files and has a duplcate of the iTunes library. The iTunes library that resides on my Macbook Pro is considered the Master library. Meaning if i want to rate a song, or create a smart playlist I do it on the Macbook Pro. It is the master simply because I use it more than any other PC I have. Whenever I make enough changes I run a bash script that will exactly duplicate the iTunes library from the Macbook Pro to the MacMini. The real way it works is it runs an Applescript that tells iTunes to quit if its open. Then it checks permissions, and finally it uses RSYNC to delete/copy files on the MacMini so its a perfect replica of the library that resides on my Macbook Pro. It runs fast, takes only 1 or 2 minutes to sync up the library.

Now that seems complicated, but its really not. I'll happily post up the script if anyone wants it.

Now, as for the MacMini's duties. Via the Connect360 helper application, it serves up the Video, iPhoto and Music libraries to all DNLA compliant devices (Xbox360, Playstation 3).
iTunes itself is set to "share" its library. Using the iTunes remote control appliation from an iOS device I can remotely control iTunes on the MacMini meaning I don't need to use the TV to play my music playlists, I just turn the Home Theater receiver on and set it to the input the Mini is hooked up to.
The MacMini also runs the Boxee application, which is set to run automatically at startup. This is the main video interface for the home theater. The Boxee application is configured to look at the iTunes library thats stored locally on the MacMini in /Users/%user%/Music/iTunes/ and it also is set to look at the USB 2TB external drive where i store ALL of the video files. Boxee can also play any DVD I put into the MacMini's drivebay. It can also play VIDEO_TS folders, and DVD iso files. Plays MKV, AVI and just about every video format out there.

So on the surface its actually really easy to use. But there are several services/applications running on a single machine that provide a single interface to the home theater.

Now here's were the iTunes libary sharing comes in. The iTunes program installed on my Linux machine via WINE looks on the network and finds the shared iTunes library hosted on the Mini. So I just connect to that and stream the music to the linux box when I want to play some music on the linux box. I also have a Mac Pro desktop that i use for video editing, ripping, encoding, photoshop...etc. If I have new movies, DVDs and such what I will do is connect to the shared folder on the USB2 drive hosted by the mac mini and simply copy the files over to the USB drive via the network. Boxee automatically recognises new files and adds them to its known library.

Now why is the Macbook Pro laptop my "Master" library? Because my iPhone is synced to the iTunes app on the laptop, and I have the laptop with me at work and at home. So i'm more likely to add music, create a playlist, or rate music on the laptop. So if I make any big changes i just use the script I wrote which will duplicate the iTunes library onto the mac mini. Here's a copy of the script. Its Stupidly simple. the script is saved and I can execute it by double clicking it from the MacMini desktop or I can invoke it from the command line via SSH from any PC on my network.

PHP Code:

osascript -e 'tell application "iTunes"' -e "quit" -e "end tell"
sudo chmod -R 777 /Users/william/Music
rsync -va --delete $user@$ipaddress:$sourcepath  $destinationpath
open -a "/Applications/iTunes.app/" 


• Line 1: closes iTunes if its open.
• Line 2: modifies permissions of the destination so there aren't any errors when writing the files. I'm not too concerned with security on my local network.
• Line 3: RSYNC will compare the source directory (Macbook Pro) with the destination director (MacMini) and then selectively delete files and copy new files from the Macbook Pro. If I delete a song on the Macbook Pro, RSYNC will delete that song on the MacMini. If I add a new song, it will copy it over. If I rename files it does the same. If I update album art or ratings it also duplicates those changes over on the MacMini. Its great because it doesn't wast time copying data that hasn't changed and already exists on the destination.
• Line 4: re-opens iTunes on the MacMini.

[rawfuls]

Holy crap, BrianG will be a happy man on New Years Day!

[BrianG]

Wow thirdgen, that's a lot of info! Thanks! It'll take me a bit to digest that...

[Thirdgen89GTA]

Was thinking about this the other day. You pretty much want simplicity. XMBC makes a "live" distribution. Meaning its basically XBMC already setup on linux, runs straight off a DVD and can be installed.

http://xbmc.org/download/

If you already have the hardware you are going to run this on, just download the "live" iso from the xbmc site, burn it to DVD, then pop it in your computer and boot from it.

Some XBMC interface pics. Yes, it does look like this on your HDTV.

[BrianG]

Thirdgen:

Again, thanks for all that info. I've actually used XBMC before and it does look/work great... if I wanted the media server right near the TV. However, my server will be sitting in the basement out of sight. As I said, I'm looking for more of a true "server" topology where the user opens a web browser to the server's URL/IP address, selects the media they want to view/hear, and the server serves up the content in whatever way the device supports. Basically, a modified version of a Netflix-like service. For greatest compatibility, transcoding into Flash might be the easiest/best, and would require the least amount of client horsepower.

I'm still looking through the options you suggested though. There might be some command-line or developer API hooks that I can take advantage of in the way I want to run things.

[Thirdgen89GTA]

Originally Posted by BrianG

Thirdgen:

Again, thanks for all that info. I've actually used XBMC before and it does look/work great... if I wanted the media server right near the TV. What I'm envisioning is more of a true "server" topology where the user opens a web browser to the server's URL/IP address, selects the media they want to view/hear, and the server serves up the content in whatever way the device supports. Basically, a modified version of a Netflix-like service. For greatest compatibility, transcoding into Flash might be the easiest/best.

Check out Orb. I used it years ago. Basically it logs into a central server, and you can visit a webpage link and your entire media library is at hand.

Winamp had an implimentation of it running. Played everything.

Worked on my iphone, mac, pc, everywhere I had internet. In-house too.

[bdebde]

Have you looked at TVersity yet?

Yeah orb was the other I was thinking of but could not remember the name.

[Thirdgen89GTA]

Originally Posted by bdebde

Have you looked at TVersity yet?

Yeah orb was the other I was thinking of but could not remember the name.

There's another one I forgot about. That does the same thing as orb.

[BrianG]

A little update on this "project":

Well, after trying several solutions, it seems that either they don't offer exactly what I'm looking for, or seem to be cobbled-together workarounds and hacks. None of the solutions are wife-approved either (meaning: simple to use).

So, I've decided to just designate the "server" as a simple HTPC. On that note, I thought XBMC would be the ticket, but there are some gotchas that I didn't care for; 1) Using netflix is a bit of a pain (it simply calls up IE/silverlight but the browsing and usage isn't as user-friendly for family use), and 2) the whole interface isn't user-friendly enough for everyone. Sure, they could "get used to it", but that's not acceptable IMO.

So, I ended up tinkering with Windows Media Center and once I added a couple plug-ins to auto-mount ISO video files and install the netflix and internet TV plugins, it works swimmingly. The interface is fairly "pretty" and couldn't be easier to use, which should satisfy everyone (including my 3 year old).

And, I can still share the music folder and connect via SMB on my smartphone to listen to music where ever I want. I'll probably install IIS (web server) and MySQL (or SQL Express) on that box as well and use it for some home apps I want to build.

I started shopping around for the actual hardware, and figured on getting a base-system something like this, install Win7 Pro "system builders licence" ($140 for the full version), and slap in a TV Tuner Card (which includes a remote). I figure a quad-core ~3.0GHz CPU and 8GB DDD3 will have more than enough horsepower, and I'll partition a 1TB drive into ~100GB for the base OS and programs, and the other ~900GB for recorded DVR content. Later on, I may get a dedicated video card and install a couple of less taxing games on it as well - just to round it out.

And a bit later on, get a couple of 2TB drives, configure them as Raid1 (for data redundancy) and start ripping all the DVDs we have as ISO files, particularly the movies watched most often. I'll probably dump all the media (like pics and music) on there as well.

[Unsullied_Spy]

Quad core and 8 GB of RAM? Is this a media center or a F@H bot? I'm looking at doing something similar for a lower-power consumption alternative to running my gaming machine 24/7 as a file server and was going to run one of those little ~$30 Sempron CPUs unlocked to a dual core and 4GB of RAM. Doesn't need a whole lot of power, my 1.83GHz C2D in my laptop can run just about anything I need it to, and the Sempron will be faster once unlocked and should still run on very little power.

I haven't checked hard drive prices in a while, have they gone down? There was some serious flooding in Taiwan which caused HDD prices to spike, the 1.5TB WD Black I bought for $120 was nearly $400 last I checked...