Hello interwebz,
My movie viewing experience has gone through several stages over the years and just recently I got a setup that I’m thoroughly enjoying more than any previous solution.
First, a few years ago I made a post that mentioned using the PS3 with Ps3MediaServer (PMS). That worked for some time, but I like digital copies of my media and when Cinavia became a big thing, and the fact I only use free ripping software and not the pay windows software that were “Cinavia compatible”, it killed my new movies.
So from there I went to a local Best Buy and got an older WD TV Live Plus and put the B-Rad modified firmware on it. I don’t believe they sell the models that work with this firmware anymore unfortunately, because it really worked great. I mounted my Movie directories from my Desktop PC to the WD TV using NFS, and when I figured out how to use MovieSheets it was beautiful. It was like browsing a netflix listing with plot, ratings, screenshots, etc. of all my movies.
The downside to the WD-TV was the MovieSheets were not automated; until SheetMaker added a cli version and I was able to Cron it but it still wasn’t perfect. Also, because the movies were in my desktop there was a few times my family would be watching something and I would reboot which would cause obvious havoc. So I decided I wanted an All-In-One solution that could house the movies, and work like the WD-TV, and also run any other media software I might want (such as SickBeard, etc.)
The end result was me building a fairly beefy HTPC:
It’s a bit on the big side, but not much bigger than most A/V Receiver equipment and doesn’t look too out of place near a TV. It’s quiet, and with the CPU cooler I have, as of yet, not had any heating issues.
So, now that I had the equipment I had to decide what to do with it. I started with going the manual route with my second favorite Linux distro, Manjaro. I followed through the information found here: Forum Link; as well as the XBMC Wiki.
I used the AUR, I tried the Frodo xbmc, the Git xbmc, and the xbmc-xvba packages. All of them worked, all had little differences, and some things didn’t work as well as others. For a while, I stuck with the xvba package as the hardware decoding was nice when I could use it. I also realized I had to use the very latest AMD drivers to work with the APU.
When I first started, I had all my media on a 4 x 1TB drive that was a manual Raid 1+0. I then decided I wanted to tinker with BTRFS again and proceeded the complicated process of breaking the Raid’s, copying the media over, building the btrfs setup, and setting it to raid 1. Personally, I am in the camp of ZFS being the “Last word in filesystems” due to the insane feature set while still being stable and at the time was unaware of ZFS On Linux and thought using ZFS on linux meant going through Fuse. So, BTRFS was my next hope.
Unfortunately over several months I had tons of issues both with BTRFS (seg faults on mounts after unclean shutdowns, data corruption, etc) and updates to the system, so I scrapped it. Also, somewhere along the way an update on my Manjaro or something I changed caused my XBMC to go haywire as well.
So, I decided as cool as it was to sport Manjaro, a rolling release distro as my XBMC server it didn’t “feel” like a media device. The manjaro splash screen still came up and the typical linux text booting on the TV on reboots. I know I could have customized this, but I like to change things up periodically anyway so I turned to XBMCbuntu to give a try.
I then reloaded my drives into a proper RAID10 but put my root and swap onto two separate Raid+0 arrays (was aiming for speed here, I can always reload and /home was somewhere else.) I also ran into major AMD issues here as the latest Ubuntu AMD drivers were not playing well at all with my APU. I had to download the latest file from AMD directly and do the manual install process to get XBMC to launch.
That’s when I came across an article at phoronix that said ZFS On Linux was ready for primetime and decided I wanted to try that! So, much to the wife’s chagrin, I shutdown the HTPC again and booted into Ubuntu LiveCD and proceeded to convert my Raid10 into a ZFS Raidz2 pool. I will link all the documents to get through this at the end, but essentially I failed out one drive, cleaned a TON of media off my drives, copied the full 1TB over to the now separate drive, and made a “degraded” 4 drive RaidZ2 with ZFS and a]14 with the intent of booting to ZFS.
Creating the pool was fairly uneventful, but initially I was trying to do a full rsync of the existing OS (XBMCbuntu) to the rpool and kept hitting errors. Upon some research I noticed quickly that was a problem with 32-bit OS’s and ZFS. Something I didn’t realize about XBMCbuntu (that it’s 32-bit.)
So that meant I had to install full Ubuntu 64-bit (the minimal CD would not let me load the ZFS repositories), but I didn’t want all the bloat. Thankfully the How-To on booting to ZFS walked through the debootstrap procedures to install a minimal system.
Using that, combined with this How-To on the XBMC Wiki, I was able to build a minimalized Ubuntu running nothing but XBMC and no Window Manager on a ZFS RaidZ2 pool.
Unfortunately I ran into issues booting to ZFS from Grub. Seemingly related to this issue. Even after a scrub I couldn’t get Grub to detect the / and any time I tried to install to the drive grub would fail telling me it can’t read /boot/grub on boot. I ended up re-partitioning my drives and including a 200M boot partition that I put into a 4 x Raid 1 with EXT4 (which is close to what the How-To says to do anyway).
With that, now my system boots faster than any of the previous configurations (ext4 + Raid 1+0, ext4 on Raid10, and BTRFS), and I have all the features of ZFS at my disposal, and with RaidZ2 I can survive up to ANY two drives failing without losing data (better than Raid10, which is 2 drives but only in separate mirrors.)
I came across this post, Link, that talked about poor performance for that guy with RaidZ2 with 4 drives so I ran his same two example tests:
`# time ( dd if=/dev/zero of=tank/tmp.dat bs=2048k count=50k ; sync ; )
time ( dd if=tank/tmp.dat of=/dev/null bs=2048k count=50k ; sync ; ); rm tank/tmp.dat`
For some reason, mine came back with insanely high numbers. I was getting 900+MB/s writes, and 2.0 GB/s reads with my drives. I have a mixture of 7200RPM SATA III drives Western Digital and Seagate. I do have compression=lz4 on all my pools.
So, in summary, my current Home Theater PC is a custom XBMC+Ubuntu 13.04 Raring+ZFS Root. It automatically searches for information on my movies, automatically gets links for Trailers, and eventually will even do Live TV for me when I get a TV Tuner. So, if anyone is out there wondering if XBMC+Linux+ZFS works well? I’d say yes.
References:
Install XBMC onto Minimal Ubuntu
A fedora guide to ZFS – It showed me the syntax for lz4 compression.
Unfortunately, this isn’t as nice a step by step as my LVM+Raid on Ubuntu was because this took me 2 evenings to get all this right and work out all the quirks and I don’t have the time anymore to do it all over again to document every command :). The biggest lesson learned that I had though was I recommend using partitions on drives instead of the whole drive which is recommended by the FAQ for performance increases. I used the whole drive first, it created a GPT partition table with 2 partitions on it and I had no end of trouble getting Grub to work with it. I eventually scrapped it and one by one broke my ZFS pool, repartitioned (1 200M boot and 1 ZFS remainder) each drive with a standard MBR table, and grub was quite a bit happier with that.
Thanks for stopping by!