When Ripping Becomes Personal

by Damian on July 6, 2011 · 41 comments

in Guides

By the title you make think this is going to be a post about the time I ate Taco Bell and the hours that followed :-) Well, as much as I would love to share “ripping” stories, this post will be about a different sort of ripping. I just received my Lord Of The Ring (LOTR) Blu-ray Extended Edition boxset, and as usual the first thing I do when I purchase a new Blu-ray disc is rip it to my Windows Home Server. Well, the LOTR boxset presented a bit of a challenge because each movie is split out onto two discs. The easy solution would be to just rip each disc individually, but I would rather have one clean rip per movie, especially since this would guarantee the best compatibility amongst the various playback software/hardware that I use and test. Merging two discs into one rip shouldn’t be difficult with the software that is available, but unfortunately I encountered some complications that turned a simple rip into a weekend project. Let’s take a look at what is involved and where things went screwy!

Step 1: Ripping The Disc:

The first thing I should mention, I do all my ripping/editing on my desktop PC, and then move the final product over to my Windows Home Server for storage/streaming.  You will need to rip each disc to your hard drive and can just follow whatever your preferred method is. I prefer to rip to a Blu-ray folder structure using AnyDVD + Clown_BD, but really you can use any method you like whether it be folder rips, ISOs, or mkv (you can read here about the variety of mkv methods).

Additional notes – you will need a decent amount of free hard disk drive space (approx 120GB per movie) since each disc will be approximately 30GB (so 30GB x 2) plus the final merged copy will be approximately 60GB. Obviously the two 30GB files can be deleted once done

Step 2: Merging Chapters (optional):

This step is completely optional. If you want to maintain chapters you will need to do so manually. If you use Clown_BD you will be left with a chapters.txt file as part of the temp files. Make sure you move the txt file from the first disc before you rip the second disc as I believe Clown_BD will simply overwrite the first chapters.txt file. Below is the temp files generated by Clown_BD, I had renamed the chapters.txt file from the first disc before ripping the second disc.

If you forget to keep the chapters you can extract by either doing some eac3to command line or use an eac3to GUI like HDBRStreamExtractor. For mkvs you can also use MKVExtractGUI-2.

Next up (and I don’t know if  there is an easier way) you will need to manually edit the Chapters.txt file doing a little math. The problem is that the Chapters file for each disc starts at 0:00 obviously. This is fine for part one of the movie, but you will need the chapter times for part two to be adjusted to start after the part one ending. To do this I simply whipped out excel, typed in all the chapter times, and did some math to figure out what the part two chapter times would look like. Once done just save a master chapter.txt file to use later on.

Step 3: Merging the Two Discs:

With both discs ripped and the merged chapters.txt file created the final step (hopefully) is to merge the two discs together. Since I want to merge into a Blu-ray folder structure I decided to use tsMuxeR. Under the Input tab click on the “add” button and navigate to the m2ts file for the first disc (should be under BDMV -> STREAM) . Next click the “join” button and navigate to the m2ts file for the second disc. If you joined properly the second disc m2ts should have a “++” in front of it. Under “Output” select “Blu-ray disk”.

Now go to the “Blu-ray” tab, choose “Custom chapters list” and copy in your merged chapter timestamps

Finally click the “Start muxing” button which will create a newly merged Blu-ray folder structure combining both discs.

You can accomplish the same thing for mkvs using mkvmerge. Under the Input tab add the first disc mkv (or individual files) by clicking the “add” button. Click the “append” button to add the files from the second disc.

To add the merged chapters go to the “Global” tab and under “Chapters” browse to the merged chapters text file.

Click “Start muxing” and soon enough you will have a merged mkv.

Step 4: When Things Go Wrong, Fixing A/V Sync Issues:

If everything went well you should have a perfect merged movie. Once the merge is done you want to play back the movie to make sure there are no issues. In particular make sure you play back clips from what would have been the first and second disc. In a majority of cases you should now be done. Unfortunately, with LOTR when you get to the second disc portion you will have terrible A/V sync issues (maybe about a 2-3 second delay from when lips start moving until you actually hear the words). It turns out the culprit is there is about a 3-5 seconds of blank video at the end of each first disc. When merged this blank video pushes the video on the second part of the merged disc ahead of the audio. The solution is to trim out this blank video and fortunately Slysoft forum member Maltese has determined what the appropriate timecodes to trim per movie are (see here for his original post). For discs where there are A/V sync issues such as LOTR you will actually need to do this step before you merge the discs together in step 3.

Go back into tsMuxeR and add the first disc. Make sure output is set to “Blu-ray disk”. Next go to the “Split&Cut tab, check “Enable cutting” and set the End timecode. Click the “Start muxing” button to begin the process. Here are the timecodes Maltese came up with for each movie:

  • LOTR – The Fellowship Of The Rings – 6338 seconds
  • LOTR – The Two Towers – 6396 seconds
  • LOTR – The Return Of The King – 7656 seconds

Once done follow the steps in Step 3 to merge the newly trimmed Disc One with Disc Two.

A similar trimming can be done for mkvs with mkvmerge. For step by step instructions, see Yraen’s post here.

Step 5: A Nasty WHS/Hardware Bug?

For most people getting through the 4 steps should get you a nice merged movie ready for playback without worries of juggling multiple discs. Of course though as always happens to me, I ran into another snafu. As I mentioned earlier I do all my ripping/video processing on my Windows 7 desktop PC, and once done I then copy over to my Windows Home Server. When I went to copy over one of the merged LOTR movies within 20 minutes the transfer speed would start to slow dramatically, and then I would get a message that the operation could not complete due to insufficient resources. I tried several more times, same problem. During the attempted transfer my Windows Home Server was being brought to a crawl. Accessing content from my media players was taking forever, and I noticed that one of my drives (most likely the one being written to) was heating up dramatically. Since it wasn’t working over the network, I decided to copy the LOTR to an external hard drive, connect that hard drive directly to WHS and do the transfer from there. 15 minutes in to the transfer same issue, failure due to insufficient resources. I have never had an issue transferring files over, but each LOTR movie was approx 60GB, much larger then the typical 15-35GB movies I have.

Ready to throw in the towel I decided to make one last attempt. This time however I decided to go back to my trusty tsMuxer and split the movie into two parts (30GB each). I then copied each part over to my Windows Homes Server without issue, and then proceeded to use tsMuxeR once again to re-merge the files. Finally success! Well, since then I did some research and found that other Windows Home Server users had encountered the same error related to the transfer of larger filesizes. Some thought the issue was related to faulty hardware (possibly insufficient RAM) while others had to use a different software (such as TeraCopy) to successfully do the transfer. Either way this little bug/issue I ran in to was the final kick in the groin for what turned out to be a weekend project!!

Update – on the last LOTR movie I decided to do the tsMuxeR merge in Step 3 directly to my Windows Home Server instead of doing to my Windows 7 desktop, and it succeeded without issue. So the only time I encountered the issue was when I tried copying the actual 60GB file over.

Additional Information:

  • These steps (specifically the timecodes) are specific to the Blu-ray Extended edition. The merging process itself however would apply to any instance where you wanted to merge multiple files.
  • Several people I have spoken to have noted that they did not encountered the a/v sync issues when merging using eac3to. I did not test this but it leaves two possibilities. (1) using mkvmerge or tsMuxeR is causing the initial audio delay that requires the trimming or (2) it is an issue with specific discs. I have the US version of the discs with others have confirmed the same issue. The folks I have spoken to so far who haven’t encountered a/v sync issues have the EU version. I already know there are some differences between the US and EU version, such as forced subtitles which are burned in on the US version but a separate subtitle track on the EU version.




Article by

Hi, my name is Damian, and I'm tech gadget addict! Although I always had some interest in technology, it wasn't until I got my EX470 and more importantly found Mediasmartserver.net, that my interest became an addiction. My goal, aside from world domination and to see the Mets/Broncos win another championship, is to set up the perfect digital home where all my media is available at the click of a button. When I am not writing for Mediasmartserver.net you can find me over at my blog at http://www.adigitalhomeblog.com or follow me on twitter


{ 40 comments }

Ryan July 6, 2011 at 3:41 pm

This is exactly why I didn’t burn these movies. My wife even said…”do you really want to waste 180 gb on 3 movies???” Add to that shocker what you just went through…hmm. I think I’ll just watch them from the disc until I move from 6tb of local storage to a separate storage server.

Query: Would this 60 gb file issue happen with flexraid or unraid???

Damian July 6, 2011 at 3:46 pm

For me it actually works out to 360GB (I have duplication enabled in WHS)! I don’t mind the space, just the price for convenience of not having to manage multiple discs. I am more impressed that you are talking GBs with the wife :-)

The issue I had does not happen to everyone, it appears there may be a variety of factors specific to my setup that caused the issue. I am not sure if this would affect other O/S (such as unRAID, etc…).

flips July 11, 2011 at 11:08 am

Just did this weekend. It transfered fine to my unraid server.

Iheartunraid! :)

dsudot July 6, 2011 at 5:30 pm

Thanks for taking the time to post your trauma, will definitely help me when i go to rip these.

Tyler July 6, 2011 at 5:59 pm

i would recommend running them through handbrake and creating an mp4 file. that way you get broad compatibility and much more manageable file sizes. you have the disks as backup in case you want reference quality to impress your friends/wife, but for casual watching, you probably wont be able to tell the difference between the original and transcoded files. i have personally-tweaked handbrake presets if you would like them. my lotr movies come in around 4.1-4.4gb per movie.

protip: you can even drop down to 720p for archival purposes and most people wont notice. i have been doing that for all of my movies on my whs, and when we have friends over to watch stuff on our 65-incher, they comment on how “crisp” the picture is. that isnt to brag, but to illustrate that even 720p blown up to 65″ still looks good.

Damian July 6, 2011 at 7:47 pm

Really just comes down to personal preference. IMO it is a waste to purchase a BluRay to just down res it (obviously a mobile device being a different scenario). Also, by going with mp4 you are losing the full HD Audio, which is just as important as PQ.As I mentioned though, everyone will have a different setup so you just have to go with what works best for you.

Nick July 6, 2011 at 8:11 pm

HD LED TV = $2.5K
BR Player = $150
BR Disc = $20
HDMI Cable = $15
Receiver = $500
Speakers = $500

My wife saying that she cant see or hear the difference between this and the old set-up with a DVD = Priceless

Damian July 6, 2011 at 8:14 pm

Haha, well everyone is different. Different setup, different ears, different acoustics in the room, etc… My wife would probably answer the same as yours, but then again I get Blu-rays like LOTR for myself. As screen size gets larger there is more chance that you will see a difference in an encode versus lossless (of course also dependent on t he encode settings, content, etc…)

Tyler July 6, 2011 at 8:31 pm

damian and nick,

i think we are on a similar page, but not the same one. since the layman cant perceive the differences, i keep transcoded copies of my movies on the whs – which stream to our win7 htpc or are transcoded on-th-fly by air video for our idevices/apple tv. plus it only takes 2 minutes to transfer the file over to the server (but to be fair, it does take 2 hours to transcode).

as far as the inferior sound, i havent noticed too much of a degradation. i guess my receiver does a good job re-imaging each channel.

much like you, when i want the full experience, i pop the blu-ray into the htpc and watch in reference quality. but i also like the convenience of smaller, less resource-intensive files for archival purposes. to each his own.

Ryan July 7, 2011 at 6:45 am

No offense Nick but you and your wife probably can’t hear a difference between the blu-ray track and regular Dolby Digital because you invested most of your budget into the television.

I guarantee if you had doubled your speaker budget you would have heard a difference. $500 for speakers is a very small budget…very small.

And you know what…I think it’s fine if that is what you wanted to spend and how to allocate as long as you understand that a $1,000 to $2,000 speaker budget would have increased your listening experience by a measure I cannot express.

I would argue that the audio is more important than the visual. But that is just me…

Nick July 6, 2011 at 8:07 pm

Damian – Nice write-up. Not your best, but I think your groin was still aching when you were typing. I remember trying to do something similar last year with an original dual-DVD of Gangs of New York and I never did get it right… once the Blu-Ray was released the problem went away as it was on one disc :-) I have a couple more dual-disc DVDs that I am going to try with your above methods… AFTER I put my groin protector on of course… Also, nice info on the file size with WHS. Wonder if the file size problems are limited to V1 or if they exist with the WHS 2011?

Damian July 6, 2011 at 8:11 pm

Hey Nick,

For DVDs you will want to do the mkvmerge route, I am not sure if there is a similar process as tsMuxeR to maintain a DVD Folder

Honestly I am still a little perplexed on exactly what the file size issue was. There doesn’t even seem to be any sort of consistency from one person to the next of who have had the issues (i.e. .like a common file size). As I mentioned I have transferred 40GB+ files and never had an issue, but the 60GB files seemed to have hit a threshold with my WHS.

Bill July 7, 2011 at 3:37 am

Damian,
I’ve had similar “insufficient resources” problem when I’ve tried copying large BR files to a back-up disk in the server. There’s plenty of room on the hard disk to copy…but it fails for lack of “space.” Any ideas on what the problem or solution is?

Damian July 7, 2011 at 6:50 am

Bill,

Do you know how large the file size was for you? I haven’t found a definitive answer. In that WGS link I provided some people thought it was RAM, others needed to use another software such as TeraCopy, etc…Really hard to say what the cause could be

Bill July 7, 2011 at 10:44 am

Damian,
Thanks, I downloaded TeraCopy and it worked fine…

Damian July 7, 2011 at 11:21 am

Interesting, so in your case it is definitely not a hardware limitation but something specific to WHS (bug or other)

Ian Robinson July 7, 2011 at 5:38 am

Hi Damian,

Thanks for another great write up. Unfortunately I read these from work at lunch time and it makes for an extremely long afternoon waiting to get home and try things out.

You seem to have moved from a prefrence of MKV to the blu ray folder structure. I know you’ve mentioned the balance was pretty even, but I missed the switch. What tipped you?

I’m restling with replacing MCE with Media Portal and also supporting a sumvision Cyclone media player in the bedroom (replacing a RROD Xbox extender) and I’m all over the place whne it comes to settling on a format for my HD discs. (Currently ISO, but AnyDVDHD has even started complaining about a number of those ripped some time ago.)

I was just about to go MKV until I read this.

Ian

Damian July 7, 2011 at 6:58 am

Hi Ian,

Yeah, I was originally an mkv guy but have primarily switched to BluRay folder rips. The main reason for me was forced subtitles. Since I use a variety of different players (whether for personal use, testing, reviews, etc…) there are just too many inconsistencies amongst the various players on how they handle (or if they even do handle) forced subtitles. By doing a Blu-ray folder rip via Clown_BD I simply choose all English subtitles without worrying if there are forced subtitles, and if present during playback they are displayed.

MKV is still a great option, just a case where with my setup Blu-ray folder rips gives me the most reliable results.

Cheers
Damian

Ian Robinson July 8, 2011 at 6:55 am

Hi Damian,

So given that storage space is not an issue, what is the advantage of a Blu-ray folder rip using Clown_BD over a straight Blu Ray folder rip of the full structure from AnyDVD?
(Though I can obviously see why you would want to re-authoring to combine to a single movie file in the LOTR case of your post.)

I assume you still need a player that can recognise the structure to use the rip, otherwise you’d be manually naviating down to the m2ts file? My understanding is that this is not an archive format that could be used with something like Media Center withouth having to launch an external player to play(?)

I like Clown_BD though – I’m experimenting with the m2ts option as an archive container. I would be willing to lose direct access to “extras” not to have to break out to TMT in Media Center and to have a format also supported by my hardware player.

Thanks again,
Ian

Damian July 8, 2011 at 7:30 am

Hi Ian,

Clown_BD creates a main movie rip (so no menus, trailers, extras, etc…) whereas with AnyDVD you get the full disc with all the extras. I have no need for all the extras/menus, so going with Clown_BD not only saves space (my guess a good 4GB+ per movie) but the movie just starts up right away when you launch (as opposed to having to go through the menu, trailers, etc…).

Correct regarding your assumption. Most players now recognize BluRay folder structures natively. Unfortunately WMC does not recognize this structure (although it does recognize DVD folder structures)

Keep in mind that in order to get your mt2ss to play properly with the internal WMC player you will need to hack it up a little (you can use a codec pack such as Shark, or do it manually shutting off Media Foundation, etc…).

I am more then g lad to answer any other questions.

Cheers
Damian

Ian Robinson July 16, 2011 at 12:57 pm

Phew.. a couple of weekend of nightmares (compounded by a Blade Runner work print HD-DVD that just wouldn’t rip turning out to be corrupt/broken)

Anyway, since you said you’d be more than glad to answer other questions. ;-)

Thanks to another older post of yours elsewhere…
Media foundation dealt with. Haali splitting all but m2ts. Graphstudio indicated all is as I would like, but VC-1 going through th MPC Video Decoder does not appear to be using DXVA.
Graphstudio says so and VC-1 video is choppy.

Any ideas?

Haven’t tried shark yet…

(Son no longer recognises me)

Cheers,
Ian

Damian July 17, 2011 at 4:38 am

Hi Ian,

What GPU are you using with you PC, AMD, NVIdia, or Intel?

Ian Robinson July 17, 2011 at 5:35 am

Hi Damian,

I’m setting up (or trying to) an Aspire R3610, so it’s an NVidia ION. Since it’s an Atom processor, the hardware acceleration is pretty key to it’s success to providing hd content.

I’ve stripped it back to a clean an up to date windows 7 home premiuminstall – there’s nothing else on there to get in the way (at least, that’s been my thinking)

Shark was a no go. Worse rather than better after that.

Cheers,
Ian

Ian Robinson July 18, 2011 at 1:14 pm

Hi Damian,

Sorry to be a pest, but I made some head way before running into another wall…

In Graph Studio, the MCP Video Decoder connects directly the Video renderer. When I connect it to the Enhanced Video Renderer, then it uses DXVA (H.264 & VC-1) and the graph plays smoothly.

Is it possible to get that preference to stick?

I could follow how to get the splitter to use the MCP Decoder when I wanted it to, but I can’t find how (outside of some codec pack screen shots) to specify a renderer.

Any pointers would be greatly appreciated.

Thanks,
Ian

Damian July 19, 2011 at 6:52 am

Hi Ian,

Hmm…. I am not sure. I thought EVR should be the default renderer. You are trying to play via the WMC internal player, correct? Have you tried using MPC HC as your player to see if that fixes the issues you are encountering?

Cheers
Damian

Ian Robinson July 19, 2011 at 11:28 am

Hi Damian,

From msdn, EVR is never the default renderer for compatability reasons. Applictions can obviously implement their own graphs to use it. MPC-HC worked well and that’s the reason I kept at trying to use its filters natively.

Last night I noticed that graph studio had the ability to set the merit of the direct show filters. So I upped EVR and happy days! It is now picked up and DXVA kicks in.

I haven’t tested that much yet (I didn’t want to diminish my small success), but it looks promising for achieving the set-up I was aiming for.

WHS has been invaluable in this for putting my Revo back to a clean slate over and over.

Thanks for your help and responses!
(Here and at adigitalhomeblog.)

Cheers,
Ian

Richard July 7, 2011 at 8:20 am

Good write-up as usual.

My comments are specifically for TeraCopy. I use it for all network transfers. Great utility!!! More control and more feedback on copy/moves as well as speedier. I highly recommend.

David July 7, 2011 at 10:35 am

I think the WHS bug you are seeing is caused by the size of the boot drive in your server. If you boot drive is 120GB, it doesn’t matter how much free space is on all the other drives in the storage pool, it is only going to allow you to copy in (120GB – the size of the OS) chunks. That is just the way that WHS v1 works and it is limiting. I had to replace my old 80GB boot drive with a 750GB drive when I started ripping and storing Blu Rays.

Damian July 7, 2011 at 10:37 am

The O/S drive in my WHS is 500GB so no issues. Also, when WHS first came out the O/S drive was used as a landing spot first when copying files over (so in that case you would have an issue transferring a file greater then your O/S drive). However, this behavior changed a while back

AWT July 12, 2011 at 7:55 pm

Wow!! That seems like a lot of work. I haven’t picked up my set yet, but when I do I will probably just do full ISO rips like I do with my other BD Titles. Besides, the break will be a good point for an intermission.

Kevin July 23, 2011 at 7:23 am

In regards to your problems with transferring large files to WHS. If your assumption is correct (that the drives are spinning too long and overheating), then consider using TeraCopy which is an app that manages copying/moving files. It’s one app I truly love for a couple reasons, but in this case, it has a “Pause” feature, I would be curious if partway through your transfer (before getting the predicted slow-down) if you pause the transfer, allow things to cool down, and then resume, if this fixes your problem.

Damian July 23, 2011 at 3:03 pm

Hey Kevin. Actually the cause is not the spinning drives. Something in WHS is causing the issue (no clue why it just started happening, but it also started affecting files sizes around 40GB which I never had problems with). The spinning drives just is a side effect.

Fortunately I installed Teracopy when the problem first started and since then no longer do I have any issues :D

Cheers
Damian

TC December 4, 2011 at 10:11 pm

Like someone else said, this seems like a lot of work. I’d rather edit the movie cover graphic to display 1 & 2 or something like that.

Damian December 5, 2011 at 4:54 am

Yes, it is extra work. However, keep in mind this is a very rare instance that you would even need to do this as nearly all Blu rays are on one disc.

Lamski January 8, 2012 at 10:54 pm

I have been working on this project this weekend and for some reason MKVmerge shows 1 warning while muxing and during playback all seems ok till I get to the “seam” where I merged the 2 files. I get a 1 sec or so flickering on the screen then play continues. Anyone run into this issue?

Im using Media Browser, MKV playback with MPC-HC.

I never could get media browser to load both parts into MPC-HC when they were in the same folder. Said “content error” and check to make sure the file path exists. So I tried the merge and have flickering on the screen now. ahhhh, what a headache.

Damian January 11, 2012 at 7:15 am

I haven’t noticed an issue with the “seam”. Honestly as long as A/V stays in sync I wouldn’t worry about it. Really the big concern was that after the seam A/V was way off

Rich February 26, 2012 at 7:51 pm

I disagree that you should only be concerned with the asynchronous A/V. I couldn’t stand the relatively long darkness and with the times given on the slysoft forums, especially when thinking of how all the other scenes transitioned in the movie. I ended up using TSPE (transport stream packet editor http://www.bitstreamtools.com/), but really any TS editor capable of m2ts would be sufficient. I haven’t completed the Two Towers and The Return of the King yet, but here are the times I used for the Fellowship:

Ended 1st Disc @ 1:45:38.37 (the last I-Frame before darkness)
Started 2nd Disc @ 0:00:02.04 (the last I-Frame before you can see/hear anything)

It is important to utilize I-Frames, else you may run into the issue of the frame selected being a P-Frame (references frames before it, a problem with the second disc) or a B-Frame (references frames before and after, an issue with either disc).

TSPE is unable to merge the two discs as far as I’ve been able to tell. So I merely edited each half with TSPE, and then merged with tsMuxer. I could also provide the Chapter timecodes to input to tsMuxer if requested.

Marcus October 6, 2012 at 2:40 am

Eac3to didn’t run into that issue because it can detect and remove those frames I believe, I’m had it detect overlapping frames a few times, and it automatically fixed it.

I do have a question though, why are you saving the BD structure on your HDD? I prefer the MKV method, because it keeps the FS cleaner, takes up less space, and most importantly you can have XML chapters, and Movie tags (shit like the actors, director, release date) Think ID3 tags, but for movies.

Damian October 6, 2012 at 4:11 am

BD structures work best for my set up (mostly Dune Media players). This gives me the most reliable playback, support for forced subs, etc… Don’t have a need for ID3tags for movies since I use jukeboxes to manage on my players, so not a big deal for me.

Marcus October 6, 2012 at 2:43 am

Oh, You can also have embedded poster art and backdrop images, like in Media center, there can be a widescreen image from the movie, that chills behind the Media Center UI.

Comments are closed, visit the forums to continue the discussion.

{ 1 trackback }

Previous post:

Next post: