Rotherham trailed after going behind to Jim Bentley's 46th-minute header but Stephen Brogan levelled from the spot on 71 minutes and then 18-year-old Yates scored twice in the last nine minutes to clinch the win.
There was more bad news for Morecambe who had keeper Joe Lewis taken to hospital with facial injuries after colliding with Rotherham striker Chris O'Grady.
David Artell, returning to Millmoor with his new club Morecambe, headed over the bar after just five minutes.
On eleven minutes, Brogan swung in a corner from which Ian Sharps hit the post with a header.
An uneventful first half had its only drama on 34 minutes when O'Grady and Lewis collided after the keeper had struggled to deal with Artell's back pass. After play was halted for seven minutes both men were replaced with Lewis stretchered off and taken straight to hospital.
Just 50 seconds into the second half, League Two's surprise package Morecambe took the lead. Michael Twiss swung in a corner to the back post and Bentley rose highest to power a header home.
Matthew Blinkhorn should have doubled the visitor's lead moments later when put clean through but he rushed his shot and hit it wide.
Bentley put the ball in the net from a similar routine but he was penalised for fouling Graham Coughlan.
Miller's manager, Mark Robins, threw on defender Ryan Cresswell upfront and he changed the game in Rotherham's favour.
On 70 minutes Artell dragged back Marcus Bean and Brogan drilled home the penalty.
Then a brilliant driving run from Marc Newsham set up Yates who fired into an empty net to give Rotherham the lead.
Six minutes after celebrating the first goal of his career, Yates then volleyed the decisive goal after Cresswell's initial effort had been blocked.