The former Chelsea man pounced on hesitancy by young Ben Turner to fire the Hammers through.
It was tough on Iain Dowie's men who had more than held their own throughout.
West Ham, with £27million of talent on the treatment table, always looked capable of stepping up a gear, but all too often they were pushed back by a Sky Blues side clearly relishing the big occasion.
The Hammers' first-half chances were few and far between. Mark Noble sent a powerful shot fizzing just over the bar when Matthew Etherington tapped a free-kick short to him just outside the penalty area.
The home side were employing Michael Mifsud wide on the right flank and his pace caused problems for George McCartney from the off.
The full-back was yellow carded in the 16th minute when he pulled the Malta international to the floor as he threatened to move on to a fine diagonal pass from Michael Doyle.
Coventry were full of endeavour, but all too often lacked any sort of killer touch inside the box. Julian Gray had a header saved and Doyle also forced Richard Wright into action in the closing moments of the half.
At the other end, Turner was caught napping by Luis Boa Morte and was lucky not to be cautioned for brining the striker down.
But the youngster had also made a goal-saving tackle when he slid in to block Noble, who had moved on to a McCartney cross on the edge of the six-yard area.
West Ham looked to up the pace after the break and were a tad unfortunate when referee Rob Styles waved away claims for an early penalty when Boa Morte again out-paced the Coventry defence.
Coventry appeared undaunted and came back strongly. Leon Best slid Jay Tabb in on the right and the former West Ham apprentice shot just over.
Best then tried his luck with a snap shot after Mifsud failed to bring the ball under control, only for his effort to fly just over.
The home side finally made the breakthrough in the 68th minute with a goal of Premier League quality.
Isaac Osbourne saw Tabb making a blindside run into the area and found him with a delightful cross which the little midfielder gleefully headed home.
But the joy was very short-lived. The noise around the Ricoh Arena had hardly died down when Boa Morte rifled in a low shot only for the ball to strike Marcus Hall and fly in past the stranded Andy Marshall.
Still Coventry pressed, but just when extra-time was looming, Turner took too long to get the ball away and Cole cashed in in the best way possible.