The summer signing from Wimbledon made it three goals in two games for his new club, while he also had a hand in Jermain Defoe's ninth minute opener.
Defoe gave the Hammers the lead, turning home the rebound after Connolly's chip had cannoned back off the crossbar, with Diamonds keeper Billy Turley well beaten.
It was Defoe's second goal in two games, with he and Connolly on target in the 2-1 win at Preston last weekend.
Connolly added a second, knocking the ball past Diamonds keeper Billy Turley after breaking into the visitors' penalty area on to Defoe's pass.
However, after Defoe had struck the woodwork, Onandi Lowe's ferocious free-kick just after the half-hour reduced the deficit to a single goal.
During the second 45 minutes the visitors pushed hard for an equaliser, but were unable to do so and were killed off 12 minutes from time by Republic of Ireland frontman Connolly.
He was on hand to net after Defoe's volley had been blocked by Turley, following Christian Dailly's header from a corner kick.
West Ham: James, Ferdinand, Repka, Dailly, Brevett, Lee, Sofiane, Noble, Etherington, Connolly, Defoe. Subs: Bywater, Garcia, Byrne, Mellor, McMahon.
Rushden & D'monds: Turley, Bignot, Underwood, Mills, Edwards, Hunter, Jack, Burgess, Bell, Hall, Lowe. Subs: Darby, Gray, Dempster, Sambrook, Talbot.
Referee: Mark Cowburn (Lancashire).
Attendance: 13,715