West Ham finished 10th last season but they have struggled so far this term, winning just one of their opening six fixtures.
The absence of injured striker Andy Carroll has hit the east London club hard as they have only been able to find the net five times.
To make matters worse, fixtures against Tottenham and Manchester City are just around the corner, but the Hammers captain is not a concerned man.
"We can't afford to panic," midfielder Nolan told West Ham's website.
"We understand the position we're in. I've been in this type of position at this stage of the season before.
"At the minute, things are just not falling for us and refereeing decisions are absolutely killing us.
"Tottenham are in a great run of form but it's a local derby and we're going to go there with nothing to lose and have a good go at them.
"Hopefully it'll be the start of the little run that we need."
Carroll's absence was felt more than ever during last weekend's 1-0 defeat at Hull, where the Hammers lacked penetration in the final third.