Fabregas converted an 89th-minute spot-kick to earn a 2-2 draw after referee Bobby Madley adjudged Ruben Loftus-Cheek had been tripped by Michail Antonio
Replays suggested if there was contact it was minimal and outside the area.
"To concede a goal that late is gutting, no matter how you concede it," Bilic said.
"To concede it from a penalty that wasn't a penalty
It's unacceptable
It was not close to the line
It was way out and simply not a penalty
It shouldn't be difficult for a referee to see it."
It was the second successive week when Bilic felt officials had ruled against his side
He believed goalkeeper Darren Randolph was fouled in the build-up to the Manchester United equaliser which forced an FA Cup quarter-final replay.
Andy Carroll thought he had enhanced the Hammers' bid for Champions League qualification when he scored one minute after coming off the substitutes' bench at Stamford Bridge.
But Fabregas netted his second after a brilliant free-kick had earlier cancelled-out Manuel Lanzini's superb opener.
West Ham remained fifth, one point behind Manchester City and without a double over Chelsea since the 2002/03 season.
Bilic added: "We could have two points more
We could go over Man City, put the pressure on them, we could make a gap with the teams that are below us queueing and we deserved it
We deserved it
So we are gutted.
"I don't look too much at the table
But what gives me confidence is the way we're playing.
"We can approach every game, no matter who we're playing against, where we're playing, no matter how difficult this league is - and it is very difficult, it's like a minefield - (and) we are capable
We are having chances against any team."
The penalty equaliser extended the Blues' unbeaten league run to 14 games
Eight of interim boss Guus Hiddink's 13 league games have been drawn.
The Dutchman felt the penalty was the correct decision.
"It was a very close call," Hiddink said.
"Loftus-Cheek was about to score, because he was going towards the goal
I thought he was on the line when he was tripped.
"And the line is part of the box, I think
When there's any doubt, the benefit of the doubt is always going for the attacking team - by this case, by rule."
Asked if Loftus-Cheek could have stayed on his feet to score, instead of inviting controversy, Hiddink said: "Normally he is able to score in this position
(But) I think it's not much of a discussion.
"He was tripped and then I think it's fair to have the benefit of the doubt, if there is any doubt, for the referee to (award) this penalty."
As West Ham chase the Champions League, Hiddink is still eager to secure Europa League qualification, dismissing the suggestion it could be beneficial not to play in the second-tier competition next term.
Chelsea are 10th, nine points behind the Hammers, who occupy the final European spot at present.
Hiddink said: "Our target now for the rest of the season is to see if we can get a European spot.
"You must get the highest position possible
If you start calculating not going into Europe for whatever the reason might be..
I'm not in favour to say 'hey, let's calculate - please let's not play Europe'
I think then you undervalue the tournaments."
The October clash between the sides, won by West Ham, was bad tempered and the conclusion of Saturday's game could be scrutinised by the Football Association.
All 20 outfield players became involved in a late squabble.
Source : PA
Source: PA