Costa picked up his fifth yellow card of the Premier League campaign in Saturday's goalless draw at Sunderland and will miss the midweek visit of Spurs with Loic Remy or Didier Drogba likely to come in.
The Spain international has 11 goals in as many top-flight games this term but Kane is hopeful a resurgent Tottenham side can make the most of his absence and record a fourth consecutive win in all competitions.
Premier League successes over Hull City and Everton have elevated Mauricio Pochettino's men in seventh place despite an inconsistent start to the former Southampton manager's tenure.
And Kane, who has also become a more prominent figure at White Hart Lane this term, hopes Costa's absence can give his team-mates an extra boost.
"We're full of confidence. Obviously we are coming off three wins in a row now in all competitions, so we'll take that confidence into the game on Wednesday," he said.
"We're looking to get the result. Chelsea are going to be trying to do the same at home, but they are missing their main man up front. So they have to change a few things, and we're feeling good.
"The Chelsea game is not the be-all or end-all, but it would add to the great momentum we have at the moment if we could beat them."
Saturday's stalemate at the Stadium of Light was only the third time Chelsea have failed to win in the league this season and they have a six-point cushion over Manchester City going into the midweek fixtures.
Jose Mourinho's title-chasers have not conceded in three consecutive outings and boast an impressive record in home games against Spurs.
Tottenham have yet to record a Premier League win at Stamford Bridge, last beating Chelsea away from home in February 1990.
Goalkeeper Thibaut Courtois will likely play a key role in maintaining that record for Chelsea and the Belgian feels Wednesday's clash is the start of an important month.
"I like to play games but it's a little bit weird for me to play over Christmas," he told Chelsea's official website.
"In Spain we normally played until December 22 and then had seven days free, so it's not a big difference, it's not a problem.
"It's an important week with two difficult games.
"First a derby against Tottenham in front of our own fans and then we go away to Newcastle, which will be tough.
"It's an important month and we will try to win as many games as we can."
Mourinho has no fresh injury worries although Kyle Walker (groin) remains some way off a return for Pochettino.
‘The pure simplicity of the way Slot has managed the squad is probably the biggest thing I could say about him. It’s not broken, so let’s get on with it’: Liverpool legend full of admiration for Jurgen Klopp's successor at Anfield
‘I trained at Spurs and thought they’d give me a chance. But I received a letter thanking me but saying they didn’t think I was good enough – I was gutted’: How Tottenham missed out on signing Wales legend John Toshack
