The visitors in Le Mans departed the MMArena with all three points, after Liverpool loanee Divock Origi's second-half penalty powered them to a 1-0 result over the 10-man hosts.
Caen defender Dennis Appiah was shown a straight red for his challenge on Origi that led to the penalty, although replays showed it was a harsh call from referee Sebastien Desiage.
Girard said his side were not deserved winners.
"It was not a good match. We didn't deserve to lose but we didn't deserve to win either," Girard told beIN Sports.
"We didn't take much pleasure from this game but I'm not worried.
"We had a difficult start to last season as well.
"There is work to be done but now we have a very important match on Wednesday."
Lille host Portuguese powerhouse Porto in the first leg of their UEFA Champions League play-off on Wednesday.
