Gus Poyet's Sunderland hover above the relegation zone on goal difference after five league fixtures without a win, but they have drawn.
While Sunderland have only lost once, Altidore is determined to avoid a repeat of last season, which saw the club fail to win any of their first eight matches.
"We need to be tougher. It is too easy sometimes," Altidore said ahead of Saturday's clash against Swansea City.
"All of us - strikers, everyone - have to be nastier; make sure, when people play us, they have to really work for it.
"We have played fairly well, but now we have to get the first win. We don't want the shadows of last year following us around this season.
"We have had a decent start and now, with the home schedule we have, we have to take three points. There is room for improvement. We have to play better.
"It was nice to score this week but I was not worried about anything anyway. I continue to go about my business and try to get into the team."
