Poyet, who was appointed in October, saw his managerial career in the Premier League start brightly as Sunderland posted home wins over Newcastle United and Manchester City.
But the former Brighton and Hove Albion boss has seen his side return to the foot of the table after a four-game winless run.
Sunderland are five points adrift of their nearest rivals and Poyet is unhappy with how things are progressing.
"Is it enjoyable? No, I can tell you it's not, but I don't think everybody gets a job and tries to enjoy the job," the 46-year-old said.
"That's a dream job. That's what I have said to the players: You want to play football? Yes. You want to earn money? Yes.
"Have you got a lovely place to train? Yes. Then you are lucky because you are enjoying your life.
"But at the moment, no, I am not really enjoying my life. But it's the way it is. I am trying to make sure I can enjoy it in a few weeks' time, that would be nice."
Poyet will hope his side return to winning ways when they visit West Ham on Saturday.
‘I don’t think Liverpool would look at Ollie Watkins, a striker isn’t a pressing issue for them – it’s Arsenal who need one’ Former Reds star explains why his old club don’t need an out-and-out forward this summer
‘He’ll still be playing at 40 at a good level because he’s in such good shape and looks after himself so well. He does everything to be a top professional’: Ex-Liverpool coach insists Mohamed Salah has plenty more miles in the tank
