Ange Postecoglou insisted Tottenham Hotspur were "not a big club" following the sacking of head coach Thomas Frank on Wednesday.
The Dane succeeded Postecoglou in the Spurs hotseat but was unable to oversee a revival in their Premier League form.
After a 2-1 defeat at home to Newcastle on Tuesday the north London club is just five points above the relegation zone.
"Having been in that position now twice in the last six months, it's tough," Postecoglou told The Overlap's Stick to Football podcast.
"You know that he can't be the only issue at the club. It's a curious club, Tottenham. It's made a major pivot at the end of last year, not just with me but with (executive chairman) Daniel (Levy) leaving as well, and you've created this whole sort of environment of uncertainty."
For all their traditional standing as a 'Big Six' club who now play in one of the best stadiums in the country, Tottenham have been crowned champions of England just twice – the same number of times as second-tier Portsmouth.
And the last of Spurs' titles came decades before the Premier League era, when a celebrated side managed by Bill Nicholson lifted both the old First Division trophy and FA Cup back to do the 'Double' back in the 1960/61 season
Long before Postecoglou took charge, high-profile managers including Terry Venables, George Graham, Harry Redknapp, Jose Mourinho and Antonio Conte tried and failed to get Spurs back to the summit of English football.
"There's no guarantee whichever manager you bring in – they've had world-class managers there and they haven't had success," Postecoglou said.
Despite overseeing the end of Spurs' 17-year wait for a major trophy when they beat Manchester United in the 2005 Europa League final, poor league form meant Postecoglou was still sacked.
The 60-year-old Australian insisted a failure to invest in players lay behind Spurs' problems.
"They've built an unbelievable stadium, unbelievable training facilities but, when you look at their expenditure and particularly their wages structure, they're not a big club," he said.
"I saw that because, when we were trying to sign players, we weren't in the market for those players."
Postecoglou added Spurs were failing to live up to their own ideals.
"When you walk into Tottenham, what you see everywhere is 'To Dare Is To Do' (the club motto), and yet their actions are almost the antithesis of that," he said. "I think they didn't realise that, to actually win, you've got to take some risks.
"I felt like Tottenham as a club were saying, 'we're one of the big boys', and the reality is I don't think they are."