A $1 million retirement kitty may work for some, but not for others. It all hinges on how much your expenses are, and how much you withdraw each year.
The only way to figure out your expenses is, well, to figure out your expenses. If you can live on $50,000 a year after taxes and Social Security, then $1 million is a good amount.
Figuring out how much to withdraw is tricky as well. You're using the 5% rule of thumb. Like most rules of thumb, the 5% rule has many, many exceptions. Can you retire worry-free on $1 million?