go to pot
Decline in quality
Go to pot means to deteriorate or fall into ruin through neglect.
A business goes to pot when management stops caring.
A garden goes to pot when no one tends it.
The phrase suggests gradual decline rather than sudden collapse: After the divorce, he really let himself go to pot.