You need an expensive visa to do this alone, so i would recommend trips. Distance between attractions is fairly long if you walk it and you lose the commentary of what we found to be very knowledgeable tour guides. The palaces are always busy, especially the Hermitage, but you have got to see the heritage, it's stunning, as it is in the three or four other palaces available to see (St Catherines, Yusopov for example). Also combine with a river/canal cruise. Russia has come a long way in 20 years.
We were there for three days and didn't see nearly enough, so I would suggest a cruise that docks there for at least this period of time.