J.L. Bell discusses how most of the new states of the union moved their capitals away from the coast during or after the Revolution, offers a couple of suggestions on why we didn't, including:
One factor is that Bostonâ€™s rivers donâ€™t go very far. New Yorkers could move their capital up the Hudson, Virginians up the James, Pennsylvanians to a spot on the Susquehanna, New Hampshirites to a spot on the Merrimack. But the big river inside Massachusetts is the Connecticut, which seemed too far and emptied out through another state.