If you have time to go the the old walled city and the waterfall across from Orleans Island, do both. If time is limited, go to the Old Walled City adjacent to the Plains of Abraham and on the north bank of the river. There is plenty to see and do there. Excellent photo opportunities.
Next Port was Saint John, New Brunswick. Rent a car and drive to St. Martin or go on a whale watch.
Take the Old Town Trolley from the port. Stop at the stops you like and enjoy Boston. Paul Reveres House, North Church, Charles River, Old Ironsides - USS CONSTITUTION, Boston Common, etc. Skip the Harbor Cruise.