I have blueberries, blackberries and pawpaws that are maintenance and issue free. I have pears, apples, and plums that are prone to disease, frost issues, and bugs. I have a handful of lemons and limes, but they can't take frost and don't like temps under 40. They're a hassle and a $5 bag of lemons from the grocery store is more than you'll get every year, but they're kind of neat.
Unless you're willing to move citrus in and out a garage, your only citrus option is what Stan McKenzie recommends.