It is income, not age, that determines when you can stop paying federal income taxes. Basically, you have to file a tax return as of publication if you and your spouse, both over 65, had combined income of more than $21,300 or half your Social Security benefits and other income was more than $32,000 combined.read more >>