TrenderX: How I Plan to Withdraw from my Accounts to Fund Early Retirement

I’ve been writing about money on MDJ for a long time – a very long time. During this tenure, most of my pieces have been about the accumulation of money and wealth.

Articles like:

As I get older and closer to early retirement, my mind has started to shift from the accumulation of wealth to how I will spend the money and where it will come from.

This is actually a fairly recent mental shift that has gained momentum with a reader comment in a recent financial freedom update.

First great job! Let’s assume you have reached your goal and quit your job, living on your dividend streams only. How would you approach this? Would you just harvest the dividend from all the accounts? Or?

I wonder if there is a optimal way of thinking about it considering taxes, the decisions you have to make as you get to 60 yrs old (CPP, AOS, …), and associated potential clawbacks.

On the topic of where the money will come from, for us, we have built our wealth around investments in the public markets. A significant amount in tax-efficient dividend stocks, and the rest in globally indexed ETFs.

When it comes time to withdraw from my portfolio, for us, the question is really how to most efficiently withdraw from the portfolio to not only fund our lifestyle but also to leave something to the estate. Perhaps a large sum to charity, and of course, some to my kids and future grandkids. Buffett puts it best along the lines of: “You should leave your children enough so they can do anything, but not enough so they can do nothing.” For others, they want to die with $0 in their bank accounts – to each their own!

Our Investment Accounts

As discussed in our financial freedom updates, we have the following accounts:

RRSP x 2
TFSA x 2
Non-registered accounts x 2
Corporate investment account x 1
Small defined benefit pension accessible at age 60

Figuring out the most efficient withdrawal strategy depends on the situation and really requires financial planning software to get the optimal answer. Here are some retirement calculators that may give you a start.

Without having an advanced piece of software, or the patience to write my own Excel spreadsheet, I did some digging around the web for some solid rules of thumb. With that, I came across a thread on The Financial Webring Forum that contains useful information about retirement accounts and withdrawals.

Within that thread, there are readers that have access to financial planning software and have run a number of common retirement scenarios leading to some general conclusions.

Some general rules of thumb for retirement account withdrawals

Keep tax-sheltered accounts as long as possible but watch out for the old age security (OAS) threshold which is $77,580 for 2019. After that income, OAS will be reduced by 15% for every dollar until it’s eliminated at income $125,696. So ideally you want to keep your retirement income under $77,580 (per spouse). If you have a spouse, the ideal scenario is to keep both incomes just under the first tax bracket. In Ontario that would be $43,906, and in NL that would be $37,591. You can see tax brackets from other provinces here. Having a tax-efficient family income of $80k would make for a comfortable retirement for most families.
Withdrawing from RRSPs early sometimes makes sense. Even though the accepted rule of thumb is to keep the RRSP as long as possible (up until Dec 31 on the year you turn 71 when you are forced to convert to an RRIF), there are a number of situations that drawing down the RRSP early has merit. For example, say your spouse has a large RRSP balance, has health issues, and unfortunately passes away early. In that scenario, the RRSP gets transferred to you tax-free. This is great, but when you reach 71 and forced to convert the RRSP to an RRIF, you’ll be required to withdraw 5.4% of the account balance in the first year and increasing after that. If you have a $1M total RRSP, that would be $54,000 in the first year of RRIF withdrawals and increasing annually. When you add CPP/OAS on top of your RRSP income, you’ll likely start seeing OAS clawback which is essentially an additional 15% tax. Not only that, when the second spouse passes away, the large RRSP balance will likely face the highest marginal tax rate. One strategy could be to retire a bit early, live on RRSP withdrawals, leave all other accounts intact to grow (or withdraw enough in addition to RRSP to fund lifestyle), all while delaying OAS/CPP for one or both spouses depending on health (up until 70). This would result in drawing down the RRSP balance to reduce forced taxable withdrawals at 71 which would hopefully be enough to mitigate OAS clawbacks. All this while letting the TFSAs continue to compound, and getting an increased CPP/OAS payout (the longer you delay CPP/OAS, the higher the payout).
Income split as much as possible. If you have a spouse, the goal is to keep incomes as equal as possible to maximize income and minimize taxes. You can convert your RRSP to an RRIF as early as age 55 which will allow you to claim the pension tax credit. However, you’ll have to wait until you turn 65 for the RRIF withdrawals to be eligible for income splitting with a spouse for tax purposes. Defined Benefit Pensions can also be split with a spouse for tax purposes – this can be a huge advantage for families where one spouse is a government employee. Here are some additional income splitting ideas.

Funding Early Retirement

Keeping those rules of thumb in mind, they will apply to every one of you in a different way. To some, they will have no issues with the OAS clawback threshold, to others, especially those with full defined benefit pensions or large RRSPs/non-registered accounts, taxation can potentially be reduced with some proper planning.

But how does this apply for someone who plans on retiring early? Let’s take a look at my personal example!

From our family’s perspective, it looks like my wife and I may reach financial independence in our early 40’s. We tend to keep our expenses fairly low which has been between $52k – $54k per year, all while raising two kids. I like to think that we live a balanced albeit comfortable life, which may not sound possible on $50k/year, but having no debt payments makes all the difference.

As you may know, we plan on living off dividends – for the most part. While our total dividends are very close to reaching the $52k-$54k mark, some of the dividends are from registered accounts. Some of which I would rather not touch, like our TFSA’s.

Here is a summary:

March(Q1) 2019 Dividend Income Update

Account	Dividends/year	Yield
SM Portfolio	$7,500	3.96%
TFSA 1	$3,500	4.54%
TFSA 2	$3,600	5.03%
Non-Registered	$3,300	4.42%
Corporate Portfolio	$20,600	3.73%
RRSP 1	$7,000	2.75%
RRSP 2	$2,700	2.37%

Total Invested: $1,332,522
Total Yield: 3.62%
Total Dividends: $48,200/year (+4.56%)

Imagining that we had enough to retire today, how would we draw down our portfolios? Being too young for seniors benefits (we both turn 40 this year), early retirement would have to be 100% funded by our own means.

To fund early retirement, my strategy would be the following:

Use dividends from non-registered accounts and corporate portfolio first. Using dividends would allow for tax-efficient income, especially for early retirees. This may get to be a challenge when you approach OAS time as dividends are grossed up by 38%. So if you make $40k in taxable eligible dividends, it will look like $55k for income testing against OAS. Something to be mindful of if you are approaching senior status. But being someone with 20+ years until senior status, the tax efficiency of dividends are welcome and realistically, it’s hard to predict old age security benefits 20 years from now.
Use RRSP’s for the difference. While dividends from with non-registered sources are not quite enough for our expenses (at the moment), I would top-up our income by drawing down the RRSP’s. While it wouldn’t deplete the RRSPs, it would help keep it to a more manageable level when we reach forced RRIF withdrawals age.
Leave TFSA’s in-tact. Combining non-registered accounts and RRSPs for spending will give us a lot of flexibility with the TFSAs. We could keep the TFSAs compounding over the years OR simply spend it. It will likely be a mix of the two. This can potentially be used as a future slush fund for luxuries such as travel, vehicles (likely one vehicle household by that point), and/or to leave to the estate/charity. Thankfully, TFSA withdrawals are not income tested for seniors benefits (as of right now!).
Delay OAS and/or CPP – With enough income to live on via dividends and drawing down on our RRSPs, we may delay OAS and/or CPP. This would depend on our health and the size of our RRSP when we reach senior status.

Realistically though, if we were to “retire” today, it would be more of a shift from full-time work to part-time (or entrepreneurial) work. So I would likely be drawing an income from somewhere. I imagine our family income coming from:

Non-registered and corporate dividends
Part-time (entrepreneurial) work
Potentially RRSPs to top up for travel and other large capital costs

Further Efficiency Needed

Going through this process has made me realize that there is an RRSP imbalance between me and the boss. If we keep going on this path, this will result in lopsided RRSP withdrawal taxation up until the age of 65 where we will be able to split the withdrawals (must be converted to RRIF first).

To mitigate this, I plan on opening a spousal RRSP (likely another account with Questrade) to help even out the account balances over the coming years. After all, the goal is to balance retirement income between spouses to minimize taxes while maximizing income.

Final Thoughts

Phewf – there sure is a lot of information in this post! There a number of factors to consider when it comes to generating retirement income. If you have some time before your retirement years, then some planning can go a long way to reduce taxation during your golden years. If you are lucky (or unlucky?) enough to have a spouse during that time, there are some strategies that can be employed as mentioned above. In the big picture though, a tax problem is a nice problem to have during retirement!