You might want to look at the 'not in labor force' numbers: https://fred.stlouisfed.org/series/LNU05026642
and also labor participation rate, but for 25-54 Yrs, so you exclude the effect of demographic changes https://fred.stlouisfed.org/series/LNS11300060
You might want to look at the 'not in labor force' numbers: https://fred.stlouisfed.org/series/LNU05026642