Who SHOULD HAVE BEEN Living at Home in 1863?

At first I thought that this blurb from my 3GGR's pension file (an part of an affidavit from my 2GRGR) might shed some light on Julius's kids -- whether any died in childhood, etc.  But upon second glance, I'm not sure that it does.

It says:

Q:  Did your father Julius Laporte remarry after the death of your mother: if so, to whom?
A.  He did. He went into the army before he re-married. He came home from the army and remarried Emily Mosse. Then he re-enlisted and went into the army leaving my sister and me with said Emily Mosse.

I thought that italicized phrase might shed some light on things as it seems to suggest that only Moses and his little sister (Sophie, two years his junior) were in the home, so it would seem like something happened to all of the other siblings -- death, perhaps.

But looking at the ages of the siblings, it seems equally, if not more plausible, that all were grown enough (by standards of the time) to have been gone from the house.

In 1863, when he re-enlisted, Amelia Melvina would have been 22, Edward would have been 21, Mary would have been 17, Louisa would have been 14.

With the exception of Louisa, all of the rest of those children would likely have been deemed plenty old to be working outside the home as servants or farm hands, or married off.