This is why Ulfric cunningly waited until Felix had grown up before rebelling. He's still pretty worried about the kids running around Skyrim but that's what the orphanage is for.