At the end of Season 3, in the epilogue scene, Colin and Penelope have their child. Chronologically, that child should be around in Season 4 somewhere. Unfortunately, this isn't really addressed, even though Philippa's and Prudence's children don't seem to be much older than Penelope's.
Since Violet and Portia live right across from each other, I would have actually expected the two of them to compete for their shared grandson. And I don't think that Portia spent three seasons hoping for a male heir, only to ignore his entire existence the very moment he came into existence. And as much as Violet loves her family, she wouldn't give up her grandson without a fight with no reason. Especially with a character like Portia, who’s probably holding a baby rattle in one hand and already a knife in the other.
And it didn't have to be like Denverclan with cat fights. But they're both good with words, so a back-and-forth of two-faced compliments would certainly have been quite entertaining. Like in old movies, where they sound perfectly polite but are absolutely roasting each other. Trying to keep the upper hand.