Jisho

That's definitely one valid meaning here, given that the story starts out with a popular idol giving birth to two children. However, it can also simply stand for the idol itself, literally "that girl that is my oshi/my favorite". See also the start of the interview here to have the author spell it out: https://tsutaya.tsite.jp/article/book-comic/599.html#01

1巻を読めばわかるんだけど“推しの子”には2つの意味があって、1つ目が「好きなアイドルを推す」という意味で、2つ目が「推しているアイドルの子ども」という意味。

So before you pick up the manga (or watch the show) you would probably think it just refers to somebody's favorite idol, but then when you read through the first chapter you would notice that there's this second meaning to the title. Clever, huh.