Receiving statutory maternity pay (SMP) from UK and moving to Germany

It is compulsory to have health insurance for everybody living in Germany.
You either have to become member of the German system, or participate in it through your continuing UK insurance (S1 form?). The German public system has no exclusions for maternity or pre-existing conditions, the private system does - thus you want to be in the public system.
If you live in Germany with your eligible child, you can also get Kindergeld and Elterngeld. These require separate applications and the paperwork is a bit bothersome. If you don't speak German, get somebody to help you with it.
There's a double taxation agreement between UK and Germany. That means you generally pay tax only in one location. But it's not you who decides where! If you live and work in Germany, you pay tax here, not in UK.