Just as a follow up to my previous post, I've been unable to find the directive which gives freedom to change jobs and to work in a freelance capacity.
I contacted the immigration office and my response from them was the following:
According to Article 12 of Directive 2009/50/EC, the restriction to one job pursuant to Section 18b, Paragraph 2, Clause 4 of the Residence Act refers to possession of an EU Blue Card in a member state. So if you come to Germany now, the 2 years will start from the beginning and you will need a job change permit for the first 2 years in Germany.
Section 18b:
holders of an EU Blue Card wishing to change jobs need permission from the foreigners authority only during the two first years of employment, provided they meet the requirements for an EU Blue Card.
https://www.gesetze-im-internet.de/engl … html#p0389
So, I find that there is some conflicting information which contradicts the original statement made above by Plasmaangel:
The card is not attached to the employer. It allows me to work 4 years on the specific position name anywhere in Germany and also be a freelancer for 2 years.