Advice on not joining the new employer

You need to terminate the employment contract you have with the employer.
The contract will specify how to terminate, how long a notice period you need to give and if there would be a penalty.
If not, just tell the employer that you changed your mind about working with them.
Then they'll cancel the EP. They cannot do anything else if you follow the rules written in the contract. But do keep a written record of the conversation with them, to show it.
However, MoM does not like job hoppers, so this will count against you in your next EP application.

Thanks Beppi for the reply. I have not yet started with the company and there is nothing in the contract anything related to not joining them. Yes, I acknowledge that it is a mistake that I did but I am sticking with my old company and plans to continue there for long. I will send them a written communication. I have my EP renewal coming in another 6 months, can it have any negative impact?

Appreciate your advice.

If the contract says nothing about termination before start of work, the same rules apply as after starting.
If the new employer has no reason to complain about you to MoM, then this will not have a negative impact.
However, in the EP online system the old employer can see (if they bother to check) that a new EP was applied for you (but they cannot see by which company). Some employers fire such illoyal people at the next opportunity.

Thanks again Beppi.

My old employer knows all the details, as I shared that with them after I decided to stay with them.

As far as the new employer is concerned, I have tried my best to share the reason of  the change in decision. Hopefully they will understand, it is just that I wanted to avoid the situation to get worse by joining them and then leaving them because of the apprehensions I have now.