Security Deposit refund

Hi,
I had paid the security deposit to a real estate agent/landlord to secure an apartment and then changed my mind to not go for it as the landlords attitude was not good regarding many things. Now the landlord is refusing to pay back the security deposit.
What should I do?
Appreciate any kind of inputs to this.

Live with the loss.
The place might well have had other customers who were refused as you secured the property with a deposit.
Should he stand the loss for your error?

WOW!! Thank you so much for your reply!  :D