Your neighbour's advice was not right. KL is city of many cultures like US and people here are friendly. There are many expats living/working here as well. I manage construction project. Working in construction site is safe as there are many friendly people around you. However, please make sure you have people accompany you when carry out visit/inspection on the building structures. It is always safe as long as you take care yourself properly and always take necessary safe /security precautions wherever/whenever you go.