Background and Upbringing play a key role in forming an adult. For the fact that you once knew someone way back does not mean you should be friends. Ask yourself if the person is really worth keeping as a friend.
There are too many people out there with different ideologies, terrible upbringing, unhealthy past experiences, which could have coined their mindsets and way of living to be in a certain way completely different from you who had a good and comfortable background and upbringing, and would deal with life differently no matter what you have been through.
Who you let into your life can change the course of your life and destiny, negatively or positively.