Does their opinion count?

Me and my boyfriend have been together for almost 2 years and recently we've been fighting A LOT. We're on the brink of breaking up, but both of us would be crushed if it happened. We want to work things out and make them better, but we've been trying to fix things for a while and its not working, but I'm not ready to let go yet and neither is he. In my family and friends' eyes he's far less than perfect. They don't like him and think I need to break up with him. My question is, should I take my family and friends' thoughts into consideration? I know its about what I want and all but I'm so confused I just need advise. Thanks!