If it is a great feature, then do what you can to keep it but if it is wrecking the rest of the house it may have to be sacrificed.
You may be able to replace it with another, possibly a gas log or Jetmaster that uses a metal flue rather than the brick chimney.