I would suggest calling a professional (heating or chimney sweep) who can check the interior of your chimney and correct all problems. I can’t give you much based on the information provided, just some basic info:
PVC vent pipes should not be inserted into the chimney flue if it is still used for other appliances venting. This may be possible with two completely isolated flues but it would need professional evaluation and approval from your local code enforcement authorities. Check here - furnace and water heater vent
Natural draft water heater (if this is what you have) should not be vented using aluminum connectors - click the following link - water heater vent
You chimney flue might be too large for venting of an orphaned water heater and require a new, smaller diameter liner. Some basic information covering flue size calculations is here - chimney flue size - but you'd need to use Gama venting tables to get exact numbers.
Just click "comment" below if you have additional questions.