Your body does need oestrogen. You must have some of it, even though you know it will feed endo, your long term health can be really severely impacted if you cut it out altogether.
Oestrogen is known to keep your heart healthier and definitely keeps the important trunk bones from getting too thin and snapping. Osteoperosis is a dreadful disease, painful and disabling and far worse than suffering with endo pains that you can take pain killers for.
If you have prematurely lost your oestrogen making machines (the ovaries) then you should be taking something to give your body a boost in the oestrogen it needs to keep the rest of you healthier. It is a small price to pay but when you see what osteoperosis does, anything you can do now to avoid getting that is worth doing. Also take on weight bearing exercises to build up bone strength.
Google Images for osteoperosis will give some good illustrations why it is worth doing anything to avoid getting that disease. Life is bad enough battling endo, without adding severe complications later on which could be avoided.
Having said that it is your choice. You cannot be forced to take HRT if you really don't want it.
I am usually pretty sceptical about taking certain drugs, but replacing missing oestrogen even knowing that it will feed endo is something I would take for the other longer term health benefits as and when i get my last remaining 'Ove' removed.
It won't stop your adhesions from causing problems, nothing does.
It might encourange endo to remain active, but any lesions which continue to cause problems can be surgically removed at a later date if that continues to be a problem you cannot cope with.
You cannot reverse heart disease and you cannot reverse osteoperosis once they take hold.
I can't decide for you, but your Doctors are giving yo the best advice, whether you choose to accept that is up to you, but do yor homework on the risks of not having HRT before you decide. Don't just concentrate on the endo (although that is currently at the forefront of your thinking), you must look at the bigger picture of living without ovaries and the impact that will have on the rest of your body.