As Camembert_Electrique pointed out, the most important thing you need to do is register with a GP. You'll probably also want to find out when your new GP wants to see your baby for check-ups, as this might vary from country to country. I also highly recommend (from personal experience) that you arrange to get paper copies of your, your wife's, and your baby's medical records to take yourself to England, as a direct doctor-to-doctor transfer across borders will likely not be possible. If you can't get copies in English, ask your doctor to write a letter in English summarizing the salient points.
If you or your wife will be working in England you'll need an NI (National Insurance) number, so your employer can prove that you've been paying the right taxes. There will be time to arrange that after you arrive, but if you're keen to get your ducks in a row that's something to think about.
There shouldn't be too much red tape involved in moving within the EU, but if you're worried you may want to contact the British embassy or consulate nearest you in Sweden--they should be able to give you a thorough answer to your question, and may even send you some informational leaflets.
Just to put your mind at ease a bit--I've lived in three countries so far and British civil servants are by far the most reasonable I've ever had to deal with. When I moved here I worked without a valid NI number for over a year because I thought the the placeholder number my employer had put on my payslips was a real one. That was all very illegal of course, but when I realized my mistake I just called up the NI people, apologized, and they just sorted me out with a number, tied it to all my previous payments, with no fuss--not even a fine. I know people who have had similar positive experiences with the Inland Revenue. That's not to say that you should deliberately let things slide, but there's no need to lie awake at night wondering if you're going to be deported or slapped with a huge fine for not doing something you didn't know you needed to.
Good luck with everything and congratulations!