Meggy.nut after my ankle fracture my ankle swelled and I did not take a step without pain for four years. Yes, it took forever to heal. It wasn't helped that I had two young children and I had to be active, go to school and all the activities and shopping for a family. I just wore the support and when it got to the point that the swelling made the support too tight I had to sit down and put the ankle up. I took off the support due to the tightness, it was getting in the way of blood flow.
I know more now than I did then. I would get a better support. Mine was not much good at all. It made it just a little harder to turn my ankle, it didn't keep me from doing it though. I would see a different doctor for a second opinion if I wasn't happy with my healing. Honestly I don't know what another doctor would have done in my case to make me heal faster.
Sheppy, do you have an aircast boot? Mom and my daughter both used them for broken feet. It did the job for them and the fact that you can take it off to bathe is very important. They didn't get the dry skin and itching that you get in a cast. A couple people on this board have reported being able to dislocate even in an aircast.

That's impressive but not in a good way. I think it is definitely worth trying though and don't even think of walking around the flat without a good support that won't allow you to turn your ankle. Just a quick walk to the toilet is all it would take to reinjure it and you have to start the healing all over. (Been there, done that several times)
Try to keep positive though. It took several years but I worked back up to walking miles and miles. One summer I was keeping track of my walking and every day was over 5 miles and many days were in the 10 - 15 mile range. It did make the ankle swell, it always will but I beat that stupid broken ankle eventually. My walking is limited now due to my hips. The ankle is doing quite well.