Must be some rust that is making the caliper hard to remove.
Removing the rear rotors with drums inside is another matter, remember to loosen up the brake shoe adjuster inside under the rubber plug. If you are lucky and don't live in the salt belt, after adjusting the brake shoe adjustment all the way, the rust ridge will not prevent you from pulling the drum. I live in Illinois and had to beat the rotor/drum assembly a lot and on one side split the drum to get it off. The shoes and hardware were trashed and I had to drill out the aluminum pop rivets that hold on the backing plate onto the axle and secure it with socket head cap screws.

Of course you will need to pull the rear axles if the backing plate rivets fail when you pull the rotors/drums.
http://s362.photobucket.com/albums/oo69 ... %20plates/Good luck, hope you don't have any salt around you.