I was in Iceland in 1994. Yes, to expensive prices. However, we camped every night, including in Reykjavik. Made our own food in the campsite. Quite doable and not nearly as expensive. Maybe two or three restaurant visits. Nearly every little hamlet has a Euro-style campground (basically an open grassy area). For a few Kroener each day we were able to use the local swimming pool and showers, including the lovely hot tubs.
We drove around the south side, then up the east, around to Akureyri (second largest city) in the north. Then down through the middle. We beat the ever loving crap out of the rented Mitsubishi station wagon (manual shift AWD). This included finding some really great locations off the main highway. Dirt roads are the most common . We crossed a lot of streams. This involved getting out and walking the route to find the way to drive through, avoiding rocks or unexpected drops. Most weren't that bad, at least where we went. Despite all that planning we managed to suck water into the engine on our way south from Akureyri. The water was just too high. Let me tell you that climbing out of the window and walking through thigh-high glacial melt water is very exciting. Managed to push the car up onto a gravel rise to get it more out of the water and out of the main "road." Luckily we got a tow to the campsite from a local with a properly equipped 4x4 pick-up. I slept like a rock that night. Then took the bus back to Reykjavik. And ate crow with the rental company, as you might imagine.
It's an absolutely beautiful country and I plan to get back there someday. I raise the above just as a word of warning - it is a rough countryside. Driving the main road around the island isn't bad, but it gets very rough very quickly once you leave the main highway. There is great hiking, incredible scenery, and you'll meet other interesting tourists and the locals are great (OK, I met one drunk local who was a jerk in Akureyri). Just be cautious with your expectations and abilities, particularly vehicles. It would very easy to ride outside of many people's abilities in Iceland and the rain/meltwater can make things less than romantic pretty fast. I did meet a German who was taking a dirtbike around the country. Saw him cross a few streams. So it's doable on a bike, but I'd keep it a lightweight model if going off the main roads to explore.
You'll love Iceland. Get there. If you are DIY and not doing a planned tour, just be prepared. Services are very sparse. Part of what makes it so beautiful.