A secondhand bike is often better value than a very cheap new one.
A lot of people have bought bikes from The Bike Station It used to be that you had to queue up on a Saturday morning, but now they are more reasonable! (Closed Sun & Mon)
Eastside usually has a selection and Soul specialises in secondhand bikes.
Some of the smaller bike shops may have bikes they have taken in part-exchange. Shops that hire will sell bikes from their fleet occasionally.
Some secondhand/junk/charity shops have bikes (none listed on this site).
Bicycle shops in Edinburgh 
Especially for kids bikes, there are also Nearly new sales like http://www.nearlynewsale.org.uk/ and lots of schools run bike sales as part of the summer fetes. Some real bargains to be had – especially if you’re looking for things like tagalongs. And the profits mostly go to charity.