A secondhand bike is often better value than a very cheap new one.
A lot of people have bought bikes from The Bike Station It used to be that you had to queue up on a Saturday morning, but now they are more reasonable! (Closed Sun & Mon)
Some of the smaller bike shops may have bikes they have taken in part-exchange. Shops that hire will sell bikes from their fleet occasionally.
Some secondhand/junk/charity shops have bikes (none listed on this site).