There's an herbalist outside of the inn or the witch Abigail that will sell ingredients. Taking herbalism is a good use for a bronze talent, so you don't have to waste money buying ingredients.
You can unlock some herbs/monster components buy talking to one of the 'old woman' that you'll see outside their huts (need to bribe them with food first). Also, do a drinking game with a drunk at the inn, and he'll give you a book that unlocks some monster ingredients if you win. --- We were going full speed ahead... but facing backwards the whole time.