i ask myself this question often.
a lot of blog posts claim 5-step processes to address it, but they're never written by serial entrepreneurs. they are written by writers.
to start, you don't come up with [good] ideas in a room with foam walls. you need to go outside.
a lot of folks, software developers especially, overemphasize their tech chops and assume they can code their way out of any problem. but often this doesn't work, because when you look into things they're almost always more complicated than they appear on the outside.
for example, suppose i'm a parent (i'm not). i could probably name 15 problems that you as a non-parent would never think of. this is called insight. but let's go a step further and suppose i want to develop a solution for one of my problems.
perhaps i want assurance my 14 year old kid gets home safely after school. i could build an app that tracks their location, but that means my 14 year old needs an iPhone. or i could see if another app already does this, like FindMyFriends or Foursquare or Facebook (my kid could "check in"). or i could just ask my kid to call me on a house landline whenever they get home from school.
here we started with a reasonable problem, knowing a child is safe. then it escalated, took a few turns, and we lost motivation because there are suitable existing solutions. at what point do we know if this was a good idea?
so first you have an insight, which manifests itself as an idea, but then you need to develop it into a solution to a problem people will pay you to fix for them.
the best way to develop insights is to observe. the best way to develop ideas is to link dissimilar things with juxtaposition. and the best way to determine if people will pay you for something, is to ask.