This problem has plagued me. I have tried to get round it by using Endnote and attaching the pdfs to my references but then I have pdfs that aren't journal articles and I have to remember which Endnote library I put the pdf in.
Last week I was introduced to Mendeley (http://www.mendeley.com/) and suddenly the future looks a little less pdf-messy.
This software is basically a place to dump our pdfs. You can download a desktop version that synchronises with the web version. This means you can also access your pdfs wherever you are - even on your smartphone. Not only that, if you are working in a small group and want to share your pdfs between you, you can easily do that too.
There are no folders in Mendeley. Instead you need to tag your pdfs so you can find them easily, though you can also order them by the keywords that were included in the original article by the author. All in all, I'm liking this. The only proviso is that there is a limit to the free option online, though at 500MB I reckon it'll be a while before that gets hit.