I've beaten P3 and P4 to a pulp, have played a little of P2 and am currently playing through P1 PSP--some thoughts on Persona generally:
--The original US P1 PS1 release is worth looking at for cheese value, but not much more IMO. The game is OK, but there are just so many game systems that were streamlined in much better ways in 3 & 4. I'm going to finish P1 PSP, but the experience has reinforced my opinions about how much better P3 and P4 (but only the PS2 versions) are.
--I admittedly have not played through P2 in a significant way, but it felt like a much more cleaned up P1, which is not a bad thing. I might give it a go at some point, since I've heard a number of people say it's one, if not the, best of the 4.
--P3...I personally love a lot about P3, but Tartarus is kind of awful. Outside of that, all the remakes make the game easier than the original, but FES has a number of quality of life improvements I'm a big fan of with the optional Hard difficulty, which is a sufficient substitute for all the nerfs from the original. The PSP version just makes things even easier, and even though I'm not a glutton for super difficult games, it really felt far, far too easy. I'd recommend FES if you can get it, but only "The Journey" if you don't like Tartarus--"The Answer" is just one giant maze of dungeons, and I thought it ruined some stuff from the normal plot.
--P4 is probably my personal favorite in a number of ways. I always play on Hard, and that has always felt like just the right amount of difficulty for me--enough to force me to prep well, but not enough to force me to power-level for hours and hours. The dungeons are much less tedious and more manageable for me in P4, and I really, really liked the individual themes of them quite a bit. It's not as dark as P3, but the plot really grew on me. The Vita version makes everything WAY too easy though (not to mention adds some things I wasn't a big fan of), so I avoid it like the plague.
P5 looks like it combines a lot of stuff I liked from P3 and P4, with some stuff I was neutral on from P1 & 2, so I'll be looking forward to possibly getting my hands on it next summer when I get some time.