eric, aside from what you already listed, a few of my personal recommendations would be:
Lumines I & II - these are extremely tight, stylish puzzle games that get madly addicting. the first game has original music, the second has licensed pop and electronic music. my copy of Lumines II has essentially fallen apart from being played so much.
Ultimate Ghosts N Goblins - brand new GnG game with some modern flair. it's tough, fun, and usually less than 5 bucks.
Loco Roco 1 & 2 - both games are fantastic and basically more of the same thing, but its probably the most vibrant, fresh, and original game on the system.
Crush - Crush is another highly underrated game. This is probably one of the coolest puzzle games you will play. You can shift from a 2D to 3D world in order to solve puzzles. creates a lot of really cool perspective puzzles in how to get to certain areas.
Wipeout Pure - Pure and Pulse are interchangeable, really, but I liked Pure's progression a bit better, and as a bonus, there's like a half dozen free packs of downloadable content out there that add a ton of new vehicles and entire new circuits. You have to get them from random websites now since the wipeout site is gone, but it is worth it.
Metal Gear Solid: Portable Ops - This game has sort of been forgotten, and it's a bit of a strange MGS game, but it does contain all the classic kind of gameplay and a better than average story. You end up collecting soldiers and you can then use them as playable characters in the game. it has an overworld map and all kinds of optional missions, not all that different from Metal Gear Acid (also one of my favorite games for the system) it's sorta weird, but neat. (also, just a note if you see Portable Ops Plus, you should likely ignore it. It's just a multiplayer expansion pack for Portable Ops, but the box does not make that terribly clear. I don't think it's worth having.)
Me & My Katamari - the revised control scheme takes some getting used to, but once you do, this is a fully fledged original game in the series. my wife enjoyed it. it does reuse some assets like most of the more recent ones but it's still original enough and fun as is.
Burnout Legends - The load times are shitty at times, but it's basically a fully fledged Burnout game with some nice shorter, quicker events for on the go gaming.
Castlevania: The Dracula X Chronicles - you basically get three games for the price of one, and this has yet to start skyrocketing in price like some other CV games from the same era.
Capcom's collections - Capcom released like 4 worthwhile collections on the system. There's two arcade collections with a lot of good content, a puzzle collection with Puzzle Fighter and Buster Bros, and Power Stone collection with both games. SNK also did some solid collections for their arcade stuff, Metal Slug, and I think one for KoF as well.
Crazy Taxi: Fare Wars - ok so i realize i may be the only one that cares about this, but it's a solid collection, honest! both Crazy Taxi 1 and 2, tons of unlockables, tons of new settings to spice up the games, and its clear they put a lot of thought into how to make this an attractive package. music is all awful no-name garbage though.
Pursuit Force: Extreme Justice - if you want a good game for a laugh, please pick this up. it's so over the top extreme in every aspect. the voiceovers and mission briefs are amazingly macho and melodramatic, you drive like a mad man and do extreme stunts to catch criminals, then dive wildly from car to car while chasing then down to beat/shoot them to death. it's not a "good" game per se, but it's pretty entertaining and surprisingly playable.
Soul Calibur: Broken Destiny - If you like SC games at all, this is basically a fancy port of SCIV with some extras. The Tekken games are likewise pretty faithful. I'd also recommend the Capcom fighters like Alpha 3 and Darkstalkers, even if i am not the biggest fan of the dpad controls on the system.
Extend Every Extra - it's a visually hypnotic shooter that really draws you in, even though it doesn't leave the best first impression. this version is completely different than the XBLA version, which i personally really did not like. it completely changed how the game was played.
There's a lot more I played but I'm blanking at the moment and others have listed some of the other good stuff. I have a small cache of games for psp i'm gearing up to start, including persona 3, metal gear acid 2, littlebigplanet, and the 3rd birthday. i probably should have also mentoned the patapon games since they are system exclusives, very unique, and some people like spook love them, but i just could never quite "get" them. they are cheap enough that it's probably at least worth giving it a once over.
this was my post from a few months ago. its not everything i liked or owned, but it's a bunch i enjoyed that struck my fancy as I was writing it out.