This is unfortunately the sort of question with a lot of answers
My first answer would be, if you've got a PC with a soundcard, there's not a massive need for any extra wee machine, you could record directly to the PC and use a piece of multi-tracking software like Audacity (
http://audacity.sourceforge.net/) to take care of the mixing. Audacity also alows for more than 4 tracks, and includes a bunch of effects, and supports VST plugins.
EDIT: Forgot to mention, Audacity is totally free, and quite easy to use
As for actual machines, there are two primary types - Analogue tape 4-tracks, and Digital 4-tracks.
The most commonly used tape model is the Tascam portastudio range. They're pretty cheap, and as a result, very basic, but they're also apparently very easy to use. They record on to regular cassette tapes, so there's likely to be a bit of hiss, as well as things like "tape compression" to give the sound a bit of "character", if you could call it that

Unfortunately to connect to PC, you'd have to opt for something fairly crude like taking a line output from the portastudio and plugging into the line-in socket on your PC sound card... Pressing record on the PC then pressing play on the tape. Here in the UK, portastudios can be bought from nearly any music shop for anywhere between £50GBP for the basic model, to around £100GBP for the more "deluxe" versions. You see a lot of them going around second hand shops/pawn shops/ebay, but although you'll get decent prices, the tape heads can well be buggered and need replacing.
Digital 4-tracks on the other hand, are generally a bit more costly, but will usually give a more clean "hi-fi" sound, and can be connected to PCs much more readily. They'll generally digitise everything and save things to either a camera-type Flash memory card or an in-built hard-drive. Hard drives however, generally only feature on more expensive models, and are mainly found on 8/16/24 track recorders. Depending on spec, a lot of digital recorders will feature some simple digital effects and amp simulators that allow you to direct record guitars and stuff. I couldn't speak for all of them, but most have the option of connecting via USB to a PC, allowing the recorder to be treated as a drive, and allowing the finished song to be carried over as a WAV file for burning to CD/Ripping to MP3 etc. I couldn't tell you as much about prices, but I'd reckon anywhere between £150-250GBP.