Pretty much what Lydia said.
I have been building my own computers for some time now (because I can usually spare more than 30% of the usual prices offered in large stores/malls, the thing to avoid because their configs are utter crap and not very good price wise). If you have some resellers and small private shops around, you might want to take an eye with them (at your own risks, do not go anywhere blindly, some people are not very trustworthy when it comes to replacement or warranty). Don't know how it is in other countries but the chinese district nearby has incredible prices compared to the rest.
- Choose your case wisely, some of them are maybe more expensive, but they can also offer a damn welcomed ergonomic side with backward storage for all the wiring, or screwless storage racks for hard drives, dvd drives, etc.
- If you are not sure to be able to handle the motherboard/cpu integration (with the delicate thermic paste part of the operation, which is quite easy actually), try to find a seller that also offers to build your config for you. If you want to learn though, if you know someone that knows what he is talking about, then ask him to show you. Or just follow carefully the user guides because eventually, it is not very hard.
- The hard part is actually to know what to buy, and in what order. I usually try to look at first which cpu i want. Depending on the CPU choice will depend the choice of the motherboard (because they are cpu socket specific). Then the motherboard should offer you the last in terms of RAM memory and USB capability, so be sure to look for latest motherboards and not old ones (but if you buy a new CPU and not an outdated one, the motherboard will be in line). Then you have done half of the work (because there is a crapload of manufacturers for motherboards, so I usually go for my favorites : Gygabite or Asus, though I prefer Gigabyte). Choosing between Intel and AMD for the CPU depends on your tastes, mostly. I like Intel because they are currently more advanced in terms of performances, but both are good. The new i3, i5, i7 generation of cpus is a good bet (have an i5 myself). The memory is the tricky part. You can now find quite cheap basic memory that works decently, or go for more powerful memory (and you will feel the difference I guarantee that, if you choose a 2133 MHz DDR3 instead of a basic 1600 MHz one). Both work in any case, depending on your budget. Then, choose your graphic card... ATI or nvidia, the eternal war. I like nvidia (and hate ATI from my own experience), but both are very performant, nvidia maybe a little more expensive but generally less heath and slightly more performant, but I may be wrong. In any case ATI is well known for their good deals/prices.
[spoiler]In more technical details :
- CPU : what is important is the number of cores of course (if you have a 2GHz cpu with 4 cores, it will be 4x 2GHz, depending if the software you use manage this or not). The slot socket standard number is the most important part : it will determine the kind motherboard on which it will actually fit. So you will go for motherboard created for this specific socket when you will come to choose it. Etching -> the lower the better (faster, less heath, more performant), current standard is around 45 nm. CPU cache L1/L2 and L3. Higher the better, it is the processor memory layer acting before the RAM memory. L1/L2 is around 1-2 Mb, a good L3 is around 8Mb.
- Motherboard : what suits you best : look for the number of USB ports, PCI internal slots if you have a lot of PCI cards to insert, PCI express (PCI-e x16) slots for the graphic card (2, 3 or 4 if you are going to put several GCs in). More generally look for all the ports offered by the motherboard and see it it suits you or not. Check if the mother board supports the SATA-3 standard (for hard drives) and USB 3. Better to go for it now, would be a shame to take an outdated model not supporting those (they are awesome). And very important : be sure the motherboard supports DDR3 memory, as you are probably going to buy DDR3 (current standard, DDR2 is outdated).
- Memory : DDR3 i suppose. Goes from around 1600 MHz (PCI 12800) up to 2300-2500 MHz for the best ones (or maybe more who knows...). The best ones are expensive. Be sure to take at least 4Gb of it (more is advised, like 6Gb, but 4 is totally ok.. 2 is too short though). CAS latency (CL) is a little advanced and specific, but if you are into it... it basically means the lower the faster. You will get 4 numbers (like 9-9-9-24). But anyway, this is what can make a little difference between 2 identical memory racks : some are more suited to AMD sockets, and somre are specifically designed for Intel sockets. Corsair or G-Skills are good brands imo. I always use G-Skill personnally, very nice memories.
- Graphic Card : check the output ports (DVI, HDMI, etc). The frequency is not very important. Best way to go for a model is to browse benchmarks on the internet between several models (ATI and nvidia), check different sites, different games, then see if they are worth their price or not. Manufacturers side : Zotac, Asus, Gigabyte, Twintech are good for nvidia ; Asus, Sapphire for ATI mostly if Im right.
- CPU fan and radiator : Totally optionnal as this is usually sold with the cpu, but the one sold with the cpu is usually pretty basic and noisy. You might want to get a better one (more durable, the basic ones tend to stop suddenly to work after one or 2 years, happened to me several times). Anyway Noctua is a very good brand (very, very silent).
- Hardrives : take SATA-3, its the current standard (and fucking fast with Sata 3 compatible motherboards, like 60Mb/s at peak on transfers). DO NOT TAKE GENERIC stuff, period. Hardrives also tend to fail after 3 or 5 years of use anyway, so if you already have some but old, maybe it is time to be cautious (or use save drives somewhere).
- Power supply : pretty much what Lydia said. The power must be sufficient to handle the GC (above 500W now for the good GCs). Check how many connectors you have available too (might be a little annoying if you can't power everything in your new shiney computer). Note that there is also a totally new kind of power supplies that is TOTALLY AWESOME : modulable power supplies -> on standard ones the supply cables are fixed on them, but here you actually have cable slots and can choose which cables you plug, and this has 2 major advantages : more versatile, and also not a fuckfest of scrambled cables everywhere. There is also another thing that is quite handy : the 80+ certification, meaning that the power supply will only consume less than 20% of its maximum consumption if it is running under 80% of its performance, which is very appreciable (and green).
Im not going for the rest of the little stuff around (dvd drives, burners, etc), not very important, take what you like. Sorry for the wall of text. Hope it helps.[/spoiler]
Rule number 1 : always decide of EVERYTHING before going to buy.