The Freedom to do as you choose.
Your goal is really just about anything you want it to be but generally, if you keep playing you will level up and collect all sorts of weapons and armor. Most likely, you'll end up looking like the guy on the front of the box. The real question of the game is whether you will be a savior or destroyer.
You'll quickly find that the wasteland isn't actually dead. There aren't just people out there, but there are towns. And of course there are mutant mole rats, mutant ants, mutant crabs, and even mutant mutants. It's pretty inhospitable place, even at the best of times but don't expect to be completely alone.
The main story is actually about you trying to find your father after he left the vault (which nobody has ever done) that you've lived in your entire life. This will take across pretty much the entire area, fighting, talking, and surviving your way to him. This is just a minor part of the game though. Really it's just one quest series that will probably take you a few hours to get through (depending on your difficulty). The story itself isn't too surprising and the voice acting is alright, except for Liam Neeson who is Liam Neeson. That isn't to say that the main quest isn't worth doing, just that it's a very small part of the game.
Where the game really shines is in the world itself. Basically, once you step out of that vault, you can go pretty much any where and do whatever you would like. There are no restrictions aside from having a big enough gun and enough ammo to get you there.
That isn't to say that there aren't consequences to the choices you make. Sometimes, you can lock yourself out of finishing a quest or even starting one depending on your actions. You can make entire towns hate you. The biggest thing I can tell you is to NOT finish the main story before you've done everything else you wanted because the game ends there and you don't return (unless you have the Broken Steel DLC). So you have the freedom to do as you choose, but you also have the freedom to make mistakes so choose wisely.
Of course, what fun are choices if they don't even let you choose how to kill (or not kill) something? RPG has always been a huge part of Fallout and it's back. You get to choose your Attributes (which modify your skills and allow you to choose certain perks), raise which skills you deem important, and choose perks every level. Perks are basically modifiers for your character. For instance, you will eventually have the option of picking up the "Pack Mule" perk which allows you to carry an extra 50 lbs of weight before your inventory slows you down. These range from simply adding a few extra points to a skill all the way up to exploding (you don't die) when your health falls below a certain threshold.
Which leaves us with combat. The combat is pretty fun even if it somewhat devolves into a simple system. You collect different clothes and armors as you go through the game that give you different bonuses and protections all the way up to Power Armor, which requires special training to wear. You'll also be able to collect a variety of weapons falling into different weapon skills. Soon, you'll discover that the most difficult choice isn't which weapons and armor to take with you, but which to drop or sell.
Most of the weapons in the game come in the form of different guns but there are some melee weapons as well. You can play the game from a first person or third person perspective. If you play from the first person perspective, it's very easy to turn the game into a first person shooter with heavy RPG elements.
But for those who are not quite as twitchy, there is a feature in the game called VATS (Vault-Tec Assisted Targeting System) where the game will pause and you'll be able to pick where you want to aim. You will be given different body parts that you can aim at and the chance that you will successfully hit that body part, then you can choose where to, cue up a few shots at once with each one eating up Action Points which refill relatively quickly, then hit go. What follows is a slow motion barrage of shots on your enemy. It's very fun to see, especially when you get better weapons and can cue up a few deadly headshots on multiple different enemies.
Unfortunately, there are a few problems with the game. One of the biggest is that the game environment is rather depressing. You'll spend a lot of time having fun, but you'll also spend very little time in gleeful awe of your surroundings. That's not to say you won't spend time in awe, because the landscape is pretty incredible and impressive, but you'll be hard pressed to like what you see.
Aside from that, the combat eventually devolves into a pretty simple system. Either the enemy will be weak enough that you'll use a precision weapon from a distance to kill it in one shot OR you'll run as close to it as you can and shoot it with your shotgun, likely killing it in one shot. There are plenty of options and you'll probably do different things for fun, but these are by far the most efficient ways of killing enemies.
And the Bugs. At this point, as long as you have all the DLC and/or are connected to the internet, you shouldn't really have an issue with bugs. Most of them have been worked out but there are still a few annoying ones that pop up. If you don't have your game updated, you are in for a veritable plague. The game runs on the same engine as Oblivion and has almost all of the same problems that that game had. It is not nearly as bad on the 360 as it can be on the PC where the bugs can EASILY (no exaggeration, you can look online for the reports) render the game unplayable. As long as you update your game you should be fine though.
Getting the GOTY edition is worth it if you're planning on getting the game anyway. The DLCs don't really change up the formula any but they add new areas, new quests, and new items to the game which is good since the game itself is pretty fun. The DLCs are just a little too expensive to be worth it on their own. It's pretty difficult to get used PC games these days so if you were to buy the game and the DLC separately at Gamestop, it would run you about $70 as opposed to $50 for the GOTY edition new. This is also opposed to $60 GOTY edition for the XBox (and PS3, didn't play that though), making this the best value even without the community made content (next paragraph).
The PC version is easily superior if you can get beyond the bugs. Bethesda has once again opened up its game to the community allowing anybody to modify it and make those mods public. You can find all sorts of mods from Infinite Ammo to entire quest lines and towns. It is highly recommended that you play the game through before you start tampering with it though. This is both so that you experience the game as it was designed and so that you know whether or not your improving or destroying the game when you change something.
Really, all in all it's a great game. Take a day when you have nothing to do, go to a game shop, buy the game for the PC and try and play it for a few hours, at least until you've done a quest or two outside of the vault. If all goes well, keep it. If not, return it to the store and say you couldn't get it to work on your PC and pick it up for one of the consoles instead.