It's called the 'Top Sites' feature. By manually removing the entries you don't actually remove the feature itself which is what most people want to do. You can disable this feature or modify it by going to Menu > Home > Firefox Home Content. From here you can choose whether these tiles appear or not and how many rows they take up.
You can also enable/disable this feature through about:config with the follow preference:
Code:
browser.newtabpage.activity-stream.feeds.topsites
= false (false is an example, it could be true and therefore enabled)
And this one allows you to modify the amount of rows the 'Top Sites' feature takes up (numbered):
Code:
browser.newtabpage.activity-stream.topSitesRows
= 1 (1 is an example, it could be 2 or 3 or 4 etc)
For better privacy it's recommended to disable this feature.
Another feature closely related is the 'Mozilla Snippets' feature which places content from Mozilla on the home-page (if not blank or custom). This can be annoying sometimes if you don't want to keep seeing stuff that might be irrelevant. Besides you won't miss updates and/or important information if you disable it as you will still get this.
You can enable/disable this feature in about:config with this preference:
Code:
browser.newtabpage.activity-stream.feeds.snippets
= false (or true, depending on your own preferences)
If you want to avoid content like this when Firefox loads you can opt to have Firefox start with a blank page everytime it starts. You can do this like all others through about:config with this preference:
Code:
browser.startup.blankWindow
= true (it can also be false if you want it to be, again, true is an example value)
This preference ensures every tab you open will be a blank page if you want things to be clean and more private:
= about.blank