Your empire is formed of countries and cities. Cities bring you income and provide military units - but only if you control the country capital. Controlling all cities in the country will also bring you bonus income (you can see it in the Country info menu or in the Finances menu).
Once you capture the country capital (marked with a star), you start receiving income from all the cities you control and are able to buy units from these cities.
