Understanding the World in Color