There's an easy way to tell the difference between sweet potatoes and yams: If you picked up the tuber from a typical American grocery store, it's probably a sweet potato. Almost all of the foods most of us think of as yams are really sweet potatoes. The root vegetable typically has brown or reddish skin with a starchy inside that's orange (though it can also be white or purple).
Yams are a different beast altogether. They are more closely related to lilies and grasses and mostly grow in tropical environments. The skin is more rough and bark-like than what you'd see on a sweet potato, and the inside is usually white or yellowish—not orange.