On round two of trying this:
Step One:
Each Alexa is named after the room (Bedroom, Kitchen, etc)
Each light is named after the room it is in + " Light" so "Bedroom Light"
I made a group for each room with the same name "Bedroom","Kitchen" and assigned its respective Light to it.
Step Two:
On the mobile App (not possible from web): goto SmartHome->Groups
select a group (the ones you named after rooms)
click Edit (a listing of Echo devices, normal devices and scenes will appear)
select the Echo device that represents that room
save it
repeat for each Group
I couldn't find this initially and it is the critical part.
Step Three:
repeat the process of discovering devices, making groups and assigning Echos for EACH user account you have in your family because Amazon doesn't share any of this info across accounts. I get that skills might use different accounts for linking, but that's a PITA if you are managing automation for a household.
Now I can say "Alexa, turn on the light" while sitting in the bedroom and the light will turn on in that room only.
If I say "Alexa, turn on the lights" it will turn on every light in the house, so keep your plural and singular in mind
I just got this working and tested it after a flurry of other posts and reading and trying things out.
It works.