Section A - Shows where to find the Template/Layout Picker on Category Pages, Product Pages, and Content Pages
Section B - Shows how to use the Template/Layout Picker.
Section A
Content Page
Go to Content > Pages
Select the Content Page you want to change

Scroll down the Page Specific Layout settings and see Section B
Product Page
Go to Catalog > Products
Find the Product you want to change and click the Edit Icon ( ), the Item Number, or the Product Name to edit the product.

Click on the "Themes" menu. It is the last item in left column and see Section B

Section B
This is the Template/Layout Picker.

You are able to use the dropdowns to assign the Content/Category/Product page to a certain Template OR Layout for each theme.
The dropdown will pull all available templates and layouts for that page type for each theme.
You can only assign EITHER a Template OR a Layout but not both.