order_items.product_id contains at least one NULL. That is enough to make the whole thing return nothing.
x NOT IN (1, 2, NULL) expands to x <> 1 AND x <> 2 AND x <> NULL, and that last comparison is UNKNOWN rather than true — so the row is never returned, for any value of x. One NULL in the subquery empties the entire result.
Use NOT EXISTS, which is null-safe and usually gets a better plan as well:
SELECT p.*
FROM products p
WHERE NOT EXISTS (
SELECT 1 FROM order_items oi WHERE oi.product_id = p.id
);
Your LEFT JOIN version works for the same reason — it compares rows rather than evaluating a list containing NULL.
Once it is fixed, the next question is why that column is nullable at all. If an order item must always reference a product, a NOT NULL constraint stops this class of bug at the source instead of in every query that touches it.