I just been to japan(osaka) with my husband and baby boy, actually, a trip for 5 days and 4 nights is optimal for a baby ,coz the trip will be very excited to him and he will spend a lot of energy (even though him have a short sleep in day time), after the trip, my baby have to have a long sleep for a few days to recover.
hence, i think as it is the first time you go to japan, tokyo and osaka will be more suitable, and also more convinent for taking a baby with you, because all you need for your baby is more easily accessable and the transportation time will be less from point to point, for a baby, most of them will feel very boring during travel therefore they will be angry.