Don't give up, as least you're try your best. It will improve when he growth up cos he will know who is his mother.
As the same time, you could do something with him, just like you could read a story for him, or sing a song before bed time. It only take you 5-10 minutes but you need to be patience cos it won't change suddenly, but time will let you know.
Also, you need to stop him if he really doing some which is dangerous for him.
-Turn off the light and sing lullabies with him.
-Read books with him before going to bed.
-Go to theme parks or zoo when you have time.
-Be more patience , hug him more even you're very tired.
-Buy him some healthy snacks or his favourite food.
-He is at 'Trouble Two', read some parenting books to help yourself.
-Tell him 'I love you!' every night.
-Hope these help!