Jax has never been or wanted to be a parent who occasionally sees his child. He has never been presented as the kind of man content to live apart from his child.
If Jax said he was going to live in New York and get Joss two weekends a month, I'd be better with that, but vague promises to see the child he has to pee in a cup to visit is pathetic on his part.
And Carly? She's not interested in her kid. So Jax looks even worse leaving his child in the care of a woman who does not give a shyte in the slightest.
He left her behind. He's a deadbeat.