Why do you think he does this?

was the split by mutual consent? could be he's feeling confused right now. if his divorce was bitter and his ex-wife used their daughter as a tool could be he thinks you might do the same. He may love you and have feeling for you which might explain why he was upset about going so long without talking to you. as for acting distant, if his divorce was bitter he might feel he's getting to close ,therefore he pushes you away in fear of getting hurt. Sometimes when previous relationship's go bad and there is bitter feelings it carries over into the next. and takes a while to get over. something like that takes alot of patience