After Matthew's call Jesus went home with him to eat. When the pharisees asked why, Jesus gave them a straight answer about their hypocrisy; people who will not recognize that they are sick won't go to the doctor.
Also, we need to see that it was Matthew's party. He had the privilege of inviting who he wanted. He invited his friends. His new friends, Jesus and His disciples and his old friends, the publicans and sinners.
There is, I believe, another reason why Jesus ate with them. Scripture dose not point this out, and I do not claim to speak for the Lord on this matter, but I can't help but feel that Jesus spent time with publicans and sinners because they were more fun to be around.
YIC
William