When you know the Father and walk in obedience to the Father, you are going to find yourself in conflict with those who do not. Jesus is speaking into why there is conflict between Him and the crowd. And the reason was this: He knows the fathers and lives according to His word and they do not. You see, the crowd simply knew OF THE FATHER. Jesus was walking in obedience TO THE FATHER.