When enabling Modern Authentication access to Office 365 Exchange a POP or EWS connection is made from your intranet to the Office 365 Cloud which then makes a call back to your ADFS as a third leg authentication.
If you have MFA (Multi Factor Authentication) enabled then this will fail as the O365 Cloud doesn’t represent that MFA back to the client.
The fix is to create an Account Control Policy to allow a group of accounts to bypass MFA, however the Access Control Policy logic is little counter intuitive. A call to MS Support ended up resulting in the following Policy.