In order to script the connection to exchange online with mfa, you must first download the Microsoft.Exchange.Management.ExoPowershellModule.dll
I assume you will have to periodically update the dll via the EAC GUI.
Open the Exchange admin center (EAC) for your Exchange Online organization.
In the EAC, go to Hybrid > Setup and click the appropriate Configure button to download the Exchange Online Remote PowerShell Module for multi-factor authentication.
In the Application Install window that opens, click Install.
After you get this working you can proceed.
Now to connect from Powershell:
Import-Module $((Get-ChildItem -Path $($env:LOCALAPPDATA+”\Apps\2.0\”) -Filter Microsoft.Exchange.Management.ExoPowershellModule.dll -Recurse ).FullName|?{$_ -notmatch “_none_”}|select -First 1)
$EXOSession = New-ExoPSSession
Import-PSSession $EXOSession
More Details found here.