to pay amounts to which the holder of the note is entitled in accordance with an insurance or sale agreement under this section accruing between the date of any payment by the borrower to the Secretary and the date of transmittal of any such payments to the holder of the note; and in the discretion of the Secretary, payments other than final payments need not be remitted to the holder until due or until the next agreed annual or semiannual remittance date;