Pay Back Act This bill prohibits the Departments of Education, Justice, or the Treasury from taking any action to cancel or forgive the outstanding balances (or portion of balances) of covered loans or otherwise modify the terms or conditions of covered loans unless expressly authorized by Congress. Covered loans refer to Federal Family Education Loans, Federal Direct Loans, Federal Perkins Loans, and loans under the Health Education Assistance Loan Program.